Brotli Compression with mod_pagespeed and ngx_pagespeed

The PageSpeed modules (not to be confused with the PageSpeed Insights site analysis service), are open-source webserver modules that optimize your site automatically. Namely, there is mod_pagespeed for the Apache server and ngx_pagespeed for the Nginx server. For example, PageSpeed can automatically create WebP versions for all your image resources, and conditionally only serve the format to clients that accept image/webp. I use it on this very blog, inspect a request to any JPEG image and see how on supporting browsers it gets served as WebP.

Chrome DevTools showing a request for a JPEG image that gets served as WebP

The impact of Brotli compression

When it comes to compression, Brotli really makes a difference. Brotli compression is only supported over HTTPS and is requested by clients by including br in the accept-encoding header. In practice, Chrome sends accept-encoding: gzip, deflate, br. As an example for the positive impact compared to gzip, check out a recent case study shared by Addy Osmani in which the web team of the hotel company Treebo share their Tale of Brotli Compression.

PageSpeed does not support Brotli yet

While both webservers support Brotli compression out of the box, Apache via mod_brotli and Nginx via ngx_brotli, one thing that PageSpeed is currently missing is native Brotli support, causing resources that went through any PageSpeed optimization step to not be Brotli-encoded 😔. PageSpeed is really smart about compression in general, for instance, it always automatically enables mod_deflate for compression, optionally adds an accept-encoding: gzip header to requests that lack it, and automatically gzips compressable resources as they are stored in the cache, but Brotli support is just not there yet. The good news is that it is being worked on, GitHub Issue #1148 tracks the effort.

Making Brotli work with PageSpeed

The even better news is that while we are waiting for native Brotli support in PageSpeed, we can just outsource Brotli compression to the underlying webserver. To do so, simply disable PageSpeed's HTTPCache Compression. Quoting from the documentation:

To configure cache compression, set HttpCacheCompressionLevel to values between -1 and 9, with 0 being off, -1 being gzip's default compression, and 9 being maximum compression.

📢 So to make PageSpeed work with Brotli, what you want in your pagespeed.conf file is a new line:

# Disable PageSpeed's gzip compression, so the server's
# native Brotli compression kicks in via `mod_brotli`
# or `ngx_brotli`.
ModPagespeedHttpCacheCompressionLevel 0

One thing to have an eye on is server load. Brotli is more demanding than gzip, so for your static resources, you probably want to serve pre-compressed content wherever possible, and for the odd chance of when you are Facebook, maybe disable Brotli for your dynamic resources.

Chrome DevTools Network panel showing traffic for this blog with resources served Brotli-compressed highlighted

Happy Brotli serving, and, by the way, in case you ever wondered, Brotli is a 🇨🇭 Swiss German word for a bread roll and literally means "small bread".

Thomas Steiner
This post appeared first on https://blog.tomayac.com/2020/01/24/brotli-compression-with-mod-pagespeed-and-ngx-pagespeed/.

Progressive Enhancement In the Age of Fugu APIs

Back in March 2003, Nick Finck and Steven Champeon stunned the web design world with the concept of progressive enhancement:

Rather than hoping for graceful degradation, [progressive enhancement] builds documents for the least capable or differently capable devices first, then moves on to enhance those documents with separate logic for presentation, in ways that don't place an undue burden on baseline devices but which allow a richer experience for those users with modern graphical browser software.

While in 2003, progressive enhancement was mostly about using presentational features like at the time modern CSS properties, unobtrusive JavaScript for improved usability, and even nowadays basic things like Scalable Vector Graphics; I see progressive enhancement in 2020 as being about using new functional browser capabilities.

Sometimes we agree to disagree

Feature support for core JavaScript language features by major browsers is great. Kangax' ECMAScript 2016+ compatibility table is almost all green, and browser vendors generally agree and are quick to implement. In contrast, there is less agreement on what we colloquially call Fugu 🐡 features. In Project Fugu, our objective is the following:

Enable web apps to do anything native apps can, by exposing the capabilities of native platforms to the web platform, while maintaining user security, privacy, trust, and other core tenets of the web.

You can see all the capabilities we want to tackle in the context of the project by having a look at our Fugu API tracker. I have also written about Project Fugu at W3C TPAC 2019.

To get an impression of the debate around these features when it comes to the different browser vendors, I recommend reading the discussions around the request for a WebKit position on Web NFC or the request for a Mozilla position on screen Wake Lock (both discussions contain links to the particular specs in question). In some cases, the result of these positioning threads might be a "we agree to disagree". And that's fine.

Progressive enhancement for Fugu features

As a result of this disagreement, some Fugu features will probably never be implemented by all browser vendors. But what does this mean for developers? Now and then, in 2003 just like in 2020, feature detection plays a central role. Before using a potentially future new browser capability like, say, the Native File System API, developers need to feature-detect the presence of the API. For the Native File System API, it might look like this:

if ('chooseFileSystemEntries' in window) {
// Yay, the Native File System API is available! 💾
} else {
// Nay, a legacy approach is required. 😔
}

In the worst case, there is no legacy approach (the else branch in the code snippet above). Some Fugu features are so groundbreakingly new that there simply is no replacement. The Contact Picker API (that allows users to select contacts from their device's native contact manager) is such an example.

But in other cases, like with the Native File System API, developers can fall back to <a download> for saving and <input type="file"> for opening files. The experience will not be the same (while you can open a file, you cannot write back to it; you will always create a new file that will land in your Downloads folder), but it is the next best thing.

A suboptimal way to deal with this situation would be to force users to load both code paths, the legacy approach and the new approach. Luckily, dynamic import() makes differential loading feasible and—as a stage 4 of the TC39 process feature—has great browser support.

Experimenting with browser-nativefs

I have been exploring this pattern of progressively enhancing a web application with Fugu features. The other day, I came across an interesting project by Christopher Chedeau, who also goes by @Vjeux on most places on the Internet. Christopher blogged about a new app of his, Excalidraw, and how the project "exploded" (in a positive sense). Made curious from the blog post, I played with the app myself and immediately thought that it could profit from the Native File System API. I opened an initial Pull Request that was quickly merged and that implements the fallback scenario mentioned above, but I was not really happy with the code duplication I had introduced.

Excalidraw web app with open "file save" dialog.

As the logical next step, I created an experimental library that supports the differential loading pattern via dynamic import(). Introducing browser-nativefs, an abstraction layer that exposes two functions, fileOpen() and fileSave(), which under the hood either use the Native File System API or the <a download> and <input type="file"> legacy approach. A Pull Request based on this library is now merged into Excalidraw, and so far it seems to work fine (only the dynamic import() breaks CodeSandbox, likely a known issue). You can see the core API of the library below.

// The imported methods will use the Native File
// System API or a fallback implementation.
import {
fileOpen,
fileSave,
} from 'https://unpkg.com/browser-nativefs';

(async () => {
// Open a file.
const blob = await fileOpen({
mimeTypes: ['image/*'],
});

// Open multiple files.
const blobs = await fileOpen({
mimeTypes: ['image/*'],
multiple: true,
});

// Save a file.
await fileSave(blob, {
fileName: 'Untitled.png',
});
})();

Polyfill or ponyfill or abstraction

Triggered by this project, I provided some feedback on the Native File System specification:

  • #146 on the API shape and the naming.
  • #148 on whether a File object should have an attribute that points to its associated FileSystemHandle.
  • #149 on the ability to provide a name hint for a to-be-saved file.

There are several other open issues for the API, and its shape is not stable yet. Some of the API's concepts like FileSystemHandle only make sense when used with the actual API, but not with a legacy fallback, so polyfilling or ponyfilling (as pointed out by my colleague Jeff Posnick) is—in my humble opinion—less of an option, at least for the moment.

My current thinking goes more in the direction of positioning this library as an abstraction like jQuery's $.ajax() or Axios' axios.get(), which a significant amount of developers still prefer even over newer APIs like fetch(). In a similar vein, Node.js offers a function fsPromises.readFile() that—apart from a FileHandle—also just takes a filename path string, that is, it acts as an optional shortcut to fsPromises.open(), which returns a FileHandle that one can then use with filehandle.readFile() that finally returns a Buffer or a string, just like fsPromises.readFile().

Thus, should the Native File System API then just have a window.readFile() method? Maybe. But more recently the trend seems to be to rather expose generic tools like AbortController that can be used to cancel many things, including fetch() rather than more specific mechanisms. When the lower-level primitives are there, developers can build abstractions on top, and optionally never expose the primitives, just like the fileOpen() and fileSave() methods in browser-nativefs that one can (but never has to) perfectly use without ever touching a FileSystemHandle.

Conclusion

Progressive enhancement in the age of Fugu APIs in my opinion is more alive than ever. I have shown the concept at the example of the Native File System API, but there are several other new API proposals where this idea (which by no means I claim as new) could be applied. For instance, the Shape Detection API can fall back to JavaScript or Web Assembly libraries, as shown in the Perception Toolkit. Another example is the (screen) Wake Lock API that can fall back to playing an invisible video, which is the way NoSleep.js implements it. As I wrote above, the experience probably will not be the same, but the next best thing. If you want, give browser-nativefs a try.

Thomas Steiner
This post appeared first on https://blog.tomayac.com/2020/01/23/progressive-enhancement-in-the-age-of-fugu-apis/.