The future of device detection on the web

Knowing what kind of device is accessing your website can enable you to scale image resolutions up or down or adapt to device capabilities, delivering a better user experience for everyone. Traditionally we've all done this by parsing and acting on the User-Agent HTTP header. More recently Client Hints has offered a new, more evolved approach. And now User-Agent headers are starting to go away. But don’t panic!

As most developers will attest, the User-Agent header is a bit of a weird beast, and after three decades of an unrestrained arms race between browser developers and web developers, the typical User-Agent header is a complete mess. This is the one the browser I'm using right now sends with every request:

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36

This is absolute madness. I'm not using "Mozilla" (at least not in the sense it refers to here!), my Mac does not contain an Intel chip, and the browser is not powered by WebKit, nor KHTML, nor Gecko. It is not Safari. The one single correct piece of information in this header value is "Chrome/120.0.0.0". The rest is pure theater intended to try and improve compatibility with naive filters.

Privacy and user-agent reduction

The length and variety of information in the User-Agent header also ends up being bad for user privacy. The W3C's Technical Architecture Group documented back in 2015 the problem of unsanctioned tracking (I later served on the TAG for two years). Combining all the values from the headers your browser is sending every time you navigate on the web, website operators could potentially identify you uniquely. 

That means, if you are known in one place (because you logged in) and then you go someplace else where you think you're anonymous, you are still transmitting the same unique fingerprint that means you're still identifying yourself, whether you want to or not.

Tackling this problem, Google initiated the User-Agent reduction program, with the aim of reducing the uniqueness of this information. That's actually why the value of my own User-Agent header (the one at the top of this post) cites that very suspiciously round-number version of Chrome, and why it says I have an Intel Mac when I don't.

User-Agent reduction is a compromise between providing a measure of useful information, and not providing so much specific data that a user ends up being unique. You can check how unique your own browser is at amiunique.org.

Device detection at the edge

At Fastly, we support this program - it's good for users and good for the web. But Fastly also offers device detection variables that automatically populate so you can easily customize experiences at the edge, and those are powered by an interpretation of the User-Agent header.  It allows you to do stuff like this:

if (client.display.width > 1000) {
   set req.http.Feature-Device-Size = "large";
} else {
   set req.http.Feature-Device-Size = "small";
}

The changes to the User-Agent header mean these variables provide less specific and less accurate information than they used to, and some data we could give you in the past we can't anymore because the information is no longer contained in (or possible to look up from) the user agent string. This is particularly true of highly granular values like screen width and height which are effectively no longer usable at the edge.

It's not the end for device detection at the edge though. Even the reduced UA header contains enough information for Fastly to populate the most important (and fortunately also the most popular!) device type variables, such as client.browser.name and client.platform.mobile. It's still fine to use these to provide experience customization.

Browsers are also now sending a new generation of device-type metadata in a set of dedicated headers:

Sec-Ch-Ua: "Not_A Brand";v="8", "Chromium";v="120", "Google Chrome";v="120"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "macOS"
Sec-Fetch-User: ?1

These present the data in a more structured way which makes it easier to Vary on it, and make it even easier for us to populate the metadata you use in your Fastly service. They even include measures to avoid overly simplistic parsing (which is what that weird "not a brand" thing is about). So… long live device detection?

Moving to Client Hints

But what if you really do need something a bit more detailed, something browsers no longer want to tell you upfront in case you use it as a fingerprint to track users? Enter client hints - another Google invention that is intended to provide a way to access that data in a privacy-preserving way.

In principle, client hints work like this. As a web developer, you make a website that has a responsive layout, and your HTML pages are customized only using the device data available in the user's first request (which we discussed earlier). However, you also include an Accept-CH response header, telling the browser that you would love to know a bit more, and this will help you provide a better user experience for subsequent resource requests:

Accept-CH: Sec-CH-Viewport-Width, Sec-CH-UA-Arch, Device-Memory, RTT

When the user's browser makes the subsequent requests for things like images, stylesheets and scripts that are required to complete the rendering of your page, the browser will (if the user approves) include the additional data you're asking for:

Sec-CH-Viewport-Width: 1000
Sec-CH-UA-Arch: "x86"
Device-Memory: 4
RTT: 125

Wow, that's quite a treasure trove of data you can use to make a great experience for users! If you're using Fastly, the edge is also an ideal place to work with client hints, especially these "high entropy" hints that might have a huge number of possible permutations. 

Imagine you have an experience that requires high computational capabilities on the device, like a WebGL scene. You might use the Device-Memory hint to determine at what resolution to render the scene, or what frame rate to target. The Device-Memory hint has six possible values, but maybe you only have a "high" and a "low" experience. If you make the decision on your server, CDNs like Fastly's will still need to cache those two variants in six cache slots, reducing the likelihood of getting a cache hit and being able to serve a user request quickly and efficiently.

Client hints aren't just available as an HTTP API either, there's also the NavigatorUAData JavaScript interface for reading them in JavaScript too.

Sounds great, but there are still some significant challenges with client hints:

• Firefox and Safari don't support them at all. This is purely a party for Chromium-based browsers at the moment, and since Google has been shopping around client hints for over a decade at this point don't expect support to land soon.

• It's very hard to use client hints to customize HTML pages. That's because you have to "ask" the browser to send them, and that happens in a response to an earlier request - normally the one for the HTML page. Attempts to mitigate this include Critical-CH (and previously Accept-CH-Lifetime), but they are not perfect solutions.

Client hints do seem here to stay though, so hopefully they will evolve to provide solutions to more use cases and enjoy broader support across all browsers.

Best practice device detection on Fastly today

The best approach combines signals from multiple places to enable the best understanding of device capabilities whilst respecting user privacy concerns. Whether using VCL or one of our Compute SDKs, there are easy ways to add headers to responses. This invites browsers to send you the client hints you care about. For example, if you have a VCL service, use a VCL snippet or custom VCL to add a list of desired client hints to vcl_deliver:

set resp.http.accept-ch = "Sec-CH-Viewport-Width, Sec-CH-UA-Arch, Device-Memory, RTT";

You can then access the hint information as HTTP headers such as req.http.sec-ch-viewport-width. It's common to combine this information with device detection variables to make the smartest possible decisions. If you simply want to log the data to a real-time logging endpoint, add something like this to your vcl_log:

log "syslog " + req.service_id + " my-endpoint-name :: {"
  {" "client-hints": { "}
    {" "viewport-width": ""} + req.http.sec-ch-viewport-width + {"","}
    {" "ua-arch": ""} + req.http.sec-ch-ua-arch + {"","}
    {" "device-memory": ""} + req.http.device-memory + {"","}
    {" "rtt": ""} + req.http.rtt + {"""}
  {" },"}
  {" "device-detection": { "}
    {" "mobile": ""} + client.platform.mobile + {"""}
    {" "touch": ""} + client.display.touchscreen + {"""}
  {" },"}
  {" "ua": { "}
    {" "sec-ch-ua": ""} + req.http.sec-ch-ua + {"","}
    {" "sec-ch-ua-mobile": ""} + req.http.sec-ch-ua-mobile + {"","}
    {" "sec-ch-ua-platform": ""} + req.http.sec-ch-ua-platform + {"""}
  {"}"}
{"}"}
;

This code brings together data provided in the new UA headers (sec-ch-ua-* headers), Fastly's device detection variables (client.* variables), and opt-in client hints like device-memory. Feel free to log it, or use it to annotate requests, trigger a special response, change the cache key or any other solution you can think of.

If you are using Fastly to provide device-specific experiences, or you want to, come chat with us over at community.fastly.com.