Extending Privacy

Chrome's Manifest V3 (MV3) changes have had a wide-ranging impact on the extension ecosystem. Primarily they are a set of API changes, that take away some of the most powerful extension APIs, replacing them with weaker, more restrictive ones.

Chrome have set a deadline for all extensions in the Chrome Web Store to migrate to the new MV3 APIs, which has meant that for the last 6 months or so, my team at DuckDuckGo have been busy migrating all the features of our Privacy Essentials extension over.

The latest feature I've been working on is our “Smarter Encryption”, which: 1. Ships a list of hostnames where we've verified that a HTTPs version of the site is available, cut to cover 90% of the most clicked domains duckduckgo.com. 2. Has an API for checking a domain against an extended list of HTTPs compatible websites.

The purpose is to ensure, that, where available, you're always on the secure version of a website, and if possible, your browser upgrades you before any insecure requests go out.

In MV2, we crammed the ~350,000 hostnames for the lookup data structure into a bloom filter. This compresses the 5.9 MB domain list down to just 1.6 MB, allowing us to keep this list in memory and synchronously upgrade all domains that match the bloom filter with the chrome.webRequest API.

MV3 removes the dynamic redirection capability of the chrome.webRequest API, in favour of declarative rules using the chrome.declarativeNetRequest API. These rules support an upgradeScheme action to support the use-case of upgrading insecure requests.

To match the behaviour of our MV2 bloom filter version, we need to write a declarativeNetRequest rule that matches the same set of ~350,000 domains as we include in our bloom filter. At a first pass, this seems simple enough: we can specify a list of requestDomains in a rule condition. The rule will match for any request that matches one of the domains in that list.

However, there's a catch. Adding example.com to the list of requestDomains will cause the rule to match for requests to example.com, as expected. It will also match requests to all subdomains of example.com (something which is not immediately obvious from the docs). Usually, this would be quite useful behaviour, but in the Smarter Encryption dataset we actually expect an exact domain match. This is because the fact that example.com supports HTTPS, does not imply that sub.example.com does. Therefore, all subdomains supporting HTTPS are explicitly included in the domain list.

This means that, having specified requestDomains, we need to prevent the rule matching for any subdomains of those domains. Luckily, my colleague kzar came up with a trick to do exactly that: 1. Group the list of domains by depth (number of .s). 2. For each group, create a rule with the list of domains in the group as requestDomains. 3. For each rule, add a regexFilter which matches only domains of the group's depth.

This allows us to match all ~350,000 SE domains in 4 declarativeNetRequest rules. These rules look like the following:

{
  "id": 1,
  "priority": 5000,
  "action": {
    "type": "upgradeScheme"
   },
  "condition": {
    "resourceTypes": [...],
    "requestDomains": [
      "en.wikipedia.org",
      "m.youtube.com",
      "m.imdb.com",
      ...
    ],
    "regexFilter": "^http://[^.]+\\.[^.]+\\.[^.]+(:|/|$)"
}

So, we have feature parity between MV2 and MV3! But at what cost? Before, we were running requests through a fast 1.6 MB bloom filter to check if an upgrade was possible. Now with MV3, we have to ship a 6.4 MB rule file, which will have to firstly match request domains against a list of ~350,000 domains, then run the URL through a regex (or do this in the reverse order). We have no control over how performant this is, as it is up to Chromium to implement efficient matching of declarativeNetRequest rules.

But that's actually not all – remember the second part of Smarter Encryption – the API for checking domains against the extended list? Well, that requires dynamic checking: when we see a new domain we need to make an asynchronous call to see whether it should be upgraded or not. This is not something declarativeNetRequest can do, so we have to listen for insecure requests via the webRequest API and trigger API calls for domains not on the bloom filter list. If the domain should be upgraded, where we used to use the webRequest redirects in MV2, we can now create a temporary 'session' declarativeNetRequest rule in MV3 to upgrade subsequent requests automatically.

The side effect of this, is that to do this in MV3, the entire MV2 code path has to remain, as we still need the bloom filter to efficiently check which requests the declarativeNetRequest rules will not upgrade. Therefore, MV3 is giving us a double performance penalty: Firstly, we have to include a very large rule file to move upgrade matching from webRequest to declarativeNetRequest, and secondly, we still have to match each request a second time in order to check requests beyond that list.

In summary: – Chrome's MV3 changes are taking away the part of the chrome.webRequest we need to implement Smarter Encryption. – We can achieve equivalent matching of the top ~350,000 domains using declarativeNetRequest. – To implement upgrades for domains beyond the 90th percentile, we still need to run our MV2 webRequest implementation in parallel. – The MV3 version is now 10x bigger for users to download and update, and spends at least double the resources to check for upgradable requests.

All this is not a surprise: at my previous job at Cliqz, we already published articles over 3 years ago debunking the privacy and performance claims of MV3.