Reading List

The most recent articles from a list of feeds I subscribe to.

Reckoning: Part 1 — The Landscape

Instead of an omnibus mega-post, this investigation into JavaScript-first frontend culture and how it broke US public services has been released in four parts. Other posts in the series:


When you live in the shadow of a slow-moving crisis, it's natural to tell people about it. At volume. Doubly so when engineers can cheaply and easily address the root causes with minor tweaks. As things worsen, it's also hard not to build empathy for Cassandra.

In late 2011, I moved to London, where the Chrome team was beginning to build Google's first "real" browser for Android.[1] The system default Android Browser had, up until that point, been based on the system WebView, locking its rate of progress to the glacial pace of device replacement.[2]

In a world where the Nexus 4's 2GB of RAM and 32-bit, 4-core CPU were the high-end, the memory savings the Android Browser achieved by reusing WebView code mattered immensely.[3] Those limits presented enormous challenges for Chromium's safer (but memory-hungry) multi-process sandboxing. Android wasn't just spicy Linux; it was an entirely new ballgame.

Even then, it was clear the iPhone wasn't a fluke. Mobile was clearly on track to be the dominant form-factor, and we needed to adapt. Fast.[4]

Browsers made that turn, and by 2014, we had made enough progress to consider how the web could participate in mobile's app-based model. This work culminated in 2015's introduction of PWAs and Push Notifications.

Disturbing patterns emerged as we worked with folks building on this new platform. A surprisingly high fraction of them brought slow, desktop-oriented JavaScript frameworks with them to the mobile web. These modern, mobile-first projects neither needed nor could afford the extra bloat frameworks included to paper over the problems of legacy desktop browsers. Web developers needed to adapt the way browser developers had, but consistently failed to hit the mark.

By 2016, frontend practice had fully lapsed into wish-thinking. Alarms were pulled, claxons sounded, but nothing changed.

It could not have come at a worse time.

By then, explosive growth at the low end was baked into the cake. Billions of feature-phone users had begun to trade up. Different brands endlessly reproduced 2016's mid-tier Androids under a dizzying array of names. The only constants were the middling specs and ever-cheaper prices. Specs that would set punters back $300 in 2016, sold for only $100 a few years later, opening up the internet to hundreds of millions along the way. The battle between the web and apps as the dominant platform was well and truly on.

<em>Tap for a larger version.</em><br>Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.<br><br>Nearly all growth in smartphone sales volume since the mid '10s occured in the 'budget' and 'low-end' categories.
Tap for a larger version.
Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.

Nearly all growth in smartphone sales volume since the mid '10s occured in the 'budget' and 'low-end' categories.

But the low-end revolution barely registered in web development circles. Frontenders poured JavaScript into the mobile web at the same rate as desktop, destroying any hope of a good experience for folks on a budget.

Median JavaScript bytes for Mobile and Desktop sites. <br>As this blog has <a href='/series/performance-inequality/'>covered at length,</a> median device specs were largely stagnant between 2014 and 2022. Meanwhile, web developers made sure the
Median JavaScript bytes for Mobile and Desktop sites.
As this blog has covered at length, median device specs were largely stagnant between 2014 and 2022. Meanwhile, web developers made sure the "i" in "iPhone" stood for "inequality."

Prices at the high end accelerated, yet average selling prices remained stuck between $300 and $350. The only way the emergence of the $1K phone didn't bump the average up was the explosive growth at the low end. To keep the average selling price at $325, three $100 low-end phones needed to sell for each $1K iPhone; which is exactly what happened.

And yet, the march of JavaScript-first, framework-centric dogma continued, no matter how incompatible it was with the new reality. Predictably, tools sold on the promise they would deliver "app-like experiences" did anything but.[5]

Billions of cheap phones that always have up-to-date browsers found their CPUs and networks clogged with bloated scripts designed to work around platform warts they don't have.

Environmental Factors

In 2019, Code for America published the first national-level survey of online access to benefits programs, which are built and operated by each state. The follow-up 2023 study provides important new data on the spread of digital access to benefits services.

One valuable artefact from CFA's 2019 research is a post by Dustin Palmer, documenting the missed opportunity among many online benefits portals to design for the coming mobile-first reality that was already the status quo in the rest of the world.

Worldwide mobile browsing surpassed desktop browsing sometime in 2016.
Worldwide mobile browsing surpassed desktop browsing sometime in 2016.
US browsing exhibited the same trend, slightly delayed, owing to comparatively high desktop and laptop ownership vs emerging markets.
US browsing exhibited the same trend, slightly delayed, owing to comparatively high desktop and laptop ownership vs emerging markets.

Moving these systems online only reduces administrative burdens in a contingent sense; if portals fail to work well on phones, smartphone-dependent folks are predictably excluded:

28% of US adults in households with less than $30K/yr income are smartphone-dependent, falling to only 19% for families making 30-70K/yr.
28% of US adults in households with less than $30K/yr income are smartphone-dependent, falling to only 19% for families making 30-70K/yr.

But poor design isn't the only potential administrative burden for smartphone-dependent users.[6]

The networks and devices folks use to access public support aren't latest-generation or top-of-the-line. They're squarely in the tail of the device price, age, and network performance distributions. Those are the overlapping conditions where the consistently falsified assumptions of frontend's lost decade have played out disastrously.

California is a rich mix of urban and hard-to-reach rural areas. Some of the poorest residents are in the least connected areas, ensuring they will struggle to use bloated sites.
California is a rich mix of urban and hard-to-reach rural areas. Some of the poorest residents are in the least connected areas, ensuring they will struggle to use bloated sites.

It would be tragic if public sector services adopted the JavaScript-heavy stacks that frontend influencers have popularised. Framework-based, "full-stack" development is now the default in Silicon Valley, but should obviously be avoided in universal services. Unwieldy and expensive stacks that have caused agony in the commercial context could never be introduced to the public sector with any hope of success.

Right?

Next: Object Lesson: a look at California's digital benefits services.

Thanks to Marco Rogers, and Frances Berriman for their encouragement in making this piece a series and for their thoughtful feedback on drafts.


  1. A "real browser", as the Chrome team understood the term circa 2012, included:

    • Chromium's memory-hungry multi-process architecture which dramatically improved security and stability
    • Winning JavaScript performance using our own V8 engine
    • The Chromium network stack, including support for SPDY and experiments like WebRTC
    • Updates that were not locked to OS versions
  2. Of course, the Chrome team had wanted to build a proper mobile browser sooner, but Android was a paranoid fiefdom separate from Google's engineering culture and systems. And the Android team were intensely suspicious of the web, verging into outright hostility at times.

    But internal Google teams kept hitting the limits of what the Android Browser could do, including Search. And when Search says "jump", the only workable response is "how high?"

    WebKit-based though it was (as was Chrome), OS-locked features presented a familiar problem, one the Chrome team had solved with auto-update and Chrome Frame. A deal was eventually struck, and when Chrome for Android was delivered, the system WebView also became a Chromium-based, multi-process, sandboxed, auto-updating system. For most, that was job done.

    This made a certain sort of sense. From the perspective of Google's upper management, Android's role was to put a search box in front of everyone. If letting Andy et al. play around with an unproven Java-based app model was the price, OK. If that didn't work, the web would still be there. If it did, then Google could go from accepting someone else's platform to having one it owned outright.[7] Win/win.

    Anyone trying to suggest a more web-friendly path for Android got shut down hard. The Android team always had legitimate system health concerns that they could use as cudgels, and they weilded them with abandon.

    The launch of PWAs in 2015 was an outcome Android saw coming a mile away and worked hard to prevent. But that's a story for another day.

  3. Android devices were already being spec'd with more RAM than contemporary iPhones, thanks to Dalvik's chonkyness. This, in turn, forced many OEMs to cut corners in other areas, including slower CPUs.

    This effect has been a silent looming factor in the past decade's divergence in CPU performance between top-end Android and iPhones. Not only did Android OEMs have to pay a distinct profit margin to Qualcomm for their chips, but they also had to dip into the Bill Of Materials (BOM) budget to afford more memory to keep things working well, leaving less for the CPU.

    Conversely, Apple's relative skimpiness on memory and burning desire to keep BOM costs low for parts it doesn't manufacture are reasons to oppose browser engine choice. If real browsers were allowed, end users might expect phones with decent specs. Apple keeps that in check, in part, by maximising code page reuse across browsers and apps that are forced to use the system WebView.

    That might dig into margins ever so slightly, and we can't have that, can we?

  4. It took browsers that were originally architected in a desktop-only world many years to digest the radically different hardware that mobile evolved. Not only were CPU speeds and memory budgets cut dramatically — nevermind the need to port to ARM, including JS engine JITs that were heavily optimised for x86 — but networks suddenly became intermittent and variable-latency.

    There were also upsides. Where GPUs had been rare on the desktop, every phone had a GPU. Mobile CPUs were slow enough that what had felt like a leisurely walk away from CPU-based rendering on desktop became an absolute necessity on phones. Similar stories played out across input devices, sensors, and storage.

    It's no exaggeration to say that the transition to mobile force-evolved browsers in a compressed time frame. If only websites had made the same transition.

  5. Let's take a minute to unpack what the JavaScript framework claims of "app-like experiences" were meant to convey.

    These were code words for more responsive UI, building on the Ajax momentum of the mid-naughties. Many boosters claimed this explicitly and built popular tools to support these specific architectures.

    As we wander through the burning wreckage of public services that adopted these technologies, remember one thing: they were supposed to make UIs better.

  6. When confronted with nearly unusable results from tools sold on the idea that they make sites easier, better, and faster to use, many technologists offer the variants of "but at least it's online!" and "it's fast enough for most people". The most insipid version implies causality, constructing a strawman but-for defense; "but these sites might not have even been built without these frameworks."[9]

    These points can be both true and immaterial at the same time. It isn't necessary for poor performance to entirely exclude folks at the margins for it to be a significant disincentive to accessing services.

    We know this because it has been proven and continually reconfirmed in commercial and lab settings.

  7. The web is unattractive to every Big Tech company in a hurry, even the ones that owe their existence to it.

    The web's joint custody arrangement rankles. The standards process is inscrutable and frustrating to PMs and engineering managers who have only ever had to build technology inside one company's walls. Playing on hard mode is unappealing to high-achievers who are used to running up the score.

    And then there's the technical prejudice. The web's languages offend "serious" computer scientists. In the bullshit hierarchy of programming language snobbery, everyone looks down on JavaScript, HTML, and CSS (in that order).

    The web's overwhelmingly successful languages present a paradox: for the comfort of the snob, they must simultaneously be unserious toys beneath the elevated palettes of "generalists" and also Gordian Knots too hard for anyone to possibly wield effectively. This dual posture justifies treating frontend as a less-than discipline, and browsers as anything but a serious application platform.

    This isn't universal, but it is common, particularly in Google's C++/Java-pilled upper ranks.[8] Endless budgetary space for projects like the Android Framework, Dart, and Flutter were the result.

  8. Someday I'll write up the tale of how Google so thoroughly devalued frontend work that it couldn't even retain the unbelievably good web folks it hired in the mid-'00s. Their inevitable departures after years of being condescended to went hand-in-hand with an inability to hire replacements.

    Suffice to say, by the mid '10s, things were bad. So bad an exec finally noticed. This created a bit of space to fix it. A team of volunteers answered the call, and for more than a year we met to rework recruiting processes and collateral, interview loop structures, interview questions, and promotion ladder criteria.

    The hope was that folks who work in the cramped confines of someone else's computer could finally be recognised for their achievements. And for a few years, Google's frontends got markedly better.

    I'm told the mean has reasserted itself. Prejudice is an insidious thing.

  9. The but-for defense for underperforming frontend frameworks requires us to ignore both the 20 years of web development practice that preceeded these tools and the higher OpEx and CapEx costs associated with React-based stacks.

    Managers sometimes offer a hireability argument, suggesting they need to adopt these univerally more expensive and harder to operate tools because they need to be able to hire. This was always nonsense, but never more so than in 2024. Some of the best, most talented frontenders I know are looking for work and would leap at the chance to do good things in an organisation that puts user experience first.

    Others sometimes offer the idea that it would be too hard to retrain their teams. Often, these are engineering groups comprised of folks who recently retrained from other stacks to the new React hotness or who graduated boot camps armed only with these tools. The idea that either cohort cannot learn anything else is as inane as it is self-limiting.

    Frontenders can learn any framework and are constantly retraining just to stay on the treadmill. The idea that there are savings to be had in "following the herd" into Next.js or similar JS-first development cul-de-sacs has to meet an evidentiary burden that I have rarely seen teams clear.

    Managers who want to avoid these messes have options.

    First, they can crib Kellan's tests for new technologies. Extra points for digesting Glyph's thoughts on "innovation tokens."

    Next, they should identify the critical user journeys in their products. Technology choices are always situated in product constraints, but until the critical user journeys are enunciated, the selection of any specific architecture is likely to be wrong.

    Lastly, they should always run bakeoffs. Once critical user journeys are outlined and agreed, bakeoffs can provide teams with essential data about how different technology options will perform under those conditions. For frontend technologies, that means evaluating them under representative market conditions.

    And yes, there's almost always time to do several small prototypes. It's a damn sight cheaper than the months (or years) of painful remediation work. I'm sick to death of having to hand-hold teams whose products are suffocating under unusably large piles of cruft, slowly nursing their code-bases back to something like health as their management belatedely learns the value of knowing their systems deeply.

    Managers that do honest, user-focused bakeoffs for their frontend choices can avoid adding their teams to the dozens I've consulted with who adopted extremely popular, fundamentally inappropriate technologies that have had disasterous effects on their businesses and team velocity. Discarding popular stacks from consideration through evidence isn't a career risk; it's literally the reason to hire engineers and engineering leaders in the first place.

Misfire

The W3C Technical Architecture Group[1] is out with a blog post and an updated Finding regarding Google's recent announcement that it will not be imminently removing third-party cookies.

The current TAG members are competent technologists who have a long history of nuanced advice that looks past the shouting to get at the technical bedrock of complex situations. The TAG also plays a uniquely helpful role in boiling down the guidance it issues into actionable principles that developers can easily follow.

All of which makes these pronouncements seem like weak tea. To grok why, we need to walk through the threat model, look at the technology options, and try to understand the limits of technical interventions.

But before that, I should stipulate my personal position on third-party cookies: they aren't great!

They should be removed from browsers when replacements are good and ready, and Google's climbdown isn't helpful. That said, we have seen nothing of the hinted-at alternatives, so the jury's out on what the impact will be in practice.[2]

So why am I dissapointed in the TAG, given that my position is essentially what they wrote? Because it failed to acknowledge the limited and contingent upside of removing third-party cookies, or the thorny issues we're left with after they're gone.

Unmasking The Problem

So, what do third-party cookies do? And how do they relate to the privacy theat model?

Like a lot of web technology, third-party cookies have both positive and negative uses. Owing to a historcal lack of platform-level identity APIs, they form the backbone of nearly every large Single Sign-On (SSO) system. Thankfully, replacements have been developed and are being iterated on.

Unfortunately, some browsers have unilaterally removed them without developing such replacements, disrupting sign-in flows across the web, harming users and pushing businesses toward native mobile apps. That's bad, as native apps face no limits on communicating with third parties and are generally worse for tracking. They're not even subject to pro-user interventions like browser extensions. The TAG should have called out this aspect of the current debate in its Finding, encouraging vendors to adopt APIs that will make the transition smoother.

The creepy uses of third-party cookies relate to advertising. Third-party cookies provide ad networks and data brokers the ability to silently reidentify users as they browse the web. Some build "shadow profiles", and most target ads based on sites users visit. This targeting is at the core of the debate around third-party cookies.

Adtech companies like to claim targeting based on these dossiers allows them to put ads in front of users most likely to buy, reducing wasted ad spending. The industry even has a shorthand: "right people, right time, right place."

Despite the bold claims and a consensus that "targeting works," there's reason to believe pervasive surveillence doesn't deliver, and even when it does, isn't more effective.

Assuming the social utility of targeted ads is low — likely much lower than adtech firms claim — shouldn't we support the TAG's finding? Sadly, no. The TAG missed a critical opportunity to call for legislative fixes to the technically unfixable problems it failed to enumerate.

Privacy isn't just about collection, it's about correlation across time. Adtech can and will migrate to the server-side, meaning publishers will become active participants in tracking, funneling data back to ad networks directly from their own logs. Targeting pipelines will still work, with the largest adtech vendors consolidating market share in the process.

This is why "give us your email address for 30% discount" popups and account signup forms are suddenly everywhere. Email addresses are stable, long-lived reidentifiers. Overt mechanisms like this are already replacing third-party cookies. Make no mistake: post-removal, tracking will continue for as long as reidentification has perceived positive economic value. The only way to change that equation is legislation; anything else is a band-aid.

Pulling tracking out of the shadows is good, but a limited and contingent good. Users have a terrible time recognising and mitigating risk on the multi-month time-scales where privacy invasions play out. There's virtually no way to control or predict where collected data will end up in most jurisdictions, and long-term collection gets cheaper by the day.

Once correlates are established, or "consent" is given to process data in ways that facilitate unmasking, re-identification becomes trivial. It only takes giving a phone number to one delivery company, or an email address to one e-commerce site to suddenly light up a shadow profile, linking a vast amount of previously un-attributed browsing to a user. Clearing caches can reset things for a little while, but any tracking vendor that can observe a large proportion of browsing will eventually be able to join things back up.

Removal of third-party cookies can temporarily disrupt this reidentification while collection funnels are rebuilt to use "first party" data, but that's not going to improve the situation over the long haul. The problem isn't just what's being collected now, it's the ocean of dormant data that was previously slurped up.[3] The only way to avoid pervasive collection and reidentification over the long term is to change the economics of correlation.

The TAG surely understands the only way to make that happen is for more jurisdictions to pass privacy laws worth a damn. It should say so.

Fire And Movement

The goal of tracking is to pick users out of crowds, or at least bucket them into small unique clusters. As I explained on Mastodon, this boils down to bits of entropy, and those bits are everywhere. From screen resolution and pixel density, to the intrinsic properties of the networks, to extensions, to language and accessibility settings that folks rely on to make browsing liveable. Every attribute that is even subtly different can be a building block for silent reidentification; A.K.A., "fingerprinting."[4]

In jurisdictions where laws allow collected data to remain the property of the collector, the risks posed by data-at-rest is only slightly attenuated by narrowing the funnel through which collection takes place.

It's possible to imagine computing that isn't fingerprintable, but that isn't what anyone is selling. For complex reasons, even the most cautious use of commodity computers is likely to be uniquely identifiable with enough time. This means that the question to answer isn't "do we think tracking is bad?", it's "given that we can't technically eliminate it, how can we rebuild privacy?". The TAG's new Finding doesn't wrestle with that question, doing the community a disservice in the process.

The most third-party cookie removal can deliver is temporary disruption. That disruption will affect distasteful collectors, costing them money in the short run. Many think of this as a win, I suspect because they fail to think through the longer-term consequences. The predictable effect will be a recalibration and entrenchment of surveillence methods. It will not put the panopticon out of business; only laws can do that.

For a preview of what this will look like, think back on Apple's "App Tracking Transparency" kayfabe, which did not visibly dent Facebook's long-term profits.

So this is not a solution to privacy, it's fire-and-movement tactics against corporate enemies. Because of the deep technical challenges in defeating fingerprinting[4:1], even the most outspoken vendors have given up, introducing "nutrition labels" to shift responsibility for privacy onto consumers.

If the best vertically-integrated native ecosystems can do is to shift blame, the TAG should call out posturing about ineffective changes and push for real solutions. Vendors should loudly lobby for stronger laws that can truly change the game and the TAG should join those calls. The TAG should also advocate for the web, rather than playing into technically ungrounded fearmongering by folks trying to lock users into proprietary native apps whilst simultaneously depriving users of more private browsers.

Finding A Way Forward

The most generous take I can muster is that the TAG's work is half-done. Calling on vendors to drop third-party cookies has the virtue of being technical and actionable, properties I believe all TAG writing should embody. But having looked deeply at the situation, the TAG should have also called on browser vendors to support further reform along several axes — particularly vendors that also make native OSes.

First, if the TAG is serious about preventing tracking and improving the web ecosystem, it should call on all OS vendors to prohibit the use of "in-app browsers" when displaying third-party content within native apps.

It is not sufficient to prevent JavaScript injection because the largest native apps can simply convince the sites to include their scripts directly. For browser-based tracking attenuation to be effective, these side-doors must be closed. Firms grandstanding about browser privacy features without ensuring users can reliably enjoy the protections of their browser need to do better. The TAG is uniquely positioned to call for this erosion of privacy and the web ecosystem to end.

Next, the TAG should have outlined the limits of technical approaches to attenuating data collection. It should also call on browser vendors to adopt scale-based interventions (rather than absolutism) in mitigating high-entropy API use.[5] The TAG should go first in moving past debates that don't acknowledge impossibilities in removing all reidentification, and encourage vendors to do the same. There's no solution to the privacy puzzle that can be solved by the purchase of a new phone, and the TAG should be clarion about what will end our privacy nightmare: privacy laws worth a damn.

Lastly, the TAG should highlight discrepancies between privacy marketing and the failure of vendors to push for strong privacy laws and enforcement. Because the threat model of privacy intrusion renders solely techincal interventions ineffective on long timeframes, this is the rare case in which the TAG should push past providing technical advice.

The TAG's role is to explain complex things with rigor and signpost credible ways forward. It has not done that yet regarding third-party cookies, but it's not too late.


  1. Praise, as well as concern, in this post is specific to today's TAG's, not the output of the group while I served. I surely got a lot of things wrong, and the current TAG is providing a lot of value. My hope here is that it can extend this good work by expanding its new Finding. ↩︎

  2. Also, James Roswell can go suck eggs. ↩︎

  3. It's neither here nor there, but the TAG also failed in these posts to encourage users and developers to move their use of digital technology into real browsers and out of native apps which invasively track and fingerprint users to a degree web adtech vendors only fantasize about.

    A balanced finding would call on Apple to stop stonewalling the technologies needed to bring users to safer waters, including PWA installation prompts. ↩︎

  4. As part of the drafting of the 2015 finding on Unsanctioned Web Tracking, the then-TAG (myself included) spent a great deal of time working through the details of potential fingerprinting vectors. What we came to realise was that only the Tor Browser had done the work to credibly analyise fingerprinting vectors and produce a coherent threat model. To the best of my knowledge, that remains true today.

    Other vendors continue to publish gussied-up marketing documents and stroppy blog posts that purport to cover the same ground, but consistently fail to do so. It's truly objectionable that those same vendors also prevent users from chosing disciplined privacy-focused browsers.

    To understand the difference, we can do a small thought experiment, enumerating what would be necessary to sand off currently-identifiable attributes of individual users. Because only 31 or 32 bits are needed to uniquely identify anybody (often less), we want a high safety factor. This means bundling users into very large crowds by removing distinct observable properties. To sand off variations between users, a truly private browser might:

    • Run the entire browser in a VM in order to:
      • Cap the number of CPU cores, frequency, and centralise on a single instruction set (e.g., emulating ARM when running on x86). Will likely result in a 2-5x slowdown.
      • Ensure (high) fixed latency for all disk access.
      • Set a uniform (low) cap on total memory.
    • Disable hardware acceleration for all graphics and media.
    • Disable JIT. Will slow JavaScript by 3-10x.
    • Only allow a fixed set of fonts, screen sizes, pixel densities, gamuts, and refresh rates; no more resizing browsers with a mouse. The web will pixelated and drab and animations will feel choppy.
    • Remove most accessibility settings.
    • Remove the ability to install extensions.
    • Eliminate direct typing and touch-based interactions, as those can leak timing information that's unique.
    • Run all traffic through Tor or a similarly high-latency VPN egress nodes.
    • Disable all reidentifying APIs (no more web-based video conferencing!)

    Only the Tor project is shipping a browser anything like this today, and it's how you can tell that most of what passes for "privacy" features in other browsers are anti-annoyance and anti-creep-factor interventions; they matter, but won't end the digital panopticon. ↩︎ ↩︎

  5. It's not a problem that sign-in flows need third-party cookies today, but it is a problem that they're used for pervasive tracking.

    Likewise, the privacy problems inherent in email collection or camera access or filesystem folders aren't absolute, they're related to scale of use. There are important use-cases that demand these features, and computers aren't going to stop supporting them. This means the debate is only whether or not users can use the web to meet those needs. Folks who push an absolutist line are, in effect, working against the web's success. This is anti-user, as the alternatives are generally much more invasive native apps.

    Privacy problems arise at scale and across time. Browsers should be doing more to discourage high-quality reidentifiaction across cache clearing and in ways that escalate with risk. The first site you grant camera access isn't the issue; it's the 10th. Similarly, speed bumps should be put in place for use of reidentifying APIs on sites across cache clearing where possible.

    The TAG can be instrumental is calling for this sort of change in approach. ↩︎

Why Browsers Get Built

There are only two-and-a-half reasons to build a browser, and they couldn't be more different in intent and outcome, even when they look superficially similar. Learning to tell the difference is helpful for browser project managers and engineers, but also working web developers who struggle to develop theories of change for affecting browser teams.

Like Platform Adjacency Theory and The Core Web Platform Loop, this post started[1] as a set of framing devices that I've been sketching on whiteboards for the best part of a decade. These lenses aren't perfect, but they provide starting points for thinking about the complex dynamics of browsers, OSes, "native" platforms, and the standards-based Web platform.

The reasons to build browsers are most easily distinguished by the OSes they support and the size and composition of their teams ("platform" vs. "product"). Even so, there are subtleties that throw casual observers for a loop. In industrial-scale engineering projects like browsers headcount is destiny, but it isn't the whole story.

Web As Platform

This is simultaneously the simplest and most vexing reason to build a browser.

Under this logic, browsers are strategically important to a broader business, and investments in platforms are investments in their future competitiveness compared with other platforms, not just other browsers. But none of those investments come good until the project has massive scale.

This strategy is exemplified by 1990s-era Andreesen's goal to render Windows "a poorly debugged set of device drivers".

The idea is that the web is where the action is, and that the browser winning more user Jobs To Be Done follows from increasing the web platform's capability. This developer-enabling flywheel aims to liberate computing from any single OS, supporting a services model.

A Web As Platform play depends on credibly keeping up with expansions in underlying OS features. The goal is to deliver safe portable, interoperable, and effective versions of important capabilities at a fast enough clip to maintain faith in the web as a viable ongoing investment.

In some sense it's a confidence-management exercise. A Web As Platform endgame requires the platform increasaes expressive capacity year over year. It must do as many new things each year as new devices can, even if the introduction of those features is delayed for the web by several years; the price of standards.

Platform-play browsers aim to grow and empower the web ecosystem, rather than contain it or treat it as a dying legacy. Examples of this strategic orientation include Netscape, Mozilla (before it lost the plot), Chrome, and Chromium-based Edge (on a good day).

Distinguishing Traits

  • Ship on many OSes and not just those owned by the sponsor
  • Large platform teams (>200 people and/or >40% of the browser team)
  • Visible, consistent investments in API leadership and capability expansion
  • Balanced benchmark focus
  • Large standards engagement footprint

The OS Agenda

There are two primary tactical modes of this strategic posture, both serving the same goal: to make an operating system look good by enabling a corpus of web content to run well on it while maintaining a competitive distance between the preferred (i.e., native, OS-specific) platform and the hopefully weaker web platform.

The two sub-variants differ in ambition owing to the market positions of their OS sponsors.

OSes treat browsers they sponsor as bridges or moats, but never the main object.
Photo by Paul Arky

Browsers as Bridges

OSes deploy browsers as a bridge for users into their environment when they're underdogs or fear disruption.

Of course, it would be better from the OS vendor's perspective if everyone simply wrote all of the software for their proprietary platform, maximising OS feature differentiation. But smart vendors also know that's not possible when an OS isn't dominant.

OS challengers, therefore, strike a bargain. For the price of developing a browser, they gain the web's corpus of essential apps and services, serving to "de-risk" the purchase of a niche device by offering broad compatibility with existing software through the web. If they do a good job, a conflicted short-term investment can yield enough browser share to enable a future turn towards moat tactics (see below). Examples include Internet Explorer 3-6 as well as Safari on Mac OS X and the first iPhone.

Conversely, incumbents fearing disruption may lower their API drawbridges and allow the web's power to expand far enough that the incumbent can gain share, even if it's not for their favoured platform; the classic example here being Internet Explorer in the late 90s. Once Microsoft knew it had Netscape well and truly beat, it simply disbanded the IE team, leaving the slowly rusting husk of IE to decay. And it would have worked, too, if it weren't for those pesky Googlers pushing IE6 beyond what was "possible"!

Browsers as Moats

Without meaningful regulation, powerful incumbents can use anti-competitive tactics to suppress the web's potential to disrupt the OS and tilt the field towards the incumbent's proprietary software ecosystem.

This strategy works by maintaining high browser share while never allowing the browser team to deliver features that are sufficient to disrupt OS-specific alternatives.

In practice, moats are arbitrage on the unwillingness of web developers to understand or play the game, e.g. by loudly demanding timely features or recommending better browsers to users. Incumbents know that web developers are easily led and are happy to invent excuses for them. It's cheap to add a few features here an there to show you're "really trying", despite underfunding browser teams so much they can never do more than a glorified PR for the OS. This was the strategy behind IE 7-11 and EdgeHTML. Even relatively low share browsers can serve as effective moats if they can't be supplanted by competitive forces.

Apple has perfected the moat, preventing competitors from even potentially offering disruptive features. This adds powerfully to the usual moat-digger's weaponisation of consensus processes. Engineering stop-energy in standards and quasi-standards bodies is nice and all, but it is so much more work than simply denying anyone the ability to ship the features that you won't.

Tipping Points

Bridge and moat tactics appear very different, but the common thread is control with an intent to suppress web platform expansion. In both cases, the OS will task the browser team to heavily prioritise integrations with the latest OS and hardware features at the expense of more broadly useful capabilities — e.g. shipping "notch" CSS and "force touch" events while neglecting Push.

Browser teams tasked to build bridges can grow quickly and have remit that looks similar to a browser with a platform agenda. Still, the overwhelming focus starts (and stays) on existing content, seldom providing time or space to deliver powerful new features to the Web. A few brave folks bucked this trend, using the fog of war to smuggle out powerful web platform improvements under a more limited bridge remit; particularly the IE 4-6 crew.

Teams tasked with defending (rather than digging) a moat will simply be starved by their OS overlords. Examples include IE 7+ and Safari from 2010 onward. It's the simplest way to keep web developers from getting uppity without leaving fingerprints. The "soft bigotry of low expectations", to quote a catastrophic American president.

Distinguishing Traits

  • Shipped only to the sponsor's OSes
  • Browser versions tied to OS versions
  • Small platform teams (<100 people and/or <30% of the browser team)
  • Skeleton standards footprint
  • Extreme focus on benchmarks of existing content
  • Consistent developer gaslighting regarding new capabilities
  • Anti-competitive tactics against competitors to maintain market share
  • Inconsistent feature leadership, largely focused on highlighting new OS and hardware features
  • Lagging quality

Searchbox Pirates

This is the "half-reason"; it's not so much a strategic posture as it is environment-surfing.

Over the years, many browsers that provide little more than searchboxes atop someone else's engine have come and gone. They lack staying power because their teams lack the skills, attitudes, and management priorities necessary to avoid being quickly supplanted by a fast-following competitor pursuing one of the other agendas.

These browsers also tend to be short-lived because they do not build platform engineering capacity. Without agency in most of their codebase, they either get washed away in unmanaged security debt, swamped by rebasing challenges (i.e., a failure to "work upstream"). They also lack the ability to staunch bleeding when their underlying engine fails to implement table-stakes features, which leads to lost market share.

Historical examples have included UC Browser, and more recently, the current crop of "secure enterprise browsers" (Chromium + keyloggers). Perhaps more controversially, I'd include Brave and Arc in this list, but their engineering chops make me think they could cross the chasm and choose to someday become platform-led browsers. They certainly have leaders who understand the difference.

Distinguishing Traits

  • Shipped to many OSes
  • Tiny platform teams (<20 people or <10% of the browser team)
  • Little benchmark interest or focus
  • No platform feature leadership
  • No standards footprint
  • Platform feature availability lags the underlying engine (e.g., UI and permissions not hooked up)
  • Platform potentially languishes multiple releases behind "upstream"

Implications

This model isn't perfect, but it has helped me tremendously in reliably predicting the next moves of various browser players, particularly regarding standards posture and platform feature pace.[2]

The implications are only sometimes actionable, but they can help us navigate. Should we hold out hope that a vendor in a late-stage browser-as-moat crouch will suddenly turn things around? Well, that depends on the priorities of the OS, not the browser team.

Similarly, a Web As Platform strategy will maximise a browser's reach and its developers' potential, albeit at the occasional expense of end-user features.

The most important takeaway for developers may be what this model implies about browser choice. Products with an OS-first agenda are always playing second fiddle to a larger goal that does not put web developers first, second, or even third. Coming to grips with this reality lets us more accurately recommend browsers to users that align with our collective interests in a vibrant, growing Web.


  1. I hadn't planned to write this now, but an unruly footnote in an upcoming post, along with Frances' constant advice to break things up, made me realise that I already had 90% of it of ready. ↩︎

  2. Modern-day Mozilla presents a puzzle within this model.

    In theory, Mozilla's aims and interests align with growing the web as a platform; expanding its power to enable a larger market for browsers, and through it, a larger market for Firefox.

    In practice, that's not what's happening. Despite investing almost everything it makes back into browser development, Mozilla has also begun to slow-walk platform improvements. It walked away from PWAs and has continued to spread FUD about device APIs and other features that would indicate an appetite for an expansive vision of the platform.

    In a sense, it's playing the OS Agenda, but without an OS to profit from or a proprietary platform to benefit with delay and deflection. This is vexing, but perhaps expected within an organisation that has entered a revenue-crunch crouch. Another way to square the circle is to note that the the Mozilla Manifesto doesn't actually speak about the web at all. If the web is just another fungible application running atop the internet (which the manifesto does center), then it's fine for the web to be frozen in time, or even shrink.

    Still, Mozilla leadership should be thinking hard about the point of maintaining an engine. Is it to hold the coats of proprietary-favouring OS vendors? Or to make the web a true competitor? ↩︎

Home Screen Advantage

Update: OWA is out with an open letter appealing to Apple to do better. If you care about the future of the web, I encourage you to sign it, particularly if you live in the EU or build products for the common market.

After weeks of confusion and chaos, Apple's plan to kneecap the web has crept into view, menacing a PWApocalypse as the March 6th deadline approaches for compliance with the EU's Digital Markets Act (DMA).

The view from Cupertino.
The view from Cupertino.

The DMA requires Apple to open the iPhone to competing app stores, and its lopsided proposal for "enabling" them is getting most of the press. But Apple knows it has native stores right where it wants them. Cupertino's noxious requirements will take years to litigate. Meanwhile, potential competitors are only that.

But Cupertino can't delay the DMA's other mandate: real browsers, downloaded from Apple's own app store. Since it can't bar them outright, it's trying to raise costs on competitors and lower their potential to disrupt Apple's cozy monopoly. How? By geofencing browser choice and kneecapping web apps, all while gaslighting users about who is breaking their web apps.

The immediate impact of iOS 17.4 in the EU will be broken apps and lost data, affecting schools, governments, startups, gamers, and anyone else with the temerity to look outside the one true app store for even a second. None of this is required by the DMA, as demonstrated by continuing presence of PWAs and the important features they enable on Windows and Android, both of which are in the same regulatory boat.

The data loss will be catastrophic for many, as will the removal of foundational features. Here's what the landscape looks like today vs. what Apple is threatening:

PWA Capability Windows Android iOS 17.3 iOS 17.4
App-like UI
Settings Integration
Reliable Storage
Push Notifications
Icon Badging
Share-to PWA
App Shortcuts
Device APIs

Apple's support for powerful web apps wasn't stellar, but this step in the wrong direction will just so happen to render PWAs useless to worldwide businesses looking to reach EU users.

Apple's interpretation of the DMA appears to be that features not available on March 6th don't need to be shared with competitors, and it doesn't want to share web apps. The solution almost writes itself: sabotage PWAs ahead of the deadline and give affected users, businesses, and competitors minimal time to react.

Cupertino's not just trying to vandalise PWAs and critical re-engagement features for Safari; it's working to prevent any browser from ever offering them on iOS. If Apple succeeds in the next two weeks, it will cement a future in which the mobile web will never be permitted to grow beyond marketing pages for native apps.

By hook or by crook, Apple's going to maintain its home screen advantage.

The business goal is obvious: force firms back into the app store Apple taxes and out of the only ecosystem it can't — at least not directly. Apple's justifications range from unfalsifiable smokescreens to blatant lies, but to know it you have to have a background in browser engineering and the DMA's legalese. The rest of this post will provide that context. Apologies in advance for the length.

If you'd like to stop reading here, take with you the knowledge that Cupertino's attempt to scuttle PWAs under cover of chaos is exactly what it appears to be: a shocking attempt to keep the web from ever emerging as a true threat to the App Store and blame regulators for Apple's own malicious choices.

And they just might get away with it if we don't all get involved ASAP.

Chaos Monkey Business

Two weeks ago, Apple sprung its EU Digital Markets Act (DMA) compliance plans on the world as a fait accomplis.

The last-minute unveil and months of radio silence were calculated to give competitors minimal time to react to the complex terms, conditions, and APIs. This tactic tries to set Apple's proposal as a negotiating baseline, forcing competitors to burn time and money arguing down plainly unacceptable terms before they can enter the market.

For native app store hopefuls, this means years of expensive disputes before they can begin to access an artificially curtailed market. This was all wrapped in a peevish, belligerant presentation, which the good folks over at The Platform Law Blog have covered in depth.

Much of the analysis has focused on the raw deal Apple is offering native app store competitors, missing the forest for the trees: the threat Apple can't delay by years comes from within.

Deep in the sub-basement of Apple's tower of tomfoolery are APIs and policies that purport to enable browser engine choice. If you haven't been working on browsers for 15 years, the terms might seem reasonable, but to these eyes they're anything but. OWA has a lengthy dissection of the tricks Apple's trying to pull.

Apple's message of hope and optimism for a better web.
Apple's message of hope and optimism for a better web.

The proposals are maximally onerous, but you don't have to take my word for it; here's Mozilla:

We are ... extremely disappointed with Apple’s proposed plan to restrict the newly-announced BrowserEngineKit to EU-specific apps. The effect of this would be to force an independent browser like Firefox to build and maintain two separate browser implementations — a burden Apple themselves will not have to bear.

Apple’s proposals fail to give consumers viable choices by making it as painful as possible for others to provide competitive alternatives to Safari.

This is another example of Apple creating barriers to prevent true browser competition on iOS.

Mozilla spokesperson

The strategy is to raise costs and lower the value of porting browsers to iOS. Other browser vendors have cited exactly these concerns when asked about plans to bring their best products to iOS. Apple's play is to engineer an unusable alternative then cite the lack of adoption to other regulators as proof that mandating real engine choice is unwise.

Instead of facilitating worldwide browser choice in good faith, Apple's working to geofence progress; classic "divide and conquer" stuff, justified with serially falsified security excuses. Odious, brazen, and likely in violation of the DMA, but to the extent that it will now turn into a legal dispute, that's a feature (not a bug) from Apple's perspective.

When you're the monopolist, delay is winning.

But Wait! There's More!

All of this would be stock FruitCo doing anti-competitive FruitCo things, but they went further, attempting to silently shiv PWAs and blame regulators for it. And they did it in the dead of the night, silently disabling important features as close to the DMA compliance deadline as possible.

It's challenging, verging on impossible, to read this as anything but extrordinary bad faith, but Apple's tactics require context to understand.

The DMA came into force in 2022, putting everyone (including Apple) on notice that their biggest platforms and products would probably be "designated", and after designation, they would have six months to "comply". The first set of designation decisions went out last Sept, obligating Android, Windows, iOS, Chrome, and Safari to comply no later than March 6th, 2024.

Apple tried everything to <a href='https://www.theregister.com/2023/11/02/apple_safari_browser/'>shrink the scope of enforcement</a> and <a href='https://www.theverge.com/2024/1/8/23961923/apple-app-store-appeal-european-union-digital-markets-act-core-platform-service-gatekeeper'>delay compliance,</a> but in the end had the same two-years of notice and six-months warning from designation as everyone else.
Apple tried everything to shrink the scope of enforcement and delay compliance, but in the end had the same two-years of notice and six-months warning from designation as everyone else.

A maximally aggressive legal interpretation might try to exploit ambiguity in what it means to comply and when responsibilities actually attach.

Does compliance mean providing open and fair access starting from when iOS and Safari were designated, or does compliance obligation only attach six months later? The DMA's text is not ironclad here:

10: The gatekeeper shall comply with the obligations laid down in Articles 5, 6 and 7 within 6 months after a core platform service has been listed in the designation decision pursuant to paragraph 9 of this Article.

DMA Article 3, Clause 10

Firms looking to comply maliciously might try to remove troublesome features just before a compliance deadline, then argue they don't need to share them with competitors becuse they weren't available before the deadline set in. Apple looks set to argue, contra everyone else subject to the DMA, that the moment from which features must be made interoperable is the end of the fair-warning period, not the date of designation.

This appears to be Apple's play, and it stinks to high heavens.

What's At Risk?

Apple's change isn't merely cosmetic. In addition to immediate data loss, FruitCo's change will destroy:

  • App-like UI:
    Web apps are no longer going to look or work like apps in the task manager, systems settings, or any other surface. Homescreen web apps will be demoted to tabs in the default browser.
  • Reliable storage:
    PWAs were the only exemption to Apple's (frankly silly) seven day storage eviction policy, meaning the last safe harbour for anyone trying to build a serious, offline-first experience just had the rug pulled out from under them.
  • Push Notifications:
    Remember how Apple gaslit web developers over Web Push for the best part of a decade? And remember how, when they finally got around to it, did a comically inept job? Recall fretting and about how shite web Push Notifications look and work for iOS users? Well, rest easy, because they're going away too.
  • App Icon Badging:
    A kissing cousin of Push, Icon Badging allows PWAs to ambiently notify users of new messages, something iOS native apps have been able to do for nearly 15 years.

Removal of one would be a crisis. Together? Apple's engineering the PWApocalypse.

You can't build credible mobile experiences without these features. A social network without notifications? A notetaking app that randomly loses data? Businesses will get the message worldwide: if you want to be on the homescreen and deliver services that aren't foundationally compromised, the only game in town is Apple's app store.

Apple understands even the most aggressive legal theories about DMA timing wouldn't support kneecapping PWAs after March 6th. Even if you believe (as I do) their obligations attached back in September, there's at least an argument to be tested. Cupertino's white-shoe litigators would be laughed out of court and Apple would get fined ridiculous amounts for non-compliance if it denied these features to other browsers after the fair-warning period. To preserve the argument for litigation, it was necessary to do the dirty deed ahead of the last plausible deadline.

Not With A Bang, But With A Beta

The first indication something was amiss was a conspicuous lack of APIs for PWA support in the BrowserEngineKit documentation, released Feb 1st alongside Apple's peevish, deeply misleading note that attempted to whitewash malicious compliance in a thin coat of security theatre.

Two days later, after developers inside the EU got their hands on the iOS 17.4 Beta, word leaked out that PWAs were broken. Nothing about the change was documented in iOS Beta or Safari release notes. Developers filed plaintive bugs and some directly pinged Apple employees, but Cupertino remained shtum. This created panic and confusion as the windows closed for DMA compliance and the inevitable iOS 17.4 final release ahead of March 6th.

Two more betas followed, but no documentation or acknowledgement of the "bug." Changes to the broken PWA behavior were introduced, but Apple failed to acknowledge the issue or confirm that it was intentional and therefore likely to persist. After two weeks of growing panic from web developers, Apple finally copped to crippling the only open, tax-free competitor to the app store.

Apple's Feb 15th statement is a masterclass in deflection and deceit. To understand why requires a deep understanding of browsers internals and how Apple's closed PWA — sorry, "home screen web app" — system for iOS works.

TL;DR? Apple's cover story is horseshit, stem to stern. Cupertino ought to be ashamed and web developers are excused for glowing incandescent with rage over being used as pawns; first ignored, then gaslit, and finally betrayed.

Lies, Damned Lies, and "Still, we regret..."

I really, really hate to do this, but Brandolini's Law dictates that to refute Apple's bullshit, I'm going to need to go through their gibberish excuses line-by-line to explain and translate.

Q: Why don’t users in the EU have access to Home Screen web apps?

Translation: "Why did you break functionality that has been a foundational part of iOS since 2007, but only in the EU?"

To comply with the Digital Markets Act, Apple has done an enormous amount of engineering work to add new functionality and capabilities for developers and users in the European Union — including more than 600 new APIs and a wide range of developer tools.

Translation: "We're so very tired, you see. All of this litigating to avoid compliance tuckered us right out. Plus, those big meanies at the EU made us do work. It's all very unfair."

It goes without saying, but Apple's burden to add APIs it should have long ago provided for competing native app stores has no bearing whatsoever on its obligation to provide fair access to APIs that browser competitors need. Apple also had the same two years warning as everyone else. It knew this was coming, and 11th hour special pleading has big "the dog ate my homework" energy.

The iOS system has traditionally provided support for Home Screen web apps by building directly on WebKit and its security architecture. That integration means Home Screen web apps are managed to align with the security and privacy model for native apps on iOS, including isolation of storage and enforcement of system prompts to access privacy impacting capabilities on a per-site basis.

Finally! A recitation of facts.

Yes, iOS has historically forced a uniquely underpowered model on PWAs, but iOS is not unique in providing system settings integration or providing durable storage or managing PWA permissions. Many OSes and browsers have created the sort of integration infrastructure that Apple describes. These systems leave the question of how PWAs are actually run (and where their storage lives) to the browser that installs them, and the sky has yet to fall. Apple is trying to gussy up preferences and present them as hard requirements without justification.

Apple is insinuating that it can't provide API surface areas to allow the sorts of integrations that others already have. Why? Because it might involve writing a lot of code.

Bless their hearts.

Without this type of isolation and enforcement, malicious web apps could read data from other web apps and recapture their permissions to gain access to a user’s camera, microphone or location without a user’s consent.

Keeping one website from abusing permissions or improperly accessing data from another website is what browsers do. It's Job #1.

Correctly separating principals is the very defintion of a "secure" browser. Every vendor (save Apple) treats subversion of the Same Origin Policy as a showstopping bug to be fixed ASAP. Unbelieveable amounts of engineering go to ensuring browsers overlay stronger sandboxing and more restrictive permissions on top of the universally weaker OS security primitives — iOS very much included.

Browser makers have become masters of origin separation because they run totally untrusted code from all over the internet. Security is paramount because browsers have to be paranoid. They can't just posture about how store reviews will keep users safe; they have to do the work.

Good browsers separate web apps better than bad ones. It's rich that Apple of all vendors is directly misleading this way. Its decade+ of under-investment in WebKit ensured Safari was less prepared for Spectre and Meltdown and Solar Winds than alternative engines. Competing browsers had invested hundreds of engineer years into more advanced Site Isolation. To this day, Apple's underfunding and coerced engine monoculture put all iOS users at risk.

With that as background, we can start to unpack Apple's garbled claims.

Cupertino is that it does not want to create APIs for syncing permission state through the thin shims every PWA-supporting OS uses to make websites first class. It doesn't want to add APIs for attributing storage use, clearing state, toggling notifications, and other common management tasks. This is a preference, but it is not responsive to Apple's DMA obligations.

If those APIs existed, Apple would still have a management question, which its misdirections also allude to. But these aren't a problem in practice. Every browser offering PWA support would happily sign up to terms that required accurate synchronization of permission state between OS surfaces and web origins, in exactly the same way they treat cross-origin subversion as a fatal bug to be hot-fixed.

Apple's excusemaking is a mirror of Cupertino's years of scaremongering about alternate browser engine security, only to take up my proposal more-or-less wholesale when the rubber hit the road.

Nothing about this is monumental to build or challenging to manage; FruitCo's just hoping you don't know better. And why would you? The set of people who understand these details generously number in the low dozens.

Browsers also could install web apps on the system without a user’s awareness and consent.

Apple know this is a lie.

They retain full control over the system APIs that are called to add icons to the homescreen, install apps, and much else. They can shim in interstitial UI if they feel like doing so. If iOS left this to Safari and did not include these sorts of precautions, those are choices Apple has made and has been given two years notice to fix.

Cupertino seems to be saying "bad things might happen if we continued to do a shit job" and one can't help but agree. However, that's no way out of the DMA's obligations.

Addressing the complex security and privacy concerns associated with web apps using alternative browser engines would require building an entirely new integration architecture that does not currently exist in iOS and was not practical to undertake given the other demands of the DMA and the very low user adoption of Home Screen web apps.

[CITATION NEEDED]

Note the lack of data? Obviously this sort of unsubstantiated bluster fails Hitchen's Razor, but that's not the full story.

Apple is counting on the opacity of its own web suppression to keep commenters from understanding the game that's afoot. Through an enervating combination of strategic underinvestment and coerced monoculture, Apple created (and still maintains) a huge gap in discoverability and friction for installing web apps vs. their native competition. Stacking the deck for native has taken many forms:

This campaign of suppression has been wildly effective. If users don't know they can install PWAs, it's because Safari never tells them, and until this time last year, neither could any other browser. Developers also struggled to justify building them because Apple's repression extended to neglect of critical features, opening and maininting a substantial capability gap.

If PWAs use on iOS is low, that's a consequence of Apple's own actions. On every other OS where I've seen the data, not only are PWAs a success, they are growing rapidly. Perhaps that's why Apple feels a need to mislead by omission and fail to provide data to back their claim.

And so, to comply with the DMA’s requirements, we had to remove the Home Screen web apps feature in the EU.

Bullshit.

Apple's embedded argument expands to:

  • We don't want to comply with the plain-letter language of the law.
  • To avoid that, we've come up with a legal theory of compliance that's favourable to us.
  • To comply with that (dubious) theory, and to avoid doing any of the work we don't want to do, we've been forced to bump off the one competitor we can't tax.

Neat, tidy, and comprised entirely of bovine excrement.

EU users will be able to continue accessing websites directly from their Home Screen through a bookmark with minimal impact to their functionality. We expect this change to affect a small number of users. Still, we regret any impact this change — that was made as part of the work to comply with the DMA — may have on developers of Home Screen web apps and our users.

Translation: "Because fuck you, that's why"

The DMA doesn't require Apple to torpedo PWAs.

Windows and Android will continue supporting them just fine. Apple apparently hopes it can convince users to blame regulators for its own choices. Cupertino's counting on the element of surprise plus the press's poorly developed understanding of the situation to keep blowback from snowballing into effective oppostion.

The Point

There's no possible way to justify a "Core Technology Fee" tax on an open, interoperable, standardsized platform that competitors would provide secure implementations of for free. What Apple's attempting isn't just some hand-wavey removal of a "low use" feature ([CITATION NEEDED]), it's sabotage of the only credible alternative to its app store monopoly.

A slide from Apple's presentation in Apple v. Epic, attempting to make the claim Epic could have just made a PWA if they didn't like the App Store terms because circa '20 Safari was <em>so</em> capable. <br><br><a href='/2021/04/progress-delayed/'>LOL.</a>
A slide from Apple's presentation in Apple v. Epic, attempting to make the claim Epic could have just made a PWA if they didn't like the App Store terms because circa '20 Safari was so capable.

LOL.

Businesses will get the message: from now on, the only reliable way to get your service under the thumb, or in the notification tray, of the most valuable users in the world is to capitulate to Apple's extortionate App Store taxes.

If the last 15 years are anything to judge by, developers will take longer to understand what's going on, but this is an attempt to pull a "Thoughts on Flash" for the web. Apple's suppression of the web has taken many forms over the past decade, but the common thread has been inaction and anti-competitive scuppering of more capable engines. With one of those pillars crumbling, the knives glint a bit more brightly. This is Apple once and for all trying to relegate web development skills to the dustbin of the desktop.

Not only will Apple render web apps unreliable for Safari users, FruitCo is setting up an argument to prevent competitors from ever delivering features that challenge the app store in future. And it doesn't care who it hurts along the way.

The Mask Is Off

This is exactly what it looks like: a single-fingered salute to the web and web developers. The removal of features that allowed the iPhone to exist at all. The end of Steve Jobs' promise that you'd be able to make great apps out of HTML, CSS, and JS.

For the past few years Apple has gamely sent $1,600/hr lawyers and astroturf lobbyists to argue it didn't need to be regulated. That Apple was really on the developer's side. That even if it overstepped occasionally, it was all in the best interest of users.

Tell that to the millions of EU PWA users about to lose data. Tell that to the public services built on open technology. Tell it to the businesses that will fold, having sweated to deliver compelling experiences using the shite tools Apple provides web developers. Apple's rug pull is anti-user, anti-developer, and anti-competition.

Now we see the whole effort in harsh relief. A web Apple can't sandbag and degrade is one it can't abide. FruitCo's fear and loathing of an open platform it can't tax is palpable. The lies told to cover for avarice are ridiculous — literally, "worthy of ridicule".

It's ok to withhold the benefit of the doubt from Safari and Apple. It's ok to be livid. These lies aren't little or white; they're directly aimed at our future. They're designed to influence the way software will be developed and delivered for decades to come.

If you're as peeved about this as I am, go join OWA in the fight and help them create the sort of pressure in the next 10 days that might actually stop a monopolist with money on their mind.

Thanks to Stuart Langride, Bruce Lawson, and Roderick Gadellaa for their feedback on drafts of this post.

The Performance Inequality Gap, 2024

The global device and network situation continues to evolve, and this series is an effort to provide an an up-to-date understanding for working web developers. So what's changed since last year? And how much HTML, CSS, and (particularly) JavaScript can a new project afford?

The Budget, 2024

In a departure from previous years, two sets of baseline numbers are presented for first-load under five seconds on 75th (P75) percentile devices and networks[1]; one set for JavaScript-heavy content, and another for markup-centric stacks.

Budget @ P75 Markup-based JS-based
Total Markup JS Total Markup JS
3 seconds 1.4MiB 1.3MiB 75KiB 730KiB 365KiB 365KiB
5 seconds 2.5MiB 2.4MiB 100KiB 1.3MiB 650KiB 650KiB

This was data was available via last year's update, but was somewhat buried. Going forward, I'll produce both as top-line guidance. The usual caveats apply:

  • Performance is a deep and nuanced domain, and much can go wrong beyond content size and composition.
  • How sites manage resources after-load can have a big impact on perceived performance.
  • Your audience may justify more stringent, or more relaxed, limits.

Global baselines matter because many teams have low performance management maturity, and today's popular frameworks – including some that market performance as a feature – fail to ward against catastrophic results.

Until and unless teams have better data about their audience, the global baseline budget should be enforced.

This isn't charity; it's how products stay functional, accessible, and reliable in a market awash in bullshit. Limits help teams steer away from complexity and towards tools that generate simpler output that's easier to manage and repair.

JavaScript-Heavy

Since at least 2015, building JavaScript-first websites has been a predictably terrible idea, yet most of the sites I trace on a daily basis remain mired in script.[2] For these sites, we have to factor in the heavy cost of running JavaScript on the client when describing how much content we can afford.

HTML, CSS, images, and fonts can all be parsed and run at near wire speeds on low-end hardware, but JavaScript is at least three times more expensive, byte-for-byte.

Most sites, even those that aspire to be "lived in", are generally experienced through short sessions, which means they can't justify much in the way of up-front code. First impressions always matter.

Most sorts of sites have shallow sessions, making up-front script costs hard to justify.
Most sorts of sites have shallow sessions, making up-front script costs hard to justify.

Targeting the slower of our two representative devices, and opening only two connections over a P75 network, we can afford ~1.3MiB of compressed content to get interactive in five seconds. A page fitting this budget can afford:

  • 650KiB of HTML, CSS, images, and fonts
  • 650KiB of JavaScript

If we set the target a more reasonable three seconds, the budget shrinks to ~730KiB, with no more than 365KiB of compressed JavaScript.

Similarly, if we keep the five second target but open five TLS connections, the budget falls to ~1MiB. Sites trying to load in three seconds but which open five connections can afford only ~460KiB total, leaving only ~230KiB for script.

Markup-Heavy

Sites largely comprised of HTML and CSS can afford a lot more, although CSS complexity and poorly-loaded fonts can still slow things down. Conservatively, to load in five seconds over two connections, we should try to keep content under 2.5MiB, including:

  • 2.4MiB of HTML, CSS, images, and fonts, and
  • 100KiB of JavaScript.

To hit the three second first-load target, we should aim for a max 1.4MiB transfer, made up of:

  • 1.325MiB of HTML, CSS, etc., and
  • 75KiB of JavaScript.

These are generous targets. The blog you're reading loads in ~1.2 seconds over a single connection on the target device and network profile. It consumes 120KiB of critical path resources to become interactive, only 8KiB of which is script.

Calculate Your Own

As in years past, you can use the interactive estimator to understand how connections and devices impact budgets. This the tool has been updated to let you select from JavaScript-heavy and JavaScript-light content composition and defaults to the updated network and device baseline (see below).

<em>Tap to try the interactive version.</em>
Tap to try the interactive version.

It's straightforward to understand the number of critical path network connections and to eyeball the content composition from DevTools or WebPageTest. Armed with that information, it's possible to use this estimator to quickly understand what sort of first-load experience users at the margins can expect. Give it a try!

Situation Report

These recommendations are not context-free, and folks can reasonably disagree.

Indeed, many critiques are possible. The five second target first load)[1:1] is arbitrary. A sample population comprised of all internet users may be inappropriate for some services (in both directions). A methodology of "informed reckons" leaves much to be desired. The methodological critiques write themselves.

The rest of this post works to present the thinking behind the estimates, both to spark more informed points of departure and to contextualise the low-key freakout taking place as INP begins to put a price on JavaScript externalities.

Another aim of this series is to build empathy. Developers are clearly out of touch with market ground-truth. Building an understanding of the differences in the experiences of the wealthy vs. working-class users can make the privilege bubble's one-way mirror perceptible from the inside.[3]

Mobile

The "i" in iPhone stands for "inequality".

Premium devices are largely absent in markets with billions of users thanks to the chasm of global wealth inequality. India's iOS share has surged to an all-time high of 7% on the back of last-generation and refurbished devices. That's a market of 1.43 billion people where Apple doesn't even crack the top five in terms of shipments.

The Latin American (LATAM) region, home to more than 600 million people and nearly 200 million smartphones, shows a similar market composition:

In <abbr>LATAM</abbr>, iPhones make up less than 6% of total device shipments.
In LATAM, iPhones make up less than 6% of total device shipments.

Everywhere wealth is unequally distributed, the haves read about it in Apple News over 5G while the have-nots struggle to get reliable 4G coverage for their Androids. In country after country (PDF) the embedded inequality of our societies sorts ownership of devices by price. This, in turn, sorts by brand.

This matters because the properties of devices defines what we can deliver. In the U.S., the term "smartphone dependence" has been coined to describe folks without other ways to access the increasing fraction of essential services only available through the internet. Unsurprisingly, those who can't afford other internet-connected devices, or a fixed broadband subscription, are also likely to buy less expensive smartphones:

Missing alt text

As smartphone ownership and use grow, the frontends we deliver remain mediated by the properties of those devices. The inequality between the high-end and low-end is only growing, even in wealthy countries. What we choose to do in response defines what it means to practice UX engineering ethically.

Device Performance

Extending the SoC performance-by-price series with 2023's data, the picture remains ugly:

<em>Tap for a larger version.</em><br>Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.
Tap for a larger version.
Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.

Not only have fruity phones extended their single-core CPU performance lead over contemporary high-end Androids to a four year advantage, the performance-per-dollar curve remains unfavourable to Android buyers.

At the time of publication, the cheapest iPhone 15 Pro (the only device with the A17 Pro chip) is $999 MSRP, while the S23 (using the Snapdrago 8 gen 2) can be had for $860 from Samsung. This nets out to 2.32 points per dollar for the iPhone, but only 1.6 points per dollar for the S23.

Meanwhile, a $175 (new, unlocked) Samsung A24 scores a more reasonable 3.1 points per dollar on single-core performance, but is more than 4.25× slower than the leading contemporary iPhone.

The delta between the fastest iPhones and moderately price new devices rose from 1,522 points last year to 1,774 today.

Put another way, the performance gap between wealthy users and budget shoppers grew more this year (252 points) than the gains from improved chips delivered at the low end (174 points). Inequality is growing faster than the bottom-end can improve. This is particularly depressing because single-core performance tends to determine the responsiveness of web app workloads.

A less pronounced version of the same story continues to play out in multi-core performance:

<em>Tap for a larger version.</em><br>Round and round we go: Android ecosystem <abbr>SoC</abbr>s are improving, but the Performance Inequality Gap continues to grow. Even the fastest Androids are 18 months (or more) behind equivalently priced iOS-ecosystem devices.
Tap for a larger version.
Round and round we go: Android ecosystem SoCs are improving, but the Performance Inequality Gap continues to grow. Even the fastest Androids are 18 months (or more) behind equivalently priced iOS-ecosystem devices.

Recent advances in high-end Android multi-core performance have closed the previous three-year gap to 18 months. Meanwhile, budget segment devices have finally started to see improvement (as this series predicted), thanks to hand-me-down architecture and process node improvements. That's where the good news ends.

The multi-core performance gap between i-devices and budget Androids grew considerably, with the score delta rising from 4,318 points last year to 4,936 points in 2023.

Looking forward, we can expect high-end Androids to at least stop falling further behind owing to a new focus on performance by Qualcomm's Snapdragon 8 gen 3 and MediaTek's Dimensity 9300 offerings. This change is long, long overdue and will take years to filter down into positive outcomes for the rest of the ecosystem. Until that happens, the gap in experience for the wealthy versus the rest will not close.

iPhone owners experience a different world than high-end Android buyers, and live galaxies apart from the bulk of the market. No matter how you slice it, the performance inequality gap is growing for CPU-bound workloads like JavaScript-heavy web apps.

Networks

As ever, 2023 re-confirmed an essential product truth: when experiences are slow, users engage less. Doing a good job in an uneven network environment requires thinking about connection availability and engineering for resilience. It's always better to avoid testing the radio gods than spend weeks or months appeasing them after the damage is done.

5G network deployment continues apace, but as with the arrival of 4G, it is happening unevenly and in ways and places that exacerbate (rather than lessen) performance inequality.[4]

Data on mobile network evolution is sketchy,[5] and the largest error bars in this series' analysis continue to reside in this section. Regardless, we can look industry summaries like the GSMA's report on "The Mobile Economy 2023" (PDF) for a directional understanding that we can triangulate with other data points to develop a strong intuition.

For instance, GSMA predicts that 5G will only comprise half of connections by 2030. Meanwhile, McKinsey predicts that high-quality 5G (networks that use 6GHz bands) will only cover a quarter of the world's population by 2030. Regulatory roadblocks are still being cleared.

As we said in 2021, "4G is a miracle, 5G is a mirage."

This doesn't mean that 4G is one thing, or that it's deployed evenly, or even that the available spectrum will remain stable within a single generation of radio technology. For example, India's network environment has continued to evolve since the Reliance Jio revolution that drove 4G into the mainstream and pushed the price of a mobile megabyte down by ~90% on every subcontinental carrier.

Speedtest.net's recent data shows dramatic gains, for example, and analysts credit this to improved infrastructure density, expanded spectrum, and back-haul improvements related to the 5G rollout — 4G users are getting better experiences than they did last year because of 5G's role in reducing contention.

India's speed test medians are moving quickly, but variance is orders-of-magnitude wide, with 5G penetration below 25% in the most populous areas.
India's speed test medians are moving quickly, but variance is orders-of-magnitude wide, with 5G penetration below 25% in the most populous areas.

These gains are easy to miss looking only at headline "4G vs. 5G" coverage. Improvements arrive unevenly, with the "big" story unfolding slowly. These effects reward us for looking at P75+, not just means or medians, and intentionally updating priors on a regular basis.

Events can turn our intuitions on their heads, too. Japan is famously well connected. I've personally experienced rock-solid 4G through entire Tokyo subway journeys, more than 40m underground and with no hiccups. And yet, the network environment has been largely unchanged by the introduction of 5G. Having provisioned more than adequately in the 4G era, new technology isn't having the same impact from pent-up demand. But despite consistent performance, the quality of service for all users is distributed in a much more egalitarian way:

Japan's network environment isn't the fastest, but is much more evenly distributed.
Japan's network environment isn't the fastest, but is much more evenly distributed.

Fleet device composition has big effects, owing to differences in signal-processing compute availability and spectrum compatibility. At a population level, these influences play out slowly as devices age out, but still have impressively positive impacts:

Device impact on network performance is visible in Opensignal's iPhone dataset.
Device impact on network performance is visible in Opensignal's iPhone dataset.

As inequality grows, averages and "generation" tags can become illusory and misleading. Our own experiences are no guide; we've got to keep our hands in the data to understand the texture of the world.

So, with all of that as prelude, what can we say about where the mobile network baseline should be set? In a departure from years prior, I'm going to use a unified network estimate (see below). You'll have to read on for what it is! But it won't be based on the sort of numbers that folks explicitly running speed tests see; those aren't real life.

Market Factors

The market forces this series previewed in 2017 have played out in roughly a straight line: smartphone penetration in emerging markets is approaching saturation, ensuring a growing fraction of purchases are made by upgrade shoppers. Those who upgrade see more value in their phones and save to buy better second and third devices. Combined with the emergence and growth of the "ultra premium" segment, average selling prices (ASPs) have risen.

2022 and 2023 have established an inflection point in the regard, with worldwide average selling prices jumping to more than $430, up from $300-$350 for much of the decade prior. Some price appreciation has been due to transient impacts of the U.S./China trade wars, but most of it appears driven by iOS ASPs which peaked above $1,000 for the first time in 2023. Android ASPs, meanwhile, continued a gradual rise to nearly $300, up from $250 five years ago.

Missing alt text

A weak market for handsets in 2023, plus stable sales for iOS, had an notable impact on prices. IDC expects global average prices to fall back below $400 by 2027 as Android volumes increase from an unusually soft 2023.

Counterpoint data shows declining sales in both 2022 and 2023.
Counterpoint data shows declining sales in both 2022 and 2023.
Shipment growth in late 2023 and beyond is coming from emerging markets like the Middle East and Africa. Samsung's A-series mid-tier is doing particularly well.
Shipment growth in late 2023 and beyond is coming from emerging markets like the Middle East and Africa. Samsung's A-series mid-tier is doing particularly well.

Despite falling sales, distribution of Android versus iOS sales remains largely unchanged:

Android sales reliably constitute 80-85% of worldwide volume.
Android sales reliably constitute 80-85% of worldwide volume.
Even in rich nations like Australia and the <a href='https://www.statista.com/statistics/262179/market-share-held-by-mobile-operating-systems-in-the-united-kingdom/'>the U.K.</a>, iPhones account for less than half of sales. Predictably, they are over-represented in analytics and logs owing to wealth-related factors including superior network access and performance hysteresis.
Even in rich nations like Australia and the the U.K., iPhones account for less than half of sales. Predictably, they are over-represented in analytics and logs owing to wealth-related factors including superior network access and performance hysteresis.

Smartphone replacement rates have remained roughly in line with previous years, although we should expect higher device longevity in future years. Survey reports and market analysts continue to estimate average replacement at 3-4 years, depending on segment. Premium devices last longer, and a higher fraction of devices may be older in wealthy geographies. Combined with discretionary spending pressure and inflationary impacts on household budgets, consumer intent to spend on electronics has taken a hit, which will be felt in device lifetime extension until conditions improve. Increasing demand for refurbished devices also adds to observable device aging.

The data paints a substantially similar picture to previous years: the web is experienced on devices that are slower and older than those carried by affluent developers and corporate directors whose purchasing decisions are not impacted by transitory inflation.

To serve users effectively, we must do extra work to live as our customers do.

Test Device Recommendations

Re-using last year's P75 device calculus, our estimate is based on a device sold new, unlocked for the mid-2020 to mid-2021 global ASP of ~$350-375.

Representative examples from that time period include the Samsung Galaxy A51 and the Pixel 4a. Neither model featured 5G,[6] and we cannot expect 5G to play a significant role in worldwide baselines for at least the next several years.[4:1]

The A51 featured eight slow cores (4x2.3 GHz Cortex-A73 and 4x1.7 GHz Cortex-A53) on a 10nm process:

Geekbench 6 scores for the Galaxy A51 versus today's leading device.
Geekbench 6 scores for the Galaxy A51 versus today's leading device.

The Pixel 4a's slow, eight-core big.LITTLE configuration was fabricated on an 8nm process:

Google spent more on the <abbr>SoC</abbr> for the Pixel 4a and enjoyed a later launch date, boosting performance relative to the A51.
Google spent more on the SoC for the Pixel 4a and enjoyed a later launch date, boosting performance relative to the A51.

Pixels have never sold well, and Google's focus on strong SoC performance per dollar was sadly not replicated across the Android ecosystem, forcing us to use the A51 as our stand-in.

Devices within the envelope of our attention are 15-25% as fast as those carried by programmers and their bosses — even in wealthy markets.

The Galaxy may be slightly faster than last year's recommendation of the Galaxy A50 for testing, but the picture is muddy:

Geekbench 5 shows almost no improvement between the A50 and the A51.
Geekbench 5 shows almost no improvement between the A50 and the A51.
Geekbench 6 shows the same story within the margin of error. The low-end is stagnant, and still <a href='https://www.statista.com/statistics/934471/smartphone-shipments-by-price-category-worldwide/' target='_new'>30% of worldwide volume</a>.
Geekbench 6 shows the same story within the margin of error. The low-end is stagnant, and still 30% of worldwide volume.

If you're building a test lab today, refurbished A51s can be had for ~$150. Even better, the newer Nokia G100 can be had for as little as $100, and it's faithful to the sluggish original in nearly every respect.[7]

If your test bench is based on last year's recommended A50 or Nokia G11, I do not recommend upgrading in 2024. The absolute gains are so slight that the difference will be hard to feel, and bench stability has a value all its own. Looking forward, we can also predict that our bench performance will be stable until 2025.

Claims about how "performant" modern frontend tools are have to be evaluated in this slow, stagnant context.

Desktop

It's a bit easier to understand the Desktop situation because the Edge telemetry I have access to provides statistically significant insight into 85+% of the market.

Device Performance

The TL;DR for desktop performance is that Edge telemetry puts ~45% of devices in a "low-end" bucket, meaning they have <= 4 cores or <= 4GB of RAM.

Device Tier Fleet % Definition
Low-end 45% Either:
<= 4 cores, or
<= 4GB RAM
Medium 48% HDD (not SSD), or
4-16 GB RAM, or
4-8 cores
High 7% SSD +
> 8 cores +
> 16GB RAM

20% of users are on HDDs (not SSDs) and nearly all of those users also have low (and slow) cores.

You might be tempted to dismiss this data because it doesn't include Macs, which are faster than the PC cohort. Recall, however, that the snapshot also excludes ChromeOS.

ChromeOS share has veered wildly in recent years, representing 50%-200% of Mac shipments in a given per quarter. In '21 and '22, ChromeOS shipments regularly doubled Mac sales. Despite post-pandemic mean reversion, according to IDC ChromeOS devices outsold Macs ~5.7M to ~4.7M in 2023 Q2. The trend reversed in Q3, with Macs almost doubling ChromeOS sales, but slow ChromeOS devices aren't going away and, from a population perspective, more than offset Macs today. Analysts also predict growth in the low end of the market as educational institutions begin to refresh their past purchases.

Networks

Desktop-attached networks continue to improve, notably in the U.S. Regulatory intervention and subsidies have done much to spur enhancements in access to U.S. fixed broadband, although disparities in access remain and the gains may not persist.

This suggests that it's time to also bump our baseline for desktop tests beyond the 5Mbps/1Mbps/28ms configuration that WebPageTest.org's "Cable" profile has defaulted to for desktop tests.

How far should we bump it? Publicly available data is unclear, and I've come to find out that Edge's telemetry lacks good network observation statistics (doh!).

But the comedy of omissions doesn't end there: Windows telemetry doesn't capture a proxy for network quality, I no longer have access to Chrome's data, the population-level telemetry available from CrUX is unhelpful, and telcos li...er...sorry, "market their products in accordance with local laws and advertising standards."

All of this makes it difficult to construct an estimate.

One option is to use a population-level assessment of medians from something like the Speedtest.net data and then construct a histogram from median speeds. This is both time-consuming and error-prone, as population-level data varies widely across the world. Emerging markets with high mobile internet use and dense populations can feature poor fixed-line broadband penetration compared with Western markets.

Another option is to mathematically hand-wave using the best evidence we can get. This might allow us to reconstruct probable P75 and P90 values if we know something about the historical distribution of connections. From there, we can gut-check using other spot data. To do this, we need to assume some data set is representative, a fraught decision all its own.[8] Biting the bullet, we could start from the Speedtest.net global survey data, which currently fails to provide anything but medians (P50):

Speedtest.net's global median values are unhelpful on their own, both because they represent users who are testing for speed (and not organic throughput) and because they don't give us a fuller understanding of the distribution.
Speedtest.net's global median values are unhelpful on their own, both because they represent users who are testing for speed (and not organic throughput) and because they don't give us a fuller understanding of the distribution.

After many attempted Stupid Math Tricks with poorly fitting curves (bandwidth seems to be a funky cousin of log-normal), I've decided to wing it and beg for help: instead of trying to be clever, I'm leaning on Cloudflare Radar's P25/P50/P75 distributions for populous, openly-connected countries with >= ~50M internet users. It's cheeky, but a weighted average of the P75 of download speeds (3/4ths of all connections are faster) should get us in the ballpark. We can then use the usual 5:1 downlink:uplink ratio to come up with an uplink estimate. We can also derive a weighted average for the P75 RTT from Cloudflare's data. Because Cloudflare doesn't distinguish mobile from desktop connections, this may be an overly conservative estimate, but it's still be more permissive than what we had been pegged to in years past:

National P75 Downlink and RTT
Country P75 Downlink (Mbps) P75 RTT (ms)
India 4 114
USA 11 58
Indonesia 5 81
Brazil 8 71
Nigeria 3 201
Pakistan 3 166
Bangladesh 5 114
Japan 17 42
Mexico 7 75
Egypt 4 100
Germany 16 36
Turkey 7 74
Philippines 7 72
Vietnam 7 72
United Kingdom 16 37
South Korea 24 26
Population Weighted Avg. 7.2 94

We, therefore, update our P75 link estimate 7.2Mbps down, 1.4Mbps up, and 94ms RTT.

This is a mild crime against statistics, not least of all because it averages unlike quantities and fails to sift mobile from desktop, but all the other methods available at time of writing are just as bad. Regardless, this new baseline is half again as much link capacity as last year, showing measurable improvement in networks worldwide.

If you or your company are able to generate a credible worldwide latency estimate in the higher percentiles for next year's update, please get in touch.

Market Factors

The forces that shape the PC population have been largely fixed for many years. Since 2010, volumes have been on a slow downward glide path, shrinking from ~350MM per year in a decade ago to ~260MM in 2018. The pandemic buying spree of 2021 pushed volumes above 300MM per year for the first time in eight years, with the vast majority of those devices being sold at low-end price points — think ~$300 Chromebooks rather than M1 MacBooks.

Lest we assume low-end means "short-lived", recent announcements regarding software support for these devices will considerably extend their impact. This low-end cohort will filter through the device population for years to come, pulling our performance budgets down, even as renewed process improvement is unlocking improved power efficiency and performance at the high end of the first-sale market. This won't be as pronounced as the diffusion of $100 smartphones has been in emerging markets, but the longer life-span of desktops is already a factor in our model.

Test Device Recommendations

Per our methodology from last year which uses the 5-8 year replacement cycle for a PC, we update our target date to late 2017 or early 2018, but leave the average-selling-price fixed between $600-700. Eventually we'll need to factor in the past couple of years of gyrations in inflation and supply chains into account when making an estimate, but not this year.

So what did $650, give or take, buy in late 2017 or early 2018?

One option was a naf looking tower from Dell, optimistically pitched at gamers, with a CPU that scores poorly versus a modern phone, but blessedly includes 8GB of RAM.

In laptops (the larger segment), ~$650 bought the Lenovo Yoga 720 (12"), with a 2-core (4-thread) Core i3-7100U and 4GB of RAM. Versions with more RAM and a faster chip were available, but cost considerably more than our budget. This was not a fast box. Here's a device with that CPU compared to a modern phone; not pretty:

The phones of wealthy developers absolutely smoke the baseline PC.
The phones of wealthy developers absolutely smoke the baseline PC.

It's considerably faster than some devices still being sold to schools, though.

What does this mean for our target devices? There's wild variation in performance per dollar below $600 which will only increase as inflation-affected cohorts grow to represent a larger fraction of the fleet. Intel's move off of 14nm (finally!) also means that gains are starting to arrive at the low end, but in an uneven way. General advice is therefore hard to issue. That said, we can triangulate based on what we know about the market:

My recommendation, then, to someone setting up a new lab today is not to spend more than $350 on new a test device. Consider laptops with chips like the N4120, N4500, or the N5105. Test devices should also have no more than 8GB of RAM, and preferably 4GB. The 2021 HP 14 is a fine proxy. The updated ~$375 version will do in a pinch, but try to spend less if you can. Test devices should preferably score no higher than 1,000 in single-core Geekbench 6 tests; a line the HP 14's N4120 easily ducks, clocking in at just over 350.

Takeaways

There's a lot of good news embedded in this year's update. Devices and networks have finally started to get faster (as predicted), pulling budgets upwards.

At the same time, the community remains in denial about the disastrous consequences of an over-reliance on JavaScript. This paints a picture of path dependence — frontend isn't moving on from approaches that hurt users, even as the costs shift back onto teams that have been degrading life for users at the margins.

We can anticipate continued improvement in devices, while network gains will level out as the uneven deployment of 5G stumbles forward. Regardless, the gap between the digital haves and have-nots continues to grow. Those least able to afford fast devices are suffering regressive taxation from developers high on DX fumes.

It's no mystery why folks in the privilege bubble are not building with empathy or humility when nobody calls them to account. What's mysterious is that anybody pays them to do it.

The PM and EM disciplines have utterly failed, neglecting to put business constraints on the enthusiasms of developers. This burden is falling, instead, on users and their browsers. Browsers have had to step in as the experience guardians of last resort, indicating a market-wide botching of the job in technology management ranks and an industry-scale principal-agent issue amongst developers.

Instead of cabining the FP crowd's proclivities for the benefit of the business, managers meekly repeat bullshit like "you can't hire for fundamentals" while bussing in loads of React bootcampers. It is not too much to ask that managers run bake-offs and hire for skills in platform fundamentals that serve businesses better over time. The alternative is continued failure, even for fellow privilege bubble dwellers.

Case in point: this post was partially drafted on airplane wifi, and I can assure you that wealthy folks also experience RTT's north of 500ms and channel capacity in the single-digit-Mbps.

Even the wealthiest users step into the wider world sometimes. Are these EMs and PMs really happy to lose that business?

<em>Tap for a larger version.</em><br>Wealthy users are going to experience networks with properties that are even worse than the 'bad' networks offered to the Next Billion Users. At an altitude of 40k feet and a ground speed for 580 MPH somewhere over Alberta, CA, your correspondent's bandwidth is scarce, lopsided, and laggy.
Tap for a larger version.
Wealthy users are going to experience networks with properties that are even worse than the 'bad' networks offered to the Next Billion Users. At an altitude of 40k feet and a ground speed for 580 MPH somewhere over Alberta, CA, your correspondent's bandwidth is scarce, lopsided, and laggy.

Of course, any trend that can't continue won't, and INP's impact is already being felt. The great JavaScript merry-go-round may grind to a stop, but the momentum of consistently bad choices is formidable. Like passengers on a cruise ship ramming a boardwalk at flank speed, JavaScript regret is dawning far too late. As the good ship Scripting shudders and lists on the remains of the ferris wheel, it's not exactly clear how to get off, but the choices that led us here are becoming visible, if only through their negative consequences.

The Great Branch Mispredict

We got to a place where performance has been a constant problem in large part because a tribe of programmers convinced themselves that it wasn't and wouldn't be. The circa '13 narrative asserted that:

  • CPUs would keep getting faster (just like they always had).
  • Networks would get better, or at least not get worse.
  • Organisations had all learned the lessons of Google and Facebook's adventures in Ajax.

It was all bullshit, and many of us spotted it a mile away.

The problem is now visible and demands a solution, but the answers will be largely social, not technical. User-centered values must contest the airtime previouly taken by failed trickle-down DX mantras. Only when the dominant story changes will better architectures and tools win.

How deep was the branch? And how many cycles will the fault cost us? If CPUs and networks continue to improve at the rate of the past two years, and INP finally forces a reckoning, the answer might be as little as a decade. I fear we will not be so lucky; an entire generation has been trained to ignore reality, to prize tribalism rather than engineering rigor, and to devalue fundamentals. Those folks may not find the next couple of years to their liking.

Frontend's hangover from the JavaScript party is gonna suck.


  1. The five second first-load target is arbitrary, and has always been higher than I would prefer. Five seconds on a modern computer is an eternity, but in 2016 I was talked down from my preferred three-second target by Googlers that despaired that "nobody" could hit that mark on the devices and networks of that era.

    This series continues to report budgets with that target, but keen readers will see that I'm also providing three-second numbers. The interactive estimation tool was also updated this year to provides the ability to configure the budget target.

    If you've got thoughts about how this should be set in future, or how it could be handled better, plesae get in touch. ↩︎ ↩︎

  2. Frontend developers are cursed to program The Devil's Computer. Web apps execute on slow devices we don't spec or provision, on runtimes we can barely reason about, lashed to disks and OSes taxed by malware and equally invasive security software, over networks with the variability of carrier pigeons.

    It's vexing, then, that contemporary web development practice has decided that the way to deliver great experiences is to lean into client CPUs and mobile networks, the most unreliable, unscalable properties of any stack.

    And yet, here we are in 2024, with Reactors somehow still anointed to decree how and where code should run, despite a decade of failure to predict the obvious, or even adapt to the world as it has been. The mobile web overtook desktop eight years ago, and the best time to call bullshit on JS-first development was when we could first see the trends clearly.

    The second best time is now. ↩︎

  3. Engineering is design under constraint, with the goal to develop products that serve users and society.

    The opposite of engineering is bullshit; substituting fairy tales for inquiry and evidence.

    For the frontend to earn and keep its stripes as an engineering discipline, frontenders need to internalise the envelope of what's possible on most devices. ↩︎

  4. For at least a decade to come, 5G will continue to deliver unevenly depending on factors including building materials, tower buildout, supported frequencies, device density, radio processing power, and weather. Yes, weather (PDF).

    Even with all of those caveats, 5G networks aren't the limiting factor in wealthy geographies; devices are. It will take years for the deployed base to be fully replaced with 5G-capable handsets, and we should expect the diffusion to be "lumpy", with wealthy markets seeing 5G device saturation at nearly all price points well in advance of less affluent countries where capital availability for 5G network roll-outs will dominate. ↩︎ ↩︎

  5. Ookla! Opensignal! Cloudflare! Akamai! I beseech thee, hear my plea and take pity, oh mighty data collectors.

    Whilst you report medians and averages (sometimes interchangeably, though I cannot speculate why), you've stopped publishing useable histogram information about the global situation, making the reports nearly useless for anything but telco marketing. Opensignal has stopped reporting meaningful 4G data at all, endangering any attempt at making sense.

    Please, I beg of you, publish P50, P75, P90, and P95 results for each of your market reports! And about the global situation! Or reach out directly and share what you can in confidence so I can generate better guidance for web developers. ↩︎

  6. Both the benchmark A51 and Pixel 4a devices were eventually sold in 5G variants (A51 5G, Pixel 4a 5G), but at a price of $500 brand-new, unlocked at launch, making them more than 40% above the price of the base models and well above our 2020-2021 ASP of $350-$375. ↩︎

  7. Samsung's lineup is not uniform worldwide, with many devices being region-specific. The closest modern (Western) Samsung device to the A51 is the Samsung A23 5G, which scores in the range of the Pixel 4a. As a result of the high CPU score and 5G modem, it's hard to recommend it — or any other current Samsung model — as a lab replacement. Try the Nokia G100 instead. ↩︎

  8. The idea that any of the publicly available data sets is globally representative should set off alarms.

    The obvious problems include (but are not limited to):

    • geographic differences in service availability and/or deployed infrastructure,
    • differences in market penetration of observation platforms (e.g., was a system properly localised? Equally advertised?), and
    • mandated legal gaps in coverage.

    Of all the hand-waving we're doing to construct an estimate, this is the biggest leap and one of the hardest to triangulate against. ↩︎