Dismantling Domain Sharding for HTTP/3

Your asset pipeline still spreads images and static files across img1., img2., and static. subdomains — a pattern that won benchmarks in 2012 — and now every cold page load burns 300–600 ms in duplicate DNS lookups and TLS/QUIC handshakes while your LCP image queues behind a congestion window that never gets warm.

Root Cause: Sharding Optimizes for a Constraint That No Longer Exists

Domain sharding was a workaround for HTTP/1.1’s concurrency model: one request per connection at a time, and a browser cap of six connections per hostname. Splitting 80 page assets across four hostnames bought 24 parallel lanes instead of 6, and the throughput gain outweighed the setup cost of the extra sockets. Every part of that trade inverts under multiplexed protocols. HTTP/2 runs 100+ concurrent streams over one TCP connection and HTTP/3 does the same over QUIC, so the concurrency ceiling sharding worked around is simply gone — what remains is only the cost column: one DNS resolution, one TCP or QUIC handshake, and one TLS negotiation per shard, roughly 100–300 ms each on a cold mobile connection, paid before the first asset byte moves.

The subtler damage is what sharding does to the transport layer’s learning process. TCP and QUIC congestion control start each connection with a small initial window (typically 10 packets, ~14 KB) and grow it as ACKs confirm capacity. One connection carrying all assets ramps once and then runs at full estimated bandwidth; four connections each start cold, probe the same bottleneck link independently, compete with each other for it, and — because each shard carries a quarter of the bytes — may finish before ever leaving slow start. Sharding also decapitates prioritization: the browser’s scheduler expresses urgency within a connection via HTTP/2 stream weights or RFC 9218 urgency, but it has no mechanism to tell shard A’s connection to yield bandwidth to the LCP image on shard B’s connection. Four independent sockets are four schedulers that cannot see each other, so your hero image races 30 below-the-fold thumbnails at equal footing. On top of that, each extra hostname is a separate 0-RTT session ticket, a separate Alt-Svc discovery, and a separate slot in the partitioned browser cache.

Browsers can rescue part of this automatically. When two shards resolve to the same IP and port and the presented certificate’s SAN list covers both hostnames, HTTP/2 and HTTP/3 clients silently route both origins over one connection — the coalescing behaviour specified in RFC 7540 §9.1.1 and RFC 9114 §3.3 and covered in depth by the parent section. But relying on it as a permanent fix fails four ways: anycast CDNs frequently hand different shards different IPs per resolver, breaking the IP-match precondition intermittently and per-geography; Safari validates SAN coverage more conservatively and opens second connections where Chrome coalesces; any HTTP/1.1 fallback traffic (old proxies, some bots, UDP-blocked-plus-h2-disabled paths) reverts to full sharded behaviour; and coalescing never refunds the per-hostname DNS lookups or reunifies the URL-keyed caches. The durable fix is to make the shards disappear from your URLs, and that is where teams get hurt — because asset URLs are cache keys, and rewriting them invalidates every copy in the browser cache, the CDN edge, and any service worker precache manifest simultaneously.

Every one of those four preconditions has to hold for a single request, on a single client, in a single geography, before coalescing saves you anything — and even when they all hold, two costs survive:

The four preconditions between a shard request and an actually coalesced connection A five-step decision chain. A request for an asset on img2.example.com must clear four gates in order: the same resolved IP and port as www, one certificate whose SAN list covers both hostnames, an HTTP/2 or HTTP/3 negotiation rather than an HTTP/1.1 fallback, and an engine willing to fold on that evidence. Failing the first two costs a second connection worth one DNS lookup plus one QUIC handshake, 100 to 300 milliseconds on cold 4G; failing the third drops the client back to six sockets per shard, 24 across four hostnames; failing the fourth splits behaviour by engine, with Safari opening a second connection where Chrome folds. Even a fully coalesced client still pays one DNS lookup per hostname and keeps a separate cache entry per URL. Will this client actually coalesce img2 onto the www connection? Browser needs an asset on img2.example.com Same resolved IP and port as www.example.com? One certificate whose SAN list covers both hostnames? Client negotiated h2 or h3? (not a proxy stuck on 1.1) Engine folds on that evidence? (Safari validates harder) Coalesced — one connection no extra handshake for this asset Second connection: +1 DNS, +1 QUIC handshake — 100–300 ms on a cold 4G link SAN gap — the client refuses to reuse the www connection and opens its own HTTP/1.1 fallback — 6 sockets per shard, 24 across the four hostnames Split by engine — Safari opens a second connection where Chrome folds Still paid either way: 1 DNS lookup per hostname, and a cache entry per URL yes yes yes yes no no no no always

Before/After: Four Sharded Connections vs One Multiplexed

Sharded vs single-origin connection waterfall under HTTP/3 Top panel: four connection lanes for the www, img1, img2 and static hostnames. Each lane spends its opening 170 milliseconds on DNS plus a QUIC handshake before any transfer starts, and each ramps its congestion window separately, so the hero image on img1 does not finish until 1,350 milliseconds. Bottom panel: one origin pays setup once, then multiplexed streams for the document, the hero image at urgency 0, the stylesheet and script bundle, and the urgency 4 thumbnails share one warm window, and the hero image finishes at 810 milliseconds. Before — 4 sharded hosts, 4 cold connections 0 700 ms 1,400 ms www. img1. img2. static. LCP image done at 1,350 ms each shard repeats DNS + QUIC, then ramps its window from cold After — 1 origin, 1 handshake, multiplexed streams 0 700 ms 1,400 ms cdn. (setup ×1) stream: document stream: hero (u=0) streams: css + js streams: thumbs (u=4) LCP image done at 810 ms one warm window; the scheduler yields u=4 bytes to the hero DNS + handshake document hero image (u=0) css + js thumbnails (u=4)

Minimal Reproduction

A fossilized sharding helper still present in many asset pipelines — round-robin hostname assignment, which also breaks URL stability between renders:

// legacy-shards.js — the 2012 pattern that now taxes every page load.
// Round-robin means the SAME image can hash to img1 on one render and
// img2 on the next, defeating the browser cache on top of the
// handshake cost. Scheduling impact: 4 cold connections, 0 shared
// congestion state, no cross-connection priority signal for the hero.
const SHARDS = ['img1', 'img2', 'img3', 'static'];
let i = 0;
export function assetUrl(path) {
  const host = `${SHARDS[i++ % SHARDS.length]}.example.com`;
  return `https://${host}${path}`;
}

Confirm the tax in three commands:

# 1. Do the shards even share an IP? (If not, coalescing is impossible.)
for h in www img1 img2 static; do echo -n "$h: "; dig +short $h.example.com | head -1; done

# 2. Does one cert cover them all? (SAN list must include every shard.)
openssl s_client -connect www.example.com:443 2>/dev/null \
  | openssl x509 -noout -ext subjectAltName

# 3. Count real connections in Chrome: Network panel → enable the
#    "Connection ID" column → hard reload. Four distinct IDs = four taxes.

Deterministic Fix Protocol

  • [ ] 1. Inventory every shard reference. grep -rhoE 'https?://(img[0-9]+|static|assets[0-9]*)\.' dist/ src/ | sort | uniq -c across built output, templates, CSS url() values, service worker precache manifests, and database-stored rich text. The database-stored URLs are the ones migrations forget.
  • [ ] 2. Bridge with coalescing before touching URLs. Point all shard CNAMEs at one endpoint and deploy a certificate whose SAN covers every shard hostname. This removes the handshake tax for h2/h3 clients on day one with zero URL churn, buying time for the real migration. Confirm it the same way every time, by checking that the shards share one Connection ID in DevTools — a shared IP alone is not proof the browser folded them.
  • [ ] 3. Verify coalescing held across engines and geographies. Check Safari specifically, and sample dig from multiple resolver locations — if any geography resolves shards to different PoP IPs, coalescing silently degrades there and step 2 is only partial. Field-verify with a RUM check that domainLookupEnd - domainLookupStart is zero for shard-hostname resources, and segment the same beacon by the nextHopProtocol value each resource reports so the http/1.1 slice — the traffic coalescing can never help — is sized rather than assumed.
  • [ ] 4. Rewrite URL generation to the single origin. Replace the shard helper with one constant base. Do it at the pipeline level (framework asset prefix, CMS media base URL, image-CDN domain setting) so shards cannot regrow from templates.
  • [ ] 5. Keep filenames byte-stable through the move. If your bundler embeds the public base URL into content hashes, changing the base regenerates every hash — a full cache wipe at browser and CDN. Configure hashing on content only, and confirm by building with old and new base and diffing the manifest: filenames must be identical.
  • [ ] 6. Warm the CDN under the new hostname before cutover. Edge caches key on hostname + path (plus Vary), so cdn.example.com/a.webp is cold even where img1.example.com/a.webp is hot. Replay the top-N asset URLs against the new hostname per PoP region, then ship the HTML that references them.
  • [ ] 7. Serve 301s from old shards through a long deprecation window. Cached HTML, RSS readers, third-party embeds, and old service worker manifests will request shard URLs for months. Redirects preserve those users at the cost of one hop; killing the DNS records instead breaks them.
  • [ ] 8. Purge stale hints and re-point priorities. Remove preconnect/dns-prefetch for retired shards (each is now a wasted socket — see strategic preconnect usage), and confirm the LCP image carries fetchpriority="high" so the now-unified scheduler actually favours it on the shared connection.
  • [ ] 9. Watch one full cache cycle. Expect a CDN hit-ratio dip and a one-visit browser-cache miss per returning user; both should recover within your asset max-age. Alert if 404s on old shard hostnames rise instead of decaying — that means a reference source from step 1 was missed.

Those nine steps collapse into five states a shard hostname passes through, each with one check that must pass before the next transition is safe. The only reversible transition is the URL rewrite, and it reverses for exactly one reason:

The five states a shard hostname passes through, and the check that gates each move A shard hostname starts live, with four IPs and four handshakes costing 1,090 milliseconds of setup. It moves to a coalesced bridge once every geography resolves the shards to one IP and DevTools shows a single Connection ID in Safari as well as Chrome, cutting setup to 260 milliseconds without touching URLs. It moves to rewritten URLs once old-base and new-base builds diff to identical filenames and the top assets are warmed on the new hostname per point of presence; this is the only reversible step, rolling back to the bridge if content hashes changed. It becomes redirect-only once no shard references remain in the build output, source or content store and the CDN hit ratio climbs back from 81 percent toward 95 percent, and is finally retired after shard 404s decay for a full max-age cycle, roughly six days in the measured migration. Lifecycle of one shard hostname, and the check that gates each move 1 · Live shard 4 IPs, 4 handshakes, 1,090 ms of setup 2 · Coalesced bridge One IP + SAN-complete cert, 260 ms 3 · URLs rewritten HTML, CSS url(), stored copy point at cdn. 4 · Redirect-only Old shards answer 301 while traffic decays 5 · Retired DNS records removed — one origin remains rollback: content hashes changed guard: every geography resolves shards to one IP verify: a single Connection ID, Safari as well as Chrome guard: old-base and new-base builds diff to identical filenames; top-N assets warmed on cdn. per PoP guard: 0 shard refs left in dist/, src/ or stored copy verify: CDN hit ratio climbing back from 81% to 95% guard: shard 404s decaying for a full max-age cycle verify: 6 days of recovery before the records go

Before/After Metrics

Catalogue site, 92 assets per page, previously sharded across four hostnames; measured p75 on 4G (170 ms RTT) two weeks after full cutover.

Metric Sharded (4 hosts) Single origin Change
DNS lookups per cold load 4 1 −3
TLS/QUIC handshakes per cold load 4 1 −3
Connection setup time (sum, cold) 1,090 ms 260 ms −76%
LCP p75 3.4 s 2.3 s −32%
Bytes in slow-start at LCP time 4 × cold cwnd 1 × warm cwnd unified ramp
0-RTT-eligible return connections 1 of 4 hosts typical all traffic ×4 surface
CDN hit ratio (cutover week) 94% 81% → 95% recovered in 6 days

The LCP gain has two roots: the hero image no longer waits behind a cold handshake to its shard, and the unified connection lets urgency signals demote thumbnail streams that previously competed from separate sockets. The temporary hit-ratio dip is the cost of step 6 done imperfectly — warming deeper than the top-N would have flattened it further.

FAQ

Can I just point all shard hostnames at the same IP and let coalescing fix it without touching URLs?

Only as a bridge. With a shared IP and a SAN-complete certificate, h2/h3 browsers fold the shards onto one connection and the handshake tax disappears — but you still pay a DNS lookup per hostname, Safari coalesces more conservatively than Chrome, HTTP/1.1 fallback clients re-open six sockets per shard, and the URL-keyed caches stay fragmented. Treat coalescing as step one of the dismantling, not the end state.

Will migrating asset URLs to one origin wipe returning visitors’ browser caches?

Yes. The HTTP cache keys on the full URL, so moving img1.example.com/a.webp to cdn.example.com/a.webp is a guaranteed miss even for identical bytes — content-hashed filenames don’t save you because the hostname changed, not the hash. Budget one cold-ish page load per returning visitor, keep old shards answering (301 or mirrored content) through the transition, and schedule the cutover away from peak campaigns.

Should I keep one shard for a separate cookieless domain?

Rarely. HPACK (h2) and QPACK (h3) header compression shrink repeated cookie headers to a few bytes after the first request on a connection, which removes most of the bandwidth argument. A second origin costs a DNS lookup, an extra handshake, a split congestion window, and a second 0-RTT session-ticket surface to manage. Trim the cookies or scope them with Path/Domain attributes instead.

Related