gw-04 — Analysis

The design-review treatment of the pool + subsetting you build in steps/, and the trade-offs to defend.

Required behaviors

Reuse by default. A second request to the same origin must not open a new connection while a warm one is idle in the pool.
Bounded pool. Per-(origin, loop) pool has a max size; over the cap, either queue with a timeout or open a temporary connection — a policy decision, stated explicitly.
Balanced subset. Across the fleet, each origin appears in ~gateways × subset / origins subsets (even coverage).
Stable subset. Adding/removing one member changes only a small, balanced fraction of assignments — re-subsetting must not itself cause a churn storm.
Lifecycle hygiene. Idle eviction (TTL) and max-lifetime recycling, with keepalive so a dead origin is detected, not handed out.

Pool keyed by (origin, eventLoopID). This is the Zuul insight made concrete: the key includes the loop so a request on loop A never touches loop B's connections — lock-free, no handoff. The lab simulates K loops with K goroutine-bound pools.
Subset selection by Van der Corput ring. Map each member to a point on [0,1) via the bit-reversed sequence; each gateway takes the subset_size origins nearest its own point. Deterministic, no coordination, balanced, and stable under membership change. The step measures coverage variance to prove balance.
h1 and h2 modes. The pool supports both: h1 pools ≈ concurrency connections per origin; h2 pools a few and multiplexes. The step shows the connection-count difference directly.
Churn measured, not asserted. The lab's whole point is the metric: connections.created counter, sampled per second, plotted with and without each technique. You reproduce the shape of the Netflix result (a big drop and a flat line), not an exact number.

Per-loop pools raise minimum connection count. With K loops you warm up to K × the connections of a shared pool. Subsetting offsets this; the two techniques are designed to be used together, and the net is still far below the no-subset baseline.
Subset size is a resilience knob. A small subset minimizes connections but means losing a few origins removes a large fraction of a gateway's capacity to that service. Size the subset so that losing f origins still leaves enough; this is a quorum-flavored argument (echoes db-17's majority reasoning).
Idle eviction vs churn. Aggressive eviction frees memory but re-handshakes on the next request — re-creating the very churn you removed. Tune the idle TTL to the traffic's inter-request gap; validate with the churn metric.
Stability vs balance under change. Perfectly balanced static assignment and minimal movement under change are in slight tension; the low-discrepancy ring is the sweet spot. Hash-mod is maximally unstable (everything moves when membership changes); fully random is unbalanced. Be able to rank the three.
The cold-pool thundering herd. A fleet deploy empties every pool; all gateways re-establish at once → a churn spike precisely during a deploy. Mitigations are operational (gw-12): staggered rollout, pre-warming, reconnect jitter, traffic ramp.

Pool acquisition backpressure integrated with admission control (gw-06): when the pool is saturated, shed load rather than queue unboundedly.
Outlier ejection inside the subset (gw-06): a bad origin in your subset is removed from rotation without re-subsetting the whole ring.
Control-plane-driven membership (gw-08 EDS / gw-09 EndpointSlices) with debounced re-subsetting so rapid pod churn doesn't thrash the ring.
Per-origin protocol negotiation (ALPN) so the pool automatically uses h2 where available and h1 otherwise.
Rich churn observability: connections.created.rate, pool-utilization, acquisition-wait histogram, eviction counters (gw-11).