gw-03 — The Hitchhiker's Guide to the API Gateway

Companion to CONCEPTS.md, with the runnable gateway in src/go/gateway/. This is the lab the role is named after; the filter-chain model here is the mental model you'll use every day on a Cloud Gateway team.

An API gateway is a reverse proxy with a programmable request lifecycle. Zuul decomposes that lifecycle into three ordered phases — inbound → endpoint → outbound — and every cross-cutting concern (auth, routing, rate limiting, retries, logging) is a filter in one of them. Build that engine once and you understand Zuul, Spring Cloud Gateway, Envoy's HTTP filters, and the Gateway API all at once.

1. The engine (gateway.go)

Gateway.ServeHTTP is the whole lifecycle in 25 lines:

build RequestContext
run INBOUND filters (sorted by Order) — STOP early if one short-circuits
run the ENDPOINT filter (only if not stopped and it applies)
run OUTBOUND filters (ALWAYS — even after a short-circuit)
write the response

Three design decisions carry the weight:

One mutable RequestContext threaded through the chain. This is Zuul's SessionContext. It carries the request, the response being built, the chosen route, an attributes map for cross-filter data (identity, cluster, status_class), and per-filter timings. The cost is shared mutable state — so filters must document what they read/write — but it matches the real codebase exactly.

Filters are interfaces with an explicit Order() and Type(), not closures. TestPhaseOrderingAndShortCircuit proves the ordering: with no Authorization header, the inbound AuthFilter (order 10) short- circuits with a 401 before RoutingFilter (order 50) or the endpoint ever run — you never open an origin connection for a request you're going to reject. Explicit ordering is the point; closures would hide it.

Outbound always runs. Notice the loop has no if !stopped guard around the outbound phase. TestPhaseOrderingAndShortCircuit asserts the access log fires even on the 401. Metrics, logs, and trace completion must never be skipped just because an earlier filter rejected the request — otherwise your dashboards undercount errors precisely when errors spike.

Panic isolation. runOne wraps each Apply in recover(): a filter that panics becomes a 502, the chain continues, and the access log still fires (TestPanicBecomes502). A gateway that can crash a connection on one filter's bug is not production-grade — and at fleet-wide blast radius, one bad filter would otherwise take down everything.

2. Routing as data (routing.go)

Routing is a table you can replace atomically, not an if/else you redeploy to change. Router holds an atomic.Pointer[RouteTable]; Swap replaces the whole table in one instruction.

TestHotReloadNoDrops is the important one: four reader goroutines continuously match requests while a fifth swaps the table in a tight loop, and the test asserts zero failed lookups across tens of thousands of matches — under -race. That lock-free swap is exactly how a data-plane node applies a control-plane config push (gw-08) without dropping a request mid-reconfiguration. The read path (1M+ rps) never takes a lock; only the rare write allocates a new table and swaps the pointer.

Specificity, not just longest-prefix

The subtle part is precedence. Match ranks routes the way the Gateway API and Envoy do: longest prefix wins, then among equal prefixes the more-constrained route (a header predicate, then a method) wins. TestLongestPrefixAndHeaderPredicate has two routes with the same prefix /v1/play — a base route and a canary route gated on x-canary: true. Naive "first match of the longest prefix" would always pick the base route and the canary would be dead config. The specificity score (len(prefix)*10 + headerBonus*2 + methodBonus) makes the canary win for canary requests — which is precisely how header-based canary steering (gw-12) is implemented. Getting this ordering right is a real maintainer detail; getting it wrong silently breaks canaries.

3. The endpoint: where the origin call lives (filters.go)

ProxyEndpoint.Apply is the terminal action. Read it for four things you must get right in any proxy:

1. A per-request timeout is the first resilience primitive. context.WithTimeout bounds the origin call; TestProxyTimeoutIs504 proves a slow origin yields 504, not a hung request. Everything in gw-06 (retries, hedging, circuit breaking) is built on top of bounded latency; without the timeout, they're built on sand.
2. Error classification. statusForError maps a deadline to 504 and anything else (connection refused, reset) to 502 (TestProxyDeadOriginIs502). The difference matters to clients and to your SLOs (gw-11).
3. Hop-by-hop hygiene. sanitizeForOrigin strips Connection, Keep-Alive, Transfer-Encoding, Upgrade, etc. (RFC 9110 §7.6.1) and appends X-Forwarded-For. TestProxyForwardsToOrigin asserts the origin never sees the client's Connection header and does see an XFF. Forwarding hop-by-hop headers is a classic correctness/security bug.
4. The Transport seam. ProxyEndpoint.Transport is an http.RoundTripper. Swap in gw-04's pooled+subsetted transport and gw-06's resilient transport here — without touching the gateway engine. That seam is the whole point: cross-cutting concerns compose.

4. The async discipline (the Zuul 2 lesson)

The CONCEPTS file explains why Zuul moved from thread-per-request to Netty's event loop: under slow origins, a thread-per-request pool exhausts and stops serving fast origins too (head-of-line at the pool); an event loop parks a cheap continuation instead. ~25% throughput and CPU win.

In Go you get this for free: a goroutine blocked in RoundTrip parks on the netpoller (gw-01), not an OS thread, so "blocking" code scales. The cost you must respect — and the thing that bites Java/Netty teams — is that on a real event loop you must never block the loop thread: no synchronous DB call, no blocking lock, in a filter. The discipline is "the expediter never picks up a knife." When you read Zuul's HttpInboundSyncFilter vs the async filter types, that distinction (CPU- only inline vs I/O off-loop) is exactly this.

5. Hands-on

cd src/go
bash ../scripts/verify.sh      # vet + build + test -race

go build -o /tmp/gateway ./cmd/gateway
python3 -m http.server 9000 &              # an origin
/tmp/gateway -listen :8080 -route /v1=http://127.0.0.1:9000 &

curl -i localhost:8080/healthz                       # 200 (auth-exempt, local endpoint)
curl -i localhost:8080/v1/thing                      # 401 (no token)
curl -i -H 'Authorization: Bearer t' localhost:8080/v1/  # 200 via origin
curl -i -H 'Authorization: Bearer t' localhost:8080/nope  # 404 (no route)

Watch the access log: every request is logged with status and route — including the 401 and 404 — because outbound always runs.

6. Where the other labs plug in

The gateway engine you built is the frame every other Phase-6 lab hangs on.

7. Exercises

1. Add a rate-limit filter (inbound, order 20, before routing) using a token bucket; short-circuit with 429 when empty. Then make it per-client (keyed by identity). Why is a per-instance bucket actually limit × instances fleet-wide, and when does that matter (gw-06)?
2. Stream the response body instead of buffering it in ProxyEndpoint. What do you gain (latency, bounded memory) and lose (the ability to inspect/transform the body, easy content-length)?
3. Add a header-rewrite outbound filter that strips a sensitive upstream header. Confirm via the access log that it runs after the proxy.
4. Implement a "shadow" filter (gw-12 preview): in the endpoint, additionally fire the request at a second cluster, discard its response, and record a diff — without affecting the user's response.
5. Make routing dynamic from a file: watch a JSON routes file and Swap the table on change. Prove with the hot-reload test pattern that live traffic sees no drops. You've now built the data-plane half of gw-08.