gw-01 — Analysis

A design-review-style look at the L4 proxy you build in steps/, and the trade-offs a senior engineer is expected to articulate.

Required behaviors

  1. Non-blocking acceptance. The acceptor must keep draining the accept queue under load; a slow accept loop manifests as ListenOverflows in nstat and as connection resets at the client.
  2. Coupled backpressure. Memory must stay bounded as throughput rises. If the origin stalls, the proxy must stop reading the client, not buffer without limit.
  3. Symmetric half-close. When one side sends FIN, forward it (CloseWrite on the peer) but keep the other direction open until it too closes. A proxy that tears down both directions on the first FIN breaks request/response protocols that close one way early.
  4. Bounded connection lifetime. Idle timeout, and a hard cap so a half-dead peer can't pin a connection forever (TCP_USER_TIMEOUT / deadlines).
  5. Graceful drain. On SIGTERM: stop accepting, fail readiness, wait for in-flight with a deadline, force-close stragglers, log them.

Design decisions

  • Go goroutines over a hand-rolled epoll loop. Go's netpoller is an epoll/kqueue event loop; "two goroutines per connection" compiles down to event-driven I/O on a small thread pool. We use it because it's idiomatic and lets the lab focus on proxy semantics, then docs/observation.md shows the epoll calls underneath with strace so you see what's really happening. In Java you'd write this on Netty with an explicit EventLoopGroup; the semantics are identical.

  • io.Copy for the data path, deliberately. io.Copy(dst, src) blocks the goroutine on a slow dst.Write, which stops src.Read — that is backpressure, for free, with a bounded internal buffer. The step then shows the failure mode of "fixing" this with an unbounded channel between read and write goroutines (the classic OOM bug).

  • Half-close via CloseWrite. We type-assert to interface{ CloseWrite() error } (satisfied by *net.TCPConn) so a FIN on one side propagates as a FIN on the other without killing the reverse direction. This is the difference between a toy proxy and one that handles real protocols.

  • PROXY protocol v1 prepend. Simple, text, human-readable in tcpdump. v2 (binary) is what you'd use in production for efficiency and for carrying TLVs (e.g., the TLS SNI), but v1 makes the concept legible in a lab.

  • Drain via context + WaitGroup. A context.Context cancels the acceptor; a sync.WaitGroup tracks in-flight connections; a select on ctx.Done() vs wg with a timeout implements wait-with-deadline.

Tradeoffs worth flagging

  • No zero-copy. The Go lab copies through userspace. Production L4 proxies use splice(2) to avoid it; we don't, because the moment you add TLS termination or inspection (gw-02/gw-07) you can't splice anyway, and the lab's next step is exactly that. We call out where splice would go and what it would save (roughly halves syscalls and removes two memory copies per buffer).

  • Per-connection goroutines, not a fixed event-loop pool. At Netflix-Pushy scale (hundreds of thousands of connections per node, gw-05) the per-goroutine stack (~8KB min) adds up; you'd profile and possibly move to a fixed-pool model or tune GOMAXPROCS/stack sizes. For an L4 proxy of moderate fan-in it's fine and far more readable.

  • L4 can't retry. Once you've forwarded bytes to an origin and it dies mid-stream, L4 cannot transparently retry — the application data is already gone and order matters. Retries are an L7 capability (gw-06). Saying this out loud in a design review is a senior signal.

  • Connection-level stickiness. An L4 LB pins a whole connection to one backend. With HTTP/1.1 keep-alive that means many requests ride one backend (uneven load); with HTTP/2 it means all multiplexed streams pin to one backend (worse). The fix is L7 LB or h2-aware L4 (gw-02, gw-06).

What "production-quality" adds beyond this lab

  • splice/sendfile zero-copy for the inspect-free path.
  • SO_REUSEPORT with one acceptor per core to remove accept-lock contention and let the kernel spread accepts.
  • Connection and request rate limiting and concurrency caps to survive a thundering herd (gw-06).
  • conntrack/ephemeral-port budgeting and TIME_WAIT management for the outbound (origin) side.
  • A real health/readiness model integrated with the LB and Kubernetes endpoint lifecycle (gw-09), so drain actually removes you from rotation before you stop accepting.
  • Observability: per-connection bytes, accept-queue depth, active connection gauge, drain duration histogram (gw-11).