gw-01 — The L4 Data Plane: TCP/UDP, Sockets, and a TCP Proxy

This lab is the floor of the gateway stack. Before HTTP, before TLS, before any "gateway" exists, there is a kernel socket, an accept queue, two file descriptors, and the job of shoveling bytes from one to the other without copying them more than you must, without letting a fast sender overrun a slow receiver, and without leaking a single connection when you redeploy. Every higher lab — the HTTP/2 demux in gw-02, the filter chain in gw-03, the connection pool in gw-04, the WebSocket fleet in gw-05 — sits on top of the primitives here.

The JD asks for "deep expertise in L4 (TCP/UDP)." That phrase means exactly the contents of this lab: you can explain the TCP state machine, the difference between an accept queue and a SYN queue, what TCP_NODELAY actually disables, why SO_REUSEPORT matters for a multi-core proxy, how zero-copy splice works, and how you drain a node without dropping in-flight requests.

You will build a non-blocking L4 TCP proxy in Go: an event-loop acceptor, bidirectional copy with backpressure, connection draining on shutdown, and the PROXY protocol so the origin learns the real client IP. You will load-test it and read the kernel counters that tell you whether it's healthy.

1. What is it?

A Layer-4 (transport) data plane moves bytes between a client connection and an origin connection without understanding the application protocol riding on top. It operates on the transport header — TCP ports and sequence numbers, or UDP datagrams — not on URLs or HTTP methods. An L4 load balancer/proxy picks a backend per connection (or per UDP flow) and then forwards the opaque byte stream.

Contrast with L7 (gw-02/gw-03), which terminates the application protocol, can route per request, can retry, and can rewrite. L4 is dumber, faster, and protocol-agnostic; L7 is smarter, slower, and protocol-specific. Real edges use both: an L4 layer for raw throughput and DDoS absorption, an L7 layer behind it for routing and policy.

The OSI/TCP layering you must be fluent in:

 L7  application   HTTP, gRPC, WebSocket, DNS        ← gw-02, gw-03, gw-05
 L4  transport     TCP (streams), UDP (datagrams)    ← THIS LAB
 L3  network       IP, routing, anycast              ← gw-09 (CNI)
 L2  link          Ethernet, ARP, veth pairs         ← gw-09

TCP in one diagram

TCP is a reliable, ordered, byte-stream protocol built on top of unreliable IP packets. The three-way handshake and the connection state machine are interview table stakes:

client                         server
  │   SYN (seq=x)                 │      CLOSED → LISTEN (server bind+listen)
  │ ────────────────────────────▶│      ── arrives in SYN queue ──
  │   SYN-ACK (seq=y, ack=x+1)    │
  │ ◀────────────────────────────│
  │   ACK (ack=y+1)               │      ── moves to ACCEPT queue ──
  │ ────────────────────────────▶│      accept() returns a new fd
  │           ESTABLISHED         │
  │ ◀══════ data both ways ══════▶│
  │   FIN ─▶  ... ◀─ ACK/FIN      │      active close → TIME_WAIT (2·MSL)

Two kernel queues sit behind one listen():

• SYN queue (incomplete): half-open connections awaiting the final ACK. Sized by net.ipv4.tcp_max_syn_backlog. SYN floods fill this; SYN cookies are the defense.
• Accept queue (completed): fully established connections waiting for your process to accept(). Sized by min(backlog, net.core.somaxconn). If your accept loop is too slow, this overflows and the kernel silently drops or resets connections — one of the most common "mysterious latency" causes at a busy gateway.

UDP — why a gateway still cares

UDP is connectionless datagrams: no handshake, no ordering, no retransmit. It matters at the edge because QUIC/HTTP3 rides on UDP (gw-02), DNS is UDP, and L4 LBs must hash UDP flows ((src ip, src port, dst ip, dst port)) to a stable backend without per-packet state. The hard part of UDP load balancing is the absence of a connection: you maintain a flow table with timeouts instead of relying on FIN.

2. Why does it matter?

• It's where p99 latency and CPU are won or lost. A gateway is an I/O machine. The difference between a thread-per-connection design (one OS thread blocked per socket; collapses past ~10k connections — the C10K problem) and an event-loop design (one thread multiplexing thousands of sockets via epoll/kqueue) is the difference between Zuul 1 and Zuul 2. The team's "Evolution of Edge" talk is, at its core, this transition.

• Connection efficiency is a top-line metric here. The connection-churn work (gw-04) is entirely about L4 behavior: keep-alive, pooling, and not paying for a TCP+TLS handshake on every request. You cannot reason about gw-04 without the socket-level model in this lab.

• Backpressure is a correctness property, not a nice-to-have. If you read from a fast client faster than a slow origin can accept, you buffer without bound and OOM the proxy. A correct L4 proxy couples the two directions: stop reading one side when the other side's write buffer is full. This is the single most common bug in homegrown proxies.

• Graceful drain is the operational core of the role. Every deploy, every scale-down, every node replacement requires draining in-flight connections without dropping requests. For a stateless L7 node that's seconds; for a stateful WebSocket node (gw-05) it's a careful dance. It starts here.

3. How does it work?

The event-loop acceptor (the C10K answer)

listen_fd = socket(); bind(); listen(backlog)
register listen_fd with epoll/kqueue for READ

loop forever:
    events = epoll_wait()            # blocks until a fd is ready
    for ev in events:
        if ev.fd == listen_fd:
            conn_fd = accept4(listen_fd, NONBLOCK)   # drain the accept queue
            origin_fd = dial(pick_backend())
            register both fds for READ
        else:
            pump(ev.fd)              # non-blocking read → write to peer

One thread, thousands of connections. Go hides the epoll loop behind goroutines + the netpoller, so the idiomatic Go proxy is "two goroutines per connection" but the runtime is doing exactly the event-loop above. Java/Netty exposes it directly as the EventLoopGroup. Know both framings.

Bidirectional copy with backpressure

The heart of an L4 proxy is two coupled copies:

client ──read──▶ [proxy] ──write──▶ origin     (upstream direction)
client ◀─write── [proxy] ◀──read── origin       (downstream direction)

The critical rule: a write that blocks must stop the corresponding read. In epoll terms, when write() returns EAGAIN you deregister READ on the source until the destination signals WRITE-ready. In Go, a blocking Write on the destination naturally throttles the Read on the source because they're in the same goroutine — io.Copy gives you backpressure for free, which is why the lab uses it and then shows you what it's hiding.

Zero-copy: don't touch the bytes

A naive proxy copies each byte twice across the userspace boundary: read() (kernel→user) then write() (user→kernel). For an L4 proxy that never inspects the payload, that's pure waste. Linux splice(2) moves bytes between two fds through a kernel pipe without copying to userspace; sendfile(2) does file→socket. This is how high-end L4 proxies hit line rate. (You can't splice once you need to inspect/TLS- terminate — that's the L4/L7 cost tradeoff in one syscall.)

naive:    socket ─read→ user buffer ─write→ socket   (2 copies, 2 syscalls/buf)
splice:   socket ════════ kernel pipe ════════ socket (0 userspace copies)

The socket options that matter

PROXY protocol — preserving the client IP

When you put a proxy in front of an origin, the origin's accept() sees the proxy's IP, not the client's. For L7 you'd add X-Forwarded-For; for L4 (no HTTP to add a header to) the standard is the PROXY protocol: a small header prepended to the byte stream before the real payload, carrying the original (src ip:port, dst ip:port). v1 is text (PROXY TCP4 1.2.3.4 5.6.7.8 56324 443\r\n); v2 is binary. Your origin (or the next proxy) parses and strips it.

Connection draining on shutdown

on SIGTERM:
    stop accepting new connections (close listen_fd)
    flip readiness probe to "not ready"     # LB stops sending new conns
    wait for in-flight connections to close, up to drain_deadline
    after deadline: force-close the stragglers, log them
    exit

The readiness-probe flip is what makes this work in Kubernetes: the endpoint is removed from the Service before the pod dies (gw-09). Get the ordering wrong — exit before the LB notices — and you drop requests on every deploy.

4. Core terminology

5. Mental models

• A proxy is a bucket brigade, not a warehouse. Bytes should flow through, not pile up. If your memory grows with throughput, you've lost backpressure and turned a brigade into a warehouse that will eventually catch fire (OOM).

• The accept queue is a checkout line. SYN queue = people walking toward the register; accept queue = people standing in line with full carts; accept() = the cashier. A slow cashier (slow accept loop) doesn't make the line longer forever — past somaxconn the store locks the doors (drops connections) and customers leave (RST/retransmit).

• TIME_WAIT is the receipt you must keep. The active closer holds TIME_WAIT for 2·MSL so a delayed duplicate segment from the old connection can't be misread by a new connection reusing the same 4-tuple. It's correct; it just means the side that closes pays the cost — a reason gateways prefer the client to close, or use keep-alive to avoid closing at all (gw-04).

• Event loop vs threads is "one chef, many pots" vs "one chef per pot." With thousands of pots (connections), hiring a chef per pot (thread-per-connection) bankrupts you on context switches and stack memory. One chef watching all the pots and stirring whichever is ready (epoll) scales. This is the entire Zuul 1 → Zuul 2 thesis.

6. Common misconceptions

• "A bigger listen() backlog fixes accept-queue drops." Only if your accept loop can drain it. A huge backlog with a slow loop just delays the drop and adds latency. Fix the loop (or add acceptors via SO_REUSEPORT), then size the queue.

• "TCP_NODELAY makes things faster." It reduces latency for small messages by disabling write coalescing — at the cost of more packets. For a request/response gateway it's almost always right; for bulk transfer it can hurt. It's a latency/throughput knob, not a "go faster" button.

• "Load balancing at L4 and L7 is the same, just different layers." No: L4 balances connections/flows (sticky for the connection's life; one slow request blocks the connection); L7 balances requests (can spread requests from one connection across backends, retry, and hedge). The mismatch is exactly why HTTP/2 multiplexing complicates L4 LBs (gw-02).

• "TLS termination is free if I have spare CPU." TLS handshakes are the expensive part (asymmetric crypto), and they happen on every new connection. This is why connection churn (gw-04) shows up as a CPU problem, and why keep-alive/pooling is a CPU optimization, not just a latency one.

• "Draining is just sleep(30) before exit." Sleeping doesn't stop the LB from sending new connections, and it doesn't bound stragglers. Correct drain is: stop accepting → fail readiness → wait-with-deadline → force-close + log.

7. Interview talking points

• "Walk me through what happens from listen() to accept()." Hit SYN queue vs accept queue, somaxconn, SYN cookies, and accept-queue overflow as a real latency cause. Mention ss -lnt shows Recv-Q (current accept queue depth) and Send-Q (its max) on a listening socket — naming the diagnostic command signals you've operated this.

• "Why did Zuul move from thread-per-request to Netty/event-loop?" C10K: threads cost stack memory (~1MB each) and context switches; past tens of thousands of connections the scheduler dominates. An event loop multiplexes thousands of connections per thread via epoll. Cost: you must never block the event-loop thread (no synchronous I/O, no blocking locks) — the discipline that makes async code hard. ~25% throughput/CPU win at Netflix.

• "How do you keep a fast client from OOMing your proxy?" Backpressure: couple the two copy directions so a stalled write pauses the corresponding read. Explain it in epoll terms (drop READ interest on EAGAIN) and Go terms (io.Copy blocks the read goroutine). Bounded buffers, not unbounded queues.

• "What's TCP_NODELAY and when would you NOT set it?" Disables Nagle. Set it for latency-sensitive small messages (almost all gateway traffic). The Nagle + delayed-ACK interaction causes ~40ms stalls — a classic war story. You might leave it off for a pure bulk-data path where packet efficiency beats latency.

• "How do you preserve the client IP through an L4 proxy?" PROXY protocol (v2 binary) at L4; X-Forwarded-For / Forwarded at L7. Note the trust boundary: only parse it from peers you trust, or a client can spoof their source IP.

• "How do you drain a node for a deploy without dropping requests?" Stop accept → fail readiness probe → LB removes endpoint → wait for in-flight with a deadline → force-close stragglers. Tie it to Kubernetes preStop hook + terminationGracePeriodSeconds (gw-09).

• "TIME_WAIT is piling up — what is it and do you care?" It's correct behavior on the active closer (prevents 4-tuple reuse hazards for 2·MSL). You care when ephemeral ports or conntrack exhaust. Fixes: keep-alive (don't close), let the client close, tune tcp_tw_reuse for outbound, widen the ephemeral range. Don't reach for the dangerous tcp_tw_recycle (removed for good reason).

8. Connections to other labs

• db-01 (storage primitives) gave you the syscall-level model — pread/pwrite, the page cache, alignment. Here the same rigor applies to sockets: read/write/splice, epoll, kernel buffers.
• gw-02 (L7 protocols) terminates the byte stream this lab forwards blindly; that's where you stop being able to splice and start paying to parse.
• gw-03 (API gateway) wraps this acceptor + copy loop in a filter chain. The Zuul event-loop model is this lab's acceptor.
• gw-04 (connection management) is the L4 optimization layer: don't re-handshake, pool and reuse the connections this lab opens.
• gw-05 (WebSockets/Pushy) is this lab's draining problem at its hardest: millions of long-lived connections that can't be dropped.
• gw-09 (Kubernetes networking) is where these packets actually flow — veth pairs, CNI, kube-proxy — and where readiness probes gate the drain.