gw-01 step 02 — PROXY protocol and socket tuning

Goal

Preserve the real client IP across the proxy with the PROXY protocol v1, and set the socket options that matter for a gateway. After this step the origin can log the true client address instead of the proxy's.

Code — emit PROXY protocol v1 toward the origin

Prepend a single header line before forwarding the first byte:

import (
	"fmt"
	"net"
)

// writeProxyV1 emits the HAProxy PROXY protocol v1 header describing the
// real client and the local (proxy) address, e.g.:
//   PROXY TCP4 203.0.113.7 10.0.0.5 56324 8080\r\n
func writeProxyV1(origin net.Conn, client net.Conn) error {
	src := client.RemoteAddr().(*net.TCPAddr)
	dst := client.LocalAddr().(*net.TCPAddr)

	fam := "TCP4"
	if src.IP.To4() == nil {
		fam = "TCP6"
	}
	header := fmt.Sprintf("PROXY %s %s %s %d %d\r\n",
		fam, src.IP.String(), dst.IP.String(), src.Port, dst.Port)
	_, err := origin.Write([]byte(header))
	return err
}

Call it once, right after dialing the origin and before the copy loop:

	if p.SendProxyHeader {
		if err := writeProxyV1(origin, client); err != nil {
			log.Printf("proxy header: %v", err)
			return
		}
	}

Code — parse PROXY protocol v1 on the receiving side

The origin (or a downstream proxy) reads and strips the header before treating the rest as payload:

import (
	"bufio"
	"errors"
	"net"
	"strconv"
	"strings"
)

type ClientInfo struct {
	SrcIP   net.IP
	SrcPort int
}

// readProxyV1 consumes a PROXY v1 header from r and returns the real
// client address. br must be used for all subsequent reads (it may hold
// buffered payload bytes).
func readProxyV1(br *bufio.Reader) (ClientInfo, error) {
	line, err := br.ReadString('\n')
	if err != nil {
		return ClientInfo{}, err
	}
	line = strings.TrimRight(line, "\r\n")
	f := strings.Split(line, " ")
	// f = ["PROXY","TCP4","203.0.113.7","10.0.0.5","56324","8080"]
	if len(f) != 6 || f[0] != "PROXY" {
		return ClientInfo{}, errors.New("bad PROXY v1 header")
	}
	port, _ := strconv.Atoi(f[4])
	return ClientInfo{SrcIP: net.ParseIP(f[2]), SrcPort: port}, nil
}

Trust boundary (say this in an interview): only parse the PROXY header from peers you trust. If an untrusted client can send it, they can spoof their source IP. In production you allowlist the proxy's IPs as trusted PROXY-protocol senders.

Socket options — what to set and why

SO_REUSEPORT control hook:

import (
	"context"
	"syscall"
	"golang.org/x/sys/unix"
)

func setReusePort(network, address string, c syscall.RawConn) error {
	return c.Control(func(fd uintptr) {
		unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEPORT, 1)
	})
}

// usage:
lc := net.ListenConfig{Control: setReusePort}
ln, err := lc.Listen(context.Background(), "tcp", p.ListenAddr)

Now you can start N copies of the proxy on the same port; the kernel spreads new connections across them.

Tasks

1. Add -proxy-header flag; when set, emit PROXY v1 toward the origin.
2. Write a tiny test origin that calls readProxyV1 and logs the real client IP. Confirm it sees the client's address, not the proxy's.
3. Enable keep-alive and SO_REUSEPORT. Start two proxy instances on :8080; hammer with wrk and confirm both accept connections (ss -tnp | grep 8080).

Acceptance

• Origin logs the genuine client IP/port via the PROXY header.
• A malformed PROXY header is rejected (your parser returns an error, doesn't crash).
• Two SO_REUSEPORT instances share :8080; load is spread across both.

Discussion prompts

• Why is PROXY protocol the L4 answer but X-Forwarded-For the L7 answer to the same problem? What can't you do at L4?
• v2 is binary and carries TLVs (e.g. the negotiated TLS SNI). When would the extra complexity of v2 pay off over v1?
• SO_REUSEPORT spreads new connections, not load. With long-lived connections (gw-05) one instance can still end up hot. Why, and what do you do about it?