gw-04 step 02 — Per-event-loop pools

Goal

Eliminate the single global pool lock and the cross-thread handoff by giving each event loop its own pool. A request handled on loop i only ever uses connections owned by loop i — lock-free, cache-local, the entire request/response on one thread. This is the specific change the Netflix talk highlights.

Code — src/go/loop_pools.go

package pool

import (
	"net"
	"time"
)

// LoopPools is a set of K independent pools, one per event loop. There
// is NO shared lock across loops: pool[i] is only ever touched by the
// goroutine(s) bound to loop i.
type LoopPools struct {
	pools []*Pool
}

func NewLoopPools(k int, dial Dialer, maxIdle int, idleTTL time.Duration) *LoopPools {
	lp := &LoopPools{pools: make([]*Pool, k)}
	for i := range lp.pools {
		lp.pools[i] = New(dial, maxIdle, idleTTL)
	}
	return lp
}

// For returns the pool owned by event loop `loopID`. The caller is the
// loop goroutine, so within one loop the *Pool's* lock is effectively
// uncontended (only this loop uses it).
func (lp *LoopPools) For(loopID int) *Pool { return lp.pools[loopID] }

// Totals aggregates churn across loops for the metric.
func (lp *LoopPools) Totals() (created, reused int64) {
	for _, p := range lp.pools {
		created += p.Created.Load()
		reused += p.Reused.Load()
	}
	return
}

Wiring — one worker goroutine per loop, pinned to its pool

This models Netty's EventLoopGroup: K workers, each draining its own request queue, each using only its own pool.

func runWorker(loopID int, lp *LoopPools, reqs <-chan Request) {
	p := lp.For(loopID) // this worker's pool — no other worker touches it
	for r := range reqs {
		c, err := p.Get(r.Origin)
		if err != nil {
			r.Done(err)
			continue
		}
		// ... write request / read response on c (same thread) ...
		p.Put(c) // returned to THIS loop's pool
		r.Done(nil)
	}
}

func main() {
	k := runtime.GOMAXPROCS(0)
	lp := NewLoopPools(k, dialTLS, 32, 30*time.Second)
	queues := make([]chan Request, k)
	for i := 0; i < k; i++ {
		queues[i] = make(chan Request, 1024)
		go runWorker(i, lp, queues[i])
	}
	// Accept connections and assign each to a loop by a stable hash so a
	// connection's requests always land on the same loop (affinity).
	acceptLoop(func(conn net.Conn) {
		loopID := stableHash(conn.RemoteAddr()) % k
		// ... read requests from conn, push to queues[loopID] ...
	})
}

The trade-off to measure

Per-loop pools mean each loop warms its own connections, so the minimum connection count rises ~K× versus a shared pool. Measure it:

shared pool:    min connections ≈ peak_concurrency_per_origin
per-loop pools: min connections ≈ K × peak_concurrency_per_loop_per_origin

This is exactly why subsetting (step 03) is needed to keep the total in check. The win is that the contention and handoff costs vanish, which matters far more at 1M+ rps than a modest rise in idle connections — and subsetting claws the count back.

Tasks

1. Convert step 01's single pool into LoopPools with K = GOMAXPROCS workers, each pinned to its own pool.
2. Under fixed load, compare lock contention: profile the shared-pool version (go test -bench with -mutexprofile, or pprof) vs the per-loop version. Show the mutex contention on the shared pool and its absence per-loop.
3. Record the connection-count trade-off: per-loop pools hold more idle connections. Note the number — step 03 will reduce it with subsetting.

Acceptance

• Per-loop version shows ~zero cross-loop mutex contention in the profile; shared version shows contention rising with K.
• You can state the connection-count trade-off with real numbers and explain why subsetting is the companion fix.

Discussion prompts

• Why is connection affinity (a connection's requests always hitting the same loop) important here? What breaks if a request can hop loops mid-flight?
• In Java/Netty this is automatic because a Channel is bound to one EventLoop for its lifetime. How does that compare to the goroutine-per-connection Go model, and where does each pay a cost?
• The per-loop trade-off raises idle connections by ~K×. Argue why that's still the right call at Netflix scale before subsetting.