gw-04 step 01 — A connection pool, and measuring churn

Goal

Build a minimal origin connection pool with keep-alive, then measure the churn drop versus a no-pool baseline. The metric is the lesson: you want to see connections.created.rate collapse.

Code — src/go/pool.go

package pool

import (
	"net"
	"sync"
	"sync/atomic"
	"time"
)

// Dialer opens a new connection to an origin (TCP+TLS happens here —
// the expensive part we want to avoid repeating).
type Dialer func(origin string) (net.Conn, error)

type Pool struct {
	dial    Dialer
	maxIdle int
	idleTTL time.Duration

	mu   sync.Mutex
	idle map[string][]*pooledConn // origin -> idle connections

	Created atomic.Int64 // THE churn metric: total connections opened
	Reused  atomic.Int64
}

type pooledConn struct {
	net.Conn
	origin   string
	idleSince time.Time
}

func New(dial Dialer, maxIdle int, idleTTL time.Duration) *Pool {
	return &Pool{dial: dial, maxIdle: maxIdle, idleTTL: idleTTL,
		idle: map[string][]*pooledConn{}}
}

// Get returns a warm connection if one is available, else dials a new
// one (and counts it as churn).
func (p *Pool) Get(origin string) (*pooledConn, error) {
	p.mu.Lock()
	q := p.idle[origin]
	for len(q) > 0 {
		c := q[len(q)-1]
		q = q[:len(q)-1]
		if time.Since(c.idleSince) > p.idleTTL { // expired: close, keep looking
			c.Conn.Close()
			continue
		}
		p.idle[origin] = q
		p.mu.Unlock()
		p.Reused.Add(1)
		return c, nil // REUSE: no handshake
	}
	p.idle[origin] = q
	p.mu.Unlock()

	conn, err := p.dial(origin) // CHURN: a new TCP+TLS handshake
	if err != nil {
		return nil, err
	}
	p.Created.Add(1)
	return &pooledConn{Conn: conn, origin: origin}, nil
}

// Put returns a connection to the pool for reuse (or closes it if full).
func (p *Pool) Put(c *pooledConn) {
	c.idleSince = time.Now()
	p.mu.Lock()
	defer p.mu.Unlock()
	q := p.idle[c.origin]
	if len(q) >= p.maxIdle {
		c.Conn.Close() // pool full: this becomes churn next time
		return
	}
	p.idle[c.origin] = append(q, c)
}

In real Go you'd often just configure http.Transport (MaxIdleConnsPerHost, IdleConnTimeout) — and the step shows that too. The hand-rolled pool exists so the Created counter is explicit and you can watch churn.

Measure it

// Churn sampler: print connections-created-per-second.
func sample(p *Pool, stop <-chan struct{}) {
	var last int64
	t := time.NewTicker(time.Second)
	for {
		select {
		case <-stop:
			return
		case <-t.C:
			now := p.Created.Load()
			rate := now - last
			last = now
			reuse := p.Reused.Load()
			fmt.Printf("connections.created/s=%d  total_created=%d  reused=%d\n",
				rate, now, reuse)
		}
	}
}

Run two experiments against a local origin under fixed load (wrk2 -R20000):

1. No pool (Put always closes / maxIdle=0): created/s tracks request rate — thousands per second.
2. With pool (maxIdle=64, sane idleTTL): created/s falls to a trickle after warmup; reused climbs to ≈ request count.

Tasks

1. Implement Pool; wire it as the Transport behind gw-03's endpoint filter (or a standalone loop that Get/Puts per request).
2. Plot connections.created/s for maxIdle=0 vs maxIdle=64 under identical load. Capture the drop (this is the gw-04 result in miniature).
3. Tune idleTTL too low (e.g. 100ms) and show churn comes back — proving over-eager eviction fights pooling.

Acceptance

• created/s collapses from ≈request-rate (no pool) to near-zero (pooled), and reused ≈ total requests.
• You can produce a churn resurgence by setting idleTTL too low and explain why.

Discussion prompts

• Why is churn a CPU problem and not only a latency problem? (TLS asymmetric crypto per handshake.)
• What's the right maxIdle? (≈ peak concurrent requests to that origin on this instance — tie to Little's law, gw-06.)
• This pool has one global lock. On a multi-loop gateway at 1M+ rps, why is that lock a problem, and what does step 02 do about it?