gw-02 step 02 — HPACK decoding and flow-control accounting

Goal

Decode HPACK-compressed headers using the static table, then track HTTP/2 flow-control windows so you can explain (and debug) stalls. These are the two stateful mechanisms that trip up gateway engineers.

Part A — HPACK static-table decoding

HPACK encodes each header field as one of: an indexed field (a single index into the static+dynamic table), or a literal field (an index for the name + a string for the value, with optional Huffman coding). The 61-entry static table starts:

 1  :authority
 2  :method GET
 3  :method POST
 4  :path /
 5  :path /index.html
 6  :scheme http
 7  :scheme https
 8  :status 200
 ...
61  www-authenticate

Use the stdlib decoder rather than reimplementing Huffman — the point is to see the table indices, not to rebuild the codec:

package h2

import "golang.org/x/net/http2/hpack"

// DecodeHeaders decodes a HEADERS frame's HPACK block into key/value
// pairs. dec must persist across frames on the same connection so the
// dynamic table stays in sync (this is the subtle part).
func DecodeHeaders(dec *hpack.Decoder, block []byte) ([]hpack.HeaderField, error) {
	var out []hpack.HeaderField
	dec.SetEmitFunc(func(hf hpack.HeaderField) { out = append(out, hf) })
	if _, err := dec.Write(block); err != nil {
		return nil, err
	}
	return out, dec.Close()
}

// NewDecoder makes a decoder with a bounded dynamic table — bounding it
// is your defense against an HPACK-table-bloat DoS.
func NewDecoder(maxTable uint32) *hpack.Decoder {
	return hpack.NewDecoder(maxTable, nil)
}

The proxy gotcha: if your gateway rewrites a header (adds x-forwarded-for, strips a hop-by-hop header), it must re-encode with its own encoder whose dynamic table tracks what it actually sent to the origin — not blindly forward the client's HPACK block. Mixing the two desynchronizes the table and corrupts every subsequent header.

Part B — flow-control accounting

Model the two-level window. A sender may transmit DATA only while both the stream window and the connection window are positive; each DATA byte debits both; WINDOW_UPDATE credits one of them.

package h2

import "errors"

type FlowController struct {
	conn    int64            // connection-level window (bytes we may still send)
	streams map[uint32]int64 // per-stream windows
}

const defaultInitialWindow = 65535 // SETTINGS_INITIAL_WINDOW_SIZE default

func NewFlowController() *FlowController {
	return &FlowController{conn: defaultInitialWindow, streams: map[uint32]int64{}}
}

func (fc *FlowController) ensure(stream uint32) {
	if _, ok := fc.streams[stream]; !ok {
		fc.streams[stream] = defaultInitialWindow
	}
}

// CanSend reports whether n DATA bytes are permitted right now.
func (fc *FlowController) CanSend(stream uint32, n int64) bool {
	fc.ensure(stream)
	return fc.conn >= n && fc.streams[stream] >= n
}

// Sent debits both windows after sending n DATA bytes.
func (fc *FlowController) Sent(stream uint32, n int64) error {
	if !fc.CanSend(stream, n) {
		return errors.New("flow control violation: window exhausted")
	}
	fc.conn -= n
	fc.streams[stream] -= n
	return nil
}

// WindowUpdate credits a window (stream==0 means connection-level).
func (fc *FlowController) WindowUpdate(stream uint32, inc int64) {
	if stream == 0 {
		fc.conn += inc
		return
	}
	fc.ensure(stream)
	fc.streams[stream] += inc
}

Tasks

  1. Decode the HEADERS frames from your step-01 capture; print each as :method GET :path /foo. Confirm a repeated header (e.g. a cookie) shows up as a short indexed reference on its second occurrence.
  2. Drive the FlowController with the DATA and WINDOW_UPDATE frames from a real download. Log the connection window over time; watch it drain toward zero and recover on each WINDOW_UPDATE.
  3. Reproduce a stall: stop emitting WINDOW_UPDATE (simulate a slow consumer) and show CanSend returns false while data is pending — the application-level backpressure that looks like a network hang.

Acceptance

  • Decoded headers match nghttp -v's header dump for the same request.
  • Your flow-control log shows windows draining and recovering, and you can produce a deliberate stall and explain it as window exhaustion, not packet loss.

Discussion prompts

  • Why is a too-small SETTINGS_INITIAL_WINDOW_SIZE a throughput bug on a high-latency (high bandwidth-delay-product) link? (Hint: the window caps in-flight bytes ≈ throughput × RTT.)
  • Why must the HPACK decoder persist for the life of the connection, and what breaks if you create a fresh one per frame?
  • How is h2 flow control different from, and layered on top of, TCP's own receive-window flow control?