gw-03 step 03 — The async proxy endpoint, timeouts, and error handling

Goal

Implement the endpoint filter that actually proxies to the origin — without blocking the event loop — with a per-request timeout, clean error handling, and the seam where gw-04 (pooling) and gw-06 (resilience) will plug in.

Code — the proxy endpoint filter

In Go, the netpoller makes a goroutine "block" cheaply (no OS thread is parked), so the idiomatic version uses context deadlines. The comments mark where the Java/Netty version would use a CompletableFuture.

package gw

import (
	"context"
	"io"
	"net"
	"net/http"
	"time"
)

// ProxyEndpoint forwards the request to the cluster chosen by routing.
// The Transport here is where gw-04's pooled, subsetted connection
// manager and gw-06's resilience policy get injected.
type ProxyEndpoint struct {
	Resolve   func(cluster string) (origin string, ok bool) // service discovery (gw-08)
	Transport http.RoundTripper                              // pooled in gw-04
	Timeout   time.Duration
}

func (ProxyEndpoint) Type() Phase { return Endpoint }
func (ProxyEndpoint) Order() int  { return 0 }
func (p ProxyEndpoint) ShouldFilter(c *RequestContext) bool {
	return c.RouteName != "" && c.RouteName != "edge-fallback"
}

func (p ProxyEndpoint) Apply(c *RequestContext) {
	origin, ok := p.Resolve(c.RouteName)
	if !ok {
		c.Resp.Status = http.StatusServiceUnavailable
		c.Resp.Body = []byte("no healthy origin")
		return
	}

	// Per-request deadline: never let a slow origin pin a request
	// forever. This is the single most important resilience primitive.
	ctx, cancel := context.WithTimeout(c.Req.Context(), p.Timeout)
	defer cancel()

	out := c.Req.Clone(ctx)
	out.URL.Scheme = "http"
	out.URL.Host = origin
	out.Host = origin
	out.RequestURI = "" // required by the client
	sanitizeForOrigin(c) // strip hop-by-hop, set X-Forwarded-For (step 02)

	// RoundTrip "blocks" this goroutine but NOT an OS thread — Go's
	// netpoller is an epoll loop. In Netty you'd return a Future and
	// resume the outbound chain in its callback; same semantics.
	resp, err := p.Transport.RoundTrip(out)
	if err != nil {
		c.Resp.Status = statusForError(err) // 504 on timeout, 502 otherwise
		c.Resp.Body = []byte("origin error: " + err.Error())
		c.Attributes["origin_error"] = err
		return
	}
	defer resp.Body.Close()

	c.Resp.Status = resp.StatusCode
	for k, vs := range resp.Header {
		for _, v := range vs {
			c.Resp.Header.Add(k, v)
		}
	}
	// For a lab we buffer; production streams the body with backpressure
	// (gw-01) to bound memory and preserve latency.
	body, _ := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
	c.Resp.Body = body
}

func statusForError(err error) int {
	var ne net.Error
	if errAs(err, &ne) && ne.Timeout() {
		return http.StatusGatewayTimeout // 504
	}
	return http.StatusBadGateway // 502
}

Code — the error filter (runs as the first outbound filter)

// ErrorNormalizer ensures every response is well-formed even after a
// filter panic or an unset status, and records the outcome class for
// metrics (gw-11).
type ErrorNormalizer struct{}

func (ErrorNormalizer) Type() Phase  { return Outbound }
func (ErrorNormalizer) Order() int   { return 1 } // earliest outbound
func (ErrorNormalizer) ShouldFilter(c *RequestContext) bool { return true }
func (ErrorNormalizer) Apply(c *RequestContext) {
	if c.Resp.Status == 0 {
		c.Resp.Status = http.StatusInternalServerError
	}
	class := c.Resp.Status / 100
	c.Attributes["status_class"] = class // 2/3/4/5 → RED metrics in gw-11
}

Tasks

1. Implement ProxyEndpoint with a default http.Transport (gw-04 replaces it with a pooled, subsetted one). Stand up a slow origin (sleep handler) and confirm a 200ms Timeout yields a 504, not a hang.
2. Kill the origin mid-flight; confirm a 502 and that the access log (step 01) still fires with the error recorded.
3. Load-test with wrk2 -R5000 through the full chain (auth → route → proxy → log). Record p50/p99. Then make 10% of origin responses slow and show how a per-request timeout bounds the tail (the motivation for gw-06 hedging/retries).

Acceptance

• Slow origin → 504 at the configured deadline; dead origin → 502; both logged with status class for metrics.
• A filter panic anywhere still produces a clean 5xx and an access log (no dropped connections).
• You can point to the Transport and Timeout seams and say exactly what gw-04 and gw-06 will inject there.

Discussion prompts

• Why is a per-request timeout the first resilience primitive, before retries or circuit breakers? (Without it, everything else is built on unbounded latency.)
• In Java/Netty, what specifically must ProxyEndpoint avoid doing on the event-loop thread, and how would you offload it if a filter truly needed blocking work?
• This step buffers the response body. When must you stream it instead, and what does streaming cost you in observability and transformation ability?