Note 04
North-south, east-west, egress: three directions, three tools
Notes from choosing a Kubernetes networking stack: the strongest component for each traffic direction instead of one product that does all three adequately.
- Kubernetes
- Cilium
- Envoy
- AWS
North-south
Gateway API as the interface, the AWS Load Balancer Controller as the implementation. The data plane stays in the AWS load balancer, so no extra proxy runs in the cluster just to receive traffic. The trade-off: target registration is an external control-plane operation and reacts more slowly than an in-cluster proxy, so test it against your availability targets.
East-west
Cilium handles L3 and L4 in the kernel with eBPF, and policy is by workload identity, not IP. Where L7 is genuinely needed (gRPC routing, retries, circuit breaking, L7 policy), traffic is redirected to a node-local Envoy, not a sidecar in every pod. The Gateway API mesh profile describes those routes in a standard API; it standardises routing, not mutual TLS, which is a separate decision.
Egress
The requirement is stronger than a stable source IP: a central point that reads the request and asks business logic before letting it out. A dedicated Envoy gateway does that, with external authorization served over a Unix socket inside the same pod, so the decision costs no network hop. Cilium's egress gateway is a different tool: L3 and L4 source-address control, useful underneath, not a replacement.
Mental models
eBPF is not L7, and the kernel cannot parse HTTP. Transparency is ergonomics; latency is hops. Gateway API is a specification, not a product. Priority classes order scheduling; they do not make one pod wait for another.
Start a conversation
Bring your infrastructure
into focus.
Tell me what you are building, operating, or trying to improve.