soft guideINFRASTRUCTURE & ADVISORY

Note 04

North-south, east-west, egress: three directions, three tools

Notes from choosing a Kubernetes networking stack: the strongest component for each traffic direction instead of one product that does all three adequately.

In the platform
Architecture review · part of every package
Experience behind it
Research and architecture notes, verified against a running cluster.
NORTH-SOUTHEAST-WESTEGRESSAWS ALBdata plane in AWSLB controllerGateway API → ALBWorkloadsCilium eBPFL3/L4 in the kernelNode-local Envoyonly where L7 is neededPolicy by identitynot by IPEnvoy egress gatewayallowlist · ext_authzCilium FQDN policysecond fenceNATfixed source IPs

North-south

Gateway API as the interface, the AWS Load Balancer Controller as the implementation. The data plane stays in the AWS load balancer, so no extra proxy runs in the cluster just to receive traffic. The trade-off: target registration is an external control-plane operation and reacts more slowly than an in-cluster proxy, so test it against your availability targets.

East-west

Cilium handles L3 and L4 in the kernel with eBPF, and policy is by workload identity, not IP. Where L7 is genuinely needed (gRPC routing, retries, circuit breaking, L7 policy), traffic is redirected to a node-local Envoy, not a sidecar in every pod. The Gateway API mesh profile describes those routes in a standard API; it standardises routing, not mutual TLS, which is a separate decision.

Egress

The requirement is stronger than a stable source IP: a central point that reads the request and asks business logic before letting it out. A dedicated Envoy gateway does that, with external authorization served over a Unix socket inside the same pod, so the decision costs no network hop. Cilium's egress gateway is a different tool: L3 and L4 source-address control, useful underneath, not a replacement.

Mental models

eBPF is not L7, and the kernel cannot parse HTTP. Transparency is ergonomics; latency is hops. Gateway API is a specification, not a product. Priority classes order scheduling; they do not make one pod wait for another.

Start a conversation

Bring your infrastructure
into focus.

Tell me what you are building, operating, or trying to improve.

Let’s talk