Nodes without internet: Bottlerocket and ECR pull-through cache

Why bother

If a node can reach the internet, so can anything that compromises it. In the Secure package, node and pod subnets have no default route at all. That only works if nodes can still boot, join the cluster and pull images.

What makes it work

• VPC endpoints for every AWS API a node needs: ECR, STS, EKS, EC2, SSM, KMS, logs. S3 and DynamoDB through gateway endpoints.
• ECR pull-through cache for the platform's own images: quay, registry.k8s.io, public ECR, GitHub and Docker Hub are mirrored into the account's ECR on first pull, scanned continuously, and expired when unused. The CNI and Flux images come from there.
• The egress gateway as the node's proxy for anything else. Bottlerocket's https-proxy setting sends containerd's pulls through the gateway by a fixed ClusterIP, so node pulls obey the same allowlist as everything else.

[settings.host-containers.admin]
enabled = false
[settings.kernel]
lockdown = "integrity"
[settings.network]
https-proxy = "http://172.20.0.50:3128"
no-proxy = [".svc", ".cluster.local", "169.254.169.254", ".amazonaws.com"]

Bottlerocket, hardened

A minimal OS with an immutable root filesystem. The admin and control host containers are off: access is through the cluster, not the host. Kernel lockdown is on, IMDSv2 is required with a hop limit of one, so a pod cannot reach the node's instance credentials, and every volume is encrypted with the cell's key.