Pod & Service networking

Kubernetes makes three promises: every pod gets its own IP, any pod can reach any other pod without NAT, and nodes can reach every pod. It doesn't implement them — the CNI plugin does (routes, veths, tunnels). Services then layer a virtual IP on top, translated on every node by kube-proxy (or eBPF). Step through a packet below and watch exactly which addresses get rewritten, and where.

Packet walk · two-node Talos cluster with Flannel

changed fields flash

step 1/7

web-1 talks to api-2 on another node

Same as before from the pod's point of view: destination 10.244.2.7, default route to the node.

IP packet

src

10.244.1.5

dst

10.244.2.7

proto

TCP :8080

Pod → pod: never NAT

Same node or cross node, overlay or routed: the source pod IP arrives intact. That's what makes pod IPs usable as identities for NetworkPolicy.

Pod → Service: DNAT on the client node

A ClusterIP is a rule, not a host. The client's node picks an endpoint and rewrites the destination; conntrack reverses it on the reply.

External → NodePort: maybe SNAT

With externalTrafficPolicy: Cluster the node may forward to another node and must masquerade, hiding the client IP. Local keeps it.

CIDRs

Where the IPs come from

Talos defaults to 10.244.0.0/16 for pods and 10.96.0.0/12 for Services. kube-controller-manager carves the pod range into per-node blocks (a /24 by default → 256 nodes × 254 IPs). Drag the node count past the limit, or shrink the block size, and see what breaks. These ranges must not overlap your LAN or VPNs — and they're effectively permanent.

podCIDR allocation

nodes6node-cidr-mask-size

podSubnets 10.244.0.0/16 → 256 × /24hover a block

pod IPs per node

254

max nodes

256

allocated

6 / 256

service IPs (/12)

1,048,576

Why per-node blocks?

Each node gets its own slice (node.spec.podCIDR), so it can assign pod IPs locally with no cluster-wide coordination, and every other node needs exactly one route per node, not per pod.

# serviceSubnets 10.96.0.0/12 (virtual — no node owns these)

10.96.0.1 → kubernetes (API server)

10.96.0.10 → kube-dns (CoreDNS)

10.96.120.15 → api (allocated on create)

machine config patch · cluster.network

# Talos machine config (cluster section) — these are the defaults
cluster:
  network:
    dnsDomain: cluster.local
    podSubnets:
      - 10.244.0.0/16      # split into per-node podCIDRs by kube-controller-manager
    serviceSubnets:
      - 10.96.0.0/12       # virtual IPs for Services; 10.96.0.10 = kube-dns
    cni:
      name: flannel        # VXLAN overlay by default
  controllerManager:
    extraArgs:
      node-cidr-mask-size-ipv4: "24"   # size of each node's block

Service → pods

EndpointSlices: the Service's live backend list

A Service's selector is evaluated continuously by the EndpointSlice controller, which writes the matching pods' IPs and readiness into EndpointSlice objects (up to 100 endpoints each, so huge Services don't create one giant object). Every node's kube-proxy watches them and regenerates its rules. Fail a readiness probe and the pod drops out of rotation within seconds — without being killed.

EndpointSlice → kube-proxy

Deployment api, replicas:click a pod to cycle Ready → NotReady → Terminating

pods (label app=api)

# EndpointSlice api-x7k2p (EndpointSlice controller)
addressType: IPv4
ports: [{port: 8080, protocol: TCP}]
endpoints:
- addresses: ["10.244.1.20"]
  nodeName: node-1
  conditions:
    ready: true
    serving: true
    terminating: false
- addresses: ["10.244.2.21"]
  nodeName: node-2
  conditions:
    ready: true
    serving: true
    terminating: false
- addresses: ["10.244.3.22"]
  nodeName: node-3
  conditions:
    ready: true
    serving: true
    terminating: false
- addresses: ["10.244.1.23"]
  nodeName: node-1
  conditions:
    ready: false
    serving: false
    terminating: false

# kube-proxy renders only ready endpoints
-A KUBE-SVC-7RQX -m statistic --mode random
   --probability 0.33333 -j SEP-api-1
-A KUBE-SVC-7RQX -m statistic --mode random
   --probability 0.50000 -j SEP-api-2
-A KUBE-SVC-7RQX -j SEP-api-3

# each ready pod: 33.3% of new connections

DNS

The ndots:5 tax

Kubelet gives every pod ndots:5 so short names like api or api.payments resolve through the search list. The side effect (replayed in the DNS scenario above): every external name with fewer than five dots burns three NXDOMAIN round-trips — times two for A and AAAA — before the real query. On chatty services that's most of CoreDNS's load.

per-pod: lower ndots (or use FQDNs with a trailing dot)

apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  dnsConfig:
    options:
      - name: ndots
        value: "2"      # names with ≥2 dots go straight to upstream
  containers:
    - name: web
      image: ghcr.io/example/web:1.4

inspect it on a real cluster

# Which /24 did each node get?
kubectl get nodes -o custom-columns=NAME:.metadata.name,PODCIDR:.spec.podCIDR

# Who backs a Service right now?
kubectl get endpointslices -l kubernetes.io/service-name=api -o wide

# kube-proxy's rules on a node (Talos has no shell — ask kube-proxy itself)
kubectl -n kube-system exec ds/kube-proxy -- iptables-save -t nat | grep -A3 KUBE-SVC

# Watch the ndots search in action
kubectl run -it --rm dig --image=nicolaka/netshoot -- dig +search +showsearch api.example.com

why it mattersFlat, NAT-free pod networking is what makes the rest possible: source IPs are meaningful, so policy, logs and observability can trust them. Overlay vs routed only changes how packets cross the underlay — not what pods see.
why it mattersServices are distributed DNAT. Each node holds rules for every Service (KUBE-SERVICES → KUBE-SVC → KUBE-SEP), the endpoint is chosen once per connection, and conntrack does the rest. Readiness, via EndpointSlices, decides who gets traffic.
why it mattersPick your CIDRs before bootstrap: the pod and service ranges can't overlap anything you route to, the per-node block caps both node count and pods per node, and changing either later means rebuilding the cluster.