Kubernetes networking is built on three core principles:
- All Pods can communicate with all other Pods on all other cluster nodes.
- Agents on a Node can communicate with all Pods on that Node.
- There's no NAT: the IP that a pod identifies as is the same IP address that others use to communicate with it.
These simple rules simplify connectivity across complex environments that might span multiple subnets and even multiple cloud providers; we just need traffic to be routable between them, and for each pod to be assigned its own unique IP address.
Kubernetes considers four different types of networking problems it has to solve:
- Tightly-coupled container to container communications, solved with pods' network namespaces.
- Pod-to-Pod communications must allow direct access without NAT in order to allow pods to access their dependencies
- Pod-to-Service, solved using Services.
- Service-to-Pod, solved using Services.
There are two networks:
- The Node network provides connectivity between all of the Nodes.
- The Pod network provides connectivity between all of the Pods running on all the Nodes. We can either deploy a layer 2 or 3 network or an overlay network here. Each pod will have a NIC connected to the node's container bridge (
cbr0), which can route packets between pods on the same node.
Pods are configured to use the cluster DNS service, which manages A/AAAA and SRV records for Services. By default, all Pods' DNS search list includes the Pod's own namespace and the cluster's default domain.
The DNS service can be used to provide service discovery to software running on Pods. For Services, two types of record are created:
- A/AAAA records of the form
- the Cluster IP of the Service for "normal" services assigned one; or
- a set containing the individual Pod IPs for "Headless" services.
- SRV records of the form
_port-name._port-protocol.service-name.namespace-name.svc.cluster-domainfor named ports, resolve to:
- the port number and the Service's A/AAAA DNS name for "normal" services; or
- an answer per Pod backing the service containing the port number and a domain name for the pod in the form
Pods get A/AAAA records in the form
pod-ip-address.namespace-name.pod.cluster-domain. Pods created by a deployment also get A/AAAA records in the form
By default, this is provided by CoreDNS, installed as a deployment to the
kube-system namespace. The configuration is provided by a ConfigMap:
$ kubectl get deployment -n kube-system -o yaml coredns [snip] spec: template: spec: containers: - volumeMounts: - mountPath: /etc/coredns name: config-volume readOnly: true [snip] volumes: - configMap: defaultMode: 420 items: - key: Corefile path: Corefile name: coredns name: config-volume [snip] $ kubectl get configmap -n kube-system -o yaml coredns data: Corefile: | [snip]