Kubernetes networking is built on three core principles:

  1. All Pods can communicate with all other Pods on all other cluster nodes.
  2. Agents on a Node can communicate with all Pods on that Node.
  3. There's no NAT: the IP that a pod identifies as is the same IP address that others use to communicate with it.

These simple rules simplify connectivity across complex environments that might span multiple subnets and even multiple cloud providers; we just need traffic to be routable between them, and for each pod to be assigned its own unique IP address.

Kubernetes considers four different types of networking problems it has to solve:

  1. Tightly-coupled container to container communications, solved with pods' network namespaces.
  2. Pod-to-Pod communications must allow direct access without NAT in order to allow pods to access their dependencies
  3. Pod-to-Service, solved using Services.
  4. Service-to-Pod, solved using Services.

There are two networks:

  1. The Node network provides connectivity between all of the Nodes.
  2. The Pod network provides connectivity between all of the Pods running on all the Nodes. We can either deploy a layer 2 or 3 network or an overlay network here. Each pod will have a NIC connected to the node's container bridge (cbr0), which can route packets between pods on the same node.

Cluster DNS

Pods are configured to use the cluster DNS service, which manages A/AAAA and SRV records for Services. By default, all Pods' DNS search list includes the Pod's own namespace and the cluster's default domain.

The DNS service can be used to provide service discovery to software running on Pods. For Services, two types of record are created:

  • A/AAAA records of the form service-name.namespace-name.svc.cluster-domain resolve to:
    • the Cluster IP of the Service for "normal" services assigned one; or
    • a set containing the individual Pod IPs for "Headless" services.
  • SRV records of the form _port-name._port-protocol.service-name.namespace-name.svc.cluster-domain for named ports, resolve to:
    • the port number and the Service's A/AAAA DNS name for "normal" services; or
    • an answer per Pod backing the service containing the port number and a domain name for the pod in the form auto-generated-name.service-name.namespace-name.svc.cluster-domain.

Pods get A/AAAA records in the form pod-ip-address.namespace-name.pod.cluster-domain. Pods created by a deployment also get A/AAAA records in the form pod-ip-address.deployment-name.namespace-name.svc.cluster-domain.

By default, this is provided by CoreDNS, installed as a deployment to the kube-system namespace. The configuration is provided by a ConfigMap:

$ kubectl get deployment -n kube-system -o yaml coredns
        - mountPath: /etc/coredns
            name: config-volume
            readOnly: true
    - configMap:
          defaultMode: 420
        - key: Corefile
            path: Corefile
          name: coredns
        name: config-volume

$ kubectl get configmap -n kube-system -o yaml coredns
  Corefile: |

Network policy

NetworkPolicies let administrators control traffic flow at OSI layer 3 or 4 and are implemented by the network plugin.

  1. CNI
  2. ExternalDNS
  3. Kubenet