For full text search please use the '?' prefix. e.g. ? Onboarding

Configuration

The cluster configuration is stored in etcd. There are two deployment models. In both models the documentation recommends that at least 3 etcd members and 3 control plane nodes exist. In particularly large clusters (with >= 7 etcd members) the etcd replication traffic can cause performance issues, hence the etcd service may be moved outside of the cluster to enable provisioning of more control plane nodes.

Data loss is likely in the event that the Kubernetes cluster configuration is lost, so backing up etcd data is critical to successful Kubernetes cluster operation. Note that because secrets are stored in the etcd configuration, backups should be treated as sensitive data. Consider backing up the cluster data using a Kubernetes CronJob.

The certificates used for authenticating with etcd are located at the following control plane node paths:

  • --cacert /etc/kubernetes/pki/ca.crt
  • --cert /etc/kubernetes/pki/etcd/server.crt
  • --key /etc/kubernetes/pki/etcd/server.key

Limitations

  • The default etcd configuration constrains the size of values to 1.5MiB. This is configurable with the --max-request-bytes switch.
  • The default storage size limit is 2GiB, configurable with the --quota-backend-bytes switch.

Stacked control plane nodes

The control plane nodes manage their own etcd members control plane components (API server, scheduler, proxy and controller manager). This is easier to configure and maintain, but the loss of a control plane node is more likely to result in data loss as the etcd member will go down with the control plane node.

In this configuration, etcd is started as a static pod, configured in the /etc/kubernetes/manifests/etcd.yaml manifest. The cluster data is hosted on the control plane nodes and mounted into the etcd Pods via a hostPath volume.

External etcd cluster

With an external etcd cluster the configuration store and control plane components are separated. In theory (depending on other fault domains) this could reduce the impact of a node failure to either a control plane node or an etcd member.

Backlinks