Cluster Infrastructure Setup
System environment preparation before installing a Kubernetes cluster, including container runtime installation, kernel parameter configuration, network settings, and other infrastructure setup.
Overview
Before installing a Kubernetes cluster, system-level environment preparation is required on all nodes (including control plane and worker nodes). This includes container runtime installation, kernel module loading, network parameter tuning, and more. The CKA exam may involve troubleshooting cluster infrastructure issues.
1. System Prerequisites
1.1 Host Requirements
| Item | Requirement |
|---|---|
| Operating System | Linux (Ubuntu 20.04+, CentOS 7+, Rocky Linux 8+) |
| CPU | At least 2 cores |
| Memory | Control plane at least 2GB, worker nodes at least 1GB |
| Disk | At least 20GB free space |
| Hostname | Unique and resolvable |
| Network | Inter-node network connectivity |
# Check system information
uname -a
cat /etc/os-release
hostnamectl
1.2 Hostname and Hosts Resolution
# Set hostname
sudo hostnamectl set-hostname control-plane-1
# Configure /etc/hosts
cat <<EOF | sudo tee -a /etc/hosts
192.168.1.10 control-plane-1
192.168.1.11 worker-1
192.168.1.12 worker-2
EOF
# Verify hostname resolution
hostname
hostname -f
getent hosts $(hostname)
1.3 Disable Swap
Kubernetes requires swap to be disabled to ensure Pod memory stability.
# Temporarily disable
sudo swapoff -a
# Permanently disable (comment out swap line in /etc/fstab)
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# Verify
free -m
swapon --show
1.4 Disable Firewall (or Open Required Ports)
# Ubuntu (ufw)
sudo ufw disable
sudo ufw status
# CentOS/RHEL (firewalld)
sudo systemctl stop firewalld
sudo systemctl disable firewalld
sudo firewall-cmd --state
# Ports to open (if not disabling firewall)
# Control plane:
# - 6443 (API Server)
# - 2379-2380 (etcd)
# - 10250 (kubelet)
# - 10259 (scheduler)
# - 10257 (controller-manager)
# Worker nodes:
# - 10250 (kubelet)
# - 30000-32767 (NodePort services)
1.5 Time Synchronization
# Install chrony
sudo apt install -y chrony # Ubuntu
sudo yum install -y chrony # CentOS/RHEL
# Start and enable
sudo systemctl enable --now chronyd
# Verify sync status
timedatectl status
chronyc tracking
1.6 Load Kernel Modules
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
# Load immediately
sudo modprobe overlay
sudo modprobe br_netfilter
# Verify
lsmod | grep overlay
lsmod | grep br_netfilter
1.7 Configure Kernel Parameters
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 1
EOF
# Apply configuration
sudo sysctl --system
# Verify
sysctl net.bridge.bridge-nf-call-iptables
sysctl net.ipv4.ip_forward
2. Container Runtime Installation
2.1 containerd
containerd is the default container runtime for Kubernetes and the recommended runtime for the CKA exam.
# --- Method 1: Install containerd via Docker ---
# Ubuntu
sudo apt update
sudo apt install -y containerd
# CentOS/RHEL
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install -y containerd.io
# --- Method 2: Install containerd from binary (recommended for exam environments) ---
# Download and install containerd
wget https://github.com/containerd/containerd/releases/download/v1.7.x/containerd-1.7.x-linux-amd64.tar.gz
sudo tar Cxzvf /usr/local containerd-1.7.x-linux-amd64.tar.gz
# Install runc
wget https://github.com/opencontainers/runc/releases/download/v1.1.x/runc.amd64
sudo install -m 755 runc.amd64 /usr/local/sbin/runc
# Install CNI plugins
wget https://github.com/containernetworking/plugins/releases/download/v1.3.x/cni-plugins-linux-amd64-v1.3.x.tgz
sudo mkdir -p /opt/cni/bin
sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.3.x.tgz
2.2 containerd Configuration
# Generate default configuration file (important!)
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
# View generated configuration
cat /etc/containerd/config.toml
2.3 SystemdCgroup Configuration
This is a critical configuration point in the CKA exam. Kubernetes requires using the systemd cgroup driver.
# Edit containerd configuration file
sudo vi /etc/containerd/config.toml
Find and modify the following configuration:
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
...
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true # Change to true
Or use sed for quick modification:
# Change SystemdCgroup to true
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
# Configure sandbox_image (modify if using an internal network)
sudo sed -i 's|sandbox_image = "registry.k8s.io/pause:.*"|sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml
# Restart containerd
sudo systemctl restart containerd
# Verify containerd status
sudo systemctl status containerd
sudo ctr version
sudo crictl version
sudo crictl info
2.4 CRI-O (Alternative)
# Ubuntu install CRI-O
sudo apt update
sudo apt install -y cri-o cri-o-runc
# CentOS/RHEL install CRI-O
sudo yum install -y cri-o
# Start CRI-O
sudo systemctl enable --now crio
# Configure CRI-O cgroup driver
sudo vi /etc/crio/crio.conf
# Set cgroup_manager = "systemd"
sudo systemctl restart crio
3. kubelet, kubeadm, kubectl Installation
# --- Ubuntu / Debian ---
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
# Add Kubernetes repository
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl # Prevent automatic version upgrades
# --- CentOS / RHEL / Rocky ---
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.31/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.31/rpm/repodata/repomd.xml.key
EOF
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo yum-mark hold kubelet kubeadm kubectl
# Start kubelet (it will fail to restart at this point because the cluster hasn't been initialized yet — this is normal)
sudo systemctl enable --now kubelet
sudo systemctl status kubelet
4. Verify Infrastructure
# Verify all prerequisites
echo "=== Hostname ==="
hostnamectl
echo "=== Swap Status ==="
swapon --show || echo "Swap is disabled ✓"
echo "=== Kernel Modules ==="
lsmod | grep -E "overlay|br_netfilter"
echo "=== Network Parameters ==="
sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward
echo "=== containerd ==="
sudo systemctl is-active containerd
sudo crictl version
echo "=== kubelet ==="
sudo systemctl is-enabled kubelet
echo "=== Port Listening ==="
sudo ss -tulpn | grep -E "6443|2379|2380|10250|10259|10257"
CKA Exam Key Points
- SystemdCgroup is a must-know -- When kubeadm init fails, first check the containerd SystemdCgroup configuration
- Swap must be disabled -- When the cluster fails to start, remember to check whether swap has been disabled
- Use
apt-mark hold-- Prevent accidental upgrades of kubelet/kubeadm/kubectl - Pre-configure containerd -- Ensure containerd is running properly before kubeadm init
- Common debugging commands:
sudo journalctl -u containerd -n 50 --no-pagersudo journalctl -u kubelet -n 50 --no-pagercrictl images-- List pulled imagescrictl ps -a-- List all containers
🧪 Complete Hands-on Example: Configure K8s Infrastructure on an Ubuntu Node
Scenario Description
On a freshly installed Ubuntu 22.04 node, complete the Kubernetes prerequisite system configuration: disable swap, load kernel modules, configure network parameters, install containerd, and set up SystemdCgroup.
Prerequisites
- A server running Ubuntu 22.04+ (physical or virtual machine)
- A user with sudo privileges
- Network access to apt repositories
Steps
Step 1: Disable swap
# Temporarily disable
sudo swapoff -a
# Permanently disable (comment out swap entry in /etc/fstab)
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
# Verify
free -m
# total used free shared buff/cache available
# Mem: 7982 1234 4567 123 2181 6543
# Swap: 0 0 0 <-- Should be 0
swapon --show
# (No output, meaning swap is disabled)
Step 2: Load kernel modules
# Configure auto-load at boot
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
# Load immediately
sudo modprobe overlay
sudo modprobe br_netfilter
# Verify
lsmod | grep -E "overlay|br_netfilter"
# overlay 147456 0
# br_netfilter 32768 0
# bridge 307200 1 br_netfilter
Step 3: Configure kernel network parameters
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
# Apply configuration
sudo sysctl --system
# Verify key parameters
sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward
# net.bridge.bridge-nf-call-iptables = 1
# net.ipv4.ip_forward = 1
Step 4: Install containerd
# Install dependencies
sudo apt-get update
sudo apt-get install -y ca-certificates curl
# Add Docker official GPG key and repository
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y containerd.io
Step 5: Configure containerd SystemdCgroup
# Generate default configuration
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
# Enable SystemdCgroup
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
# Restart containerd
sudo systemctl restart containerd
# Verify configuration
sudo systemctl status containerd --no-pager
# ● containerd.service - containerd container runtime
# Loaded: loaded (/lib/systemd/system/containerd.service; enabled; vendor preset: enabled)
# Active: active (running)
sudo crictl info | grep cgroupManager
# "cgroupManager": "systemd"
Verification Results
# One-click verify all prerequisites
echo "=== swap ===" && swapon --show || echo "Swap is disabled"
echo "=== Kernel Modules ===" && lsmod | grep -E "overlay|br_netfilter"
echo "=== Network Parameters ===" && sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward
echo "=== containerd ===" && sudo systemctl is-active containerd && ctr version
Exam Tips
- SystemdCgroup is a common cause of kubeadm init failure in CKA -- make sure to configure it before initialization
swapoff -aonly takes effect temporarily; you must also modify/etc/fstabto ensure it persists after reboot- Use
crictl infoto verify containerd configuration, notctr version - When kubeadm init fails, first check
journalctl -u kubeletand the containerd SystemdCgroup configuration