In this blog, I will demonstrate how we can upgrade a Kubernetes cluster version with the help of kubeadm. To know about a Kubernetes installation and configuration I recommend you to go through my previous articles.
Kubernetes installation and configuration – Part1
How to remove a node from the Kubernetes cluster – Part2
The cluster up-gradation consists of two parts one is the up-gradation of the control plane or master node and the other one is the up-gradation of worker nodes. Before proceeding with the upgrade I hope you have a Kubernetes cluster is up and running. Below are the details of my Kubernetes cluster.
| Machine name | OS | Size | IP address |
| control-node-1 | Ubuntu 18.04 LTS | 2cpu, 4g memory | 50.1.0.5 |
| worker-node-1 | Ubuntu 18.04 LTS | 2cpu, 4g memory | 50.1.0.7 |
| worker-node-2 | Ubuntu 18.04 LTS | 2cpu, 4g memory | 50.1.0.8 |
The current version of kubelet, kubeadm, and kubectl is 1.21.0 and we will upgrade it to version 1.21.1 (Minor version upgrade).
Upgrade control plane node
To reduce the downtime we will perform the upgrade node by node. Log in to the control plane node and check the current version.
techies@control-node-1:~$ kubectl version Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:31:21Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:25:06Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"} techies@control-node-1:~$
To upgrade the control plane node first we have to drain the node from the cluster (In my previous post I have written about the node draining process.)
Execute the below command to drain the control plane node.
kubectl drain control-node-1 --ignore-daemonsets
Few systems and networking pods will be evicted as part of this activity but all these pods will come back once we add back the control plane node to the cluster.
Execute the below command to upgrade the kubeadm package.
sudo apt-get update && sudo apt-get install -y --allow-change-held-packages kubeadm=1.21.1-00
Check the version kubeadm
techies@control-node-1:~$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:17:27Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}
From the above output, we can confirm the kubeadm package is now upgraded to v1.21.1. Next, we have to upgrade all other cluster components with the help of kubeadm. To upgrade the control plane execute the below command.
sudo kubeadm upgrade plan v1.21.1
The “kubeadm upgrade plan” command shows what all are the component going to be upgraded and to apply the changes execute the below command.
sudo kubeadm upgrade apply v1.21.1
The command may take a few minutes to complete. If there is no error in the upgrade process, you will get a message similar to below.
[upgrade/successful] SUCCESS! Your cluster was upgraded to “v1.21.1”. Enjoy!
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven’t already done so.
Now we have kubectl and kubelet is pending for the upgrade, to upgrade these packages execute the below command.
sudo apt-get update && sudo apt-get install -y --allow-change-held-packages kubelet=1.21.1-00 kubectl=1.21.1-00The argument “–allow-change-held-packages” overwrite package upgrade. I have held the cluster package upgrade at the time of installation.
Reload and restart the kubelet daemon
sudo systemctl daemon-reload
sudo systemctl restart kubeletControl plane upgrade activity has been completed and we can now add back the node to the cluster. To add the node execute the below command.
kubectl uncordon control-node-1
Execute kubectl get nodes command to know the status of each node.
techies@control-node-1:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION control-node-1 Ready control-plane,master 4d1h v1.21.1 worker-node-1 Ready <none> 4d v1.21.0 worker-node-2 Ready <none> 4d v1.21.0
We have successfully upgraded the Control node version to v1.21.1. Next, we will upgrade worker node one by one to reduce the downtime.
Upgrade worker node
To upgrade a worker node first we have to drain the node from the cluster. Execute the below command from the Control plane node.
kubectl drain worker-node-1 --ignore-daemonsets --force
the –ignore-daemonsets command evicts all pods running on the node. The pods running on this worker node will be moved to worker-node-2. In my cluster, there were 2 Nginx pods were running and all are moved to the worker-node-2.
To upgrade worker node login to worker node and start upgrading kubeadm package first.
sudo apt-get update && sudo apt-get install -y --allow-change-held-packages kubeadm=1.21.1-00sudo kubeadm upgrade node
Next, upgrade kubectl kubelet packages
sudo apt-get update && sudo apt-get install -y --allow-change-held-packages kubelet=1.21.1-00 kubectl=1.21.1-00Don’t forget to reload and restart the kubelet daemon.
sudo systemctl daemon-reload
sudo systemctl restart kubelet
Worker node upgrade is completed. Next, add this node back to the cluster. For this execute the below command from the control plane node.
techies@control-node-1:~$ kubectl uncordon worker-node-1 node/worker-node-1 uncordoned techies@control-node-1:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION control-node-1 Ready control-plane,master 5d14h v1.21.1 worker-node-1 Ready <none> 5d13h v1.21.1 worker-node-2 Ready <none> 5d13h v1.21.0
Execute the “kubectl get nodes” command and make sure that worker-node is upgraded.
Repeat the steps for worker-node-2 as well. Once all the worker-node has upgraded execute the “kubectl get nodes” command to check node status.
techies@control-node-1:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION control-node-1 Ready control-plane,master 5d14h v1.21.1 worker-node-1 Ready <none> 5d13h v1.21.1 worker-node-2 Ready <none> 5d13h v1.21.1
We have completed our Kubernetes cluster upgrade. If you have any questions regarding the upgrade feel free to ask in the comment section. Thanks!
