In our latest tutorial, we shared how to conduct a rolling Kubernetes upgrade. Now that you’ve mastered the basics, we’re going to dive into how to perform advanced upgrades with fine-grained controls. By having fine-grained control of your upgrade, it speeds up the upgrade process and minimizes any disruption to your deployment environment.
Mar 23, 2021
Alex Hisaka
D2iQ
3 min read
D2iQ Konvoy provides controls to easily upgrade Kubernetes itself, Kubernetes add-ons, like Prometheus, or the Konvoy CLI independently. Best of all, upgrades can be performed in place without disruption.
This tutorial will cover how to upgrade node pools, or specify the upgrade based on specific node pools. We’ll also walk through how to upgrade in parallel, or specify the number of concurrent nodes to upgrade.
Step 1: Determine the Number of Nodes in Each Node Pool
To get started, take a look at the cluster config. Use the Konvoy node pool command to see how many nodes you have allocated to each node pool.
In the screenshot below, you can see the cluster has four node pools. It has a control plane node pool with three nodes in it; a GPU worker node pool with four nodes in it, and a worker pool that has 20 nodes in it. There’s currently no nodes in the bastion node pool.
These pools are described with Kubernetes Labels. The labels can be found by describing the node.
Let’s take a look at the first node. At the very top, you’ll see the Kubernetes labels. The label in the screenshot below describes the node as part of the GPU worker node pool.
Step 2: Upgrade the GPU Node Pool
Next, upgrade the GPU node pool.
With D2iQ Konvoy, it will automatically upgrade the control plane followed by the four GPU nodes in this node pool. And it will leave the other nodes alone.
Begin the upgrade with the D2iQ Konvoy$ konvoy up --upgrade command. Adding the target node pull option allows you to specify which node pool you want to upgrade first. This can be a comma separated list of node pools. For this particular tutorial, we’re going to upgrade the GPU worker node pool.
Once the upgrade is finished, check to see if the service version is upgraded. In this case, it should be reporting 117.8. When you look at the nodes on your right terminal, you should see that the control plane, or master nodes, have been upgraded, as well as a few of the worker nodes.
Step 3: Specify the Node Pool to Upgrade
Remember that only the GPU nodes have been upgraded at this point. You can see in the screenshot below that the worker node poll has 20 nodes in it.
Next, specify this node pool to upgrade. This can be specified either as an integer or a percent. In this case, we’ve specified 25% which will upgrade 5 of the 20 nodes at a time. You can watch the progress in your right terminal window.
When upgrading by node pulls, it’s important to consider how much free capacity you have in your cluster. For example, by picking a 25% parallel upgrade, you will need 25% of free resources in your cluster, as Konvoy migrates workloads off the nodes for the upgrade process.
With D2iQ Konvoy, it will drain and cordon off five nodes at a time. You can see that all nodes have been upgraded to version 117.8 in your right terminal window.
