Upgrade a Cluster

Upgrading Clusters¶

There are three areas of a Cluster than need to be upgraded when new versions are released.

1. The Kubernetes cluster software
2. The custom Kelvin pods
3. Applications and Connections deployed to a Cluster

1. Kubernetes Cluster Upgrades¶

If you choose the option that Kelvin installs the Kubernetes cluster, Kelvin installs a customized K3s installation. The K3s can manage its own upgrades using the system-upgrade-controller.

If you choose to use your own Kubernetes cluster, then you should refer to its documentation on how to manage the updates.

2. Custom Kelvin Pod Upgrades¶

All Nodes in a Cluster have Kelvin specific pods running behind the scenes managing all the Kelvin related services.

From time to time these will need to be upgraded with the latest versions for both new features and for security updates. You can apply these upgrades automatically or manually.

Automatic Upgrades¶

Automatic upgrades will check for new versions at regular intervals and upgrade when new versions are available.

You can turn on and manage the frequency of upgrades from Kelvin UI.

From Orchestration menu click on a single Cluster, then under the Configuration tab you can set or disable the auto update feature and the frequency to check for updates.

If you have Auto Update set, then Kelvin will automatically regularly check and upgrade all Kelvin pods when available.

Manual Upgrades¶

In circumstances when you want to apply upgrades manually, for example during maintenance shifts or when your assets are not in operation, you can turn off automatic upgrades.

For manual upgrades you have three options;

• Option 1: Download and upgrade simultaneously.
• Option 2: Download and upgrade separately. Download the updates but do not immediately apply them. At a later date you can separately initiate the actual upgrade process.
• Option 3: Upgrade directly on the Edge System. Use the kelvin-provision cluster k3s commands on the Edge System. This option is only available if Kelvin installed the k3s cluster, and it does not use the Cluster keys.

Option 1: Download and Upgrade Simultaneously¶

Step 1: Understand the Cluster keys

Before applying the changes, it is important that you understand the Cluster key options available in Kelvin API.

Step 2: Turn off automatic upgrades

You can do this from Kelvin API or Kelvin UI.

1
2
3
4
5
6
7
8

curl -X "POST" \
  "https://<url.kelvin.ai>/api/v4/orchestration/clusters/<Your Cluster Name>/update" \
  -H "Authorization: Bearer <Your Current Token>" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
  "manifests_scrape_enabled": false
}'

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

from kelvin.api.client import Client

# Login
client = Client(url="https://<url.kelvin.ai>", username="<your_username>")
client.login(password="<your_password>")

# Set or Update the Default Cluster in an Instance
cluster_updates =  client.orchestration.update_orchestration_clusters(
    cluster_name="sales-01-cluster", 
    data={
    "manifests_scrape_enabled": False
    })

Step 3: Set the upgrade keys

There are two ways to do a manual upgrade where the download and upgrade is done simultaneously.

You will need to set the correct values for the keys for the Cluster to assign the way you want the upgrades to be performed.

Select the manual upgrade option in the table above then use these commands to set your choice in Kelvin.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

curl -X "POST" \
  "https://<url.kelvin.ai>/api/v4/orchestration/clusters/docs-demo-cluster-k3s/update" \
  -H "Authorization: Bearer <Your Current Token>" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
  "upgrade": {
    "instantly_apply": <Boolean Value>,
    "pre_download": <Boolean Value>
  }
}'

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

from kelvin.api.client import Client

# Login
client = Client(url="https://<url.kelvin.ai>", username="<your_username>")
client.login(password="<your_password>")

# Set or Update the Default Cluster in an Instance
cluster_updates =  client.orchestration.update_orchestration_clusters(
    cluster_name="sales-01-cluster", 
    data={
        "instantly_apply": <Boolean Value>,
        "pre_download": <Boolean Value>
    })

Option 2: Download and Upgrade Separately¶

Step 1: Understand the Cluster keys

Before applying the changes, it is important that you understand the Cluster key options available in Kelvin API.

Step 2: Turn off automatic upgrades

You can do this from Kelvin API or Kelvin UI.

1
2
3
4
5
6
7
8

curl -X "POST" \
  "https://<url.kelvin.ai>/api/v4/orchestration/clusters/<Your Cluster Name>/update" \
  -H "Authorization: Bearer <Your Current Token>" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
  "manifests_scrape_enabled": false
}'

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

from kelvin.api.client import Client

# Login
client = Client(url="https://<url.kelvin.ai>", username="<your_username>")
client.login(password="<your_password>")

# Set or Update the Default Cluster in an Instance
cluster_updates =  client.orchestration.update_orchestration_clusters(
    cluster_name="sales-01-cluster", 
    data={
    "manifests_scrape_enabled": False
    })

Step 3: Set the upgrade keys

You will first need to set the correct values for the keys for the Cluster to ensure the download and upgrades processes are done by two different commands.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

curl -X "POST" \
  "https://<url.kelvin.ai>/api/v4/orchestration/clusters/docs-demo-cluster-k3s/update" \
  -H "Authorization: Bearer <Your Current Token>" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
  "upgrade": {
    "instantly_apply": false,
    "pre_download": true
  }
}'

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

from kelvin.api.client import Client

# Login
client = Client(url="https://<url.kelvin.ai>", username="<your_username>")
client.login(password="<your_password>")

# Set or Update the Default Cluster in an Instance
cluster_updates =  client.orchestration.update_orchestration_clusters(
    cluster_name="sales-01-cluster", 
    data={
        "instantly_apply": False,
        "pre_download": True
    })

Step 4: Trigger the upgrade from your computer, or on the Edge System

After the keys are set, you can trigger the upgrade to start from your computer.

There are a few conditions that must be met to ensure your upgrade request succeeds.

• The cluster must be connected.
• The cluster must be configured with the pre_download == true and instantly_apply = false

1
2
3
4
5

curl -X "GET" \
  "https://<url.kelvin.ai>/api/v4/orchestration/clusters/docs-demo-cluster-k3s/edge-apps/version/apply" \
  -H "Authorization: Bearer <Your Current Token>" \
  -H "Accept: application/json" \
  -H "Accept: application/json"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from kelvin.api.client import Client

# Login
client = Client(url="https://<url.kelvin.ai>", username="<your_username>")
client.login(password="<your_password>")

# Set or Update the Default Cluster in an Instance
response =  client.orchestration.apply_orchestration_clusters_edge_apps_version(
    cluster_name="sales-01-cluster"
    )

Alternatively, you can trigger the upgrade to start in the terminal of the Edge System. This is very useful if the upgrade is done while the Cluster is disconnected from the Internet.

1

kelvin-provision cluster apply

Option 3: Upgrade Directly on the Edge System¶

This option does not use the Cluster keys. It is only available if you let Kelvin install the k3s cluster, because only then do you have the kelvin-provision tools on the Edge System.

It is not the same as the Option 2 trigger on the Edge System, which applies a pending upgrade with kelvin-provision cluster apply.

Step 1: Open the Kelvin Provision cluster k3s command

1

sudo kelvin-provision cluster k3s

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

Manage the k3s version across the cluster

Usage:
  kelvin-provision cluster k3s [command]

Available Commands:
  status      Show the current and target k3s versions for every node
  upgrade     Upgrade k3s on this node and then every worker node

Flags:
  -h, --help   help for k3s

Global Flags:
      --debug         Show debug logs
  -d, --dry-run       Just print the commands without execute them
  -k, --insecure      Skip SSL verification
  -o, --omit-stdout   Omit StdOut from bash scripts

Use "kelvin-provision cluster k3s [command] --help" for more information about a command.

Step 2: Check the versions

Check the current and target versions of every Node with the status command.

1

sudo kelvin-provision cluster k3s status

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

Reading cluster config...[OK]
Connecting to cluster API...[OK]
Test cluster connection...[OK]
+-------+--------+--------------+--------------+---------------+
| Name  | Status | Current k3s  | Target k3s   | Needs upgrade |
+-------+--------+--------------+--------------+---------------+
| node2 | Ready  | v1.36.1+k3s1 | v1.36.1+k3s1 | no            |
+-------+--------+--------------+--------------+---------------+

This node's k3s config is already normalized.

Step 3: Start the upgrade

Start the upgrade with the upgrade command. It upgrades this Node first and then every worker Node.

1

sudo kelvin-provision cluster k3s upgrade

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

Reading cluster config...[OK]
Connecting to cluster API...[OK]
Test cluster connection...[OK]

0 of 1 node(s) need upgrading to v1.36.1+k3s1
Wrote /etc/rancher/k3s/registries.yaml

Node role:      server
Current k3s:    v1.36.1+k3s1
Target k3s:     v1.36.1+k3s1
Config changes: none
This node already runs v1.36.1+k3s1 with a normalized config. Nothing to do.

No worker nodes need upgrading. All done!

3. Applications, Docker Apps and Connections Upgrades¶

Workloads and Connections must be upgraded manually. You can send a new deploy for the new version and the old version currently running will be managed depending on the staged key.

Applications can be upgraded from the Kelvin UI. You can read full documentation on this feature here.