vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Metadata updates N/A N/A Updated the worker node version fix pack metadata for internal documentation purposes. 







 Change log for worker node fix pack 1.19.7_1533, released 1 February 2021 

The following table shows the changes that are in the worker node fix pack 1.19.7_1533. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.19.7_1532

 Component Previous Current Description
* [Resetting the host key](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-locationhost-key-reset)
* [Migrating your control plane from RHEL 7 to RHEL 8](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-locationmigrate-cp-rhel8)



[Updating hosts that are assigned as worker nodes](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers)



* [Checking if a version update is available for worker node hosts](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers-check)



* [Checking if a version update is available with the IBM Cloud CLI](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers-check-cli)
* [Checking if a version update is available from the IBM Cloud console](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers-check-console)



* [Identifying worker node hosts](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-identify)
* [Apply version updates to worker node hosts without detaching them](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers-inplace)



* [Apply version updates to worker node hosts one at a time](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers-individually)
* [Apply version updates to your worker node hosts with a ConfigMap](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers-rolling)
Metadata updates N/A N/A Updated the worker node version fix pack metadata for internal documentation purposes. 







 Change log for worker node fix pack 1.18.15_1539, released 1 February 2021 

The following table shows the changes that are in the worker node fix pack 1.18.15_1539. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.18.15_1538

 Component Previous Current Description
Ensure that the worker nodes in your cluster that you want to use in your storage configuration have at least one available local disk in addition to the disks required by Satellite. The extra disks must be unformatted.
4. [Get the device details of your worker nodes](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-local-volume-blocksat-storage-block-local-devices).
5. [Label the worker nodes](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-local-volume-blocksat-storage-block-local-labels) that have an available disk and that you want to use in your configuration. The local storage drivers are installed only on the labeled worker nodes.





 Getting the device details for your local block storage configuration 

When you create your local block storage configuration, you must specify which devices that you want to use. The device paths that you retrieve in the following steps are specified as parameters when you create your configuration.



1. Log in to your cluster and get a list of available worker nodes. Make a note of the worker nodes that you want to use in your configuration.

oc get nodes
2. Log in to each worker node that you want to use for your local storage configuration.

oc debug node/<node-name>
3. When the debug pod is deployed on the worker node, run the following commands to list the available disks on the worker node.



1. Allow host binaries.

chroot /host
2. List your devices.

lsblk
3. Get the details of your devices. Verify that the devices that you want to use are unmounted and unformatted.

fdisk -l



4. List available block storage disks on your worker node. You must use unmounted disks for the local storage configuration. In the following example output from the lsblk command, the nvme2n1 disk is unmounted and has no partitions.

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:3 0 100G 0 disk
1. Keep in mind that the [worker nodes reserve certain amounts of CPU and memory resources](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodesresource_limit_node) to run required components, such as the operating system or container runtime.
2. Consider whether your app pulls large or many images, which can take up local storage on the worker node.
3. Decide whether you want to [integrate services](https://cloud.ibm.com/docs/containers?topic=containers-supported_integrationssupported_integrations) into your cluster, such as [Helm](https://cloud.ibm.com/docs/containers?topic=containers-helm) or [Prometheus](https://github.com/prometheus-operator/kube-prometheus). These integrated services and add-ons spin up pods that consume cluster resources.







 What type of availability do I want my workload to have? 

Don't forget that you want your workload to be up as much as possible! As such, you might need replicas of your workload pods.



1. Plan out your strategy for [highly available clusters](https://cloud.ibm.com/docs/containers?topic=containers-ha_clustersha_clusters), such as deciding between single or multizone clusters.
2. Review [highly available deployments](https://cloud.ibm.com/docs/containers?topic=containers-plan_deployhighly_available_apps) to help decide how you can make your app available.







 How many worker nodes do I need to handle my workload? 

Now that you have a good idea of what your workload looks like, let's map the estimated usage onto your available cluster configurations.



1. Estimate the max worker node capacity, which depends on what type of cluster you have. You don't want to max out worker node capacity in case a surge or other temporary event happens.



* Single zone clusters: Plan to have at least three worker nodes in your cluster.
Troubleshooting worker nodes in Critical or NotReady state 

Cluster worker nodes go into a Critical or NotReady state when they stop communicating with the cluster master. When this occurs, your worker nodes are marked as Critical in the IBM Cloud UI or when you run ibmcloud ks worker commands, and as NotReady in the Kubernetes dashboards and when you run kubectl get nodes. There are several reasons why communication stops between worker nodes and the cluster master. Follow these steps to troubleshoot worker nodes in these states.

Check the IBM Cloud health and status dashboard for any notifications or maintenance updates that might be relevant to your worker nodes. These notifications or updates might help determine the cause of the worker node failures.



 Check the common causes of worker node failures 

There are several reasons why communication stops between worker nodes and the cluster master. Check whether the following common issues are causing the disruption.

The worker was deleted, reloaded, updated, replaced, or rebooted
: Worker nodes might temporarily show a Critical or NotReady state when they are deleted, reloaded, updated, or replaced. If any of these actions have been initiated on your worker node, whether manually or as part of an automation setup such as cluster autoscaler, wait until the actions are complete. Then, check the status of your worker nodes again. If any workers remain in the Critical or NotReady state, [reload](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli&interface=uics_worker_reload) or [replace](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli&interface=uicli_worker_replace) the affected workers.
: If a worker node was reloaded or replaced and initially works correctly, but then after some time goes back into a Critical or NotReady state, then it is likely that some workload or component on the worker is causing the issue. See [Debugging worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-debug-kube-nodes) to isolate the problem workload.
Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.19.15_1562

 Component Previous Current Description 

 Ubuntu 18.04 packages N/A N/A Updated worker node images with package updates. Contains fix for cloud-init performance problem. 
 Worker-pool taint automation N/A N/A Fixes known issue related to worker-pool taint automation that prevents workers from getting providerID. 







 Change log for worker node fix pack 1.19.15_1562, released 11 October 2021 

The following table shows the changes that are in the worker node fix pack patch update 1.19.15_1562. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.19.15_1561

 Component Previous Current Description 

 Containerd v1.4.9 v1.4.11 See the [security bulletin](https://www.ibm.com/support/pages/node/6501867) and the [change logs](https://github.com/containerd/containerd/releases/tag/v1.4.11). 
 Ubuntu 18.04 packages 4.15.0-158 4.15.0-159 Updated worker node images and kernel with package updates [CVE-2021-3778](https://nvd.nist.gov/vuln/detail/CVE-2021-3778) and [CVE-2021-3796](https://nvd.nist.gov/vuln/detail/CVE-2021-3796). 







 Change log for worker node fix pack 1.19.15_1561, released 27 September 2021 

The following table shows the changes that are in the worker node fix pack patch update 1.19.15_1561. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.19.14_1559

 Component Previous Current Description
ibmcloud iam oauth-tokens
5. Retrieve the ID of the worker node that you want to attach to the Block Storage for Classic instance. Make sure to select a worker node that is located in the same zone as your Block Storage for Classic volume.

ibmcloud oc worker ls --cluster <cluster_name_or_ID>
6. Use a POST request to attach your Block Storage for Classic volume to the worker node.

Example request

curl -X POST "https://containers.cloud.ibm.com/global/v2/storage/createAttachment" -H "accept: application/json" -H "Authorization: <IAM_token>" -H "X-Auth-Resource-Group-ID: <resource_group>" -H "Content-Type: application/json" -d "{ "cluster": "<cluster_name_or_ID>", "volumeID": "<volume_ID>", "worker": "<worker_ID>"}"

IAM_token
: The IAM OAuth token for your current session. You can retrieve this value by running ibmcloud iam oauth-tokens.

cluster_name_or_ID
: The unique ID or the name that is assigned to your cluster. You can retrieve this ID by running ibmcloud oc cluster ls.

worker_ID
: The unique ID that is assigned to the worker node where you want to attach your volume. You can retrieve this value by running ibmcloud oc worker ls -c <cluster_name>.

volume_ID
: The unique ID that is assigned to your Block Storage for Classic volume. You can retrieve a list of your Block Storage for Classic volumes by running ibmcloud is volumes.

Example response.

{
"id": "0111-1aaa11a1-aa1a-111a-111b-1111a1dad1bc",
"volume": {
"name": "my-vol",
Reloading a worker node applies patch version updates to your worker node, but not major or minor updates. To see the changes from one patch version to the next, review the [Version change log](https://cloud.ibm.com/docs/containers?topic=containers-changelog) documentation.

Before you reload your worker node, make sure that you have enough capacity in other worker nodes to reschedule the pods on the worker node. Rescheduling pods helps to avoid downtime for your app or data corruption on your worker node.



1. List all worker nodes in your cluster and note the name of the worker node that you want to reload.

kubectl get nodes

The name that is returned in this command is the private IP address that is assigned to your worker node. You can find more information about your worker node when you run the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command and look for the worker node with the same Private IP address.
2. Reload the worker node. As part of the reload process, the pods that run on the worker node are drained and rescheduled onto remaining worker nodes in the cluster. The worker node is also cordoned, or marked as unavailable for future pod scheduling. Use the worker node ID that is returned from the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command.

ibmcloud ks worker reload --cluster <cluster_name_or_ID> --worker <worker_name_or_ID>
3. Wait for the reload to complete. When your worker node is in a Normal state, the worker node becomes available for pod scheduling again.



ibmcloud ks worker reload --cluster CLUSTER --worker WORKER_ID [--skip-master-healthcheck] [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

-w, --worker WORKER
: Specify a worker node ID.
ibm-cloud.kubernetes.io/zone: us-east-2
kubernetes.io/arch: amd64
kubernetes.io/hostname: satellite-ibm-host-3
kubernetes.io/os: linux
node-role.kubernetes.io/master: ""
node-role.kubernetes.io/worker: ""
node.kubernetes.io/instance-type: upi
node.openshift.io/os_id: rhel
privateVLAN: "1"
topology.kubernetes.io/region: us-east
topology.kubernetes.io/zone: us-east-2
4. Create a ConfigMap and define the unavailability rules for your worker nodes. The following example shows two checks, the defaultcheck.json and a check template. You can use this example check to define rules for all worker nodes that don't match any of the checks that you defined in the ConfigMap (defaultcheck.json). Use the check template to create your own check. For every check, to identify a worker node, you must choose one of the worker node labels that you retrieved in the previous step.

For every check, you can set only one value for NodeSelectorKey and NodeSelectorValue. Define up to 10 checks in a ConfigMap. If you add more checks, they are ignored.

Example

apiVersion: v1
kind: ConfigMap
metadata:
name: ibm-cluster-update-configuration
namespace: kube-system
data:
drain_timeout_seconds: "120"
defaultcheck.json: 
{
"MaxUnavailablePercentage": 20
}
<check_name>: 
{
"MaxUnavailablePercentage": <value_in_percentage>,
"NodeSelectorKey": "<node_selector_key>",
"NodeSelectorValue": "<node_selector_value>"
Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.20.11_1555

 Component Previous Current Description 

 Ubuntu 18.04 packages N/A N/A Updated worker node images with package updates. Contains fix for cloud-init performance problem. 
 Worker-pool taint automation N/A N/A Fixes known issue related to worker-pool taint automation that prevents workers from getting providerID. 







 Change log for worker node fix pack 1.20.11_1555, released 11 October 2021 

The following table shows the changes that are in the worker node fix pack patch update 1.20.11_1555. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.20.11_1554

 Component Previous Current Description 

 Containerd v1.4.9 v1.4.11 See the [security bulletin](https://www.ibm.com/support/pages/node/6501867) and the [change logs](https://github.com/containerd/containerd/releases/tag/v1.4.11). 
 Ubuntu 18.04 packages 4.15.0-158 4.15.0-159 Updated worker node images and kernel with package updates [CVE-2021-3778](https://nvd.nist.gov/vuln/detail/CVE-2021-3778) and [CVE-2021-3796](https://nvd.nist.gov/vuln/detail/CVE-2021-3796). 







 Change log for master fix pack 1.20.11_1553, released 28 September 2021 

The following table shows the changes that are in the master fix pack patch update 1.21.1_1553. Master patch updates are applied automatically.



Changes since version 1.20.10_1550

 Component Previous Current Description
The worker nodes that exist in the same zone as this subnet are configured to receive traffic from the VPC NLB. To see subnets in all resource groups, run ibmcloud ks subnets --provider vpc-gen2 --vpc-id <vpc> --zone <zone>.

service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-node-selector
: Optional: Annotation to specify a worker node label selector. Within the same zone as the dedicated subnet for the VPC NLB, you can configure specific worker nodes to receive traffic by selecting one of the supported label selector keys. Note that you can include only one label selector in the annotation, and that the selector must be specified in the "key=value" format. If this annotation is not specified, all worker nodes in the same zone as the VPC subnet that you specified in the service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets annotation are configured to receive traffic from the VPC NLB. If specified, any dedicated: edge labels on worker nodes are ignored.
: The following keys are permitted.



* ibm-cloud.kubernetes.io/internal-ip
* ibm-cloud.kubernetes.io/machine-type
* ibm-cloud.kubernetes.io/os
* ibm-cloud.kubernetes.io/region
* ibm-cloud.kubernetes.io/subnet-id
* ibm-cloud.kubernetes.io/worker-pool-id
* ibm-cloud.kubernetes.io/worker-pool-name
* ibm-cloud.kubernetes.io/zone
* kubernetes.io/arch
* kubernetes.io/hostname
* kubernetes.io/os
* node.kubernetes.io/instance-type
* topology.kubernetes.io/region
* topology.kubernetes.io/zone
Add worker nodes to the worker pool by resizing the pool.

ibmcloud oc worker-pool resize --cluster <cluster_name_or_ID> --worker-pool <pool_name> --size-per-zone <number_of_workers_per_zone>

If you want to remove worker nodes that use the previous network metadata, change the number of workers per zone to double the previous number of workers per zone. Later in these steps, you can cordon, drain, and remove the previous worker nodes.
6. Verify that new worker nodes are created with Public IP and Private IP addresses in the output.

ibmcloud oc worker ls --cluster <cluster_name_or_ID> --worker-pool <pool_name>
7. Optional: Remove the worker nodes with the previous network metadata from the worker pool.



1. In the output of the previous step, note the ID of the worker nodes that you want to remove from the worker pool.
2. Remove the worker node.

ibmcloud oc worker rm --cluster <cluster_name_or_ID> --worker <worker_name_or_ID>
3. Verify that the worker node is removed.

ibmcloud oc worker ls --cluster <cluster_name_or_ID> --worker-pool <pool_name>
4. Rebalance the worker pool.

ibmcloud oc worker-pool rebalance --cluster <cluster_name_or_ID> --worker-pool <pool_name>



8. Optional: You can repeat steps 2 - 7 for each worker pool in your cluster. After you complete these steps, all worker nodes in your cluster are set up with the new VLANs.
9. Move networking services to the new VLANs. The networking services in your cluster are still bound to the old VLAN because their IP addresses are from a subnet on that VLAN.
oc apply -f enable-ssh.yaml
5. Use the private or public network to access the worker node by using your SSH key.





 Private network 

Create a new or choose an existing server instance that has access to the same private network as the worker node. For VPC clusters, the [virtual server instance](https://cloud.ibm.com/vpc-ext/compute/vs) must exist in the same VPC as the worker node.

For classic clusters, the [device](https://cloud.ibm.com/gen1/infrastructure/devices) can access the worker node from any private VLAN if a [Virtual Router Function (VRF)](https://cloud.ibm.com/docs/account?topic=account-vrf-service-endpointvrf) or [VLAN spanning](https://cloud.ibm.com/docs/vlans?topic=vlans-vlan-spanningvlan-spanning) is enabled. Otherwise, the device must exist on the same private VLAN as the worker node.



1. Copy your SSH private key from step 1 from your local machine to this server instance.

scp <SSH_private_key_location> <user@host>:/.ssh/id_rsa_worker_private
2. SSH into the server instance.
3. Set the correct permissions for using the SSH private key that you copied.

chmod 400 /.ssh/id_rsa_worker_private
4. Use the private key to SSH into the worker node that you found in step 2.

ssh -i /.ssh/id_rsa_worker_private root@<WORKER_PRIVATE_IP>







 Public network classic clusters that are connected to a public VLAN only 

Debug classic clusters that are connected to a public VLAN by logging in to your worker nodes.



1.
* [IBM-managed master to customer-provided worker nodes for the Satellite location control plane](https://cloud.ibm.com/docs/satellite?topic=satellite-host-latency-testhost-latency-test-master-worker)
* [Customer-provided worker nodes in the Satellite location control plane to worker nodes that run Satellite-enabled IBM Cloud services, such as Red Hat OpenShift clusters in the same location](https://cloud.ibm.com/docs/satellite?topic=satellite-host-latency-testhost-latency-test-woker-worker)
* [Customer-provided worker nodes that are assigned to the same resource, such as the Satellite location control plane or a cluster](https://cloud.ibm.com/docs/satellite?topic=satellite-host-latency-testhost-latency-test-customer-provided)
* [Testing the latency between IBM Cloud and the Satellite location control plane hosts](https://cloud.ibm.com/docs/satellite?topic=satellite-host-latency-testhost-latency-mzr)



[Checking your host setup](https://cloud.ibm.com/docs/satellite?topic=satellite-host-network-checkhost-network-check)







 Creating locations 

[Creating a Satellite location](https://cloud.ibm.com/docs/satellite?topic=satellite-locationslocations)



* [Create location overview](https://cloud.ibm.com/docs/satellite?topic=satellite-locationssatloc-prereq)
* [Automating your location setup with a Schematics template](https://cloud.ibm.com/docs/satellite?topic=satellite-locationssatloc-template)
* [Manually creating Satellite locations](https://cloud.ibm.com/docs/satellite?topic=satellite-locationslocation-create-manual)



* [Manually creating locations from the console](https://cloud.ibm.com/docs/satellite?topic=satellite-locationslocation-create-console)
For example, to verify that the app pod deployed to a worker node on a specific VLAN, view the VLAN that the worker node is on by running ibmcloud ks worker get --cluster <cluster_name_or_ID> --worker <worker_ID>.
4. In the output, verify that the worker node with the private IP address that you identified in the previous step is deployed in this worker pool.









 Deploying an app on a GPU machine 

If you have a [GPU machine type](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodesplanning_worker_nodes), you can accelerate the processing time required for compute intensive workloads such as AI, machine learning, inferencing and more.

IBM Cloud VPC worker nodes with GPUs are available for allowlisted accounts only. To request that your account be allowlisted, see [Requesting access to allowlisted features](https://cloud.ibm.com/docs/openshift?topic=openshift-get-help). Be sure to include the data centers, the VPC infrastructure profile, and the number of workers that you want use. For example 12 worker nodes in us-east-1 of VPC profile gx2-16x128xv100.

In the following steps, you learn how to deploy workloads that require the GPU. You can also deploy apps that don't need to process their workloads across both the GPU and CPU.

In the following steps, you learn how to deploy workloads that require the GPU. You can also deploy apps that don't need to process their workloads across both the GPU and CPU. After, you might find it useful to play around with mathematically intensive workloads such as the [TensorFlow](https://www.tensorflow.org/) machine learning framework with [this Kubernetes demo](https://github.com/pachyderm/pachyderm/tree/master/examples/ml/tensorflow).

Before you begin