vania-janet/multiturn-rag-retrieval-data huggingface dataset:
POST/v2/rebalanceWorkerPool Rebalance workers in a worker pool. containers-kubernetes.cluster.operate containers-kubernetes.account.update 
 POST/v2/removeWorker Delete a worker node from a cluster. containers-kubernetes.cluster.operate containers-kubernetes.account.delete 
 POST/v2/removeWorkerPool Removes a worker pool. containers-kubernetes.cluster.operate containers-kubernetes.account.delete 
 POST/v2/replaceWorker Replace a worker node with a new worker node. containers-kubernetes.cluster.operate containers-kubernetes.account.update 
 POST/v2/resizeWorkerPool Resize an existing worker pool. containers-kubernetes.cluster.operate containers-kubernetes.workerpool.update 
 POST/v2/setWorkerPoolLabels Set custom labels for a worker pool. containers-kubernetes.cluster.operate containers-kubernetes.account.update 
 POST/v2/setWorkerPoolTaints Set custom taints for a worker pool. containers-kubernetes.cluster.operate containers-kubernetes.account.update 
 POST/v2/vpc/createWorkerPool Create a worker pool for a VPC cluster. containers-kubernetes.cluster.operate containers-kubernetes.account.create 
 POST/v2/vpc/createWorkerPoolZone Create a zone in the specified worker pool for a VPC cluster. containers-kubernetes.cluster.operate containers-kubernetes.account.create 
 POST/v2/vpc/replaceWorker Replace a worker node with a new worker node. containers-kubernetes.cluster.operate containers-kubernetes.account.create 
 PUT/v1/clusters/{idOrName}/workers/{workerId} Reboot, reload, or update a worker node for a cluster. containers-kubernetes.cluster.operate containers-kubernetes.worker.update
Instead, apply [labels](https://cloud.ibm.com/docs/containers?topic=containers-add_workersworker_pool_labels) or [taints](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cliworker_pool_taint) at the worker pool level so that the replacement worker node gets these attributes.
* Automatic rebalancing: A replacement worker node is not created if the worker pool does not have [automatic rebalancing enabled](https://cloud.ibm.com/docs/containers?topic=containers-auto-rebalance-off).



Before you begin, make sure that your cluster has enough other worker nodes so that your pods can be rescheduled and continue to run.



1. List all worker nodes in your cluster and note the name of the worker node that you want to replace.

kubectl get nodes

The name that is returned in this command is the private IP address that is assigned to your worker node. You can find more information about your worker node when you run the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command and look for the worker node with the same Private IP address.
2. Replace the worker node. As part of the replace process, the pods that run on the worker node are drained and rescheduled onto remaining worker nodes in the cluster. The worker node is also cordoned, or marked as unavailable for future pod scheduling. Use the worker node ID that is returned from the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command.

ibmcloud ks worker replace --cluster <cluster_name_or_ID> --worker <worker_node_ID>
3. Verify that the worker node is replaced.

ibmcloud ks worker ls --cluster <cluster_name_or_ID>



ibmcloud ks worker replace --cluster CLUSTER_NAME_OR_ID --worker WORKER_ID [--update] [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service.
RHEL 7 Packages Worker 3.10.0-1127.el7 3.10.0-1127.8.2.el7 Updated worker node images with kernel package updates for [CVE-2017-18595](https://nvd.nist.gov/vuln/detail/CVE-2017-18595), [CVE-2019-19768](https://nvd.nist.gov/vuln/detail/CVE-2019-19768), and [CVE-2020-10711](https://nvd.nist.gov/vuln/detail/CVE-2020-10711). 







 Change log for worker node fix pack 3.11.216_1550_openshift, released 11 May 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.216_1550_openshift. Worker node patch updates can be applied by updating or reloading the worker node



Changes since version 3.11.200_1549_openshift

 Component Previous Current Description 

 Red Hat OpenShift node 3.11.200 3.11.216 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/3.11/release_notes/ocp_3_11_release_notes.htmlocp-3-11-216). 







 Change log for worker node fix pack 3.11.200_1549_openshift, released 27 April 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.200_1549_openshift. Worker node patch updates can be applied by updating or reloading the worker node



Changes since version 3.11.200_1546_openshift

 Component Previous Current Description 

 HA proxy 1.8.25-30b675 1.8.25-adb65d Update addresses [CVE-2020-1967](https://nvd.nist.gov/vuln/detail/CVE-2020-1967).
ibmcloud ks worker reboot --cluster <cluster_name_or_ID> --worker <worker_name_or_ID>
4. Wait about 5 minutes before you make your worker node available for pod scheduling to ensure that the reboot is finished. During the reboot, the state of your worker node does not change. The reboot of a worker node is usually completed in a few seconds.
5. Make your worker node available for pod scheduling. Use the name for your worker node that is returned from the kubectl get nodes command.

kubectl uncordon <worker_name>



ibmcloud ks worker reboot [--hard] --cluster CLUSTER --worker WORKER_ID [--skip-master-healthcheck] [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--hard
: Optional: Use this option to force a hard restart of a worker node by cutting off power to the worker node. Use this option if the worker node is unresponsive or the worker node's container runtime is unresponsive.

-w, --worker WORKER
: Specify a worker node ID. To reload multiple worker nodes, use multiple options, such as -w worker1_id -w worker2_id.

--skip-master-healthcheck
: Skip a health check of your master before reloading or rebooting your worker nodes.

-f
: Optional: Force the command to run with no user prompts.

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker reboot command
ibmcloud oc worker reboot --cluster <cluster_name_or_ID> --worker <worker_name_or_ID>
4. Wait about 5 minutes before you make your worker node available for pod scheduling to ensure that the reboot is finished. During the reboot, the state of your worker node does not change. The reboot of a worker node is usually completed in a few seconds.
5. Make your worker node available for pod scheduling. Use the name for your worker node that is returned from the oc get nodes command.

oc adm uncordon <worker_name>



ibmcloud oc worker reboot [--hard] --cluster CLUSTER --worker WORKER_ID [--skip-master-healthcheck] [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--hard
: Optional: Use this option to force a hard restart of a worker node by cutting off power to the worker node. Use this option if the worker node is unresponsive or the worker node's container runtime is unresponsive.

-w, --worker WORKER
: Specify a worker node ID. To reload multiple worker nodes, use multiple options, such as -w worker1_id -w worker2_id.

--skip-master-healthcheck
: Skip a health check of your master before reloading or rebooting your worker nodes.

-f
: Optional: Force the command to run with no user prompts.

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker reboot command
Key Management Service provider v1.0.1 v1.0.3 Updated Go version for [CVE-2020-16845](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-16845) and [CVE-2020-24553](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-24553). 
 Red Hat OpenShift 3.11.248 3.11.272 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/3.11/release_notes/ocp_3_11_release_notes.htmlocp-3-11-272). 







 Change log for worker node fix pack 3.11.272_1566_openshift, released 14 September 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.272_1566_openshift. Worker node patch updates can be applied by updating or reloading the worker node



Changes since version 3.11.272_1565_openshift

 Component Previous Current Description 

 Master proxy 1.8.25-384f42 1.8.26-561f1a See the [HA proxy change log](https://www.haproxy.org/download/1.8/src/CHANGELOG). 
 RHEL 7 packages N/A N/A Updated worker node image with package updates. 







 Change log for worker node fix pack 3.11.272_1565_openshift, released 31 August 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.272_1565_openshift. Worker node patch updates can be applied by updating or reloading the worker node



Changes since version 3.11.248_1564_openshift

 Component Previous Current Description 

 RHEL 7 packages 3.10.0-1127.18.2.el7 3.10.0-1127.19.1.el7 Updated worker node image with kernel and package updates.
For more information, see the [IBM Cloud blog](https://www.ibm.com/cloud/blog/announcements/new-bare-metal-servers-available-for-kubernetes-and-openshift-clusters). 
 Pod instances You can run 110 pods per worker node. If you have worker nodes with 11 CPU cores or more, you can support 10 pods per core, up to a limit of 250 pods per worker node. The number of pods includes kube-system and ibm-system pods that run on the worker node. For improved performance, consider limiting the number of pods that you run per compute core so that you don't overuse the worker node. For example, on a worker node with a b3c.4x16 flavor, you might run 10 pods per core that use no more than 75% of the worker node total capacity. 
 Worker node quota You can't exceed 500 worker nodes across all clusters in a region, per [infrastructure provider](https://cloud.ibm.com/docs/containers?topic=containers-infrastructure_providers). If you need more of the resource, [contact IBM Support](https://cloud.ibm.com/docs/get-support?topic=get-support-using-avatar). In the support case, include the new quota limit for the region and infrastructure provider that you want. 
 Worker pool size You must always have a minimum of 1 worker node in your cluster and each worker pool. For more information, see [What is the smallest size cluster that I can make?](https://cloud.ibm.com/docs/containers?topic=containers-faqssmallest_cluster). You can't scale worker pools down to zero. Because of the worker node quota, you are limited in the number of worker pools per cluster and number of worker nodes per worker pool. For example, with the default worker node quota of 500 per region, you might have up to 500 worker pools of 1 worker node each in a region with only 1 cluster. Or, you might have 1 worker pool with up to 500 worker nodes in a region with only 1 cluster.
If the issue persists, see [Debugging your worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_worker_nodesworker-debug-steps) for additional steps.



1. [Update your cluster and worker nodes to the latest version](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate).
2. [Update your command line tools](https://cloud.ibm.com/docs/openshift?topic=openshift-cli-update).
3. [Run tests in the Diagnostics and Debug Tool add-on](https://cloud.ibm.com/docs/openshift?topic=openshift-debug-tool).







 Debugging your worker node 



 Step 1: Get the worker node state 

If your cluster is in a Critical, Delete failed, or Warning state, or is stuck in the Pending state for a long time, review the state of your worker nodes.

ibmcloud oc worker ls --cluster <cluster_name_or_id>





 Step 2: Review the worker node state 

Review the State and Status field for every worker node in your CLI output.

For more information, see [Worker node states](https://cloud.ibm.com/docs/openshift?topic=openshift-worker-node-state-reference).





 Step 3: Get the details for each worker node 

Get the details for the worker node. If the details include an error message, review the list of [common error messages for worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-common_worker_nodes_issues) to learn how to resolve the problem.

ibmcloud oc worker get --cluster <cluster_name_or_id> --worker <worker_node_id>





 Step 4: Review the infrastructure provider for the worker node 

Review the infrastructure environment to check for other reasons that might cause the worker node issues.



1.
Change log for worker node fix pack 3.11.306_1572_openshift, released 9 November 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.306_1572_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 3.11.306_1571_openshift

 Component Previous Current Description 

 RHEL 7 Packages N/A N/A Updated worker node image with package updates for [CVE-2020-15999](https://nvd.nist.gov/vuln/detail/CVE-2020-15999). 







 Change log for worker node fix pack 3.11.306_1571_openshift, released 26 October 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.306_1571_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 3.11.286_1570_openshift

 Component Previous Current Description 

 Red Hat OpenShift node 3.11.286 3.11.306 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/3.11/release_notes/ocp_3_11_release_notes.htmlocp-3-11-306). 
 RHEL 7 Packages 3.10.0-1160.2.1.el7 3.10.0-1160.2.2.el7 Updated worker node images with kernel and package updates for [CVE-2020-12351](https://nvd.nist.gov/vuln/detail/CVE-2020-12351) and [CVE-2020-12352](https://nvd.nist.gov/vuln/detail/CVE-2020-12352). 







 Change log for master fix pack 3.11.286_1571_openshift, released 26 October 2020
Change log for worker node fix pack 4.9.38_1543_openshift, released 20 June 2022 

The following table shows the changes that are in the worker node fix pack 4.9.38_1543_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.9.36_1541_openshift

 Component Previous Current Description 

 RHEL 7 Packages N/A N/A Worker node package updates for [CVE-2022-1271](https://nvd.nist.gov/vuln/detail/CVE-2022-1271). 
 HAProxy 468c09 04f862 [CVE-2022-1271](https://nvd.nist.gov/vuln/detail/CVE-2022-1271). 
 Red Hat OpenShift on IBM Cloud. 4.9.36 4.9.38 For more information, see the [change log](https://docs.openshift.com/container-platform/4.9/release_notes/ocp-4-9-release-notes.htmlocp-4-9-38). 







 Change log for worker node fix pack 4.9.36_1541_openshift, released 7 June 2022 

The following table shows the changes that are in the worker node fix pack 4.9.36_1541_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.9.33_1539_openshift

 Component Previous Current Description 

 RHEL 7 Packages N/A N/A Worker node package updates for [CVE-2022-24903](https://nvd.nist.gov/vuln/detail/CVE-2022-24903). 
 Red Hat OpenShift on IBM Cloud.
Change log for worker node fix pack 4.8.57_1597_openshift, released 27 March 2023 

The following table shows the changes that are in the worker node fix pack 4.8.57_1597_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.8.57_1594_openshift

 Component Previous Current Description 

 RHEL 7 Packages N/A N/A N/A 
 Red Hat OpenShift on IBM Cloud. N/A N/A N/A 
 Haproxy af5031 8398d1 [CVE-2023-23916](https://nvd.nist.gov/vuln/detail/CVE-2023-23916). 







 Change log for worker node fix pack 4.8.57_1594_openshift, released 13 March 2023 

The following table shows the changes that are in the worker node fix pack 4.8.57_1594_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.8.57_1590_openshift

 Component Previous Current Description
Change log for worker node fix pack 4.4.31_1526_openshift, released 7 December 2020 

The following table shows the changes that are in the worker node fix pack update 4.4.31_1526_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.4.30_1525_openshift

 Component Previous Current Description 

 HAProxy 1.8.26-384f42 db4e6d Added provenance labels for source tracking. 
 Red Hat OpenShift 4.4.30 4.4.31 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/4.4/release_notes/ocp-4-4-release-notes.htmlocp-4-4-31). 
 RHEL 7 Packages N/A N/A Updated worker node image with package updates. 







 Change log for worker node fix pack 4.4.30_1525_openshift, released 23 November 2020 

The following table shows the changes that are in the worker node fix pack update 4.4.30_1525_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.4.29_1524_openshift

 Component Previous Current Description 

 Ephemeral storage reservations N/A N/A Local ephemeral storage is reserved on the Kubernetes data disk for system components. For more information, see [Worker node resource reserves](https://cloud.ibm.com/docs/openshift?topic=openshift-planning_worker_nodesresource_limit_node). 
 Red Hat OpenShift node 4.4.29 4.4.30 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/4.4/release_notes/ocp-4-4-release-notes.htmlocp-4-4-30).
IBM Cloud File Storage for Classic plug-in and monitor 376 377 Fixed a bug that prevents persistent volume claim (PVC) creation failures from being retried. 
 Key Management Service provider v1.0.0 v1.0.1 Updated image for [CVE-2020-15586](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15586). 
 Red Hat OpenShift 3.11.232 3.11.248 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/3.11/release_notes/ocp_3_11_release_notes.htmlocp-3-11-248). 







 Change log for worker node fix pack 3.11.248_1564_openshift, released 17 August 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.248_1564_openshift. Worker node patch updates can be applied by updating or reloading the worker node



Changes since version 3.11.248_1561_openshift

 Component Previous Current Description 

 RHEL 7 packages N/A N/A Updated worker node images with package updates. 







 Change log for worker node fix pack 3.11.248_1561_openshift, released 3 August 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.248_1561_openshift. Worker node patch updates can be applied by updating or reloading the worker node



Changes since version 3.11.232_1558_openshift

 Component Previous Current Description 

 Red Hat OpenShift node 3.11.232 3.11.248 See the [Red Hat OpenShift release notes](https://docs.openshift.com/container-platform/3.11/release_notes/ocp_3_11_release_notes.htmlocp-3-11-248).
[Why can't I create a new admin certificate?](https://cloud.ibm.com/docs/containers?topic=containers-ts_admin-certts_admin-cert)





 Worker nodes 

[Debugging worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesdebug_worker_nodes)



* [Check worker node notifications and maintenance updates](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-notifs)
* [Quick steps to resolve worker node issues](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-quick)
* [Debugging your worker node](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-steps)



* [Step 1: Get the worker node state](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-get-state)
* [Step 2: Review the worker node state](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-rev-state)
* [Step 3: Get the details for each worker node](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-get-details)
* [Step 4: Review the infrastructure provider for the worker node](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-rev-infra)





[Debugging worker nodes with Kubernetes API](https://cloud.ibm.com/docs/containers?topic=containers-debug-kube-nodesdebug-kube-nodes)
cpu_used_percent Greater than 65% for 24 hours. 
 Memory utilization of the worker node over threshold. memory_used_percent Greater than 80 % for 1 hour. 
 Memory utilization of the worker node over threshold. memory_used_percent Greater than 65% for 24 hours. 
 Amount of memory used over threshold. memory_bytes_used Greater than NUMBER_OF_BYBTES. 
 Nodes with disk pressure exist. kube_node_status_condition Greater than or equal to 1 for 10 minutes. 
 Kubernetes nodes not ready exist. kube_node_status_ready >= 1 





 Resolving worker node alerts 

Reloading or rebooting the worker can resolve the issue. However, you might need add more workers to increase capacity.



1. Get your worker nodes and review the [state](https://cloud.ibm.com/docs/openshift?topic=openshift-worker-node-state-reference).

kubectl get nodes
2. If all the worker nodes are not in the Ready state, [add worker nodes to your cluster](https://cloud.ibm.com/docs/containers?topic=containers-add_workers).
3. If all the worker nodes are in the Ready state, [reload](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload) or [reboot](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reboot) your worker nodes.



1. Describe your worker node and review the Events section for common error messages.

kubectl describe node <node>
2. Cordon the node that isn't Ready so that you can start investigating.
3. Drain the worker node. Review the [Kubernetes documentation to safely drain pods from your worker node](https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/).

kubectl drain <node>
Change log for worker node fix pack 4.12.21_1547_openshift, released 19 June 2023 

The following table shows the changes that are in the worker node fix pack 4.12.21_1547_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.12.19_1546_openshift

 Component Previous Current Description 

 Red Hat OpenShift on IBM Cloud. 4.12.19 4.12.21 see [change logs](https://docs.openshift.com/container-platform/4.12/release_notes/ocp-4-12-release-notes.htmlocp-4-12-21). 
 RHEL 8 Packages N/A N/A Worker node package updates for [CVE-2023-32067](https://nvd.nist.gov/vuln/detail/CVE-2023-32067),[CVE-2023-24329](https://nvd.nist.gov/vuln/detail/CVE-2023-24329). 







 Change log for worker node fix pack 4.12.19_1546_openshift, released 5 June 2023 

The following table shows the changes that are in the worker node fix pack 4.12.19_1546_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.12.16_1544_openshift

 Component Previous Current Description 

 Red Hat OpenShift on IBM Cloud. 4.12.16 4.12.19 see [change logs](https://docs.openshift.com/container-platform/4.12/release_notes/ocp-4-12-release-notes.htmlocp-4-12-19). 
 RHEL 7 Packages N/A N/A N/A