vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Change log for worker node fix pack 1.16.10_1534, released 8 June 2020 

The following table shows the changes that are in the worker node fix pack 1.16.10_1534. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.16.10_1533

 Component Previous Current Description 

 Ubuntu 18.04 packages N/A N/A Updated worker node images with package updates for [CVE-2019-1547](https://nvd.nist.gov/vuln/detail/CVE-2019-1547), [CVE-2019-1549](https://nvd.nist.gov/vuln/detail/CVE-2019-1549), [CVE-2019-1551](https://nvd.nist.gov/vuln/detail/CVE-2019-1551), [CVE-2019-1563](https://nvd.nist.gov/vuln/detail/CVE-2019-1563), and [CVE-2020-12762](https://nvd.nist.gov/vuln/detail/CVE-2020-12762). 
 Ubuntu 16.04 packages 4.4.0-178-generic 4.4.0-179-generic Updated worker node images with package and kernel updates for [CVE-2019-1547](https://nvd.nist.gov/vuln/detail/CVE-2019-1547), [CVE-2019-1551](https://nvd.nist.gov/vuln/detail/CVE-2019-1551), [CVE-2019-1563](https://nvd.nist.gov/vuln/detail/CVE-2019-1563), and [CVE-2020-12762](https://nvd.nist.gov/vuln/detail/CVE-2020-12762). 







 Change log for 1.16.10_1533, released 26 May 2020
If the affected worker nodes are all from the same zone, subnet, or VLAN, continue to the next section.



1. Get the details of the specific node.

kubectl describe node <node-IP-address>
2. In the output, check the Conditions section to determine if the node is experiencing any memory, disk or PID issues. This information might indicate that the node is running low on that type of resource. This situation might occur for one of the following reasons:



* Memory or CPU exhaustion caused by a lack of proper requests and limits on your pods.
* Worker disks are full, sometimes because of large pod logs or pod output to the node itself.
* Slow memory leaks that build up over time, which can cause problems for workers that have not been updated in over a month.
* Bugs and crashes that affect the Linux kernel.



3. If you are able to determine the cause of the issue from the information in the Conditions section, follow the steps in [Debugging worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-debug-kube-nodes) to isolate the problem workload.
4. If the previous steps do not solve the issue, [reload](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli&interface=uics_worker_reload) or [replace](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli&interface=uicli_worker_replace) the affected workers one at a time.



If some, but not all, of your worker nodes frequently enter a Critical or NotReady state, consider enabling [worker autorecovery](https://cloud.ibm.com/docs/containers?topic=containers-health-monitorautorecovery) to automate these recovery steps.





 If all worker nodes in a single zone, subnet, or VLAN are affected
* Image and version updates: Worker node updates, such as security patches to the image or Red Hat OpenShift versions, are provided by IBM for you. However, you choose when to apply the updates to the worker nodes. For more information, see [Updating clusters, worker nodes, and cluster components](https://cloud.ibm.com/docs/containers?topic=containers-update).
* Temporary modifications: If you log in to a pod or use some other process to modify a worker node setting, the modifications are temporary. Worker node lifecycle operations, such as autorecovery, reloading, updating, or replacing a worker node, change any modifications back to the default settings.
* Persistent modifications: For modifications to persist across worker node lifecycle operations, create a daemon set that uses an init container. For more information, see [Modifying default worker node settings to optimize performance](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelworker).



Modifications to the operating system are not supported. If you modify the default settings, you are responsible for debugging and resolving the issues that might occur.





 Hardware changes 

To change the compute hardware, such as the CPU and memory per worker node, choose among the following options.



* [Create a worker pool](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workers). The instructions vary depending on the type of infrastructure for the cluster, such as classic, VPC, Satellite, or gateway clusters.
* [Update the flavor](https://cloud.ibm.com/docs/containers?topic=containers-updatemachine_type) in your cluster by creating a worker pool and removing the previous worker pool.









 Modifying worker node settings to optimize performance 



 Modifying worker node settings by using the Node Tuning Operator 

You can use the node tuning operator to tune worker node performance by creating custom profiles. For more information, see the Red Hat [Node Tuning Operator](https://docs.openshift.com/container-platform/4.7/scalability_and_performance/using-node-tuning-operator.html) docs.
: The label value that the worker node must have to be considered for the rule that you define.
5. Create the ConfigMap in your cluster.

kubectl apply -f <filepath/configmap.yaml>
6. Verify that the ConfigMap is created.

kubectl get configmap --namespace kube-system
7. Update the worker nodes by listing them by ID.

ibmcloud ks worker update --cluster <cluster_name_or_ID> --worker <worker_node1_ID> --worker <worker_node2_ID>
8. Optional: Verify the events that are triggered by the ConfigMap and any validation errors that occur. The events can be reviewed in the Events section of your CLI output.

kubectl describe -n kube-system cm ibm-cluster-update-configuration
9. Confirm that the update is complete by reviewing the Kubernetes version of your worker nodes.

kubectl get nodes

If the update failed, you must [apply version updates by replacing hosts](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workershost-update-workers-minor).
10. Verify that you don't have duplicate worker nodes. Sometimes, older clusters might list duplicate worker nodes with a NotReady status after an update. To remove duplicates, see [troubleshooting](https://cloud.ibm.com/docs/containers?topic=containers-cs_duplicate_nodes).









 Applying version updates to worker nodes by replacing hosts 

Hosts that are attached to a location do not update automatically. To apply a version update, you can first attach and assign new hosts to your [Satellite-enabled IBM Cloud service](https://cloud.ibm.com/docs/satellite?topic=satellite-managed-services) and then remove the old hosts.
Change log for worker node fix pack 1.21.4_1530, released 13 September 2021 

The following table shows the changes that are in the worker node fix pack patch update 1.21.4_1530. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.21.2_1529

 Component Previous Current Description 

 Ubuntu 18.04 packages 4.15.0-154 4.15.0-156 Updated worker node images and kernel with package updates for [CVE-2021-3653](https://nvd.nist.gov/vuln/detail/CVE-2021-3653), [CVE-2021-3656](https://nvd.nist.gov/vuln/detail/CVE-2021-3656), [CVE-2021-38185](https://nvd.nist.gov/vuln/detail/CVE-2021-38185) , and [CVE-2021-40153](https://nvd.nist.gov/vuln/detail/CVE-2021-40153). 







 Change log for worker node fix pack 1.21.4_1529, released 30 August 2021 

The following table shows the changes that are in the worker node fix pack patch update 1.21.4_1529. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.21.2_1524

 Component Previous Current Description 

 Ubuntu 18.04 packages 4.15.0-153 4.15.0-154 Updated worker node images and kernel with package updates for [CVE-2021-3711](https://nvd.nist.gov/vuln/detail/CVE-2021-3711) and [CVE-2021-3712](https://nvd.nist.gov/vuln/detail/CVE-2021-3712). 







 Change log for master fix pack 1.21.4_1528, released 25 August 2021
Change log for worker node fix pack 1.16.11_1537, released 6 July 2020 

The following table shows the changes that are in the worker node fix pack 1.16.11_1537. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.16.11_1536

 Component Previous Current Description 

 HAProxy 1.8.25-30b675 2.0.15-afe432 See the [HAProxy change log](https://www.haproxy.org/download/2.0/src/CHANGELOG). 
 Ubuntu 18.04 packages 4.15.0-106-generic 4.15.0-109-generic Updated worker node images with kernel and package updates for [CVE-2019-12380](https://nvd.nist.gov/vuln/detail/CVE-2019-12380), [CVE-2019-16089](https://nvd.nist.gov/vuln/detail/CVE-2019-16089), [CVE-2019-19036](https://nvd.nist.gov/vuln/detail/CVE-2019-19036), [CVE-2019-19039](https://nvd.nist.gov/vuln/detail/CVE-2019-19039), [CVE-2019-19318](https://nvd.nist.gov/vuln/detail/CVE-2019-19318), [CVE-2019-19642
Change log for worker node fix pack 1.22.15_1577, released 07 November 2022 

The following table shows the changes that are in the worker node fix pack 1.22.15_1577. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.22.15_1575

 Component Previous Current Description 

 Ubuntu 18.04 packages N/A N/A Worker node package updates for [CVE-2022-32221](https://nvd.nist.gov/vuln/detail/CVE-2022-32221), [CVE-2022-40284](https://nvd.nist.gov/vuln/detail/CVE-2022-40284), [CVE-2022-42010](https://nvd.nist.gov/vuln/detail/CVE-2022-42010), [CVE-2022-42011](https://nvd.nist.gov/vuln/detail/CVE-2022-42011), [CVE-2022-42012](https://nvd.nist.gov/vuln/detail/CVE-2022-42012). 
 Kubernetes N/A N/A N/A
Kubernetes N/A N/A N/A 







 Change log for worker node fix pack 1.22.17_1583, released 19 December 2022 

The following table shows the changes that are in the worker node fix pack 1.22.17_1583. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.22.16_1580

 Component Previous Current Description 

 Containerd 1.5.14 1.5.16 See the [1.5.16 change log](https://github.com/containerd/containerd/releases/tag/v1.5.16) and the [1.5.15 change log](https://github.com/containerd/containerd/releases/tag/v1.5.15), the security bulletin for [CVE-2022-23471](https://www.ibm.com/support/pages/node/6850799).
If the affected worker nodes are all from the same zone, subnet, or VLAN, continue to the next section.



1. Get the details of the specific node.

oc describe node <node-IP-address>
2. In the output, check the Conditions section to determine if the node is experiencing any memory, disk or PID issues. This information might indicate that the node is running low on that type of resource. This situation might occur for one of the following reasons:



* Memory or CPU exhaustion caused by a lack of proper requests and limits on your pods.
* Worker disks are full, sometimes because of large pod logs or pod output to the node itself.
* Slow memory leaks that build up over time, which can cause problems for workers that have not been updated in over a month.
* Bugs and crashes that affect the Linux kernel.



3. If you are able to determine the cause of the issue from the information in the Conditions section, follow the steps in [Debugging worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-debug-kube-nodes) to isolate the problem workload.
4. If the previous steps do not solve the issue, [reload](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli&interface=uics_worker_reload) or [replace](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli&interface=uicli_worker_replace) the affected workers one at a time.







 If all worker nodes in a single zone, subnet, or VLAN are affected 

If all worker nodes in a single zone, subnet, or VLAN are in a Critical or NotReady state, but all other worker nodes in the cluster are functioning normally, there might be an issue with a networking component.
Use the worker node ID that is returned from the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command.

ibmcloud oc worker reload --cluster <cluster_name_or_ID> --worker <worker_name_or_ID>
3. Wait for the reload to complete. When your worker node is in a Normal state, the worker node becomes available for pod scheduling again.



ibmcloud oc worker reload --cluster CLUSTER --worker WORKER_ID [--skip-master-healthcheck] [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

-w, --worker WORKER
: Specify a worker node ID. To reload multiple worker nodes, use multiple options, such as -w worker1_id -w worker2_id.

--skip-master-healthcheck
: Skip a health check of your master before reloading or rebooting your worker nodes.

-f
: Optional: Force the command to run with no user prompts.

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker reload command 

ibmcloud oc worker reload --cluster my_cluster -w kube-dal10-cr18a61a63a6a94b658596aa93d087aaa9-w1 -w kube-dal10-cr18a61a63a6a94b658596aa93d087aaa9-w2







 ibmcloud oc worker replace 

Virtual Private Cloud

Classic infrastructure

Delete a worker node and replace it with a new worker node in the same worker pool.

The replacement worker node is created in the same zone and has the same flavor as the old worker node, but might be assigned new public or private IP addresses.
Change log for worker node fix pack 1.24.10_1553, released 13 February 2023 

The following table shows the changes that are in the worker node fix pack 1.24.10_1553. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.24.10_1552

 Component Previous Current Description 

 Ubuntu 18.04 packages 4.15.0-202 4.15.0-204 Worker node kernel & package updates for [CVE-2022-28321](https://nvd.nist.gov/vuln/detail/CVE-2022-28321), [CVE-2022-3437](https://nvd.nist.gov/vuln/detail/CVE-2022-3437), [CVE-2022-4304](https://nvd.nist.gov/vuln/detail/CVE-2022-4304), [CVE-2022-4450](https://nvd.nist.gov/vuln/detail/CVE-2022-4450), [CVE-2022-45142](https://nvd.nist.gov/vuln/detail/CVE-2022-45142), [CVE-2022-47016](https://nvd.nist.gov/vuln/detail/CVE-2022-47016), [CVE-2023-0215](https://nvd.nist.gov/vuln/detail/CVE-2023-0215), [CVE-2023-0286](https://nvd.nist.gov/vuln/detail/CVE-2023-0286).
Although the worker nodes might be normal, other infrastructure resources, such as [networking](https://cloud.ibm.com/docs/containers?topic=containers-coredns_lameduck) and [storage](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_file), might still need attention. If you just created the cluster, some parts of the cluster that are used by other services such as Ingress secrets or registry image pull secrets, might still be in process.





 Pending 

Review the following description of the Pending cluster state. To view the state of your cluster, run ibmcloud oc cluster get --cluster <cluster_name_or_ID>.

The Kubernetes master is deployed. The worker nodes are provisioning and aren't available in the cluster yet. You can access the cluster, but you can't deploy apps to the cluster.





 Requested 

Review the following description of the Requested cluster state. To view the state of your cluster, run ibmcloud oc cluster get --cluster <cluster_name_or_ID>.

A request to create the cluster and order the infrastructure for the Kubernetes master and worker nodes is sent. When the deployment of the cluster starts, the cluster state changes to Deploying. If your cluster is stuck in the Requested state for a long time, open an [IBM Cloud support case](https://cloud.ibm.com/docs/openshift?topic=openshift-get-help).





 Updating 

Review the following description of the Updating cluster state. To view the state of your cluster, run ibmcloud oc cluster get --cluster <cluster_name_or_ID>.

The Kubernetes API server that runs in your Kubernetes master is being updated to a new Kubernetes API version. During the update, you can still access and change the cluster. However, you cannot initiate concurrent master operations, such as enabling API server auditing. Worker nodes, apps, and resources that the user deployed aren't modified and continue to run.
},
"flow_job_status": {
"updated_at": "0001-01-01T00:00:00Z"
}
},
"data": {
"job_type": "",
"workspace_job_data": {
"updated_at": "0001-01-01T00:00:00Z"
},
"action_job_data": {
"updated_at": "0001-01-01T00:00:00Z",
"inventory_record": {
"created_at": "0001-01-01T00:00:00Z",
"updated_at": "0001-01-01T00:00:00Z"
}
},
"system_job_data": {
"updated_at": "0001-01-01T00:00:00Z"
},
"flow_job_data": {
"workitems": [
{
"command_object_name": "basic-resource-group",
"source": {
"source_type": "git_hub",
"git": {
"git_repo_url": "https://github.com/Cloud-Schematics/blueprint-example-modules/tree/main/IBM-DefaultResourceGroup",
"git_branch": "main"
},
"catalog": {},
"cos_bucket": {}
},
"outputs":
{
"name": "resource_group_name",
"metadata": {}
},
{
"name": "resource_group_id",
"metadata": {}
}
],
"last_job": {},
"updated_at": "0001-01-01T00:00:00Z"
},
{
Allow worker nodes to be created in your cluster. ALL <br><br> * <br><br><br> CIDR block 161.26.0.0/16 
 Allow worker nodes to communicate with other IBM Cloud services that support private cloud service endpoints. ALL <br><br> * <br><br><br> CIDR block 166.8.0.0/14 
 Allow all worker nodes in this cluster to communicate with each other. ALL <br><br> * <br><br><br> Security group kube-<cluster_ID> 
 Allow outbound traffic to be sent to the Virtual private endpoint gateway which is used to talk to the Kubernetes master. ALL <br><br> * <br><br><br> Virtual private endpoint gateway IP addresses. The Virtual private endpoint gateway is assigned an IP address from a VPC subnet in each of the zones where your cluster has a worker node. For example, if the cluster spans 3 zones, there are up to 3 IP addresses assigned to each Virtual private endpoint gateway. To find the Virtual private endpoint gateway IPs:<br><br><br><br>1. Go to the [Virtual private cloud dashboard](https://cloud.ibm.com/vpc-ext/network/vpcs).<br>2. Click Virtual private endpoint gateways, then select the Region where your cluster is located.<br>3. Find your cluster, then click the IP addresses in the IP Address column to copy them.<br><br><br> 
 Allow the worker nodes to connect to the Ingress LoadBalancer. To find the IPs needed to apply this rule, see [Allow worker nodes to connect to the Ingress LoadBalancer](https://cloud.ibm.com/docs/containers?topic=containers-vpc-security-groupvpc-security-group-loadbalancer-outbound). TCP 443 Ingress load balancer IPs 









 Required inbound and outbound rules for VPC ALBs
Allow worker nodes to be created in your cluster. ALL <br><br> * <br><br><br> CIDR block 161.26.0.0/16 
 Allow worker nodes to communicate with other IBM Cloud services that support private cloud service endpoints. ALL <br><br> * <br><br><br> CIDR block 166.8.0.0/14 
 Allow all worker nodes in this cluster to communicate with each other. ALL <br><br> * <br><br><br> Security group kube-<cluster_ID> 
 Allow outbound traffic to be sent to the Virtual private endpoint gateway which is used to talk to the Kubernetes master. ALL <br><br> * <br><br><br> Virtual private endpoint gateway IP addresses. The Virtual private endpoint gateway is assigned an IP address from a VPC subnet in each of the zones where your cluster has a worker node. For example, if the cluster spans 3 zones, there are up to 3 IP addresses assigned to each Virtual private endpoint gateway. To find the Virtual private endpoint gateway IPs:<br><br><br><br>1. Go to the [Virtual private cloud dashboard](https://cloud.ibm.com/vpc-ext/network/vpcs).<br>2. Click Virtual private endpoint gateways, then select the Region where your cluster is located.<br>3. Find your cluster, then click the IP addresses in the IP Address column to copy them.<br><br><br> 
 Allow the worker nodes to connect to the Ingress LoadBalancer. To find the IPs needed to apply this rule, see [Allow worker nodes to connect to the Ingress LoadBalancer](https://cloud.ibm.com/docs/containers?topic=containers-vpc-security-group&interface=uivpc-security-group-loadbalancer-outbound). TCP 443 Ingress load balancer IPs 









 Required inbound and outbound rules for VPC ALBs
Change log for worker node fix pack 1.18.4_1518, released 6 July 2020 

The following table shows the changes that are in the worker node fix pack 1.18.4_1518. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.18.4_1517

 Component Previous Current Description 

 HA proxy 1.8.25-30b675 2.0.15-afe432 See the [HA proxy change log](https://www.haproxy.org/download/2.0/src/CHANGELOG). 
 Ubuntu 18.04 packages 4.15.0-106-generic 4.15.0-109-generic Updated worker node images with kernel and package updates for [CVE-2019-12380](https://nvd.nist.gov/vuln/detail/CVE-2019-12380), [CVE-2019-16089](https://nvd.nist.gov/vuln/detail/CVE-2019-16089), [CVE-2019-19036](https://nvd.nist.gov/vuln/detail/CVE-2019-19036), [CVE-2019-19039](https://nvd.nist.gov/vuln/detail/CVE-2019-19039), [CVE-2019-19318](https://nvd.nist.gov/vuln/detail/CVE-2019-19318), [CVE-2019-19642
If a match is found, check the current environment variable values and the Hadoop worker node memory resources by using following commands:



* To check the "DOCPROC_MAX_MEMORY" variable in the orchestrator container:

oc exec oc get po -l run=orchestrator -o 'jsonpath={.items[0].metadata.name}' env | grep DOCPROC_MAX_MEMORY
* To check the "YARN_NODEMANAGER_RESOURCE_MEMORY_MB" variable in the Hadoop worker node container:

oc exec oc get po -lrun=hdp-worker -o 'jsonpath={.items[0].metadata.name}' -c hdp-worker -- env | grep YARN_NODEMANAGER_RESOURCE_MEMORY_MB
* To check the memory resource of the hdp-worker container:

oc get po -l run=hdp-worker -o 'jsonpath=requests are {.items[].spec.containers[?(.name=="hdp-worker")].resources.requests.memory}, limits are {.items[].spec.containers[?(.name=="hdp-worker")].resources.limits.memory}'



7. Patch the environment variable resources gradually by using the following command:

oc patch wd oc get wd -o 'jsonpath={.items[0].metadata.name}' --type=merge --patch='{"spec":{"orchestrator":{"docproc":{"maxMemory":"4g"}}, "hdp":{"worker":{"nm":{"memoryMB":12000}, "resources":{"limits":{"memory":"20Gi"}, "requests":{"memory":"20Gi"}}}}}}'

The default values for the resources are as follows:



* docproc.maxMemory: 2g

Increase in increments of 2g at a time.
* nm.memoryMB: 10,240
Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.19.10_1546

 Component Previous Current Description 

 Containerd v1.4.5 v1.4.6 See the [containerd release notes](https://github.com/containerd/containerd/releases/tag/v1.4.6). The update resolves CVE-2021-30465 (see the [IBM security bulletin](https://www.ibm.com/support/pages/node/6456943)). 
 HA proxy e0fa2f 26c5cc Updated image with fixes for [CVE-2020-26116](https://nvd.nist.gov/vuln/detail/CVE-2020-26116), [CVE-2020-27619](https://nvd.nist.gov/vuln/detail/CVE-2020-27619), [CVE-2021-23336](https://nvd.nist.gov/vuln/detail/CVE-2021-23336), [CVE-2021-3177](https://nvd.nist.gov/vuln/detail/CVE-2021-3177), [CVE-2019-3842](https://nvd.nist.gov/vuln/detail/CVE-2019-3842), [CVE-2020-13776](https://nvd.nist.gov/vuln/detail/CVE-2020-13776), [CVE-2019-18276](https://nvd.nist.gov/vuln/detail/CVE-2019-18276), [CVE-