vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Reservation: The reservation contains details such as the container platform, worker node flavor, location, and infrastructure provider.

Contract: The example reservation has two contracts for different terms. One contract is for 10 worker nodes across 3 years. The other contract is for 5 worker nodes across 1 year. Therefore, the total capacity of the reservation is 15 worker nodes.

Clusters: Each cluster is in the same multizone region as the reservation, but can be in a different resource group. Reserved worker nodes from different reservations can be used in different clusters. A cluster can have a mix of reserved and regular, as-needed worker pools.

Worker pools: You can create multiple worker pools in multiple clusters with your reserved worker nodes. Reserved worker nodes are used for the entire worker pool; you can't mix and match between regular and reserved worker nodes in the same worker pool. Reserved worker nodes are used on a first-available basis, which means if you provision a worker pool that uses all your reserved worker nodes, you have no more reserved worker nodes remaining for other worker pools. Therefore, you might create separate reservations for each team that owns a worker pool or cluster.

Zones: Because the reservation is created in an IBM Cloud multizone region, the reserved worker nodes can be used for clusters in any of the zones. For example, reservations in Washington, DC (US East region) can be used to create classic worker nodes in wdc04, wdc06, or wdc07.

Reserved worker nodes that are used in the worker pools: In this scenario, the development and production environments share a reservation that has a mix of contracts that are used for a mix of worker pools and teams. The reservation still has one unused, 3-year term reserved worker node that can be used for other worker pools or to scale up the existing worker pools. If the production environment needs more than one more worker node, you might resize down some worker pools in development, create a worker pool that uses on demand worker nodes, or add more contracts to the reservation. If you are concerned about a development environment using up your production environment resources, consider creating separate reservations for the different environments.



 Reservation usage and lifecycle 

How can I use my reservation?
: You can use your reservation to create worker pools in new or existing clusters.
Ubuntu 16.04 kernel and packages 4.4.0-161-generic 4.4.0-164-generic Updated worker node images with kernel and package updates for [CVE-2019-14835)](https://nvd.nist.gov/vuln/detail/CVE-2019-14835). 







 Change log for worker node fix pack 1.15.3_1517, released 16 September 2019 

The following table shows the changes that are in the worker node fix pack 1.15.3_1517.



Table 1. Changes since version 1.15.3_1516

 Component Previous Current Description 

 containerd v1.2.8 v1.2.9 See the [containerd release notes](https://github.com/containerd/containerd/releases/tag/v1.2.9). Update resolves [CVE-2019-9515](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9515).
HAProxy 6514a2 c1634f [CVE-2022-32206](https://nvd.nist.gov/vuln/detail/CVE-2022-32206), [CVE-2022-32208](https://nvd.nist.gov/vuln/detail/CVE-2022-32208) 







 Change log for worker node fix pack 4.10.26_1530_openshift, released 16 August 2022 

The following table shows the changes that are in the worker node fix pack 4.10.26_1530_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.10.24_1529_openshift

 Component Previous Current Description
: Worker node patch updates are available for Kubernetes [1.15.4_1519](https://cloud.ibm.com/docs/containers?topic=containers-115_changelog1154_1519_worker), [1.14.7_1535](https://cloud.ibm.com/docs/containers?topic=containers-114_changelog1147_1535_worker), [1.13.11_1538](https://cloud.ibm.com/docs/containers?topic=containers-113_changelog11311_1538_worker), [1.12.10_1569](https://cloud.ibm.com/docs/containers?topic=containers-112_changelog11210_1569_worker), and Red Hat OpenShift [3.11.146_1527_openshift](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_changelog_311311146_1527).





 3 October 2019 

Ingress ALB change log
: Updated the ALB [nginx-ingress image to build 579 and ingress-auth image build to 341](https://cloud.ibm.com/docs/containers?topic=containers-cluster-add-ons-changelogkube_ingress_changelog).

DevOps toolchain
: You can now use the DevOps tab on the cluster details page to configure your DevOps toolchain. For more information, see [Setting up a continuous delivery pipeline for a cluster](https://cloud.ibm.com/docs/containers?topic=containers-cicdcontinuous-delivery-pipeline).

Security for VPC clusters
: Added information for how to achieve [network segmentation and privacy in VPC clusters](https://cloud.ibm.com/docs/containers?topic=containers-securitynetwork_segmentation_vpc).





 1 October 2019 

End of service of IBM Log Analysis and IBM Cloud Monitoring
Updated image for [CVE-2021-33194](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-33194). 







 Change log for worker node fix pack 1.17.17_1566, released 22 June 2021 

The following table shows the changes that are in the worker node fix pack 1.17.17_1566. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.17.17_1565

 Component Previous Current Description 

 HA proxy 26c5cc d3dc33 Updated image with fixes for [CVE-2020-24977](https://nvd.nist.gov/vuln/detail/CVE-2020-24977), [CVE-2020-13434](https://nvd.nist.gov/vuln/detail/CVE-2020-13434), [CVE-2020-15358](https://nvd.nist.gov/vuln/detail/CVE-2020-15358), [CVE-2020-29361](https://nvd.nist.gov/vuln/detail/CVE-2020-29361), [CVE-2020-29362](https://nvd.nist.gov/vuln/detail/CVE-2020-29362), [CVE-2020-29363](https://nvd.nist.gov/vuln/detail/CVE-2020-29363), [CVE-2019-2708](https://nvd.nist.gov/vuln/detail/CVE-2019-2708), [CVE-2019-13012
ibmcloud oc worker reload --cluster my_cluster -w kube-dal10-cr18a61a63a6a94b658596aa93d087aaa9-w1 -w kube-dal10-cr18a61a63a6a94b658596aa93d087aaa9-w2







 ibmcloud oc worker replace 

Virtual Private Cloud

Classic infrastructure

Delete a worker node and replace it with a new worker node in the same worker pool.

The replacement worker node is created in the same zone and has the same flavor as the old worker node, but might be assigned new public or private IP addresses. You might replace a worker node if you can't reload or update the worker node, such as if it enters a troubled state.

You can also use this command to update the Kubernetes version of the worker node to match the major and minor version of the Kubernetes master by including the --update option. If you don't include the --update option, patch version updates are applied to your worker node, but not major or minor updates. To see the changes from one major, minor, or patch version to the next, review the [Version change log](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_versions) documentation. Remember that your worker nodes can be only up to two versions behind the master version (n-2).

When you replace a worker node, keep in mind the following considerations.

To update Satellite worker nodes, see [Updating hosts that are assigned as worker nodes to Satellite-enabled services](https://cloud.ibm.com/docs/satellite?topic=satellite-host-update-workers).



* Multiple worker nodes are replaced concurrently: If you replace multiple worker nodes at the same time, they are deleted and replaced concurrently, not one by one. Make sure that you have enough capacity in your cluster to reschedule your workloads before you replace worker nodes.
* Worker node subnet CIDRs: If you anticipate changing the number of worker nodes in your cluster frequently, such as if you enable the [cluster autoscaler](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-scaling-classic-vpc), you might not want to update your allowlist for each new worker node. Instead, you can add the VPC subnets that the cluster uses. Keep in mind that the VPC subnet might be shared by worker nodes in other clusters.



1. Get the Worker Zones and VPCs that your cluster is created in.

ibmcloud oc cluster get -c <cluster>

Example output

...
Worker Zones: us-south-1, us-south-2, us-south-3
Ingress Subdomain: vpc-prod.us-south.containers.appdomain.cloud
Ingress Secret: vpc-prod
Creator: -
Public Service Endpoint URL: https://c2.us-south.containers.cloud.ibm.com:20267
Private Service Endpoint URL: https://c2.private.us-south.containers.cloud.ibm.com:20267
Pull Secrets: enabled in the default namespace
VPCs: ff537d43-a5a4-4b65-9627-17eddfa5237b
...
2. For the subnets in the zones and VPC that your cluster is in, note the Subnet CIDR.

ibmcloud is subnets

Example output

ID Name Status Subnet CIDR Addresses ACL Public Gateway VPC Zone
5f5787a4-f560-471b-b6ce-20067ac93439 vpc-prod-dal1 available 10.240.0.0/24 183/256 allow-all-network-acl-ff537d43-a5a4-4b65-9627-17eddfa5237b - prod us-south-1
Out of the box, your worker nodes are set up with an IBM-managed container runtime, separate compute resources, networking, and a volume service. The built-in security features provide isolation, resource management capabilities, and worker node security compliance.

The worker nodes and all the worker node components are dedicated only to you, and are not shared with other IBM customers. However, if you use a worker node virtual machine, the underlying hardware might be shared with other customers depending on the level of hardware isolation that you choose. For more information, see [Virtual machines](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodesvm).

Modifying default worker node components such as the kubelet is not supported and might cause unexpected results.

The following tables describe the components of a worker node.





 kube-system namespace 

ibm-master-proxy
: The ibm-master-proxy forwards requests from the worker node to the IP addresses of the highly available master replicas. In single zone clusters, the master has three replicas on separate hosts with one master IP address and domain name. For clusters that are in a multizone-capable zone, the master has three replicas that are spread across zones. As such, each master has its own IP address that is registered with DNS, with one domain name for the entire cluster master.

konnectivity-agent
: The Konnectivity agent works with the Konnectivity server to securely connect the master to the worker node. This connection supports apiserver proxy calls to your pods and services, and kubectl exec, attach, and logs calls to the kubelet.

kubelet
: The kubelet is a pod that runs on every worker node and is responsible for monitoring the health of pods that run on the worker node and for watching the events that the Kubernetes API server sends. Based on the events, the kubelet creates or removes pods, ensures liveness and readiness probes, and reports back the status of the pods to the Kubernetes API server.

coredns
Worker nodes that were already created in the pool continue to use the previous public or private VLAN, but new worker nodes in the pool use the new network data.

ibmcloud oc zone network-set --zone ZONE --cluster CLUSTER --private-vlan PRIVATE_VLAN [--worker-pool WORKER_POOL] [--public-vlan PUBLIC_VLAN] [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

--zone ZONE
: Required: The zone that you want to add. The zone must be a [multizone-capable zone](https://cloud.ibm.com/docs/openshift?topic=openshift-regions-and-zoneszones-mz) within the cluster's region.

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

-p, --worker-pool WORKER_POOL
: The name of the worker pool to add the zone to. To specify multiple worker pools, use multiple options, such as -p pool1 -p pool2.

--private-vlan PRIVATE_VLAN
: The ID of the private VLAN. This value is required, whether you want to use the same or a different private VLAN than the one that you used for your other worker nodes. New worker nodes are added to the VLAN that you specify, but the VLANs for any existing worker nodes are not changed. The private and public VLANs must be compatible, which you can determine from the Router ID prefix.

--public-vlan PUBLIC_VLAN
: The ID of the public VLAN. This value is required only if you want to change the public VLAN for the zone. To change the public VLAN, you must always provide a compatible private VLAN. New worker nodes are added to the VLAN that you specify, but the VLANs for any existing worker nodes are not changed. The private and public VLANs must be compatible, which you can determine from the Router ID prefix.

-f
Change log for worker node fix pack 1.19.14_1558, released 30 August 2021 

The following table shows the changes that are in the worker node fix pack patch update 1.19.14_1558. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.19.13_1556

 Component Previous Current Description 

 Ubuntu 18.04 packages 4.15.0-153 4.15.0-154 Updated worker node images and kernel with package updates for [CVE-2021-3711](https://nvd.nist.gov/vuln/detail/CVE-2021-3711) and [CVE-2021-3712](https://nvd.nist.gov/vuln/detail/CVE-2021-3712). 







 Change log for master fix pack 1.19.14_1557, released 25 August 2021 

The following table shows the changes that are in the master fix pack patch update 1.19.14_1557. Master patch updates are applied automatically.



Changes since version 1.19.13_1554

 Component Previous Current Description 

 Calico N/A N/A Updated the rolling update configuration for the calico-node daemonset in the kube-system namespace. The configuration is now the same regardless of the number of cluster worker nodes. 
 Cluster health image v1.2.14 v1.2.15 Updated to use Go version 1.15.15. Updated universal base image (UBI) to the latest 8.4 version to resolve CVEs. 
 Gateway-enabled cluster controller 1348 1444 Updated image for [CVE-2021-36159](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-36159). 
 GPU device plug-in and installer 82ee77b a9461a8 Updated to use Go version 1.16.6. 
 IBM Calico extension 747 763 Updated to use Go version 1.16.6.
HAPROXY 6514a2 c1634f [CVE-2022-32206](https://nvd.nist.gov/vuln/detail/CVE-2022-32206), [CVE-2022-32208](https://nvd.nist.gov/vuln/detail/CVE-2022-32208) 







 Change log for worker node fix pack 4.7.55_1576_openshift, released 16 August 2022 

The following table shows the changes that are in the worker node fix pack 4.7.55_1576_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.7.55_1575_openshift

 Component Previous Current Description
Worker node setup in Red Hat OpenShift on IBM Cloud excluding network security

CIS-compliant RHEL image
: Every worker node is set up with a Red Hat Enterprise Linux operating system that implements the benchmarks that are published by the Center of Internet Security (CIS). The user or the owner of the machine can't change this operating system to another operating system. To review the current RHEL version, run oc get nodes -o wide. IBM works with internal and external security advisory teams to address potential security compliance vulnerabilities. Security updates and patches for the operating system are made available through Red Hat OpenShift on IBM Cloud and must be installed by the user to keep the worker node secure. Red Hat OpenShift on IBM Cloud uses a Red Hat Enterprise Linux kernel for worker nodes. You can run containers based on any Linux distribution in Red Hat OpenShift on IBM Cloud. Check with your container image vendor to verify that your container images can run on a Red Hat Enterprise kernel.

Continuous monitoring by Site Reliability Engineers (SREs)
: The image that is installed on your worker nodes is continuously monitored by IBM Site Reliability Engineers (SREs) to detect vulnerabilities and security compliance issues. To address vulnerabilities, SREs create security patches and fix packs for your worker nodes. Make sure to apply these patches when they are available to ensure a secure environment for your worker nodes and the apps that you run on them.

CIS Kubernetes worker node benchmark
: To configure Red Hat OpenShift on IBM Cloud, IBM engineers follow relevant cybersecurity practices from the Kubernetes worker node benchmark that is published by the [Center of Internet Security (CIS)](https://www.cisecurity.org/benchmark/kubernetes/). You can review the compliance of worker nodes against [CIS Kubernetes benchmark](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmarkcis-worker-test) and [Red Hat OpenShift benchmark](https://cloud.ibm.com/docs/openshift?topic=openshift-benchmark-comparison) standards.

Compute isolation
You can find more information about your worker node when you run the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and look for the worker node with the same Private IP address.
2. Replace the worker node. As part of the replace process, the pods that run on the worker node are drained and rescheduled onto remaining worker nodes in the cluster. The worker node is also cordoned, or marked as unavailable for future pod scheduling. Use the worker node ID that is returned from the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command.

ibmcloud oc worker replace --cluster <cluster_name_or_ID> --worker <worker_node_ID>
3. Verify that the worker node is replaced.

ibmcloud oc worker ls --cluster <cluster_name_or_ID>



ibmcloud oc worker replace --cluster CLUSTER_NAME_OR_ID --worker WORKER_ID [--update] [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service.

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--worker WORKER
: Required: The name or ID of a worker node.

--update
: Include this option to update the worker node to the same major and minor version of the master and the latest patch.

-f
: Optional: Force the command to run with no user prompts.

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker replace command 

ibmcloud oc worker replace --cluster my_cluster --worker kube-dal10-cr18a61a63a6a94b658596aa93d087aaa9-w1







 ibmcloud oc worker rm 

Virtual Private Cloud

Classic infrastructure

Remove one or more worker nodes from a cluster.
* Worker zones: User defined subnet in each zone of the region
* Worker nodes per zone: 1
* Flavor: mx2.4x32 - 4 vCPU, 32 GB Memory



2. After provisioning completes, [access the Red Hat OpenShift on IBM Cloud cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-access_clustervpc_private_se).
3. Taint the worker pool. By providing a taint on the worker pool, it ensures that only the monitoring stack runs on the worker pool. For more information on taints and tolerations, see the [Red Hat OpenShift on IBM Cloud documentation](https://docs.openshift.com/container-platform/4.6/nodes/scheduling/nodes-scheduler-taints-tolerations.htmlnodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations). A taint can be set on a worker pool with the following ibmcloud CLI command:

ibmcloud oc worker-pool taint set --worker-pool <WORKER_POOL> --cluster <CLUSTER> --taint KEY=VALUE:EFFECT

An example taint setting of logging-monitoring=node:NoExecute, can be set by using the following ibmcloud cli command:

ibmcloud oc worker-pool taint set --worker-pool <WORKER_POOL> --cluster <CLUSTER> --taint logging-monitoring=node:NoExecute







 Step 3: Configure the Red Hat OpenShift on IBM Cloud monitoring stack 



1. For conceptual information about the monitoring stack, see [Understanding the monitoring stack](https://docs.openshift.com/container-platform/4.6/monitoring/understanding-the-monitoring-stack.html).
2.
This is the number of worker nodes that are provisioned at the time the cluster is created. However, you can use a maximum number of worker nodes that should be added to the Spectrum LSF cluster defined by worker_node_max_count. This is to limit the number of machines that can be added to Spectrum LSF cluster when the auto scaling configuration is used. This property can be used to manage the cost associated with Spectrum LSF cluster instance.

The following example shows worker_node_min_count=2 and worker_node_max_count=10.



1. To check the two worker nodes, run the following command:

bhosts -w

Example output:

Zoom

![Two worker nodes](https://cloud.ibm.com/docs-content/v1/content/74a831a53989757bd05bd0dea7bb42c9ede61d0f/ibm-spectrum-lsf/images/original_workernodes.png)

Two worker nodes
2. To try the auto scaling function, run a job that requires more than two nodes. For example, this job requires five jobs to sleep for 10 seconds:

bsub -n 5 -R "span[ptile=1]" sleep 10
3. The job is submitted.
4. After a minute, check the nodes by running the following command:

bhosts -w

You can see that now five nodes were added to your cluster:

Zoom

![Two worker nodes](https://cloud.ibm.com/docs-content/v1/content/74a831a53989757bd05bd0dea7bb42c9ede61d0f/ibm-spectrum-lsf/images/autoscaling.png)

Five worker nodes added
5. The difference of nodes that were created by the auto scaling function are destroyed automatically after 10 minutes of not being used.







 Step 7: Customize your HPC cluster for your workloads 

You have two options to add software packages in the cluster for your workload.
Classic: Why do I see a timeout error when I try to log in to a pod on a new worker node? 

Classic infrastructure

 What’s happening 

You deleted a worker node in your cluster and then added a worker node. When you deployed a pod or Kubernetes service, the resource can't access the newly created worker node, and the connection times out.

 Why it’s happening 

If you delete a worker node from your cluster and then add a worker node, the new worker node might be assigned the private IP address of the deleted worker node. Calico uses this private IP address as a tag and continues to try to reach the deleted node.

 How to fix it 

Manually update the reference of the private IP address to point to the correct node.



1. Confirm that you have two worker nodes with the same Private IP address. Note the Private IP and ID of the deleted worker.

ibmcloud oc worker ls --cluster <cluster_name_or_id>

ID Public IP Private IP Machine Type State Status Zone Version
kube-dal10-cr9b7371a7fcbe46d08e04f046d5e6d8b4-w1 169.xx.xxx.xxx 10.xxx.xx.xxx b3c.4x16 normal Ready dal10 1.26
kube-dal10-cr9b7371a7fcbe46d08e04f046d5e6d8b4-w2 169.xx.xxx.xxx 10.xxx.xx.xxx b3c.4x16 deleted - dal10 1.26
2. Install the [Calico CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-network_policiescli_install).
3. List the available worker nodes in Calico. Replace <path_to_file> with the local path to the Calico configuration file.

calicoctl get nodes --config=filepath/calicoctl.cfg

NAME