vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* If other backends or users are authenticated, log in by using one of those accounts as explained in the Red Hat OpenShift documentation, for example, by running the command ./oc login.



3. Run ./oc get nodes to see the new worker. If it is not yet in the Ready state, you must repeatedly check for pending CSRs by using the command ./oc get csr, and then approve the CSRs by using the command ./oc adm certificate approve <csr_name>. Continue to check until the configuration is complete and the new worked is in the Ready state.
4. After the new worker is in Ready state, it can be used by Red Hat OpenShift.



8. Power on the VM.



* After the VM is powered on, you can monitor the VM to see whether a problem exists. The VM gets an IP address, then processes the ignition file, and then opens a login prompt.
* After the login prompt is shown, it might be covered by console log messages. If required, press the Enter key a few times on the console. If the login prompt is present but covered with console log messages, press Enter to display the login prompt again.
* If the login prompt does not display, one of these items occurred.



* The VM is not getting an IP address. Check the network that the VM is connected to. Check the DHCP binding settings also.
* The base64 value from worker.ign.64 is not correct. It might be missing some characters or it has extra characters. Check the value to confirm.



If you must change any settings because the VM didn't display a login prompt, power off the VM, change the necessary settings, and power it back on.









 Related links 



* [Ordering services for vCenter Server instances](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-vc_addingservices)
ibmcloud oc worker ls -c <cluster_name_or_ID>

Example output

ID Public IP Private IP Flavor State Status Zone Version
kube-blrs3b1d0p0p2f7haq0g-mycluster-default-000001f7 169.xx.xxx.xxx 10.xxx.xx.xxx u3c.2x4.encrypted deployed Ready dal10 1.26
3. Verify that the prerequisite steps for your Ingress controller creation are completed.



* Classic clusters: Get the details of the ibm-cloud-provider-vlan-ip-config config map.



oc describe cm ibm-cloud-provider-vlan-ip-config -n kube-system



* If the config map shows IP addresses, continue to the next step.
* If the Events section shows a warning message similar to ErrorSubnetLimitReached: There are already the maximum number of subnets permitted in this VLAN, see the [VLAN capacity troubleshooting topic](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_subnet_limit_43).



Example output of a config map populated with IP addresses:

NAME: ibm-cloud-provider-vlan-ip-config
Namespace: kube-system
Labels: <none>
Annotations: <none>

Data
====
reserved_public_vlan_id:
----

vlanipmap.json:
----
{
"vlans": [
{
"id": "2234947",
"subnets":
{
"id": "2215454",
"ips":
"10.XXX.XXX.XXX",
"10.XXX.XXX.XXX",
"10.XXX.XXX.XXX",
"10.XXX.XXX.XXX",
See [Managing encryption for the worker nodes in your cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-encryptionworker-encryption).

-q
: Optional: Do not show the message of the day or update reminders.

--secondary-storage STORAGE
: Optional: The storage option for the flavor. For example, 900gb.5iops-tier. When you add a secondary disk, that disk is used for the container runtime, while the primary disk is used for the operating system. To view the storage options for a flavor, run the ibmcloud oc flavor get --flavor FLAVOR --zone ZONE --provider vpc-gen2 command.

--security-group GROUP
: Optional. Specify up to five security group IDs to apply to all workers in the worker pool.

--output json
: Optional: Prints the command output in JSON format.



 Example worker-pool create vpc-gen2 command 

ibmcloud oc worker-pool create vpc-gen2 --name my_pool --cluster my_cluster --flavor bx2.4x16 --size-per-zone 3







 ibmcloud oc worker-pool get 

Virtual Private Cloud

Classic infrastructure

View the details of a worker pool.

ibmcloud oc worker-pool get --worker-pool WORKER_POOL --cluster CLUSTER [--output json] [-q]

Minimum required permissions
: Viewer platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-p, --worker-pool WORKER_POOL
: Required: The name of the worker node pool that you want to view the details of. To list available worker pools, run ibmcloud oc worker-pool ls --cluster <cluster_name_or_ID>.

-c, --cluster CLUSTER
: Required: The name or ID of the cluster where the worker pool is located.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.
ssh root@<WORKER_PUBLIC_IP>
5. Run debug commands to help you gather information and troubleshoot issues, such as ip, ifconfig, nc, tcpdump, ping, ps, and curl. You can also install other tools, such as mtr, by running yum install <tool>.







 Cleaning up after debugging 

After you finish debugging, clean up resources to disable SSH access.



1. Delete the SSH enablement pod.

kubectl delete pod enable-ssh-<NODE_NAME>
2. If you accessed the worker node via the public network, delete the Calico policy so that port 22 is blocked again.

calicoctl delete gnp ssh-open [-c <path_to_calicoctl_cfg>/calicoctl.cfg]
3. [Reload your classic worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_worker_reload) or [replace your VPC worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clicli_worker_replace) so that the original SSH configuration is used and the SSH key that you added is removed.
IBM Cloud Kubernetes Service automatically adds the failure-domain.beta.kubernetes.io/region label for the region and the failure-domain.beta.kubernetes.io/zone label for the zone to each worker node. The Kubernetes scheduler uses these labels to spread pods across zones within the same region.

If you have multiple worker pools in your cluster, add the zone to all them so that worker nodes are spread evenly across your cluster. Note that when you add worker nodes to your cluster, the new worker nodes run the same major.minor version as the cluster master, but the latest worker node patch of that major.minor version.

Before you begin: Make sure that you have the [Operator or Administrator IBM Cloud IAM platform access role](https://cloud.ibm.com/docs/openshift?topic=openshift-users).



1. Get the Location of your cluster, and note the existing Worker Zones and VPC ID.

ibmcloud ks cluster get --cluster <cluster_name_or_ID>

Example output

...
VPC ID: <VPC_ID>
Workers: 3
Worker Zones: us-south-1
...
Location: Dallas
2. List available zones for your cluster's location to see what other zones you can add.

ibmcloud ks zone ls --provider vpc-gen2 | grep <location>
3. List available VPC subnets for each zone that you want to add. If you don't have a VPC subnet in the zone, [create a VPC subnet](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-vpc-resources-with-cli-and-api&interface=clicreate-a-subnet-cli). VPC subnets provide IP addresses for your worker nodes and load balancer services in the cluster, so [create a VPC subnet with enough IP addresses](https://cloud.ibm.com/docs/containers?topic=containers-vpc-subnetsvpc_basics_subnets), such as 256.
ssh root@<WORKER_PUBLIC_IP>
5. Run debug commands to help you gather information and troubleshoot issues, such as ip, ifconfig, nc, tcpdump, ping, ps, and curl. You can also install other tools, such as mtr, by running yum install <tool>.







 Cleaning up after debugging 

After you finish debugging, clean up resources to disable SSH access.



1. Delete the SSH enablement pod.

oc delete pod enable-ssh-<NODE_NAME>
2. If you accessed the worker node via the public network, delete the Calico policy so that port 22 is blocked again.

calicoctl delete gnp ssh-open [-c <path_to_calicoctl_cfg>/calicoctl.cfg]
3. [Reload your classic worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_worker_reload) or [replace your VPC worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clicli_worker_replace) so that the original SSH configuration is used and the SSH key that you added is removed.
Change log for worker node fix pack 4.11.28_1543_openshift, released 27 February 2023 

The following table shows the changes that are in the worker node fix pack 4.11.28_1543_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.11.26_1542_openshift

 Component Previous Current Description 

 RHEL 8 Packages 4.18.0-425.10.1 4.18.0-425.13.1 Worker node & kernel package updates for [CVE-2020-10735](https://nvd.nist.gov/vuln/detail/CVE-2020-10735), [CVE-2021-28861](https://nvd.nist.gov/vuln/detail/CVE-2021-28861), [CVE-2022-2873](https://nvd.nist.gov/vuln/detail/CVE-2022-2873), [CVE-2022-40897](https://nvd.nist.gov/vuln/detail/CVE-2022-40897), [CVE-2022-41222](https://nvd.nist.gov/vuln/detail/CVE-2022-41222), [CVE-2022-43945](https://nvd.nist.gov/vuln/detail/CVE-2022-43945), [CVE-2022-4415](https://nvd.nist.gov/vuln/detail/CVE-2022-4415), [CVE-2022-45061](https://nvd.nist.gov/vuln/detail/CVE-2022-45061), [CVE-2022-48303](https://nvd.nist.gov/vuln/detail/CVE-2022-48303). 
 Red Hat OpenShift on IBM Cloud.
Review the host labels that the worker pools use to request compute capacity. You have several options.



* [Create a worker pool in a Satellite cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-satellite-clusters) with the host labels that you want to use for auto assignment.
* Review existing worker pools for their host labels. Note that you cannot update the host labels that a worker pool has. You can review the Host labels by running the ibmcloud oc worker-pool get -c <cluster> --worker-pool <worker_pool> command.
* Review the host labels that a Satellite-enabled IBM Cloud service cluster uses to request resources from the Satellite-enabled IBM Cloud service instance console.



2. Review the host labels that your available hosts have. Remember that hosts automatically get cpu and memory labels when you attach the host to your Satellite location.



1. Get the Satellite location name.

ibmcloud sat location ls
2. List the available (unassigned) hosts in your location, and note the IDs of the hosts that you want to check the labels for.

ibmcloud sat host ls --location <location_name_or_ID> | grep unassigned
3. For each host that you want to check, get the host details and note the Labels in the output.

ibmcloud sat host get --location <location_name_or_ID> --host <host_name_or_ID>

Example output

...
Labels
app webserver
cpu 4
memory 3874564
os RHCOS
...



3. Update the labels of your available host to match all the labels of the worker pool that you want the host to be available for automatic assignment. You can optionally specify a zone to make sure that the host only gets automatically assigned to that zone.

The cpu, os, and memory labels on the host are applied automatically and cannot be edited.
: For a public network, map the location to its respective geographic area (us-east, us-south, eu-de, lon, tor, syd, or tok).

CLOUD_REGION when you use a private network
: For a private network, map the location to its respective geographic area (us-east, us-south, eu-de, eu-gb, ca-tor, au-syd, jp-tok, jp-osa, br-sao, or ca-mon).
5. On the tile for the workspace, click View Instances.
6. In the list of the virtual server instances, click each of the cluster nodes, and take a note of each ID.
7. Set these IDs as POWERVSI_01 and POWERVSI_02.







 Preparing the nodes for RHEL HA Add-On installation 

The following section describes basic preparation steps on the cluster nodes. Make sure that you follow the steps on both nodes.

Log in as root to each of the cluster nodes.



 Populating entries for each node in the hosts file 

On both nodes, use the following information to populate entries.

Add the IP addresses and hostnames of both nodes to the hosts file /etc/hosts.

For more information, see [Setting up /etc/hosts files on RHEL cluster nodes](https://access.redhat.com/solutions/81123).





 Preparing environment variables 

To simplify the setup process, prepare the following environment variables for user root on both nodes.

On both nodes, create a file with the following environment variables and update to your environment.

export CLUSTERNAME=SAP_CLUSTER Cluster Name

export APIKEY=<APIKEY> API Key of the ServiceID
export IBMCLOUD_CRN=<IBMCLOUD_CRN> CRN of workspace
export GUID=<GUID> GUID of workspace
export CLOUD_REGION=<CLOUD_REGION> Region of workspace
export PROXY_IP=<IP_ADDRESS> IP address of proxy server

export NODE1=<HOSTNAME_01> <Hostname of virtual server instance 1
export NODE2=<HOSTNAME_02> <Hostname of virtual server instance 2
](https://nvd.nist.gov/vuln/detail/CVE-2019-7175), [CVE-2019-7397](https://nvd.nist.gov/vuln/detail/CVE-2019-7397), [CVE-2019-7398](https://nvd.nist.gov/vuln/detail/CVE-2019-7398), [CVE-2019-9503](https://nvd.nist.gov/vuln/detail/CVE-2019-9503), [CVE-2019-9924](https://nvd.nist.gov/vuln/detail/CVE-2019-9924), [CVE-2019-9956](https://nvd.nist.gov/vuln/detail/CVE-2019-9956), [CVE-2020-1702](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1702), and[CVE-2020-8945](https://nvd.nist.gov/vuln/detail/CVE-2020-8945). 







 Change log for worker node fix pack 3.11.188_1545_openshift, released 30 March 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.188_1545_openshift. Worker node patch updates can be applied by updating or reloading the worker node



Changes since version 3.11.170_1544_openshift

 Component Previous Current Description
* [Hardware changes](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelworker-default-hw)



* [Modifying worker node settings to optimize performance](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelworker)



* [Modifying worker node settings by using the Node Tuning Operator](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelworker-nto)
* [Modifying worker node kernel settings](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelworker-kernel-ds)



* [Optimizing network keepalive sysctl settings](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelkeepalive-iks)
* [Changing the Calico maximum transmission unit (MTU)](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelcalico-mtu)



* [Updating the Calico installation](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelcalico-mtu-43)



* [Disabling the port map plug-in](https://cloud.ibm.com/docs/openshift?topic=openshift-kernelcalico-portmap)



[Tuning performance for Red Hat CoreOS worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-rhcos-performancerhcos-performance)



* [Deploying the Node Feature Discovery Operator](https://cloud.ibm.com/docs/openshift?topic=openshift-rhcos-performancerhcos-node-feature-discovery)
2020-12402](https://nvd.nist.gov/vuln/detail/CVE-2020-12402), [CVE-2020-12403](https://nvd.nist.gov/vuln/detail/CVE-2020-12403), [CVE-2018-20843](https://nvd.nist.gov/vuln/detail/CVE-2018-20843), [CVE-2019-15903](https://nvd.nist.gov/vuln/detail/CVE-2019-15903), [CVE-2019-14834](https://nvd.nist.gov/vuln/detail/CVE-2019-14834), [CVE-2019-11068](https://nvd.nist.gov/vuln/detail/CVE-2019-11068), [CVE-2019-18197](https://nvd.nist.gov/vuln/detail/CVE-2019-18197), [CVE-2019-16935](https://nvd.nist.gov/vuln/detail/CVE-2019-16935), [CVE-2019-20386](https://nvd.nist.gov/vuln/detail/CVE-2019-20386), [CVE-2019-17498](https://nvd.nist.gov/vuln/detail/CVE-2019-17498), and [CVE-2020-14365](https://nvd.nist.gov/vuln/detail/CVE-2020-14365). 







 Change log for worker node fix pack 3.11.286_1569_openshift, released 30 September 2020 

The following table shows the changes that are in the worker node fix pack update 3.11.286_1569_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 3.11.286_1568_openshift
](https://nvd.nist.gov/vuln/detail/CVE-2019-19642), [CVE-2019-19813](https://nvd.nist.gov/vuln/detail/CVE-2019-19813), [CVE-2019-3689](https://nvd.nist.gov/vuln/detail/CVE-2019-3689), [CVE-2020-0543](https://nvd.nist.gov/vuln/detail/CVE-2020-0543), [CVE-2020-10711](https://nvd.nist.gov/vuln/detail/CVE-2020-10711), [CVE-2020-13143](https://nvd.nist.gov/vuln/detail/CVE-2020-13143), [CVE-2020-8177](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8177), [CVE-2019-19377](https://nvd.nist.gov/vuln/detail/CVE-2019-19377), and [CVE-2019-19816](https://nvd.nist.gov/vuln/detail/CVE-2019-19816). 
 Ubuntu 16.04 packages N/A N/A Updated worker node images with package updates for [CVE-2019-3689](https://nvd.nist.gov/vuln/detail/CVE-2019-3689) and [CVE-2020-8177](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8177). 
 Worker node drain automation N/A N/A Fixes a race condition that can cause worker node drain automation to fail. 







 Change log for 1.15.12_1543, released 22 June 2020 

The following table shows the changes that are in the master and worker node update 1.15.12_1543.
[CVE-2022-21499](https://nvd.nist.gov/vuln/detail/CVE-2022-21499), [CVE-2022-23960](https://nvd.nist.gov/vuln/detail/CVE-2022-23960), [CVE-2022-26373](https://nvd.nist.gov/vuln/detail/CVE-2022-26373), [CVE-2022-27950](https://nvd.nist.gov/vuln/detail/CVE-2022-27950), [CVE-2022-28390](https://nvd.nist.gov/vuln/detail/CVE-2022-28390), [CVE-2022-28893](https://nvd.nist.gov/vuln/detail/CVE-2022-28893), [CVE-2022-29581](https://nvd.nist.gov/vuln/detail/CVE-2022-29581), [CVE-2022-36946](https://nvd.nist.gov/vuln/detail/CVE-2022-36946), [CVE-2022-24448](https://nvd.nist.gov/vuln/detail/CVE-2022-24448). 







 Change log for worker node fix pack 4.11.12_1530_openshift, released 07 November 2022 

The following table shows the changes that are in the worker node fix pack 4.11.12_1530_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.11.9_1526_openshift

 Component Previous Current Description
oc get pods -n kube-system -l app=vpn
2. Check the OpenVPN logs, and check for an ERROR message such as WORKERIP:<port>, such asWORKERIP:10250, that indicates that the VPN tunnel does not work.

oc logs -n kube-system <vpn_pod> --tail 10
3. If you see the worker IP error, check if worker-to-worker communication is broken. Log in to a calico-node pod in the calico-system project, and check for the same WORKERIP:10250 error.

oc exec -n calico-system <calico-node_pod> -- date
4. If the worker-to-worker communication is broken, make sure that you enable [VRF or VLAN spanning](https://cloud.ibm.com/docs/openshift?topic=openshift-subnetsbasics_segmentation).
5. If you see a different error from either the OpenVPN or calico-node pod, restart the OpenVPN pod.

oc delete pod -n kube-system <vpn_pod>
6. If the OpenVPN still fails, check the worker node that the pod runs on.

oc describe pod -n kube-system <vpn_pod> | grep "Node:"
7. Cordon the worker node so that the OpenVPN pod is rescheduled to a different worker node.

oc cordon <worker_node>
8. Check the OpenVPN pod logs again. If the pod no longer has an error, the worker node might have a network connectivity issue. Reload the worker node.

ibmcloud oc worker reload -c <cluster_name_or_ID> -w <worker_node_ID>







 Step 9: Refresh the cluster master 

Refresh the cluster master to set up the default Red Hat OpenShift components. After you refresh the cluster, wait a few minutes to allow the operation to complete.
View the details of a worker pool.

ibmcloud ks worker-pool get --worker-pool WORKER_POOL --cluster CLUSTER [--output json] [-q]

Minimum required permissions
: Viewer platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-p, --worker-pool WORKER_POOL
: Required: The name of the worker node pool that you want to view the details of. To list available worker pools, run ibmcloud ks worker-pool ls --cluster <cluster_name_or_ID>.

-c, --cluster CLUSTER
: Required: The name or ID of the cluster where the worker pool is located.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker-pool get command 

ibmcloud ks worker-pool get --worker-pool pool1 --cluster my_cluster







 ibmcloud ks worker-pool label rm 

Virtual Private Cloud

Classic infrastructure

Remove all custom Kubernetes labels from all worker nodes in a worker pool.

To remove an individual label from a worker pool, you can run the ibmcloud ks worker-pool label set command with only the custom labels that you want to keep.

ibmcloud ks worker-pool label rm --cluster CLUSTER --worker-pool POOL [-f] [-q]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster where the worker pool is located.

-p, --worker-pool WORKER_POOL
: Required: The name of the worker node pool that you want to view the details of. To list available worker pools, run ibmcloud ks worker-pool ls --cluster <cluster_name_or_ID>.

-f
: Optional: Force the command to run with no user prompts.

-q
: Optional: Do not show the message of the day or update reminders.
Make sure you include the --hard option.

ibmcloud ks worker reboot --cluster <cluster_name_or_id> --worker <worker_id> --hard
4. Mark the worker as available to be scheduled.

kubectl uncordon <worker_id>



For workers with the cx2d., bx2d., or mx2d. flavors:



1. Cordon the worker.

kubectl cordon <worker_id>
2. Drain the worker.

kubectl drain <worker_id>
3. Replace the worker. A new worker node is provisioned on a host that is not undergoing maintenance.

ibmcloud ks worker replace --cluster <cluster_name_or_ID> --worker <worker_node_ID>
Change log for worker node fix pack 4.11.12_1530_openshift, released 07 November 2022 

The following table shows the changes that are in the worker node fix pack 4.11.12_1530_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.11.9_1526_openshift

 Component Previous Current Description 

 RHEL 8 Packages 4.18.0-372.26.1 4.18.0-372.32.1 Worker node kernel & package updates for [CVE-2020-35525](https://nvd.nist.gov/vuln/detail/CVE-2020-35525), [CVE-2020-35527](https://nvd.nist.gov/vuln/detail/CVE-2020-35527), [CVE-2022-0494](https://nvd.nist.gov/vuln/detail/CVE-2022-0494), [CVE-2022-1353](https://nvd.nist.gov/vuln/detail/CVE-2022-1353), [CVE-2022-23816](https://nvd.nist.gov/vuln/detail/CVE-2022-23816), [CVE-
Debugging clusters 

Virtual Private Cloud

Classic infrastructure

Review the options to debug your clusters and find the root causes for failures.



1.  List your cluster and find the State of the cluster.

ibmcloud oc cluster ls
2.  Review the State of your cluster. If your cluster is in a Critical, Delete failed, or Warning state, or is stuck in the Pending state for a long time. For more information, see [cluster states](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-states-reference).
3.  Review the state of each worker node. For more information, see [worker node states](https://cloud.ibm.com/docs/openshift?topic=openshift-worker-node-state-reference).

ibmcloud oc worker ls -c CLUSTER
4.  Review the [common worker node issues](https://cloud.ibm.com/docs/openshift?topic=openshift-common_worker_nodes_issues).
5.  Start [debugging the worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_worker_nodes).
6.  Review the site map for additional [troubleshooting information](https://cloud.ibm.com/docs/openshift?topic=openshift-sitemapsitemap_troubleshooting).