vania-janet/multiturn-rag-retrieval-data huggingface dataset:
In your VPC infrastructure dashboard, the VPC load balancer reports as healthy only the two worker nodes that run the Ingress controller replica pods, because these worker nodes are configured as the listeners for the VPC load balancer. Even though only the listener worker nodes are reported as healthy, the listeners' backend pool of worker nodes is kept up-to-date by Red Hat OpenShift on IBM Cloud so that all worker nodes in your cluster can still receive requests from the VPC load balancer.



 Traffic flow in a classic single-zone cluster 

The following diagram shows how a router directs network traffic from the internet to an app in a single-zone, classic cluster.

Zoom

![Expose an app in a single-zone Red Hat OpenShift cluster by using a router](https://cloud.ibm.com/docs-content/v1/content/bce0f0917a9eea684d1b4b704ac6343a1f65f446/openshift/images/roks-router.png)

Figure 1. Expose an app in a single-zone cluster by using a router



1. A request to your app uses the route hostname that you set up for your app.
2. A DNS service resolves the subdomain to the portable public IP address of router service.
3. The router receives the request, and forwards it to the private IP address of the app pod over the private network. The source IP address of the request package is changed to the public IP address of the worker node where the router pod runs. If multiple app instances are deployed in the cluster, the router sends the requests between the app pods.
4. When the app returns a response packet, it uses the IP address of the worker node where the router that forwarded the client request exists. The router then sends the response packet through the load balancer service to the client.







 Traffic flow in a classic multizone cluster 

The following diagram shows how a router directs network traffic from the internet to an app in a multizone, classic cluster.

Zoom

!
You can manage the encryption of the local disks in your worker nodes by using a [key management service (KMS) provider](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkms) such as [Key Protect](https://cloud.ibm.com/docs/key-protect?topic=key-protect-getting-started-tutorial). The way that you manage encryption for worker nodes depends on the infrastructure provider.



 Classic worker nodes 

Classic infrastructure: Classic worker nodes have two disks, and you can manage encryption for the second disk.



* The primary disk has the kernel images to boot your worker node. This disk is unencrypted.
* The secondary disk has the container file system and locally pulled images. This disk is AES 256-bit encrypted with an IBM-managed LUKS encryption key that is unique to the worker node and stored as a Kubernetes secret in your cluster. When you reload or update your worker nodes, the LUKS keys are rotated.







 VPC worker nodes 

VPC infrastructure: By default, the one primary disk of VPC worker nodes is AES-256 bit encrypted at rest by the [underlying VPC infrastructure provider](https://cloud.ibm.com/docs/vpc?topic=vpc-block-storage-aboutvpc-storage-encryption).

You can manage the encryption of the worker nodes by enabling a KMS provider at the worker pool level.



1. Complete the same [prerequisite steps](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkms_prereqs) for enabling a KMS provider at the cluster level, including to create your own KMS instance and root key. You don't have to enable encryption at the cluster level, but you might want to so that you manage the encryption of cluster secrets.
2. Make sure that you have [service authorization policies in IBM Cloud IAM](https://cloud.ibm.com/iam/authorizations), created under the account where the KMS instance resides, with the following details:



* Required service access policy for Kubernetes Service and the KMS provider
If you run privileged containers, run commands on the workers, or execute untrusted code, then you might be at risk.

Some change logs are for worker node fix packs, and apply only to worker nodes. You must [apply these patches](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_update) to ensure security compliance for your worker nodes. These worker node fix packs can be at a higher version than the master because some build fix packs are specific to worker nodes. Other change logs are for master fix packs, and apply only to the cluster master. Master fix packs might not be automatically applied. You can choose to [apply them manually](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_update). For more information about patch types, see [Update types](https://cloud.ibm.com/docs/containers?topic=containers-cs_versionsupdate_types).





 Version 1.22 change log 

Review the version 1.22 change log.



 Change log for worker node fix pack 1.22.17_1584, released 02 January 2023 

The following table shows the changes that are in the worker node fix pack 1.22.17_1584. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.22.17_1583

 Component Previous Current Description 

 Ubuntu 18.04 packages N/A N/A N/A 
 Ubuntu 20.04 packages N/A N/A N/A 
 Kubernetes N/A N/A N/A 







 Change log for worker node fix pack 1.22.17_1583, released 19 December 2022 

The following table shows the changes that are in the worker node fix pack 1.22.17_1583. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.22.16_1580
In addition, changed the cloud provider configuration to better handle updates for load balancer services with externalTrafficPolicy set to local. 
 IBM Cloud File Storage for Classic plug-in 334 338 Updated incubator version to 1.8. File storage is provisioned to the specific zone that you select. You can't update an existing (static) PV instance's labels, unless you are using a multizone cluster and need to add the region and zone labels. <br>Removed the default NFS version from the mount options in the IBM-provided file storage classes. The host's operating system now negotiates the NFS version with the IBM Cloud infrastructure NFS server. To manually set a specific NFS version, or to change the NFS version of your PV that was negotiated by the host's operating system, see [Changing the default NFS version](https://cloud.ibm.com/docs/containers?topic=containers-file_storagenfs_version_class). 
 Kubernetes v1.9.9 v1.9.10 See the Kubernetes [release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.9.10) 
 Kubernetes Heapster configuration N/A N/A Increased resource limits for the heapster-nanny container. 







 Change log for worker node fix pack 1.9.9_1522, released 23 August 2018 

The following table shows the changes that are in the worker node fix pack 1.9.9_1522.



Table 1. Changes since version 1.9.9_1521

 Component Previous Current Description 

 systemd 229 230 Updated systemd to fix cgroup leak. 
 Kernel 4.4.0-127 4.4.0-133 Updated worker node images with kernel update for [CVE-2018-3620,CVE-2018-3646](https://ubuntu.com/security/notices/USN-3741-1) 







 Change log for worker node fix pack 1.9.9_1521, released 13 August 2018 

The following table shows the changes that are in the worker node fix pack 1.9.9_1521.
Add the zone to your worker pool. When you add a zone to a worker pool, the worker nodes that are defined in your worker pool are provisioned in the zone and considered for future workload scheduling. IBM Cloud® Kubernetes Service automatically adds the failure-domain.beta.kubernetes.io/region label for the region and the failure-domain.beta.kubernetes.io/zone label for the zone to each worker node. The Kubernetes scheduler uses these labels to spread pods across zones within the same region.



1. To add a zone to one worker pool: Replace <pool_name> with the name of your worker pool, and fill in the cluster ID, zone, and VLANs with the information you previously retrieved. If you don't have a private and a public VLAN in that zone, don't specify this option. A private and a public VLAN are automatically created for you.

If you want to use different VLANs for different worker pools, repeat this command for each VLAN and its corresponding worker pools. Any new worker nodes are added to the VLANs that you specify, but the VLANs for any existing worker nodes are not changed.

ibmcloud ks zone add classic --zone <zone> --cluster <cluster_name_or_ID> --worker-pool <pool_name> --private-vlan <private_VLAN_ID> --public-vlan <public_VLAN_ID>
2. To add the zone to multiple worker pools: Add multiple worker pools to the ibmcloud ks zone add classic command. To add multiple worker pools to a zone, you must have an existing private and public VLAN in that zone. If you don't have a public and private VLAN in that zone, consider adding the zone to one worker pool first so that a public and a private VLAN are created for you. Then, you can add the zone to other worker pools. It is important that the worker nodes in all your worker pools are provisioned into all the zones to ensure that your cluster is balanced across zones.
Consider storing your data on persistent storage outside of the worker node.
* Make sure that you have the [Operator or Administrator IBM Cloud IAM platform access role](https://cloud.ibm.com/docs/openshift?topic=openshift-userschecking-perms).



To update flavors:



1. List available worker nodes and note their private IP address.

1. List available worker pools in your cluster.
 {: pre}
ibmcloud oc worker-pool ls --cluster CLUSTER

2. List the worker nodes in the worker pool. Note the ID and Private IP.
 {: pre}
ibmcloud oc worker ls --cluster CLUSTER --worker-pool WORKER-POOL

3. Get the details for a worker node. In the output, note the zone and either the private and public VLAN ID for classic clusters or the subnet ID for VPC clusters.
 {: pre}
ibmcloud oc worker get --cluster CLUSTER --worker WORKER-ID

2. List available flavors in the zone.

ibmcloud oc flavors --zone <zone>
3. Create a worker node with the new machine type.



1. Create a worker pool with the number of worker nodes that you want to replace.



* Classic clusters:

ibmcloud oc worker-pool create classic --name WORKER-POOL --cluster CLUSTER --flavor FLAVOR --size-per-zone NUMBER-OF-WORKERS-PER-ZONE
* VPC Generation 2 clusters:

ibmcloud oc worker-pool create vpc-gen2 --name NAME --cluster CLUSTER --flavor FLAVOR --size-per-zone NUMBER-OF-WORKERS-PER-ZONE --label LABEL



2. Verify that the worker pool is created.

ibmcloud oc worker-pool ls --cluster CLUSTER
3. Add the zone to your worker pool that you retrieved earlier. When you add a zone, the worker nodes that are defined in your worker pool are provisioned in the zone and considered for future workload scheduling.
* Classic clusters: If you restrict network traffic to edge worker nodes, ensure that at least two [edge worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-edgeedge) are enabled in each zone so that NLBs deploy uniformly.
* Gateway-enabled classic clusters: Ensure that at least two gateway worker nodes in the gateway worker pool are enabled in each zone so that NLBs deploy uniformly.



* When cluster nodes are reloaded or when a cluster master update includes a new keepalived image, the load balancer virtual IP is moved to the network interface of a new node. When this occurs, any long-lasting connections to your load balancer must be re-established. Consider including retry logic in your application so that attempts to re-establish the connection are made quickly.



To set up an NLB 2.0 in a multizone cluster:



1. [Deploy your app to the cluster](https://cloud.ibm.com/docs/containers?topic=containers-deploy_appapp_cli). Ensure that you add a label to your deployment in the metadata section of your configuration file. This custom label identifies all pods where your app runs to include them in the load balancing.
2. Create a load balancer service for the app that you want to expose to the public internet or a private network.



1. Create a service configuration file that is named, for example, myloadbalancer.yaml.
2. Define a load balancer service for the app that you want to expose. You can specify a zone, a VLAN, and an IP address.

apiVersion: v1
kind: Service
metadata:
name: myloadbalancer
annotations:
service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: <public_or_private>
service.kubernetes.io/ibm-load-balancer-cloud-provider-zone: "<zone>"
service.kubernetes.io/ibm-load-balancer-cloud-provider-vlan: "<vlan_id>"
ibmcloud oc worker-pool create vpc-gen2 --name NAME --cluster CLUSTER --flavor FLAVOR --size-per-zone NUMBER-OF-WORKERS-PER-ZONE --label LABEL



2. Verify that the worker pool is created.

ibmcloud oc worker-pool ls --cluster CLUSTER
3. Add the zone to your worker pool that you retrieved earlier. When you add a zone, the worker nodes that are defined in your worker pool are provisioned in the zone and considered for future workload scheduling. If you want to spread your worker nodes across multiple zones, choose a [classic](https://cloud.ibm.com/docs/openshift?topic=openshift-regions-and-zoneszones-mz) or [VPC](https://cloud.ibm.com/docs/openshift?topic=openshift-regions-and-zoneszones-vpc) multizone location.



* Classic clusters:

ibmcloud oc zone add classic --zone ZONE --cluster CLUSTER --worker-pool WORKER-POOL --private-vlan PRIVATE-VLAN-ID --public-vlan PUBLIC-VLAN-ID
* VPC clusters:

ibmcloud oc zone add vpc-gen2 --zone ZONE --cluster CLUSTER --worker-pool WORKER-POOL --subnet-id VPC-SUBNET-ID





4. Wait for the worker nodes to be deployed. When the worker node state changes to Normal, the deployment is finished.

ibmcloud oc worker ls --cluster CLUSTER
5. Remove the old worker node. Note: If you are removing a flavor that is billed monthly (such as bare metal), you are charged for the entire the month.



1. Remove the worker pool with the old machine type. Removing a worker pool removes all worker nodes in the pool in all zones. This process might take a few minutes to complete.

ibmcloud oc worker-pool rm --worker-pool WORKER-POOL --cluster CLUSTER
2. Verify that the worker pool is removed.
Keep in mind that some services such as Ingress might require multiple worker nodes for high availability, and you might not be able to run these services or your apps in the smallest size cluster possible.

Classic clusters only: Initially, you can create a classic cluster with only 1 worker node. This operation is allowed for multizone clusters, so that you are not forced to create 2 worker nodes per zone. If you have a single zone cluster, resize the worker pool to 2. Also, note that after resizing a worker pool to 2, you can't later resize back down to 1 worker node.





 Which Red Hat OpenShift versions does the service support? 

Red Hat OpenShift on IBM Cloud concurrently supports multiple versions of Red Hat OpenShift. When a new version (n) is released, versions up to 2 behind (n-2) are supported. Versions more than 2 behind the latest (n-3) are first deprecated and then unsupported.

For more information about supported versions and update actions that you must take to move from one version to another, see [Red Hat OpenShift on IBM Cloud version information](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_versions).





 Which worker node operating systems does the service support? 

For a list of supported worker node operated systems by cluster version, see [Red Hat OpenShift on IBM Cloud version information](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_versions).





 Where is the service available? 

Red Hat OpenShift on IBM Cloud is available worldwide. You can create clusters in every supported Red Hat OpenShift on IBM Cloud region.

For more information about supported regions, see [Locations](https://cloud.ibm.com/docs/openshift?topic=openshift-regions-and-zonesregions-and-zones).





 Is the service highly available? 

Yes. By default, Red Hat OpenShift on IBM Cloud sets up many components such as the cluster master with replicas, anti-affinity, and other options to increase the high availability (HA) of the service.
Optional: Verify the events that are triggered by the config map and any validation errors that occur. The events can be reviewed in the Events section of your CLI output.

oc describe -n kube-system cm ibm-cluster-update-configuration
9. Confirm that the update is complete by reviewing the Kubernetes version of your worker nodes.

oc get nodes
10. Verify that you don't have duplicate worker nodes. Sometimes, older clusters might list duplicate worker nodes with a NotReady status after an update. To remove duplicates, see [troubleshooting](https://cloud.ibm.com/docs/containers?topic=containers-cs_duplicate_nodes).



Next steps:



* Repeat the update process with other worker pools.
* Inform developers who work in the cluster to update their oc CLI to the version of the Kubernetes master.
* If the Kubernetes dashboard does not display utilization graphs, [delete the kube-dashboard pod](https://cloud.ibm.com/docs/containers?topic=containers-cs_dashboard_graphs).







 Updating classic worker nodes in the console 

After you set up the config map for the first time, you can then update worker nodes by using the IBM Cloud console.

To update worker nodes from the console:



1. Complete the [prerequisite steps](https://cloud.ibm.com/docs/openshift?topic=openshift-updateworker-up-prereqs) and [set up a config map](https://cloud.ibm.com/docs/openshift?topic=openshift-updateworker_node) to control how your worker nodes are updated.
2. From the [IBM Cloud console](https://cloud.ibm.com/) menu !
Classic: Why can't I access resources in my cluster? 

Classic infrastructure

  What’s happening 

When the worker nodes in your cluster can't communicate on the private network, you might see various different symptoms.



*  Sample error message when you run kubectl exec, attach, logs, proxy, or port-forward:

Error from server: error dialing backend: dial tcp XXX.XXX.XXX:10250: getsockopt: connection timed out
*  Sample error message when kubectl proxy succeeds, but the Kubernetes dashboard is not available:

timeout on 172.xxx.xxx.xxx
*  Sample error message when kubectl proxy fails or the connection to your service fails:

Connection refused

Connection timed out

Unable to connect to the server: net/http: TLS handshake timeout



  Why it’s happening 

To access resources in the cluster, your worker nodes must be able to communicate on the private network. You might have a Vyatta or another firewall set up, or customized your existing firewall settings in your IBM Cloud infrastructure account.

IBM Cloud Kubernetes Service requires certain IP addresses and ports to be opened to allow communication from the worker node to the Kubernetes master and vice versa. If your worker nodes are spread across multiple zones, you must allow private network communication by enabling VLAN spanning. Communication between worker nodes might also not be possible if your worker nodes are stuck in a reloading loop.

  How to fix it 

Review the worker node state.



1.  List the worker nodes in your cluster and verify that your worker nodes are not stuck in a Reloading state.

ibmcloud ks worker ls --cluster <cluster_name_or_id>
2.  If you have a multizone cluster and your account is not enabled for VRF, verify that you [enabled VLAN spanning](https://cloud.ibm.com/docs/containers?topic=containers-subnetssubnet-routing) for your account.
3.  If you have a Vyatta or custom firewall settings, make sure that you [opened up the required ports](https://cloud.ibm.com/docs/containers?topic=containers-firewallfirewall_outbound) to allow the cluster to access infrastructure resources and services.
The Red Hat OpenShift Controller Manager converts your Ingress resources to Route resources and the Red Hat OpenShift Ingress controller processes those Routes.





 Choosing among load balancing solutions 

Now that you understand what [options](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_network_planningexternal) you have to expose apps in your Red Hat OpenShift cluster, choose the best solution for your workload.

The following table compares the features of each app exposure method.



Comparison of external networking for apps in Red Hat OpenShift clusters.

 Characteristics NodePort LoadBalancer (Classic - NLB) LoadBalancer (VPC load balancer) Ingress controller 

 Stable external IP Yes Yes 
 External hostname Yes Yes Yes 
 SSL termination Yes* Yes* Yes 
 HTTP(S) load balancing Yes 
 Custom routing rules Yes 
 Multiple apps per route or service Yes 
 Consistent hybrid multicloud deployment Yes 



* SSL termination is provided by ibmcloud oc nlb-dns commands. In classic clusters, these commands are supported for public NLBs only.





 Planning public external load balancing 

Publicly expose an app in your cluster to the internet.

In classic clusters, your worker nodes are connected to a public VLAN. The public VLAN determines the public IP address that is assigned to each worker node, which provides each worker node with a public network interface. Public networking services connect to this public network interface by providing your app with a public IP address and, optionally, a public URL.

In VPC clusters, your worker nodes are connected to private VPC subnets only. However, when you create public networking services, a VPC load balancer is automatically created. The VPC load balancer can route public requests to your app by providing your app a public URL. When an app is publicly exposed, anyone that has the public URL can send a request to your app.

When an app is publicly exposed, anyone that has the public service IP address or the URL that you set up for your app can send a request to your app. For this reason, expose as few apps as possible.
: The number of workers that you want to have in each zone. This value is required, and must be 1 or greater. For more information, see [What is the smallest size cluster that I can make?](https://cloud.ibm.com/docs/containers?topic=containers-faqssmallest_cluster).

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker-pool resize command 

ibmcloud ks worker-pool resize --cluster my_cluster --worker-pool my_pool --size-per-zone 3







 ibmcloud ks worker-pool rm 

Virtual Private Cloud

Classic infrastructure

Remove a worker pool from your cluster. All worker nodes in the pool are deleted. Your pods are rescheduled when you delete. To avoid downtime, be sure that you have enough workers to run your workload.

ibmcloud ks worker-pool rm --worker-pool WORKER_POOL --cluster CLUSTER [-q] [-f]

Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-p, --worker-pool WORKER_POOL
: Required: The name of the worker node pool that you want to remove.

-c, --cluster CLUSTER
: Required: The name or ID of the cluster that you want to remove the worker pool from.

-q
: Optional: Do not show the message of the day or update reminders.

-f
: Optional: Force the command to run with no user prompts.



 Example worker-pool rm command 

ibmcloud ks worker-pool rm --cluster my_cluster --worker-pool pool1







 ibmcloud ks worker-pool taint 

Set or remove Kubernetes taints for all the worker nodes in a worker pool. After you set up taints, pods without a matching toleration can't run on the worker pool. You might use taints to dedicate your worker pool exclusively for a certain type of workload, such as for networking edge nodes or the cluster autoscaler.