vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Next, you can follow the steps in [Setting up an NLB 2.0 in a multizone cluster](https://cloud.ibm.com/docs/containers?topic=containers-loadbalancer-v2ipvs_multi_zone_config) or [in a single-zone cluster](https://cloud.ibm.com/docs/containers?topic=containers-loadbalancer-v2ipvs_single_zone_config).





 Setting up an NLB 2.0 in a multizone cluster 

Before you begin:



* Important: Complete the [NLB 2.0 prerequisites](https://cloud.ibm.com/docs/containers?topic=containers-loadbalancer-v2ipvs_provision).
* To create public NLBs in multiple zones, at least one public VLAN must have portable subnets available in each zone. To create private NLBs in multiple zones, at least one private VLAN must have portable subnets available in each zone. You can add subnets by following the steps in [Configuring subnets for clusters](https://cloud.ibm.com/docs/containers?topic=containers-subnets).
* Ensure that you have the [Writer or Manager IBM Cloud IAM service access role](https://cloud.ibm.com/docs/containers?topic=containers-userschecking-perms) for the default namespace.
* Ensure you have the required number of worker nodes:



* Classic clusters: If you restrict network traffic to edge worker nodes, ensure that at least two [edge worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-edgeedge) are enabled in each zone so that NLBs deploy uniformly.
* Gateway-enabled classic clusters: Ensure that at least two gateway worker nodes in the gateway worker pool are enabled in each zone so that NLBs deploy uniformly.



* When cluster nodes are reloaded or when a cluster master update includes a new keepalived image, the load balancer virtual IP is moved to the network interface of a new node. When this occurs, any long-lasting connections to your load balancer must be re-established.
Classic: Why is the bare metal instance ID inconsistent with worker records? 

Classic infrastructure

  What’s happening 

When you use ibmcloud oc worker commands with your bare metal worker node, you see a message similar to the following.

The worker node instance ID changed. Reload the worker node if bare metal hardware was serviced.

  Why it’s happening 

The machine ID can become inconsistent with the Red Hat OpenShift on IBM Cloud worker record when the machine experiences hardware issues. When IBM Cloud infrastructure resolves this issue, a component can change within the system that the service does not identify.

  How to fix it 

For Red Hat OpenShift on IBM Cloud to re-identify the machine, [reload the bare metal worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_worker_reload). Note that reloading also updates the machine's [patch version](https://cloud.ibm.com/docs/containers?topic=containers-changelog).

You can also [delete the bare metal worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_cluster_rm). Remember that bare metal instances are billed monthly.
Any new worker nodes are added to the VPC subnets that you specify, but the VPC subnets for any existing worker nodes are not changed.

ibmcloud oc zone add vpc-gen2 --zone <zone> --subnet-id <subnet_id> --cluster <cluster_name_or_ID> --worker-pool <worker_pool_name>
6. Verify that the zone is added to your cluster. Look for the added zone in the Worker Zones field of the output. Note that the total number of workers in the Workers field has increased as new worker nodes are provisioned in the added zone.

ibmcloud oc cluster get --cluster <cluster_name_or_ID>

Example output

Workers: 9
Worker Zones: us-south-1, us-south-2, us-south-3
7. To expose apps with [Ingress](https://cloud.ibm.com/docs/openshift?topic=openshift-ingress-about-roks4), you must [update the VPC load balancer that exposes the router](https://cloud.ibm.com/docs/openshift?topic=openshift-router-mzr-error) to include the subnet for the new zone in your cluster.









 Adding worker nodes in classic clusters 

Add worker nodes to your classic cluster.

Want to save on your classic worker node costs? [Create a reservation](https://cloud.ibm.com/docs/containers?topic=containers-reservations) to lock in a discount over 1 or 3 year terms! Then, create your worker pool by using the reserved instances. Note that autoscaling can't be enable on worker pools that use reservations.



 Creating a new worker pool 

You can add worker nodes to your classic cluster by creating a new worker pool.

Before you begin, make sure that you have the [Operator or Administrator IBM Cloud IAM platform access role](https://cloud.ibm.com/docs/openshift?topic=openshift-users).



1.
Consider the following example scenario to understand how clusters might become orphaned.



1. You have an IBM Cloud Pay-As-You-Go account.
2. You create a cluster named Cluster1. The worker nodes and other infrastructure resources are provisioned into the infrastructure account that comes with your Pay-As-You-Go account.
3. Later, you find out that your team uses a legacy or shared classic IBM Cloud infrastructure account. You use the ibmcloud ks credential set command to change the IBM Cloud infrastructure credentials to use your team account.
4. You create another cluster named Cluster2. The worker nodes and other infrastructure resources are provisioned into the team infrastructure account.
5. You notice that Cluster1 needs a worker node update, a worker node reload, or you just want to clean it up by deleting it. However, because Cluster1 was provisioned into a different infrastructure account, you can't modify its infrastructure resources. Cluster1 is orphaned.
6. You follow the resolution steps in the following section, but don't set your infrastructure credentials back to your team account. You can delete Cluster1, but now Cluster2 is orphaned.
7. You change your infrastructure credentials back to the team account that created Cluster2. Now, you no longer have an orphaned cluster!



 How to fix it 

Follow the steps to review your infrastructure credentials and determine why you are seeing the credentials error.



1. Log in to the [Kubernetes clusters console](https://cloud.ibm.com/kubernetes/clusters).
2. [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster).
3. Check which infrastructure account the region that your cluster is in currently uses to provision clusters. Replace REGION with the IBM Cloud region that the cluster is in.

ibmcloud ks credential get --region REGION

If you see a message similar to the following, then the account uses the default, linked infrastructure account.
Instead, use the [worker replace --update CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clicli_worker_replace) or [API operation](https://containers.cloud.ibm.com/global/swagger-global-api//beta/replaceWorker) to replace worker nodes that are outdated or in a troubled state. 
 Cluster networking Unlike classic infrastructure, the worker nodes of your VPC cluster are attached to VPC subnets and assigned private IP addresses. The worker nodes are not connected to the public network, which instead is accessed through a public gateway, floating IP, or VPN gateway. For more information, see [Overview of VPC networking in Red Hat OpenShift on IBM Cloud](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-subnetsvpc_basics). 
 Apps and container platform You can choose to create [community Kubernetes or Red Hat OpenShift clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-faqscontainer_platforms) to manage your containerized apps. Your app build processes don't differ because of the infrastructure provider, but how you expose the app does. For more information, see [Choosing an app exposure service](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_network_planning). 
 App networking All pods that are deployed to a worker node are assigned a private IP address in the 172.30.0.0/16 range and are routed between worker nodes on the worker node private IP address of the private VPC subnet. To expose the app on the public network, you can create a Kubernetes LoadBalancer service, which provisions a VPC load balancer and public hostname address for your worker nodes. For more information, see [Exposing apps with VPC load balancers](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-lbaas).
satdemo-cp3 8fg4ff8faaa1fa622bb5 assigned Ready zone-3 satdemo-cluster sat-satdemocp3-d46581b509357ea4b429fddc38a18b155463bf1c 169.62.42.182
2. List your current hosts that are assigned as worker nodes to your Satellite-enabled IBM Cloud service and make note of their IDs.

ibmcloud ks worker ls -c <cluster_name_or_ID>

Review the example output.

ID Primary IP Flavor State Status Zone Version
sat-satellitei-5bd83963cdfedc7668c9040879c88113a134c4e7 10.241.0.4 upi normal Ready us-east-2 4.7.55_1575_openshift
sat-satellitei-854beae4556401b5761e34ed849ba64c4b0a674c 10.241.128.4 upi normal Ready us-east-1 4.7.55_1575_openshift
sat-satellitei-bf1fc9b135011d5c0d9d500855db6e489d15610b 10.241.64.4 upi normal Ready us-east-3 4.7.55_1575_openshift







 Apply version updates to worker node hosts without detaching them 

You can update your worker node hosts without detaching them from the location. You can also perform a rolling update of your worker node hosts with a ConfigMap.

Before you begin



* Verify that all your worker nodes are in a healthy state.
* If you are using persistent block storage volumes, you must detach these volumes from the node before you start your updates. Move the persistent volumes to a different worker node that does not require updates. Then, cordon and drain the workload from the worker node to update with the kubectl drain NODENAME command.
: Required: The name or ID of the cluster with the worker pool that you want to taint.

-f
: Optional: Force the command to run with no user prompts.





 Example worker-pool taint rm command 

ibmcloud ks worker-pool taint rm --cluster my_cluster --worker-pool pool1







 ibmcloud ks worker-pool zones 

Virtual Private Cloud

Classic infrastructure

View the zones that are attached to a worker pool.

ibmcloud ks worker-pool zones --worker-pool WORKER_POOL --cluster CLUSTER [-q] [-f]

Minimum required permissions
: Viewer platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster where the worker pool exists.

-p, --worker-pool WORKER_POOL
: Required: The name of the worker node pool that you want to see zones for.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker-pool zones command 

ibmcloud ks worker-pool zones --cluster my_cluster --worker-pool pool1









 zone commands 



 ibmcloud ks zone add classic 

Classic infrastructure

After you create a classic cluster or worker pool, you can add a zone. When you add a zone, worker nodes are added to the new zone to match the number of workers per zone that you specified for the worker pool. You can add more than one zone only if your cluster is in a multizone metro.

To add a zone to worker pools in a VPC cluster, use the [ibmcloud ks zone add vpc-gen2 command](https://cloud.ibm.com/docs/cli?topic=cli-kubernetes-service-clicli_zone-add-vpc-gen2).
: Required: The name or ID of the cluster with the worker pool that you want to taint.

-f
: Optional: Force the command to run with no user prompts.





 Example worker-pool taint rm command 

ibmcloud ks worker-pool taint rm --cluster my_cluster --worker-pool pool1







 ibmcloud ks worker-pool zones 

Virtual Private Cloud

Classic infrastructure

View the zones that are attached to a worker pool.

ibmcloud ks worker-pool zones --worker-pool WORKER_POOL --cluster CLUSTER [-q] [-f]

Minimum required permissions
: Viewer platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster where the worker pool exists.

-p, --worker-pool WORKER_POOL
: Required: The name of the worker node pool that you want to see zones for.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.



 Example worker-pool zones command 

ibmcloud ks worker-pool zones --cluster my_cluster --worker-pool pool1









 zone commands 



 ibmcloud ks zone add classic 

Classic infrastructure

After you create a classic cluster or worker pool, you can add a zone. When you add a zone, worker nodes are added to the new zone to match the number of workers per zone that you specified for the worker pool. You can add more than one zone only if your cluster is in a multizone metro.

To add a zone to worker pools in a VPC cluster, use the [ibmcloud ks zone add vpc-gen2 command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clicli_zone-add-vpc-gen2).
IBM Cloud Block Storage driver and plug-in 1.17.2 v2.0.0 Updated to use the universal base image (UBI), to use Go version 1.15.5, to run with a least privileged security context, and to improve logging. Updated image to implement additional IBM security controls. 
 IBM Cloud Controller Manager v1.17.15-1 v1.17.16-1 Updated to support the Kubernetes 1.17.16 release. 
 IBM Cloud File Storage for Classic plug-in N/A N/A Updated to run with a privileged security context. 
 IBM Cloud RBAC Operator c148a8a f859228 Updated image for [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971) and [CVE-2020-24659](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-24659). 







 Change log for worker node fix pack 4.3.40_1549_openshift, released 21 December 2020 

The following table shows the changes that are in the worker node fix pack update 4.3.40_1549_openshift. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.3.40_1548_openshift

 Component Previous Current Description 

 HAProxy db4e6d 9b2dca Image update for [CVE-2020-1971](https://nvd.nist.gov/vuln/detail/CVE-2020-1971) and [CVE-2020-24659](https://nvd.nist.gov/vuln/detail/CVE-2020-24659).
: If you don't create a config map or the map is configured incorrectly, the Kubernetes default is applied. By default, only 20% of the worker nodes in the cluster can be unavailable at a time. You can override the default value by adding the default check to your config map. In the example, every worker node that is not specified in the zone and region checks (dal13 or us-south) can be unavailable during the update.

MaxUnavailablePercentage
: The maximum number of nodes that are allowed to be unavailable for a specified label key and value, which is specified as a percentage. A worker node is unavailable during the deploying, reloading, or provisioning process. The queued worker nodes are blocked from updating if it exceeds any defined maximum unavailable percentages.

NodeSelectorKey
: The label key of the worker node for which you want to set a rule. You can set rules for the default labels that are provided by IBM, as well as on worker node labels that you created. If you want to add a rule for worker nodes that belong to one worker pool, you can use the ibm-cloud.kubernetes.io/machine-type label.

NodeSelectorValue
: The label value that the worker node must have to be considered for the rule that you define.
5. Create the configuration map in your cluster.

kubectl apply -f <filepath/configmap.yaml>
6. Verify that the config map is created.

kubectl get configmap --namespace kube-system
7. Update the worker nodes.

ibmcloud ks worker update --cluster CLUSTER --worker WORKER-NODE-1-ID --worker WORKER-NODE-2-ID
8. Optional: Verify the events that are triggered by the config map and any validation errors that occur. The events can be reviewed in the Events section of your CLI output.

kubectl describe -n kube-system cm ibm-cluster-update-configuration
9. Confirm that the update is complete by reviewing the Kubernetes version of your worker nodes.
ibmcloud ks zone add classic --zone <zone> --cluster <cluster_name_or_ID> --worker-pool <pool_name> --private-vlan <private_VLAN_ID> --public-vlan <public_VLAN_ID>
7. Verify that worker nodes provision in the zone that you added. Your worker nodes are ready when the status changes from provision_pending to normal.

ibmcloud ks worker ls --cluster <cluster_name_or_ID> --worker-pool <pool_name>

Example output

ID Public IP Private IP Machine Type State Status Zone Version
kube-dal10-crb20b637238ea471f8d4a8b881aae4962-w7 169.xx.xxx.xxx 10.xxx.xx.xxx b3c.4x16 provision_pending Ready dal10 1.26
kube-dal10-crb20b637238ea471f8d4a8b881aae4962-w8 169.xx.xxx.xxx 10.xxx.xx.xxx b3c.4x16 provision_pending Ready dal10 1.26







 Adding a zone to a worker pool 

You can span your classic cluster across multiple zones within one region by adding a zone to your existing worker pool.

When you add a zone to a worker pool, the worker nodes that are defined in your worker pool are provisioned in the new zone and considered for future workload scheduling. IBM Cloud Kubernetes Service automatically adds the failure-domain.beta.kubernetes.io/region label for the region and the failure-domain.beta.kubernetes.io/zone label for the zone to each worker node. The Kubernetes scheduler uses these labels to spread pods across zones within the same region.

If you have multiple worker pools in your cluster, add the zone to all them so that worker nodes are spread evenly across your cluster. Note that when you add worker nodes to your cluster, the new worker nodes run the same major.minor version as the cluster master, but the latest worker node patch of that major.minor version.
After the cluster state shows Normal, the cluster network and load balancing components take about 10 more minutes to deploy and update the cluster domain that you use for the Red Hat OpenShift web console and other routes. Before you continue, wait until the cluster is ready by checking that the Ingress Subdomain follows a pattern of <cluster_name>.<globally_unique_account_HASH>-0001.<region>.containers.appdomain.cloud.



The Red Hat-provided OperatorHub source images require access to the registry.redhat.io and quay.io registries. If your cluster runs on a restricted network, such as in a VPC without a public gateway or classic worker nodes on only a private VLAN, these images are not accessible. To mirror the OperatorHub source images for use in your cluster, see [Disabling and mirroring OperatorHub catalog source images](https://cloud.ibm.com/docs/openshift?topic=openshift-operatorsmirror-operatorhub).

The worker node can take a few minutes to provision, but you can see the progress in the Worker nodes tab. When the status reaches Ready, you can start working with your cluster by [deploying your first app](https://cloud.ibm.com/docs/openshift?topic=openshift-getting-starteddeploy-app)!





 Creating classic clusters in the Red Hat OpenShift on IBM Cloud CLI 

Review the sample commands for creating classic clusters in the CLI. For more detailed steps and information about creating clusters, see [Creating classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-create-classic&interface=cli). For information about planning your cluster set up, see [Preparing to create clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-clusters&interface=cli).

Create a classic cluster on a shared virtual machine.
* If you use the certificate from your cluster's kubeconfig file in your own service such as Travis or Jenkins, or if you use calicoctl to manage Calico network policies, update your services and automation to use the new certificates.



6. Verify that the timestamps on your new certificates are later than the timestamp that you found in step 3. To check the date on your certificates, you can use a tool such as [KeyCDN](https://tools.keycdn.com/ssl).
7. [Reload your classic worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_worker_reload) or [replace your VPC worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clicli_worker_replace) to pick up the certificates that are signed by the new CA.
8. Rotate the old certificates with the new certificates. The old CA certificates in your cluster are removed.

ibmcloud oc cluster ca rotate -c CLUSTER
9. Check the status of the CA certificate rotation.

ibmcloud oc cluster ca status -c CLUSTER

Example output

Status: CA certificate rotation complete.
Action Started: 2020-08-30T16:37:56+0000
Action Completed: 2020-08-30T16:41:13+0000









 Worker node 

Worker nodes carry the deployments and services that make up your app. When you host workloads in the public cloud, you want to ensure that your app is protected from being accessed, changed, or monitored by an unauthorized user or software.



 Who owns the worker node and am I responsible to secure it? 

The ownership of a worker node depends on the type of cluster that you create and the infrastructure provider that you choose.



* Classic clusters: Worker nodes are provisioned in to your IBM Cloud account.
After the cluster state shows Normal, the cluster network and load balancing components take about 10 more minutes to deploy and update the cluster domain that you use for the Red Hat OpenShift web console and other routes. Before you continue, wait until the cluster is ready by checking that the Ingress Subdomain follows a pattern of <cluster_name>.<globally_unique_account_HASH>-0001.<region>.containers.appdomain.cloud.



The Red Hat-provided OperatorHub source images require access to the registry.redhat.io and quay.io registries. If your cluster runs on a restricted network, such as in a VPC without a public gateway or classic worker nodes on only a private VLAN, these images are not accessible. To mirror the OperatorHub source images for use in your cluster, see [Disabling and mirroring OperatorHub catalog source images](https://cloud.ibm.com/docs/openshift?topic=openshift-operatorsmirror-operatorhub).

The worker node can take a few minutes to provision, but you can see the progress in the Worker nodes tab. When the status reaches Ready, you can start working with your cluster by [deploying your first app](https://cloud.ibm.com/docs/openshift?topic=openshift-getting-started&interface=uideploy-app)!





 Creating classic clusters in the Red Hat OpenShift on IBM Cloud CLI 

Review the sample commands for creating classic clusters in the CLI. For more detailed steps and information about creating clusters, see [Creating classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-create-classic&interface=cli). For information about planning your cluster set up, see [Preparing to create clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-clusters&interface=cli).

Create a classic cluster on a shared virtual machine.