vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* [Adding worker nodes by resizing an existing worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersresize_pool)
* [Adding worker nodes in VPC clusters](https://cloud.ibm.com/docs/containers?topic=containers-add_workersvpc_pools)



* [Creating a new worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersvpc_add_pool)
* [Adding a zone to a worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersvpc_add_zone)



* [Adding worker nodes in classic clusters](https://cloud.ibm.com/docs/containers?topic=containers-add_workersclassic_pools)



* [Creating a new worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersadd_pool)
* [Adding a zone to a worker pool](https://cloud.ibm.com/docs/containers?topic=containers-add_workersadd_zone)



* [Deprecated: Adding stand-alone worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-add_workersstandalone)
* [Installing SGX drivers and platform software on SGX-capable worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-add_workersinstall-sgx)



* [Installing with a script](https://cloud.ibm.com/docs/containers?topic=containers-add_workersintel-sgx-script)



* [Adding tags to existing clusters](https://cloud.ibm.com/docs/containers?topic=containers-add_workerscluster_tags)



* [Adding tags to clusters with the console](https://cloud.ibm.com/docs/containers?topic=containers-add_workersadd-tags-console)
List the details for the worker node to find the details for the failure by running ibmcloud oc worker get --cluster <cluster_name_or_id> --worker <worker_node_id>.





 Normal state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Normal state means that your worker node is fully provisioned and ready to be used in the cluster. This state is considered healthy and does not require an action from the user.

Although the worker nodes might be normal, other infrastructure resources, such as [networking](https://cloud.ibm.com/docs/containers?topic=containers-coredns_lameduck) and [storage](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_file), might still need attention.





 Provisioned state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Provisioned state means that your worker node completed provisioning and is part of the cluster. Billing for the worker node begins. The worker node state soon reports a regular health state and status, such as normal and ready.





 Provisioning state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Provisioning state means that your worker node is being provisioned and is not available in the cluster yet. You can monitor the provisioning process in the Status column of your CLI output. If your worker node is stuck in this state for a long time, check whether a problem occurred during the provisioning.





 Provision pending state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Provision pending state means that another process is completing before the worker node provisioning process starts.
Worker node names worker-nodes Config A comma separated list of the worker node names where you want to deploy ODF. Leave this field blank to deploy ODF across all worker nodes in your cluster. The minimum number of worker nodes is 3. You can find your worker node names by running oc get nodes. false N/A 
 Upgrade odf-upgrade Config If you are upgrading an existing ODF installation, set to true. true false 
 Billing type billing-type Config The billing type you want to use. Choose from essentials or advanced. true advanced 
 IBM COS endpoint ibm-cos-endpoint Config The IBM COS regional public endpoint. false N/A 
 IBM COS location constraint ibm-cos-location Config The location constraint that you want to use when creating your COS bucket. For example: us-east-standard. false N/A 
 Access key ID ibm-cos-access-key Secret Your IBM COS HMAC access key ID . false N/A 
 Secret access key ibm-cos-secret-key Secret Your IBM COS HMAC secret access key. false N/A 
 Encryption enabled cluster-encryption Config Set to true if you want to enable cluster-wide encryption. true false 
 IAM API key iam-api-key Secret Your IAM API key. true N/A 
 Perform Cleanup perform-cleanup Config Set to true if you want to perform complete cleanup of ODF on assignment deletion true false 







 4.10 parameter reference 



Table 2. 4.10 parameter reference

 Display name CLI option Type Description Required? Default value 

 Automatic storage volume discovery auto-discover-devices Config Automatically discover and use the storage volumes on your worker nodes. If set to false then you must explicitly provide the volumes IDs. true true 
 OSD volume IDs osd-device-path Config The local storage device path. Example: /dev/sdc. Required when auto-discover-devices is set to false. false N/A
GET/v1/clusters/{idOrName}/workers/{workerId} View details of a worker node. containers-kubernetes.cluster.read N/A 
 GET/v2/classic/getWorker View details of a worker node for classic cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/classic/getWorkerPool View details of a worker pool for a classic cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/classic/getWorkerPools View details of a worker pool for a classic cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/classic/getWorkers View all workers for a classic cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/getWorker View details of a worker node for cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/getWorkerPool View details of a worker pool for a cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/getWorkerPools View details of a worker pool for a cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/getWorkers View all workers for cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/vpc/getWorker View details of a worker node for VPC cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/vpc/getWorkerPool View details of a worker pool for a VPC cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/vpc/getWorkerPools View details of a worker pool for a VPC cluster. containers-kubernetes.cluster.read N/A 
 GET/v2/vpc/getWorkers View all workers for VPC cluster. containers-kubernetes.cluster.read N/A
oc get nodes
2. Get the details of each node and make a note of the zone that the node is in. For example: eastus-1.

oc get nodes <node-name> -o yaml | grep zone
3. Label your worker nodes with the zone value that you retrieved earlier. Replace <node-name> and <zone> with the node name and zone of your worker node. For example, if you have a worker node in zone: eastus-1, use the following commands to add eastus-1 as a label to the worker node in the eastus-1 zone.

oc label node <node-name> topology.kubernetes.io/zone-
oc label node <node-name> topology.kubernetes.io/zone=<zone> --overwrite
4. Repeat the previous steps for each worker node
5. [Sign in to your Azure account](https://azure.microsoft.com/en-us/get-started/) and retrieve the required parameters. For more information about the parameters, see [Cluster config](https://cloud-provider-azure.sigs.k8s.io/install/configs/cluster-config).



Before you begin, review the [parameter reference](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-azuredisk-csi-driverazuredisk-csi-driver-parameter-reference) for the template version that you want to use.







 Creating and assigning a configuration in the console 



1. [From the Locations console](https://cloud.ibm.com/satellite/locations), select the location where you want to create a storage configuration.
2. Select Storage > Create storage configuration
3. Enter a name for your configuration.
4. Select the Storage type.
5. Select the Version and click Next
6. If the Storage type that you selected accepts custom parameters, enter them on the Parameters tab.
7. If the Storage type that you selected requires secrets, enter them on the Secrets tab.
In the [IBM Cloud console resource list](https://cloud.ibm.com/resources), worker nodes on classic infrastructure are classified as compute resources, or virtual machines. Sometimes a user might not realize that these resources function as cluster worker nodes, and might unintentionally power the worker nodes down. Worker nodes that are powered down might show up in the Critical or NotReady state. Ensure that the affected worker nodes are not powered down.





 Troubleshooting steps 

If your worker nodes remain in the Critical or NotReady state after addressing the [common causes](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-common), continue with the following troubleshooting steps.

If a worker node that you previously reloaded or replaced is in the deploy_failed or provision_failed state when you run ibmcloud ks workers, follow the steps in the [All worker nodes in a cluster are affected](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-steps-all) section, even if not all nodes are affected. If a different state is indicated, see [Worker node states](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-reference) for steps to troubleshoot the new worker. Do not replace or reload any additional worker nodes.



 If one or some worker nodes are affected 

If only some, but not all, of the worker nodes in your cluster are in a Critical or NotReady state, follow these steps to determine the cause of the disruption and resolve the issue. If the affected worker nodes are all from the same zone, subnet, or VLAN, continue to the next section.



1. Get the details of the specific node.

oc describe node <node-IP-address>
2. In the output, check the Conditions section to determine if the node is experiencing any memory, disk or PID issues. This information might indicate that the node is running low on that type of resource. This situation might occur for one of the following reasons:
Run the px_logcollect.sh script. You can collect logs from all your worker nodes, or you can specify the --workers option and pass the private IP addresses of the worker nodes from where you want to collect logs. If you specify the --workers option, the log files are saved in the /tmp/pxlogs/<worker_node_IP> directory with the private IP address of each worker node as the folder name. To get the private IP addresses of your worker nodes, run the oc get nodes command.



* Collect the logs from all worker nodes in your cluster.

sudo ./px_logcollect.sh
* Collect the logs from only certain worker nodes in your cluster.

sudo ./px_logcollect.sh --workers <worker-IP> <worker-IP> <worker-IP>



4. Review the log files locally. If you can't resolve your issue by reviewing the logs, [open a support ticket](https://cloud.ibm.com/docs/openshift?topic=openshift-get-help) and provide the log information that you collected.
Every label consists of a NodeSelectorKey and a NodeSelectorValue. You can use the labels to specify which worker nodes to update.

kubectl get nodes -o yaml

Example output

labels:
arch: amd64
beta.kubernetes.io/arch: amd64
beta.kubernetes.io/instance-type: upi
beta.kubernetes.io/os: linux
failure-domain.beta.kubernetes.io/region: us-east
failure-domain.beta.kubernetes.io/zone: us-east-2
ibm-cloud.kubernetes.io/iaas-provider: upi
ibm-cloud.kubernetes.io/internal-ip: 10.241.0.4
ibm-cloud.kubernetes.io/machine-type: upi
ibm-cloud.kubernetes.io/os: REDHAT_7_64
ibm-cloud.kubernetes.io/region: us-east
ibm-cloud.kubernetes.io/worker-id: sat-satellitei-5bd83963cdfedc7668c9040879c88113a134c4e7
ibm-cloud.kubernetes.io/worker-pool-id: cbtljodw089nltg8k210-9a3f763
ibm-cloud.kubernetes.io/worker-pool-name: default
ibm-cloud.kubernetes.io/worker-version: 4.7.59_1583_openshift
ibm-cloud.kubernetes.io/zone: us-east-2
kubernetes.io/arch: amd64
kubernetes.io/hostname: satellite-ibm-host-3
kubernetes.io/os: linux
node-role.kubernetes.io/master: ""
node-role.kubernetes.io/worker: ""
node.kubernetes.io/instance-type: upi
Any data that was stored only on the worker node, such as container images, are also deleted.





 Deployed state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

Updates are successfully deployed to your worker node. After updates are deployed, Red Hat OpenShift on IBM Cloud starts a health check on the worker node. After the health check is successful, the worker node goes into a Normal state. Worker nodes in a Deployed state usually are ready to receive workloads, which you can check by running oc get nodes and confirming that the state shows Normal.





 Deploying state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Deploying state means that when you update the Kubernetes version of your worker node, your worker node is redeployed to install the updates. If you reload or reboot your worker node, the worker node is redeployed to automatically install the latest patch version. If your worker node is stuck in this state for a long time, check whether a problem occurred during the deployment.





 Deploy_failed state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Deploy_failed state means that your worker node could not be deployed. List the details for the worker node to find the details for the failure by running ibmcloud oc worker get --cluster <cluster_name_or_id> --worker <worker_node_id>.





 Normal state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Normal state means that your worker node is fully provisioned and ready to be used in the cluster. This state is considered healthy and does not require an action from the user.
Worker node states 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.



 Critical state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A worker node can go into a Critical state for many reasons. See [Troubleshooting worker nodes in Critical or NotReady state](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notready) for more information and troubleshooting steps.





 Deleting state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Deleting state means that you requested to delete the worker node, possibly as part of resizing a worker pool or autoscaling the cluster. Other operations can't be issued against the worker node while the worker node deletes. You can't reverse the deletion process. When the deletion process completes, you are no longer billed for the worker nodes.





 Deleted state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Deleted state means that your worker node is deleted, and no longer is listed in the cluster or billed. This state can't be undone. Any data that was stored only on the worker node, such as container images, are also deleted.





 Deployed state 

You can view the current worker node state by running the ibmcloud oc worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

Updates are successfully deployed to your worker node. After updates are deployed, Red Hat OpenShift on IBM Cloud starts a health check on the worker node. After the health check is successful, the worker node goes into a Normal state.
In the [IBM Cloud console resource list](https://cloud.ibm.com/resources), worker nodes on classic infrastructure are classified as compute resources, or virtual machines. Sometimes a user might not realize that these resources function as cluster worker nodes, and might unintentionally power the worker nodes down. Worker nodes that are powered down might show up in the Critical or NotReady state. Ensure that the affected worker nodes are not powered down.





 Troubleshooting steps 

If your worker nodes remain in the Critical or NotReady state after addressing the [common causes](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-common), continue with the following troubleshooting steps.

If a worker node that you previously reloaded or replaced is in the deploy_failed or provision_failed state when you run ibmcloud ks workers, follow the steps in the [All worker nodes in a cluster are affected](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-steps-all) section, even if not all nodes are affected. If a different state is indicated, see [Worker node states](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-reference) for steps to troubleshoot the new worker. Do not replace or reload any additional worker nodes.



 If one or some worker nodes are affected 

If only some, but not all, of the worker nodes in your cluster are in a Critical or NotReady state, follow these steps to determine the cause of the disruption and resolve the issue. If the affected worker nodes are all from the same zone, subnet, or VLAN, continue to the next section.



1. Get the details of the specific node.

kubectl describe node <node-IP-address>
2. In the output, check the Conditions section to determine if the node is experiencing any memory, disk or PID issues. This information might indicate that the node is running low on that type of resource. This situation might occur for one of the following reasons:
If the details include an error message, review the list of [common error messages for worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-common_worker_nodes_issues) to learn how to resolve the problem.

ibmcloud ks worker get --cluster <cluster_name_or_id> --worker <worker_node_id>





 Step 4: Review the infrastructure provider for the worker node 

Review the infrastructure environment to check for other reasons that might cause the worker node issues.



1. Check with your networking team to make sure that no recent maintenance, such as firewall or subnet updates, might impact the worker node connections.
2. Review [IBM Cloud](https://cloud.ibm.com/status/) for IBM Cloud Kubernetes Service and the underlying infrastructure provider, such as Virtual Servers for classic, VPC related components, or Satellite.
3. If you have access to the underlying infrastructure, such as classic Virtual Servers, review the details of the corresponding machines for the worker nodes.
Your Flow Logs for VPC gathers information from the VPC, VPC subnet, or VPC load balancer level. However, you can use the flow logs to gather information that is specific to your worker nodes. Separate flow log files are created for ingress and egress traffic.



1. In the CLI, find the ibm-cloud.kubernetes.io/instance-id label value for the worker node.

oc describe node <worker_node_ip> | grep instance-id

Example output.

ibm-cloud.kubernetes.io/instance-id=1010_a1aa1010-a1a0-1010-a1aa-aa1a1-a1-aa1
2. In the IBM Cloud UI, click your IBM Cloud Object Storage instance in the Resource list.
3. Click the bucket where your flow logs are collected.
4. Download and decompress the flow log object.
5. Open the file and navigate through the file directory until you reach directories that begin with instance-id=.
6. Find the file directory that contains the instance ID found in the first step. The ID is included at the end of the file directory name. Example.

instance-id=crn%3AV1%...%3Ainstance%3A1010_a1aa1010-a1a0-1010-a1aa-aa1a1-a1-aa1
7. In the instance=id= directory, locate the record-type=ingress and record-type=egress files. Your ingress and egress traffic logs are located here.
How many worker nodes do I need to handle my workload? 

Now that you have a good idea of what your workload looks like, let's map the estimated usage onto your available cluster configurations.



1. Estimate the max worker node capacity, which depends on what type of cluster you have. You don't want to max out worker node capacity in case a surge or other temporary event happens.



* Single zone clusters: Plan to have at least three worker nodes in your cluster. Further, you want one extra node's worth of CPU and memory capacity available within the cluster.
* Multizone clusters: Plan to have at least two worker nodes per zone, so six nodes across three zones in total. Additionally, plan for the total capacity of your cluster to be at least 150% of your total workload's required capacity, so that if one zone goes down, you have resources available to maintain the workload.



2. Align the app size and worker node capacity with one of the [available worker node flavors](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodesplanning_worker_nodes). To see available flavors in a zone, run ibmcloud ks flavors --zone <zone>.



* Don't overload worker nodes: To avoid your pods competing for CPU or running inefficiently, you must know what resources your apps require so that you can plan the number of worker nodes that you need. For example, if your apps require less resources than the resources that are available on the worker node, you can limit the number of pods that you deploy to one worker node. Keep your worker node at around 75% capacity to leave space for other pods that might need to be scheduled. If your apps require more resources than you have available on your worker node, use a different worker node flavor that can fulfill these requirements. You know that your worker nodes are overloaded when they frequently report back a status of NotReady or evict pods due to the lack of memory or other resources.
* Larger versus smaller worker node flavors: Larger nodes can be more cost efficient than smaller nodes, particularly for workloads that are designed to gain efficiency when they process on a high-performance machine.