vania-janet/multiturn-rag-retrieval-data huggingface dataset:
If the issue is still not resolved, follow the steps to [gather your worker node data](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-gather) and open a support ticket.









 Gathering data for a support case 

If you are unable to resolve the issue with the troubleshooting steps, gather information about your worker nodes. Then, [open a support ticket](https://cloud.ibm.com/unifiedsupport/cases/form) and include the worker node information you gathered.

Before you open a support ticket, review the information and follow any troubleshooting steps in [Debugging worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodes), [Worker node states](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-reference), and [Troubleshooting worker nodes in Critical or NotReady state](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready).

If all worker nodes in a cluster, or in a single zone, subnet, or VLAN are affected, you can open an initial support ticket without gathering data. However, you might later be asked to gather the relevant data. If only one or some of your worker nodes are affected, you must gather the relevant data to include in your support ticket.



 Before you begin 

Check the conditions of your worker nodes and cluster before you gather data.



1. Check the CPU and memory level of your nodes. If any node is over 80% in either CPU or memory usage, consider provisioning more nodes or reducing your workload.

kubectl top node

Example output.

NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
10.001.1.01 640m 16% 6194Mi 47%
10.002.2.02 2686m 68% 4024Mi 30%
10.003.3.03 2088m 53% 10735Mi 81%
2.
5. If the issue is still not resolved, follow the steps to [gather your worker node data](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-gather) and open a support ticket.









 Gathering data for a support case 

If you are unable to resolve the issue with the troubleshooting steps, gather information about your worker nodes. Then, [open a support ticket](https://cloud.ibm.com/unifiedsupport/cases/form) and include the worker node information you gathered.

Before you open a support ticket, review the information and follow any troubleshooting steps in [Debugging worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodes), [Worker node states](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-reference), and [Troubleshooting worker nodes in Critical or NotReady state](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready).

If all worker nodes in a cluster, or in a single zone, subnet, or VLAN are affected, you can open an initial support ticket without gathering data. However, you might later be asked to gather the relevant data. If only one or some of your worker nodes are affected, you must gather the relevant data to include in your support ticket.



 Before you begin 

Check the conditions of your worker nodes and cluster before you gather data.



1. Check the CPU and memory level of your nodes. If any node is over 80% in either CPU or memory usage, consider provisioning more nodes or reducing your workload.

oc top node

Example output.

NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
10.001.1.01 640m 16% 6194Mi 47%
10.002.2.02 2686m 68% 4024Mi 30%
10.003.3.03 2088m 53% 10735Mi 81%
* [If all worker nodes in a single zone, subnet, or VLAN are affected](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-steps-zone)
* [If all worker nodes in a cluster are affected](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-steps-all)
* [If worker nodes switch between normal and critical states](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-steps-switch)



* [Gathering data for a support case](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-gather)



* [Before you begin](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-gather-before)
* [Gathering data](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-critical-notreadyts-critical-notready-gather-steps)





[VPC: Why can't I create worker nodes on dedicated hosts?](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-worker-dedicatedts-worker-dedicated)

[Why doesn't replacing a worker node create a worker node?](https://cloud.ibm.com/docs/openshift?topic=openshift-auto-rebalance-offauto-rebalance-off)

[Classic: Why is the bare metal instance ID inconsistent with worker records?](https://cloud.ibm.com/docs/openshift?topic=openshift-bm_machine_idbm_machine_id)
Worker nodes <br><br> * Provide worker node patch operating system (OS), version, and security updates.<br> * Fulfill automation requests to update and recover worker nodes.<br><br><br> <br><br> * Use the API, CLI, or console tools to [apply](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate) the provided worker node updates that include operating system patches; or to request that worker nodes are rebooted, reloaded, or replaced.<br><br><br> 
 Cluster version <br><br> * Provide a suite of tools to automate cluster management, such as the IBM Cloud Kubernetes Service [API](https://containers.cloud.ibm.com/global/swagger-global-api//), [CLI plug-in](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli), and [console](https://cloud.ibm.com/kubernetes/clusters).<br> * Automatically apply Kubernetes master patch OS, version, and security updates.<br> * Make major and minor updates for master nodes available for you to apply.<br> * Provide worker node major, minor, and patch OS, version, and security updates.<br> * Fulfill automation requests to update cluster master and worker nodes.<br><br><br> <br><br> * Use the API, CLI, or console tools to [apply](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate) the provided major and minor Kubernetes master updates and major, minor, and patch worker node updates.<br><br><br> 







 Identity and access management 

You and IBM share responsibilities for controlling access to your IBM Cloud Kubernetes Service instances. For IBM Cloud® Identity and Access Management responsibilities, consult that product's documentation. You are responsible for identity and access management to your application data.



Table 4. Responsibilities for identity and access management

 Resource IBM responsibilities Your responsibilities
Worker nodes <br><br> * Provide worker node patch operating system (OS), version, and security updates.<br> * Fulfill automation requests to update and recover worker nodes.<br><br><br> <br><br> * Use the API, CLI, or console tools to [apply](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate) the provided worker node updates that include operating system patches; or to request that worker nodes are rebooted, reloaded, or replaced.<br><br><br> 
 Cluster version <br><br> * Provide a suite of tools to automate cluster management, such as the Red Hat OpenShift on IBM Cloud [API](https://containers.cloud.ibm.com/global/swagger-global-api//), [CLI plug-in](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli), and [console](https://cloud.ibm.com/kubernetes/clusters).<br> * Automatically apply Red Hat OpenShift master patch OS, version, and security updates.<br> * Make major and minor updates for master nodes available for you to apply.<br> * Provide worker node major, minor, and patch OS, version, and security updates.<br> * Fulfill automation requests to update cluster master and worker nodes.<br><br><br> <br><br> * Use the API, CLI, or console tools to [apply](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate) the provided major and minor Red Hat OpenShift master updates and major, minor, and patch worker node updates.<br><br><br> 







 Identity and access management 

You and IBM share responsibilities for controlling access to your Red Hat OpenShift on IBM Cloud instances. For IBM Cloud® Identity and Access Management responsibilities, consult that product's documentation. You are responsible for identity and access management to your application data.



Table 4. Responsibilities for identity and access management

 Resource IBM responsibilities Your responsibilities
Run the px_logcollect.sh script. You can collect logs from all your worker nodes, or you can specify the --workers option and pass the private IP addresses of the worker nodes from where you want to collect logs. If you specify the --workers option, the log files are saved in the /tmp/pxlogs/<worker_node_IP> directory with the private IP address of each worker node as the folder name. To get the private IP addresses of your worker nodes, run the kubectl get nodes command.



* Collect the logs from all worker nodes in your cluster.

sudo ./px_logcollect.sh
* Collect the logs from only certain worker nodes in your cluster.

sudo ./px_logcollect.sh --workers <worker-IP> <worker-IP> <worker-IP>



4. Review the log files locally. If you can't resolve your issue by reviewing the logs, [open a support ticket](https://cloud.ibm.com/docs/containers?topic=containers-get-help) and provide the log information that you collected.
Worker nodes <br><br> * Provide worker node patch operating system (OS), version, and security updates.<br> * Fulfill automation requests to update and recover worker nodes.<br><br><br> <br><br> * Use the API, CLI, or console tools to [apply](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate) the provided worker node updates that include operating system patches; or to request that worker nodes are rebooted, reloaded, or replaced.<br><br><br> 
 Cluster version <br><br> * Provide a suite of tools to automate cluster management, such as the Red Hat OpenShift on IBM Cloud [API](https://containers.cloud.ibm.com/global/swagger-global-api//), [CLI plug-in](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-cli), and [console](https://cloud.ibm.com/kubernetes/clusters).<br> * Automatically apply Red Hat OpenShift master patch OS, version, and security updates.<br> * Make major and minor updates for master nodes available for you to apply.<br> * Provide worker node major, minor, and patch OS, version, and security updates.<br> * Fulfill automation requests to update cluster master and worker nodes.<br><br><br> <br><br> * Use the API, CLI, or console tools to [apply](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate) the provided major and minor Red Hat OpenShift master updates and major, minor, and patch worker node updates.<br><br><br> 







 Identity and access management 

You and IBM share responsibilities for controlling access to your Red Hat OpenShift on IBM Cloud instances. For IBM Cloud® Identity and Access Management responsibilities, consult that product's documentation. You are responsible for identity and access management to your application data.



Table 4. Responsibilities for identity and access management

 Resource IBM responsibilities Your responsibilities
[Adding worker nodes and zones to clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersadd_workers)



* [Adding worker nodes by resizing an existing worker pool](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersresize_pool)
* [Adding worker nodes in VPC clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersvpc_pools)



* [Creating a new worker pool](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersvpc_add_pool)
* [Adding a zone to a worker pool](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersvpc_add_zone)



* [Adding worker nodes in classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersclassic_pools)



* [Creating a new worker pool](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersadd_pool)
* [Adding a zone to a worker pool](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workersadd_zone)



* [Managing Satellite worker pools](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workerssatcluster-worker-pools)



* [Creating Satellite worker pools with host labels for autoassignment](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workerssat-pool-create-labels)
* [Maintaining Satellite worker pools](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workerssat-pool-maintenance)
* [Step 4: Review the infrastructure provider for the worker node](https://cloud.ibm.com/docs/containers?topic=containers-debug_worker_nodesworker-debug-rev-infra)





[Debugging worker nodes with Kubernetes API](https://cloud.ibm.com/docs/containers?topic=containers-debug-kube-nodesdebug-kube-nodes)

[Worker node states](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-state-reference)



* [Critical state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-critical)
* [Deleting state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-deleting)
* [Deleted state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-deleted)
* [Deployed state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-deployed)
* [Deploying state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-deploying)
* [Deploy_failed state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-deploy-failed)
* [Normal state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-normal)
* [Provisioned state](https://cloud.ibm.com/docs/containers?topic=containers-worker-node-state-referenceworker-node-provisioned)
List the details for the worker node to find the details for the failure by running ibmcloud ks worker get --cluster <cluster_name_or_id> --worker <worker_node_id>.





 Normal state 

You can view the current worker node state by running the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Normal state means that your worker node is fully provisioned and ready to be used in the cluster. This state is considered healthy and does not require an action from the user.

Although the worker nodes might be normal, other infrastructure resources, such as [networking](https://cloud.ibm.com/docs/containers?topic=containers-coredns_lameduck) and [storage](https://cloud.ibm.com/docs/containers?topic=containers-debug_storage_file), might still need attention.





 Provisioned state 

You can view the current worker node state by running the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Provisioned state means that your worker node completed provisioning and is part of the cluster. Billing for the worker node begins. The worker node state soon reports a regular health state and status, such as normal and ready.





 Provisioning state 

You can view the current worker node state by running the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Provisioning state means that your worker node is being provisioned and is not available in the cluster yet. You can monitor the provisioning process in the Status column of your CLI output. If your worker node is stuck in this state for a long time, check whether a problem occurred during the provisioning.





 Provision pending state 

You can view the current worker node state by running the ibmcloud ks worker ls --cluster <cluster_name_or_ID> command and locating the State and Status fields.

A Provision pending state means that another process is completing before the worker node provisioning process starts.
* [If all worker nodes in a cluster are affected](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-steps-all)
* [If worker nodes switch between normal and critical states](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-steps-switch)



* [Gathering data for a support case](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-gather)



* [Before you begin](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-gather-before)
* [Gathering data](https://cloud.ibm.com/docs/containers?topic=containers-ts-critical-notreadyts-critical-notready-gather-steps)





[VPC: Why can't I create worker nodes on dedicated hosts?](https://cloud.ibm.com/docs/containers?topic=containers-ts-worker-dedicatedts-worker-dedicated)

[Why doesn't replacing a worker node create a worker node?](https://cloud.ibm.com/docs/containers?topic=containers-auto-rebalance-offauto-rebalance-off)

[Classic: Why is the bare metal instance ID inconsistent with worker records?](https://cloud.ibm.com/docs/containers?topic=containers-bm_machine_idbm_machine_id)

[After deleting all worker nodes, why don't my pods start on new worker nodes?](https://cloud.ibm.com/docs/containers?topic=containers-zero_nodes_calico_failurezero_nodes_calico_failure)

[After a worker node updates or reloads, why do duplicate nodes and pods appear?]
xvdb 202:16 0 2G 0 disk
-xvdb1 202:17 0 2G 0 part
xvdc 202:32 0 100G 0 disk
xvde 202:64 0 50G 0 disk /var/data
xvdh 202:112 0 64M 0 disk
5. Repeat the previous steps for each worker node that you want to use for your local file storage configuration.







 Labeling your worker nodes when using local file storage 

After you have [retrieved the device paths for the disks that you want to use in your configuration](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-local-volume-filesat-storage-file-local-devices), label the worker nodes where the disks are located.



1. Get the worker node IP addresses.

oc get nodes
2. Label the worker nodes that you retrieved earlier. The local storage drivers are deployed to the worker nodes with this label. You can use the storage=local-file label in the example command or you can create your own label in the key=value format.

oc label nodes <worker-IP> <worker-IP> "storage=local-file"

Example output

node/<worker-IP> labeled
node/<worker-IP> labeled
3. Verify that the label is added to the worker nodes that you want to use. Run the following command to display the labels on your worker nodes and highlight the label that you added in the previous step.

oc get nodes --show-labels | grep --color=always storage=local-file



Before you begin, review the [parameter reference](https://cloud.ibm.com/docs/satellite?topic=satellite-storage-local-volume-filelocal-volume-file-parameter-reference) for the template version that you want to use.







 Creating and assigning a configuration in the console 



1. [From the Locations console](https://cloud.ibm.com/satellite/locations), select the location where you want to create a storage configuration.
2. Select Storage > Create storage configuration
3. Enter a name for your configuration.
4.
[Updating clusters, worker nodes, and cluster components](https://cloud.ibm.com/docs/openshift?topic=openshift-updateupdate)



* [Updating the master](https://cloud.ibm.com/docs/openshift?topic=openshift-updatemaster)



* [About updating the master](https://cloud.ibm.com/docs/openshift?topic=openshift-updatemaster-about)
* [Steps to update the cluster master](https://cloud.ibm.com/docs/openshift?topic=openshift-updatemaster-steps)



* [Updating classic worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-updateworker_node)



* [Prerequisites](https://cloud.ibm.com/docs/openshift?topic=openshift-updateworker-up-prereqs)
* [Updating classic worker nodes in the CLI with a configmap](https://cloud.ibm.com/docs/openshift?topic=openshift-updateworker-up-configmap)
* [Updating classic worker nodes in the console](https://cloud.ibm.com/docs/openshift?topic=openshift-updateworker_up_console)



* [Updating VPC worker nodes](https://cloud.ibm.com/docs/openshift?topic=openshift-updatevpc_worker_node)



* [Prerequisites](https://cloud.ibm.com/docs/openshift?topic=openshift-updatevpc_worker_prereqs)
* [Updating VPC worker nodes in the CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-updatevpc_worker_cli)
* [Debugging your worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_worker_nodesworker-debug-steps)



* [Step 1: Get the worker node state](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_worker_nodesworker-debug-get-state)
* [Step 2: Review the worker node state](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_worker_nodesworker-debug-rev-state)
* [Step 3: Get the details for each worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_worker_nodesworker-debug-get-details)
* [Step 4: Review the infrastructure provider for the worker node](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_worker_nodesworker-debug-rev-infra)





[Debugging worker nodes with Kubernetes API](https://cloud.ibm.com/docs/openshift?topic=openshift-debug-kube-nodesdebug-kube-nodes)

[Worker node states](https://cloud.ibm.com/docs/openshift?topic=openshift-worker-node-state-referenceworker-node-state-reference)



* [Critical state](https://cloud.ibm.com/docs/openshift?topic=openshift-worker-node-state-referenceworker-node-critical)
* [Deleting state](https://cloud.ibm.com/docs/openshift?topic=openshift-worker-node-state-referenceworker-node-deleting)
You can also find the zones of your worker nodes by using the kubectl CLI. Navigate to the Access panel and follow the instructions under Gain access to your cluster to connect to your cluster by using the IBM Cloud and kubectl CLI tools. When you are connected, use the command kubectl get nodes --show-labels to get the full list of nodes and zones of your cluster. You will be to find the zone that each worker node is located after zone field under the LABELS column.





1. To create a node within a specific zone, provide the zone name to the [Create an ordering service](https://cloud.ibm.com/apidocs/blockchaincreate-an-ordering-service) or [Create a peer](https://cloud.ibm.com/apidocs/blockchaincreate-a-peer) API calls by using the zone field of the request body. The anti-affinity policy of the IBM Blockchain Platform console will automatically deploy your component to different worker nodes within each zone based on the resources available.









 Creating a node with a custom configuration 

If you are using the IBM Blockchain APIs to deploy a CA, peer, or ordering node, you have the option of customizing the node configuration by using a configuration override JSON string. The nodes that are deployed by the IBM Blockchain Console and APIs are configured with the default Fabric values that are provided in the fabric-ca-server-config.yaml, orderer.yaml, and core.yaml files. You can customize your node settings by providing a configuration override JSON to the APIs that create or update your nodes. You can use the configuration override to deploy a High Availability CA or use a Hardware Security Module (HSM) while using the IBM Blockchain APIs. For more information about the configuration override, High Availability CAs, or HSMs, see [Advanced deployment options](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-adv-deploymentibp-console-adv-deployment).



 Example: Creating a custom Certificate Authority