vania-janet/multiturn-rag-retrieval-data huggingface dataset:
To review the health of Red Hat OpenShift on IBM Cloud clusters that run in your Satellite location, see the [Red Hat OpenShift on IBM Cloud documentation](https://cloud.ibm.com/docs/openshift?topic=openshift-health-monitorstates).





 Viewing Kubernetes resources in clusters 

When you add your clusters to Satellite Configuration, the Kubernetes resources are automatically added to an inventory that you can review. For more information, see [Managing apps with Satellite configurations](https://cloud.ibm.com/docs/satellite?topic=satellite-setup-clusters-satconfig).

Adding clusters to Satellite Configuration does not automatically set up logging and monitoring solutions, such as IBM Log Analysis and IBM Cloud Monitoring.





 Viewing Satellite Config registration status for clusters 

You can view the registration status of clusters that are enabled for use with Satellite Config. Keep in mind that some of these clusters might be in a public cloud location, not your Satellite location.



1. List clusters that are registered with Satellite Config. Note the output in the Status and Location columns.

ibmcloud sat cluster ls
2. Review the following Satellite Config registration statuses.





Host health states.

 Status Description 

 active Satellite Config components for the location are installed in the cluster, and at least one resource is being watched. 
 inactive Satellite Config components were manually removed from the cluster, or are installed but are no longer responding to Satellite Config. For example, network connectivity might be disconnected. Existing resources, if any, continue to run but do not receive updates. To resolve the issue, try debugging your [Satellite location](https://cloud.ibm.com/docs/satellite?topic=satellite-ts-locations-debug) or [cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_clusters). 
 registered Satellite Config components are installed in the cluster, but no resources are currently watched.
Migrating to a new Red Hat Enterprise Linux version 

Virtual Private Cloud

Classic infrastructure

Satellite

With the release of RHEL 8, the use of RHEL 7 worker nodes is deprecated in clusters that run version 4.10 and is not supported in clusters that run 4.11 or greater. You cannot upgrade RHEL 7 worker nodes to RHEL 8. Instead, you must provision a new worker pool and then delete the previous worker pool. In versions 4.10 and later, worker nodes in the new worker pool run RHEL 8 by default.

To find your worker node operating system, run the ibmcloud oc worker-pools -c CLUSTER command.



 Migration actions by cluster version 

If you have Portworx installed, follow the steps to upgrade your installation to use at least image version 2.11.4. For more information, see [Upgrading Portworx to a specific version](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_about).

Version 4.10+ clusters
: By default, clusters that run version 4.10 and later are created with RHEL 8 worker nodes.
: The RHEL 7 operating system is deprecated for version 4.10.
: Existing 4.10 clusters must be migrated to RHEL 8 before support for RHEL 7 in version 4.10 ends on 6 December 2022.

Version 4.9 clusters
: Version 4.9 supports both RHEL 7 and 8 until end of support for 4.9. RHEL 7 remains the default version when creating new clusters.
: If you want to upgrade a 4.9 cluster to 4.10, make sure to migrate your worker nodes from RHEL 7 to RHEL 8 before you upgrade.
: For more information about updating to 4.9, see the [Version information and update actions](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_versions_49).

Version 4.8 clusters
: RHEL 7 is supported until end of support for these cluster versions.
Optional: Decide if you want to set up [encryption for your Portworx volumes](https://cloud.ibm.com/docs/containers?topic=containers-storage_portworx_encryption).
5. Follow the instructions to [install Portworx](https://cloud.ibm.com/docs/containers?topic=containers-storage_portworx_deploy) with the disaster recovery plan in both of your clusters. If you installed Portworx without the disaster recovery plan in one of your clusters already, you must re-install Portworx in that cluster with the disaster recovery plan. Make sure that you select Databases for etcd from the Portworx metadata key-value store drop-down and that you enter the same Databases for etcd API endpoint and Kubernetes secret name in both of your clusters.
6. Continue following the [Portworx documentation](https://docs.portworx.com/portworx-install-with-kubernetes/disaster-recovery/px-metro/2-pair-clusters/) to pair your clusters, sync data between them, and try out a failover of an application.



Asynchronous DR:



1. Choose at least two Kubernetes clusters that are located in different regions. If you have one cluster only, you can still configure this cluster for asynchronous disaster recovery, but Portworx can't do a proper failover until a second cluster is configured.
2. Make sure that all your clusters have sufficient [raw and unformatted block storage](https://cloud.ibm.com/docs/containers?topic=containers-utilitiesmanual_block) so that you can build your Portworx storage layer.
3. Review your [options to configure a Portworx key-value store](https://cloud.ibm.com/docs/containers?topic=containers-storage_portworx_kv_store). Because both clusters are in different regions, each cluster must use its own key-value store.
Each option offers a set of predefined dashboards that you can use to monitor the health of your resources. You can also select to monitor by host.



 Monitoring Red Hat OpenShift clusters by cluster 

When you choose to monitor your clusters by cluster, you can select more filters to display data by node or by namespace, or you can choose any of the following dashboards:



* Workload Status & Performance
* Node Status & Performance
* Pod Rightsizing & Workload Capacity Optimization
* Cluster Capacity Planning
* Cluster / Namespace Available Resources
* Cluster Overview
* CPU Allocation Optimization
* Memory Allocation Optimization



Zoom

![Advisor predefined dashboards by cluster](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/images/overview-tab-clusters.png)

Advisor predefined dashboards by cluster

For more information on how to interpret this view, see [About Clusters Overview](https://docs.sysdig.com/en/docs/sysdig-monitor/advisor/overview/clusters-data/).





 Monitoring clusters by node 

When you choose to monitor your clusters by node, you can choose any of the following dashboards:



* Node Status & Performance
* Pod Scheduling Troubleshooting
* Node Overview
* CPU Allocation Optimization
* Memory Allocation Optimization



For more information on how to interpret this view, see [About Nodes Overview](https://docs.sysdig.com/en/docs/sysdig-monitor/advisor/overview/nodes-data/).





 Monitoring clusters by namespace 

When you choose to monitor your clusters by namespace, you can select more filters to display data by workload, or you can choose any of the following dashboards:



* Workload Status & Performance
* Pod Status & Performance
* Pod Rightsizing & Workload Capacity Optimization
* Namespace Overview
* Workloads CPU Usage and Allocation
* Workloads Memory Usage and Allocation
Expose multiple apps in a cluster by setting up routing with the [Ingress](https://cloud.ibm.com/docs/containers?topic=containers-managed-ingress-about) application load balancer (ALB). The ALB uses a secured and unique public or private entry point, an Ingress subdomain, to route incoming requests to your apps. You can use one subdomain to expose multiple apps in your cluster as services. Ingress consists of three components:



* The Ingress resource defines the rules for how to route and load balance incoming requests for an app.
* The ALB listens for incoming HTTP, HTTPS, or TCP service requests. It forwards requests across the apps' pods based on the rules that you defined in the Ingress resource.
* The multizone load balancer (MZLB) for classic clusters or the VPC load balancer for VPC clusters handles all incoming requests to your apps and load balances the requests among the ALBs in the various zones. It also enables health checks for the public Ingress IP addresses.



Zoom

![Ingress service traffic in Classic clusters](https://cloud.ibm.com/docs-content/v1/content/4d497aff9e27b92fbbc2ee4e343a9ec4549cb1d7/containers/images/classic-ingress-traffic.svg)

Ingress service traffic in Classic clusters.

Zoom

![Ingress service traffic in VPC clusters](https://cloud.ibm.com/docs-content/v1/content/4d497aff9e27b92fbbc2ee4e343a9ec4549cb1d7/containers/images/vpc-ingress-traffic.svg)

Ingress service traffic in VPC clusters.







 Planning public external load balancing 

Publicly expose an app in your cluster to the internet.

In classic clusters, you can connect worker nodes to a public VLAN. The public VLAN determines the public IP address that is assigned to each worker node, which provides each worker node with a public network interface.
Connecting Lone Symphony and SMC Cluster 

After the Lone Symphony cluster is provisioned, add the SMC CIDR block to the Security Group of all previously created Lone symphony clusters to allow traffic between the SMC cluster and the Lone symphony cluster. When you created the SMC cluster by using the existing_lone_vpc and existing_lone_vpc_region and the Lone symphony VPC was added to the SMC Transit Gateway.



1.  On DNS Zones, add these forwarding rules:



*  Lone symphony clusters to SMC using domain_name
*  SMC to lone symphony using dns_server_ip



2.  Add a forwarding rule within each Lone symphony cluster to enable traffic between Lone symphony clusters.
3.  Verify the connection by pinging the host names of the SMC and Lone symphony clusters from each other.
* [Setting up public routes in VPC clusters with a private cloud service endpoint only](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_routesroutes-public-vpc-privse)



* [Setting up private routes](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_routesprivate-routes)



* [Setting up private routes in classic clusters or in VPC clusters with a public cloud service endpoint](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_routesprivate-routes-setup-43)
* [Setting up private routes in VPC clusters with a private cloud service endpoint only](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_routesroutes-private-vpc-privse)



* [Moving Ingress controller services across VLANs in classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_routesmigrate-router-vlan-classic)







 Exposing apps with load balancers 

[Quick start for load balancers](https://cloud.ibm.com/docs/openshift?topic=openshift-loadbalancer-qsloadbalancer-qs)



* [Exposing an app by using an NLB in a classic cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-loadbalancer-qslb_qs_classic)
* [Exposing an app by using a VPC load balancer in a VPC cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-loadbalancer-qslb_qs_vpc)



[Exposing apps with load balancers for VPC](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-lbaasvpc-lbaas)
Both clusters need to recognize each other, so you need to modify /opt/ibm/lsf/conf/lsf.shared. This configuration file should be identical in both clusters.

...
Begin Cluster
ClusterName Servers Keyword modified
HPCCluster (icgen2host-10-244-128-37) modified
OnPremiseCluster (on-premise-management) modified
End Cluster
...
3. The two clusters are configured to have different lsb.queues files. In the cloud cluster, you need to append the following lines to /opt/ibm/lsf/conf/lsbatch/HPCCluster/configdir/lsb.queues to register a receive queue:

...
Begin Queue
QUEUE_NAME=recv_q
RCVJOBS_FROM=OnPremiseCluster
PRIORITY=30
NICE=20
RC_HOSTS=all
End Queue

The on-premises cluster is configured to have a send queue at /opt/ibm/lsf/conf/lsbatch/OnPremiseCluster/configdir/lsb.queues:

...
Begin Queue
QUEUE_NAME=send_q
SNDJOBS_TO=recv_q@HPCCluster
PRIORITY=30
NICE=20
End Queue
4. Restart both clusters by running the following command:

$ lsfrestart
5. After you restart both clusters, you can now forward jobs from on-premises to the cloud. At your on-premises cluster, you can test the following job:

$ bsub -q send_q sh -c 'echo $HOSTNAME > /home/lsfadmin/shared/mc-test.txt'

You can see that the job comes to the HPCCluster at 10.244.128.37.

$ bjobs -aw

JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME
For a list of available flavors, see [VPC Gen 2 flavors](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-flavors) or [Classic flavors](https://cloud.ibm.com/docs/openshift?topic=openshift-classic-flavors).





 Do I use multiple clusters, or just add more workers to an existing cluster? 

The number of clusters that you create depends on your workload, company policies and regulations, and what you want to do with the computing resources. You can also review security information about this decision in [Container isolation and security](https://cloud.ibm.com/docs/openshift?topic=openshift-securitycontainer).

Multiple clusters: You need to set up [a global load balancer](https://cloud.ibm.com/docs/openshift?topic=openshift-ha_clustersmultiple-clusters-glb) and copy and apply the same configuration YAML files in each to balance workloads across the clusters. Therefore, multiple clusters are generally more complex to manage, but can help you achieve important goals such as the following.



* Comply with security policies that require you to isolate workloads.
* Test how your app runs in a different version of Red Hat OpenShift or other cluster software such as Calico.
* Create a cluster with your app in another region for higher performance for users in that geographical area.
* Configure user access on the cluster-instance level instead of customizing and managing multiple RBAC policies to control access within a cluster at the namespace level.



Fewer or single clusters: Fewer clusters can help you to reduce operational effort and per-cluster costs for fixed resources. Instead of making more clusters, you can add worker pools for different flavors of computing resources available for the app and service components that you want to use. When you develop the app, the resources it uses are in the same zone, or otherwise closely connected in a multizone, so that you can make assumptions about latency, bandwidth, or correlated failures. However, it becomes even more important for you to organize your cluster by using namespaces, resource quotas, and labels.
Optional: Decide if you want to set up [encryption for your Portworx volumes](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_encryption).
5. Follow the instructions to [install Portworx](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_deploy) with the disaster recovery plan in both of your clusters. If you installed Portworx without the disaster recovery plan in one of your clusters already, you must re-install Portworx in that cluster with the disaster recovery plan. Make sure that you select Databases for etcd from the Portworx metadata key-value store drop-down and that you enter the same Databases for etcd API endpoint and Kubernetes secret name in both of your clusters.
6. Continue following the [Portworx documentation](https://docs.portworx.com/portworx-install-with-kubernetes/disaster-recovery/px-metro/2-pair-clusters/) to pair your clusters, sync data between them, and try out a failover of an application.



Asynchronous DR:



1. Choose at least two Red Hat OpenShift clusters that are located in different regions. If you have one cluster only, you can still configure this cluster for asynchronous disaster recovery, but Portworx can't do a proper failover until a second cluster is configured.
2. Make sure that all your clusters have sufficient [raw and unformatted block storage](https://cloud.ibm.com/docs/openshift?topic=openshift-utilitiesmanual_block) so that you can build your Portworx storage layer.
3. Review your [options to configure a Portworx key-value store](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_portworx_kv_store).
* [Worker-to-worker communication for private clusters](https://cloud.ibm.com/docs/containers?topic=containers-plan_basicsprivate_clusters-worker)
* [Worker-to-master and user-to-master communication for private clusters](https://cloud.ibm.com/docs/containers?topic=containers-plan_basicsprivate_clusters-master)
* [Worker communication to other services or networks for private clusters](https://cloud.ibm.com/docs/containers?topic=containers-plan_basicsprivate_clusters-services)
* [External communication to apps that run on worker nodes for private clusters](https://cloud.ibm.com/docs/containers?topic=containers-plan_basicsprivate_clusters-external)





[Planning your cluster for high availability](https://cloud.ibm.com/docs/containers?topic=containers-ha_clustersha_clusters)



* [Single zone clusters](https://cloud.ibm.com/docs/containers?topic=containers-ha_clusterssingle_zone)



* [Is my master highly available in a single zone cluster?](https://cloud.ibm.com/docs/containers?topic=containers-ha_clusterssz-master-ha)
* [How can I protect my workloads against a single zone failure?](https://cloud.ibm.com/docs/containers?topic=containers-ha_clusterssz-workload-failover)



* [Multizone cluster](https://cloud.ibm.com/docs/containers?topic=containers-ha_clustersmz-clusters)



* [Why do I need worker nodes in three zones?](https://cloud.ibm.com/docs/containers?topic=containers-ha_clustersmz-cluster-zones)
* [How is my IBM Cloud Kubernetes Service master set up?](https://cloud.ibm.com/docs/containers?topic=containers-ha_clustersmz-master-setup)
containers-kubernetes.account-api-key.reset The API key that is used for all clusters in the region and resource group is set to the requesting user's API key credentials. 
 containers-kubernetes.account-customer-root-key.list Root keys from key management service instances for the IBM Cloud account are listed. 
 containers-kubernetes.account-datacenter-vlan.list VLANs in an IBM Cloud account for a particular data center are listed. 
 containers-kubernetes.account-infra-credential.delete Classic infrastructure credentials for managing clusters in the region and resource group are deleted. 
 containers-kubernetes.account-infra-credential.get Details on the classic infrastructure credentials that are set for managing clusters in the region and resource group are returned. 
 containers-kubernetes.account-infra-credential.set Classic infrastructure credentials for managing clusters in the region and resource group are set. 
 containers-kubernetes.account-infra-permission.get Details on the compute, networking, and storage classic infrastructure permissions that are set for managing clusters in the region and resource group are returned. 
 containers-kubernetes.account-key-management-service-instance.list Key management service instances in the IBM Cloud account are listed. 
 containers-kubernetes.account-quota.get The quota for resources such as clusters or worker nodes for the IBM Cloud account is returned. 
 containers-kubernetes.account-reservation-contract.add A contract for a specific term and number of worker nodes is added to a reservation. 
 containers-kubernetes.account-reservation-contract.list Contracts for a reservation are listed. 
 containers-kubernetes.account-subnet.list Subnets in the IBM Cloud classic infrastructure account are listed. 
 containers-kubernetes.account-subnet-vlan-spanning.get Details on whether the IBM Cloud account has VLAN spanning enabled are returned. 
 containers-kubernetes.account-user-config.get Details on whether a user can create in a certain region and resource group are returned. 
 containers-kubernetes.account-vpc.get Details for a virtual private cloud (VPC) instance are returned.
The different versions of the reviews microservice are still returned randomly because the Istio gateway manages the virtual service and destination routing rules for microservices.





 Exposing BookInfo by using an IBM-provided subdomain with TLS 

When you enable the BookInfo add-on in your cluster, the Istio gateway bookinfo-gateway is created for you. The gateway uses Istio virtual service and destination rules to configure a load balancer, istio-ingressgateway, that publicly exposes the BookInfo app. In the following steps, you create a subdomain for the istio-ingressgateway load balancer IP address in classic clusters or the hostname in VPC clusters through which you can publicly access BookInfo. You also use the SSL certificate to enable HTTPS connections to the BookInfo app.



1. Register the IP address in classic clusters or the hostname in VPC clusters for the istio-ingressgateway load balancer by creating a DNS subdomain.



* Classic:

ibmcloud ks nlb-dns create classic --ip $INGRESS_IP --secret-namespace istio-system --cluster <cluster_name_or_id>
* VPC:

ibmcloud ks nlb-dns create vpc-gen2 --lb-host $GATEWAY_URL --secret-namespace istio-system --cluster <cluster_name_or_id>



2. Verify that the subdomain is created and note the name of your SSL secret in the SSL Cert Secret Name field.

ibmcloud ks nlb-dns ls --cluster <cluster_name_or_id>

Example output for classic clusters.

Hostname IP(s) Health Monitor SSL Cert Status SSL Cert Secret Name
mycluster-a1b2cdef345678g9hi012j3kl4567890-0001.us-south.containers.appdomain.cloud ["168.1.1.1"] None created <certificate>

Example output for VPC clusters
If your cluster is not in a healthy state, review the [Debugging clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_clusters) guide for help. For example, if your cluster is provisioned in an account that is protected by a firewall gateway appliance, you must [configure your firewall settings to allow outgoing traffic to the appropriate ports and IP addresses](https://cloud.ibm.com/docs/openshift?topic=openshift-firewall).
5. Find your cluster's service endpoint.



* Classic or VPC clusters: In the output of the cluster details from the previous step, check the Public or Private Service Endpoint URL of the cluster.



* Public Service Endpoint URL only: Continue with [Accessing clusters through the public cloud service endpoint](https://cloud.ibm.com/docs/openshift?topic=openshift-access_clusteraccess_public_se).
* Both service endpoint URLs: You can access your cluster either through the [public](https://cloud.ibm.com/docs/openshift?topic=openshift-access_clusteraccess_public_se) or the [private](https://cloud.ibm.com/docs/openshift?topic=openshift-access_clusteraccess_private_se) service endpoint.



* IBM Cloud Satellite clusters: Continue with [Accessing clusters on Satellite](https://cloud.ibm.com/docs/openshift?topic=openshift-access_clusteraccess_cluster_sat).



6. If any users in your account use multifactor authentication (MFA) like TOTP, make sure that you [enable it for all users at the account level](https://cloud.ibm.com/docs/account?topic=account-enablemfa). To use MFA, it must be enabled at the account level to avoid authentication errors.







 Accessing clusters through the public cloud service endpoint