vania-janet/multiturn-rag-retrieval-data huggingface dataset:
For more information, see [Managing classic infrastructure API keys](https://cloud.ibm.com/docs/account?topic=account-classic_keysclassic_keys).
2. Click the Classic infrastructure tab and then click the Permissions tab.
3. If the user doesn't have each category checked, you can use the Permission sets drop-down list to assign the Super User role. Or you can expand each category and give the user the required [infrastructure permissions](https://cloud.ibm.com/docs/containers?topic=containers-classic-roles).



* For VPC clusters, assign the user the [Administrator platform access role for VPC Infrastructure](https://cloud.ibm.com/docs/vpc?topic=vpc-iam-getting-started).











 Understanding access to the infrastructure portfolio 

Determine whether your account has access to the IBM Cloud infrastructure portfolio and learn about how IBM Cloud Kubernetes Service uses the API key to access the portfolio.

Does the classic or VPC infrastructure provider for my cluster affect what access I need to the portfolio?
: Access to IBM Cloud infrastructure works differently in classic and VPC clusters. Infrastructure resources for classic clusters are created in a separate IBM Cloud infrastructure account. Usually, your Pay-As-You-Go or Subscription account is linked to the IBM Cloud infrastructure account so that account owners can access classic IBM Cloud infrastructure automatically. To authorize other users to access classic compute, storage, and networking resources, you must assign [classic infrastructure roles](https://cloud.ibm.com/docs/containers?topic=containers-classic-roles).

VPC infrastructure resources are integrated into IAM and as such, you must have the IBM Cloud IAM Administrator platform access role to the [VPC Infrastructure service](https://cloud.ibm.com/docs/vpc?topic=vpc-iam-getting-started) to create and list VPC resources.

For both [classic and VPC clusters](https://cloud.ibm.com/docs/containers?topic=containers-infrastructure_providers), the credentials to access infrastructure resources are stored in an API key for the region and resource group of the cluster.
Why does the Ingress status show an ERRADRUH error? 

Virtual Private Cloud

Classic infrastructure

You can use the the ibmcloud ks ingress status-report ignored-errors add command to add an error to the ignored-errors list. Ignored errors still appear in the output of the ibmcloud ks ingress status-report get command, but are ignored when calculating the overall Ingress Status.

 What’s happening 

When you check the status of your cluster's Ingress components by running the ibmcloud ks ingress status-report get command, you see an error similar to the following example.

One or more ALB pod is not in running state (ERRADRUH).

 Why it’s happening 

One or more ALBs have replicas that are not running.

 How to fix it 

Complete the following steps to verify your cluster setup.



1. List your ALB pods.

kubectl get pods -n kube-system | grep -E "public-cr|private-cr"
2. Describe the ALB pods that are not running and review the Events section.

kubectl describe pod POD
3. If you notice scheduler problems, follow the steps:



1. List your ALBs using the ibmcloud ks ingress alb ls[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_albs).
2. List your workers using the ibmcloud ks worker ls[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_workers).
3. Classic clusters: Ensure you have at least two worker nodes in the VLANs where your ALBs are deployed. See [Adding worker nodes and zones to clusters](https://cloud.ibm.com/docs/containers?topic=containers-add_workers).
4. VPC clusters: Ensure you have at least two worker nodes in the zones where your ALBs are deployed.
ALBs listen for incoming HTTP, HTTPS, or TCP service requests and then forward those requests to the appropriate app pod according to the rules you define in the Ingress resource. When you create a standard cluster with either classic or VPC infrastructure, one public and one private ALB is automatically created for you in each zone.



 ALBs in classic clusters 

When you create a classic cluster, one public and one private ALB is automatically created for you in each zone. Public and private ALBs in classic clusters are assigned a static IP address that does not change for the life of the cluster.

Public ALBs share the same IBM-provided Ingress subdomain that is registered for you when you provision a cluster, and each public ALB's individual IP address is linked to this subdomain. To find a public ALB's IP address, run ibmcloud ks ingress alb ls and check the ALB IP field in the output.

Private ALBs in classic clusters do not use the IBM-provided Ingress subdomain and are not automatically enabled. You must first enable private ALBs in the CLI and then specify the private-iks-k8s-nginx class in your Ingress resource. After the private ALBs are enabled, you can find the private IP addresses by running ibmcloud ks ingress alb ls and checking the ALB IP field in the output.

If a public or private ALB pod is rescheduled, it keeps the same IP address. However, removing a zone with an ALB, or removing all workers on a VLAN in that zone, removes that ALB's IP address.





 ALBs in VPC 

When you create a VPC cluster, one public and one private ALB is automatically created for you in each zone. In addition, one public VPC load balancer is automatically created outside of your cluster in your VPC. When you enable private ALBs in your VPC cluster, a private VPC load balancer is also created.

IP addresses for ALBs in VPC clusters are not static and might change over time, therefore the VPC load balancers put the public or private IP address of your ALBs behind a static hostname. Separate hostnames are applied for public or private ALBs.
If you have a multizone cluster, note that the Ingress controller service in the first zone where you have workers nodes is always named router-default, and Ingress controller services in the zones that you subsequently add to your cluster have names such as router-dal12.





 VPC clusters: Ingress controller hostnames 

When you create a VPC cluster, one public and one private multizone VPC load balancer are automatically created outside of your cluster in your VPC. The public VPC load balancer creates a hostname to register the public Ingress controller, and the private VPC load balancer creates a hostname to register the private Ingress controller. In VPC clusters, a hostname is assigned to the Ingress controllers because external IP addresses are not static and might change over time. Note that this Ingress controller hostname is different than the default Ingress subdomain for your cluster.

The Ingress subdomain for your cluster is automatically linked to the VPC load balancer hostname for your public Ingress controller. Note that the Ingress subdomain for your cluster, which looks like <cluster_name>.<hash>-0000.<region>.containers.appdomain.cloud, is different than the VPC load balancer-assigned hostname for your public Ingress controller, which looks like 01ab23cd-<region>.lb.appdomain.cloud. The Ingress subdomain is the public route through which users access your app from the internet, and can be configured to use TLS termination. The assigned hostname for your public Ingress controller is what the VPC load balancer uses to forward traffic to the Ingress controller service.

You can find the hostname that is assigned to your public Ingress controller and the hostname that is assigned to your private Ingress controller by running oc get svc -n openshift-ingress and looking for the EXTERNAL IP field.

In your VPC infrastructure dashboard, the VPC load balancer reports as healthy only the two worker nodes that run the Ingress controller replica pods, because these worker nodes are configured as the listeners for the VPC load balancer.
Select the Users tab, click on the user. The required infrastructure permissions vary depending on what type of [cluster infrastructure provider](https://cloud.ibm.com/docs/openshift?topic=openshift-infrastructure_providers) you use, classic or VPC.



* For classic clusters



1. In the API keys pane, verify that the user has a Classic infrastructure API key, or click Create an IBM Cloud API key. For more information, see [Managing classic infrastructure API keys](https://cloud.ibm.com/docs/account?topic=account-classic_keysclassic_keys).
2. Click the Classic infrastructure tab and then click the Permissions tab.
3. If the user doesn't have each category checked, you can use the Permission sets drop-down list to assign the Super User role. Or you can expand each category and give the user the required [infrastructure permissions](https://cloud.ibm.com/docs/openshift?topic=openshift-classic-roles).



* For VPC clusters, assign the user the [Administrator platform access role for VPC Infrastructure](https://cloud.ibm.com/docs/vpc?topic=vpc-iam-getting-started).











 Understanding access to the infrastructure portfolio 

Determine whether your account has access to the IBM Cloud infrastructure portfolio and learn about how Red Hat OpenShift on IBM Cloud uses the API key to access the portfolio.

Does the classic or VPC infrastructure provider for my cluster affect what access I need to the portfolio?
: Access to IBM Cloud infrastructure works differently in classic and VPC clusters. Infrastructure resources for classic clusters are created in a separate IBM Cloud infrastructure account. Usually, your Pay-As-You-Go or Subscription account is linked to the IBM Cloud infrastructure account so that account owners can access classic IBM Cloud infrastructure automatically. To authorize other users to access classic compute, storage, and networking resources, you must assign [classic infrastructure roles](https://cloud.ibm.com/docs/openshift?topic=openshift-classic-roles).
Controlling traffic with ACLs 

Virtual Private Cloud

Control inbound and outbound traffic to your cluster by creating and applying access control lists (ACLs) to each subnet that your cluster is attached to. Note that these steps outline the minimum ACL rules that are required for basic cluster functionality. You might need to create additional ACL rules based on your use case.

Looking for a simpler security setup? Leave the default ACL for your VPC as-is, and modify the [default security group](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-security-group) instead.

Level of application
: Virtual Private Cloud subnet

Default behavior
: When you create a VPC, a default ACL is created in the format allow-all-network-acl-<VPC_ID> for the VPC. The ACL includes an inbound rule and an outbound rule that allow all traffic to and from your subnets. Any subnet that you create in the VPC is attached to this ACL by default. ACL rules are applied in a particular order. When you allow traffic in one direction by creating an inbound or outbound rule, you must also create a rule for responses in the opposite direction because responses are not automatically permitted.

Use case
: If you want to specify which traffic is permitted to the worker nodes on your VPC subnets, you can create a custom ACL for each subnet in the VPC. For example, you can create the following set of ACL rules to block most inbound and outbound network traffic of a cluster, while allowing communication that is necessary for the cluster to function.

Limitations
: If you create multiple clusters that use the same subnets in one VPC, you can't use ACLs to control traffic between the clusters because they share the same subnets. You can use [Calico network policies](https://cloud.ibm.com/docs/openshift?topic=openshift-network_policiesisolate_workers) to isolate your clusters on the private network.
Note: Sharing persistent storage across clusters is available only if the cluster and the storage instance are located in the same zone.
4. Share persistent storage across namespaces in the same cluster: You provisioned persistent storage in a namespace of your cluster. You want to use the same storage instance for an app pod that is deployed to a different namespace in your cluster.







 Storage classes 

To dynamically provision persistent storage, you must define the type and configuration of the storage that you want.

A [Kubernetes storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/) is used to abstract the underlying storage platform that is supported in IBM Cloud so that you don't have to know all the details about supported sizes, IOPS, or retention policies to successfully provision persistent storage in a cluster. Red Hat OpenShift on IBM Cloud provides pre-defined storage classes for every type of storage that is supported. Each storage class is designed to abstract the supported storage tier while giving you the choice to decide on the size, IOPS, and retention policy that you want.

Not finding what you are looking for? You can also create your own customized storage class to provision the type of storage that you want.





 Next steps 

Get started using storage in your cluster by reviewing the following docs.



* [Block Storage for Classic](https://cloud.ibm.com/docs/openshift?topic=openshift-block_storage)
* [Block Storage for VPC](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-block)
* [File Storage for Classic for Classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-file_storage)
* [File Storage for VPC](https://cloud.ibm.com/docs/openshift?topic=openshift-storage-file-vpc-install)
* [IBM Cloud Object Storage](https://cloud.ibm.com/docs/openshift?topic=openshift-storage-cos-understand)
The web UI opens.



It may take some time before you see the cluster entry while the information is initally collected and processed by the monitoring agent.

You only can monitor one instance per browser. You could have multiple tabs for the same instance.





 Step 4: Monitor your cluster 

In the Advisor tab, you can monitor and troubleshoot the health, risk, and capacity of hosts and Kubernetes clusters.

Zoom

![Advisor tab](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/images/advisor-tab.png)

Advisor tab



* Data is refreshed every 10 minutes.
* Metrics are prioritized by event count and severity.
* For more information, see [Advisor](https://docs.sysdig.com/en/docs/sysdig-monitor/advisor/).



In the Advisor section, you can choose to monitor your Kubernetes clusters by cluster, by node, by namespace, or by workload. Each option offers a set of predefined dashboards that you can use to monitor the health of your resources. You can also select to monitor by host.



 Monitoring Kubernetes clusters by cluster 

When you choose to monitor your Kubernetes clusters by cluster, you can select more filters to display data by node or by namespace, or you can choose any of the following dashboards:



* Workload Status & Performance
* Node Status & Performance
* Pod Rightsizing & Workload Capacity Optimization
* Cluster Capacity Planning
* Cluster / Namespace Available Resources
* Cluster Overview
* CPU Allocation Optimization
* Memory Allocation Optimization



Zoom

![Advisor predefined dashboards by cluster](https://cloud.ibm.com/docs-content/v1/content/220cdcbf6f4421bd7fee83fd1028048dba3f16c8/monitoring/images/overview-tab-clusters.png)
Controlling traffic with ACLs 

Virtual Private Cloud

Control inbound and outbound traffic to your cluster by creating and applying access control lists (ACLs) to each subnet that your cluster is attached to. Note that these steps outline the minimum ACL rules that are required for basic cluster functionality. You might need to create additional ACL rules based on your use case.

Looking for a simpler security setup? Leave the default ACL for your VPC as-is, and modify the [default security group](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-security-group) instead.

Level of application
: Virtual Private Cloud subnet

Default behavior
: When you create a VPC, a default ACL is created in the format allow-all-network-acl-<VPC_ID> for the VPC. The ACL includes an inbound rule and an outbound rule that allow all traffic to and from your subnets. Any subnet that you create in the VPC is attached to this ACL by default. ACL rules are applied in a particular order. When you allow traffic in one direction by creating an inbound or outbound rule, you must also create a rule for responses in the opposite direction because responses are not automatically permitted.

Use case
: If you want to specify which traffic is permitted to the worker nodes on your VPC subnets, you can create a custom ACL for each subnet in the VPC. For example, you can create the following set of ACL rules to block most inbound and outbound network traffic of a cluster, while allowing communication that is necessary for the cluster to function.

Limitations
: If you create multiple clusters that use the same subnets in one VPC, you can't use ACLs to control traffic between the clusters because they share the same subnets. You can use [Calico network policies](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesisolate_workers) to isolate your clusters on the private network.

When you use the following steps to create custom ACLs, only network traffic that is specified in the ACL rules is permitted to and from your VPC subnets.
Note that all Intel® x86-64 servers have Hyper-Threading enabled by default. For more information, see [Intel Hyper-Threading Technology](https://cloud.ibm.com/docs/vpc?topic=vpc-profilesvpc-intel-hyper-threading). 
 Security Clusters on shared hardware run in an isolated environment in the public cloud. Clusters on dedicated hosts do not run in a shared environment, instead only your clusters are present on your hosts. Network access control lists protect the subnets that provide the floating IPs for your worker nodes. For more information, see the [VPC documentation](https://cloud.ibm.com/docs/vpc?topic=vpc-about-vpc). 
 High availability The master includes three replicas for high availability. Further, if you create your cluster in a multizone metro, the master replicas are spread across zones and you can also spread your worker pools across zones. For more information, see [High availability for Red Hat OpenShift on IBM Cloud](https://cloud.ibm.com/docs/openshift?topic=openshift-ha). 
 Reservations Reservations aren't available for VPC. 
 Cluster administration VPC clusters can't be reloaded or updated. Instead, use the [worker replace --update CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clicli_worker_replace) or [API operation](https://containers.cloud.ibm.com/global/swagger-global-api//beta/replaceWorker) to replace worker nodes that are outdated or in a troubled state. 
 Cluster networking Unlike classic infrastructure, the worker nodes of your VPC cluster are attached to VPC subnets and assigned private IP addresses. The worker nodes are not connected to the public network, which instead is accessed through a public gateway, floating IP, or VPN gateway.
What actions you need to take 

You must ensure that any cluster workers that have not been reloaded, replaced, or rebooted since 5 July 2022 are reloaded or replaced before 15 December 2022. For more information about how to reload workers for classic clusters, see [ibmcloud ks worker reload](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload). For more information about how to replace workers for VPC clusters, see [ibmcloud ks worker replace](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clicli_worker_replace). You can also reboot affected workers but this action is not recommended due to the risks associated with rebooting. For more information about rebooting workers, see [ibmcloud ks worker reboot](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reboot).

If you have any other custom configuration that uses the old Container Registry private IP addresses, such as for firewall rules or Calico policies, ensure that you update your configuration to use the new Container Registry IP addresses and remove the old IP addresses. Both the old and new private IP addresses are documented in the IBM Cloud Kubernetes Service documentation, see [Permit worker nodes to communicate with IBM Cloud Container Registry](https://cloud.ibm.com/docs/containers?topic=containers-firewallfirewall_private_container_registry).

The IBM Cloud Kubernetes Service [Calico policy samples](https://github.com/IBM-Cloud/kube-samples/tree/master/calico-policies) will be updated to remove the old IP addresses on 15 December 2022. If you use these samples, ensure that you pull in the most recent samples after this date.
Before you can attach your RHCOS hosts to your location, your must create a Red Hat CoreOS enabled location. Note that Red Hat CoreOS enabled locations are currently available only in new locations that are managed from Dallas (us-south) or Frankfurt (eu-de). You can't migrate existing locations. For more information, see [Creating a location](https://cloud.ibm.com/docs/satellite?topic=satellite-locations).

Create and manage dedicated hosts in VPC Gen 2.
: You can now create clusters on dedicated hosts in VPC Gen 2. Note that support for dedicated hosts is available only for allowlisted accounts. For more information, see [Creating and managing dedicated hosts](https://cloud.ibm.com/docs/openshift?topic=openshift-dedicated-hosts).





 11 April 2022 

New! IBM Cloud® Secrets Manager
: With the deprecation of IBM Cloud® Certificate Manager, you can now manage certificates and secrets with IBM Cloud® Secrets Manager. You can integrate your own Secrets Manager instances with your Red Hat OpenShift clusters. Secrets Manager instances can be used across multiple clusters, and a single cluster can have more than one instance.

HPCS Router add-on
: Versions 4.6.0_1468, 4.7.0_1469, 4.8.0_1470, and 4.9.0_1471 of the [HPCS Router add-on](https://cloud.ibm.com/docs/openshift?topic=openshift-hpcs-router-changelog) are released. Cloud Image Key Synchronizer add-on
: Version 1.0.0_927 of the [Cloud Image Key Synchronizer add-on](https://cloud.ibm.com/docs/openshift?topic=openshift-image-key-synchronizer-changelog) is released.

Block Storage for VPC add-on.
Check out this guidance on [custom address prefixes](https://cloud.ibm.com/docs/vpc?topic=vpc-configuring-address-prefixes). If you use custom-range subnets for your worker nodes, you must [ensure that your worker node subnets don't overlap with your cluster's pod subnet](https://cloud.ibm.com/docs/containers?topic=containers-vpc-subnetsvpc-ip-range).

Do not delete the subnets that you attach to your cluster during cluster creation or when you add worker nodes in a zone. If you delete a VPC subnet that your cluster used, any load balancers that use IP addresses from the subnet might experience issues, and you might be unable to create new load balancers.

When you create VPC subnets for your clusters, keep in mind the following features and limitations. For more information about VPC subnets, see [Characteristics of subnets in the VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-about-networking-for-vpcsubnets-in-the-vpc).



* The default CIDR size of each VPC subnet is /24, which can support up to 253 worker nodes. If you plan to deploy more than 250 worker nodes per zone in one cluster, consider creating a subnet of a larger size.
* After you create a VPC subnet, you can't resize it or change its IP range.
* Multiple clusters in the same VPC can share subnets.
* VPC subnets are bound to a single zone and can't span multiple zones or regions.
* After you create a subnet, you can't move it to a different zone, region, or VPC.
* If you have worker nodes that are attached to an existing subnet in a zone, you can't change the subnet for that zone in the cluster.