vania-janet/multiturn-rag-retrieval-data huggingface dataset:
This design uses vSphere High Availability (HA) in the initial cluster and in the additional clusters to detect compute failures and recover VMs that run in a cluster. The vSphere HA feature in this design is configured with both the Host Monitoring and Admission Control options that are enabled in the cluster. Additionally, the initial cluster reserves one node’s resources as spare capacity for the admission control policy.

You are responsible to adjust the admission control policy when the cluster is later expanded or contracted.

By default, the VM restart priority option is set to medium and the Host isolation response option is disabled. Additionally, VM monitoring is disabled and the Datastore Heartbeating feature is configured to include any of the cluster data stores. This approach uses the NAS data stores if they are present.





 Enhanced vMotion compatibility 

To simplify vMotion compatibility across cluster nodes with potentially differing CPU capabilities, Enhanced vMotion Compatibility (EVC) mode is enabled at the highest available level that is supported by the vSphere version. This setting ensures vMotion compatibility across cluster nodes when newer processors arrive within IBM Cloud® inventory. The setting allows for cluster expansion in the future if the original processor is no longer in inventory. An exception to this rule is that EVC mode is not set for a management cluster with Cascade Lake processors where Cascade Lake EVC is not supported by the vSphere version.







 IBM CloudDriver 

Cornerstone to these solutions is automation. Automation reduces the complexity of deployment, drastically reduces deployment time, and ensures the VMware instance is deployed in a consistent manner.

IBM CloudDriver is an ephemeral IBM Cloud VM virtual server instance (VSI), which is deployed as needed for initial deployment and for Day 2 operations such as adding hosts, clusters, or add-on services to your VMware instance. CloudDriver is deleted when the all current processes are completed.

CloudDriver is deployed only on the private network that connects to the IBM management plane over a private message queue. They are IBM developed components, are not user accessible, and have the following attributes and functions:



* Deployment and configuration of the vCenter Server instance within the user account.
* Add and remove hosts from the vCenter Server clusters.
* Add and remove clusters from vCenter Server instances.
* Private service endpoint only: Communication to master from both worker nodes and cluster users is established over the private network through the private cloud service endpoint. Cluster users must either be in your VPC network or connect through a [VPC VPN connection](https://cloud.ibm.com/docs/containers?topic=containers-vpc-vpnaas).







 Worker communication to other services or networks 

After the cluster is created, you create a worker pool that is deployed only to the zone where subnet has an attached public gateway. Any app pods that are deployed to the worker nodes in this pool can make requests to a public endpoint through the public gateway. For example, if you want your app pods to send logs to IBM Cloud® Activity Tracker (which does not support private endpoints), you can add an affinity rule for the worker pool ID label to the app deployment. This affinity rule ensures that the app pods which require public egress are confined to only one worker pool on one subnet. If you deploy other apps that don't require public egress, you can instead add anti-affinity rules to the app deployment so that the app pods deploy to only worker pools on subnets without a public gateway.

If your app workload requires other IBM Cloud services that support private cloud service endpoints, your worker nodes can automatically, securely communicate with these services over the private VPC network without using a public gateway.





 External communication to apps that run on worker nodes 

After you test your app, you can expose it to the internet by creating a public Kubernetes LoadBalancer service or using the default public Ingress application load balancers (ALBs). The VPC load balancer that is automatically created in your VPC outside of your cluster when you use one of these services routes traffic to your app. You can improve the security of your cluster and control public traffic apps by modifying the default VPC security group for your cluster. Security groups consist of rules that define which inbound traffic is permitted for your worker nodes. For example, you can use inbound rules to control incoming public traffic to your apps through the VPC load balancer, and outbound rules to control outgoing requests from your apps through the public gateway.

Ready to get started with a cluster for this scenario?
Check the [VPC quotas documentation](https://cloud.ibm.com/docs/vpc?topic=vpc-quotas) for VPC resource quotas across all your VPC clusters in your VPC.



 How to fix it 

Verify that no VPC security groups are blocking traffic to your cluster and that the VPC load balancer is available.



2. Verify that the VPC load balancer for the Kubernetes LoadBalancer service exists. In the output, look for the VPC load balancer that is formatted kube-<cluster_ID>-<kubernetes_lb_service_UID>. You can get the Kubernetes LoadBalancer service UID by running oc get svc <service_name> -o yaml.

ibmcloud is load-balancers



* If the VPC load balancer is not listed, it does not exist for one of the following reasons:





* You reached the maximum number of VPC load balancers permitted per account. Across all your VPC clusters in your VPC, a maximum of 20 VPC load balancers can be created. One VPC load balancer is created for each Kubernetes LoadBalancer service that you create, and it routes requests to that Kubernetes LoadBalancer service only.
* The VPC load balancer was deleted through the VPC console or the CLI. To re-create the VPC load balancer for your Kubernetes LoadBalancer service, restart the Kubernetes master by running ibmcloud oc cluster master refresh --cluster <cluster_name_or_id>.



If you want to remove the load balancing setup for an app in your VPC cluster, delete the Kubernetes LoadBalancer service by running oc delete svc <kubernetes_lb_service_name>. The VPC load balancer that is associated with the Kubernetes LoadBalancer service is automatically deleted from your VPC.
3.
Make sure to [attach a public gateway to all the VPC subnets](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-vpc-resources-with-cli-and-api&interface=cliattach-public-gateway-cli) that the cluster uses so that you can provision Block Storage for VPC.<br> * [IBM Cloud Object Storage](https://cloud.ibm.com/docs/containers?topic=containers-storage_cos_install) is available as a Helm chart.<br><br><br> 
 Volume attachments See [Volume attachment limits](https://cloud.ibm.com/docs/vpc?topic=vpc-attaching-block-storagevol-attach-limits). 
 Portworx Review the [Portworx limitations](https://cloud.ibm.com/docs/containers?topic=containers-storage_portworx_planportworx_limitations). 
 Block Storage for VPC The default storage class in VPC clusters can not be changed. However, you can [create a custom storage class](https://cloud.ibm.com/docs/containers?topic=containers-vpc-blockvpc-customize-storage-class).
[ibmcloud is instance-group-delete](https://cloud.ibm.com/docs/vpc?topic=vpc-infrastructure-cli-plugin-vpc-referenceinstance-group-delete) Delete instance groups. 
 [ibmcloud is instance-group-membership](https://cloud.ibm.com/docs/vpc?topic=vpc-infrastructure-cli-plugin-vpc-referenceinstance-group-membership) View details of a member of an instance group. 
 [ibmcloud is instance-group-membership-delete](https://cloud.ibm.com/docs/vpc?topic=vpc-infrastructure-cli-plugin-vpc-referenceinstance-group-membership-delete) Delete membership for an instance group. 
 [ibmcloud is instance-group-memberships](https://cloud.ibm.com/docs/vpc?topic=vpc-infrastructure-cli-plugin-vpc-referenceinstance-group-memberships) List all members for an instance group. 
 [ibmcloud is instance-group-memberships-delete](https://cloud.ibm.com/docs/vpc?topic=vpc-infrastructure-cli-plugin-vpc-referenceinstance-group-memberships-delete) Delete all memberships for an instance group. 



Optionally, you can set auto scaling policies for your instance group to dynamically add or remove virtual server instances from your group. For more information, see [Creating an instance group for auto scaling](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-auto-scale-instance-group).
Setting up Block Storage for VPC for unmanaged clusters 

The following documentation covers the steps to deploy the Block Storage for VPC driver on unmanaged OpenShift Container Platform clusters in IBM Cloud. This process is unsupported and any issues with the steps or the driver must be recreated in a Kubernetes cluster to receive support.

Virtual Private Cloud

Want to use Block Storage for VPC in an IBM Cloud Kubernetes Service or Red Hat OpenShift on IBM Cloud cluster? See [Setting up Block Storage for VPC](https://cloud.ibm.com/docs/containers?topic=containers-vpc-block) for more information.



 Prerequisites 

To use the Block Storage for VPC driver, complete the following tasks:



* Create an IBM Cloud Kubernetes Service cluster on VPC infrastructure.
* [Label your worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-vpc-block-storage-driver-unmanagedvpc-block-label-um).
* [Create your storage secret](https://cloud.ibm.com/docs/containers?topic=containers-vpc-block-storage-driver-unmanagedvpc-block-create-storage-secret).
* [Create an image pull secret that uses your IAM credentials](https://cloud.ibm.com/docs/containers?topic=containers-vpc-block-storage-driver-unmanagedvpc-block-create-storage-secret).





 Labeling your worker nodes 

Before you can deploy the Block Storage for VPC driver, you must prepare your worker nodes by adding the required labels.

Before you begin, [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)



1. Retrieve the following details of your VPC instance. These parameters are used to apply labels to your worker nodes.



* <instanceID> - The VPC instance ID. To retrieve this value, run ibmcloud is ins.
[Architecture overview for a bastion VPC with VMware Solutions](https://cloud.ibm.com/docs-content/v1/content/5bdb37c0149145723c5437e8176651928565e722/vmwaresolutions/images/arch-pattern-bastion-vpc.svg)

Figure 1. Architecture overview for a bastion VPC with VMware® Solutions

This architecture pattern deployment is summarized as follows:



1. [Create a VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-vpc-resources-with-cli-and-api&interface=cli) for the Bastion hosts. [Choose a VPC prefix](https://cloud.ibm.com/docs/vpc?topic=vpc-choosing-ip-ranges-for-your-vpc), which does not overlap with your Classic. For example, the default VPC prefix of the MZR and Zone typically works on this step. Provision a subnet with a size of your preference, for example, /27 or larger.
2. [Provision client-to-site VPNaaS](https://cloud.ibm.com/docs/vpc?topic=vpc-vpn-client-to-site-overview) for OpenVPN based VPN connectivity. Configure your preferred authentication method for the clients. Use a prefix for your clients, which does not overlap with the rest. You might use SNAT when the VPN clients communicate with your connected resources. Advertise 10.0.0.0/8 to your client, or smaller prefix based on your preference.
3. Use [classic connectivity](https://cloud.ibm.com/docs/vpc?topic=vpc-setting-up-access-to-classic-infrastructure), or provision a [IBM Cloud Transit Gateway](https://cloud.ibm.com/docs/transit-gateway?topic=transit-gateway-about) and add your VPC and Classic as connections to it.
VMware virtual machines on VPC subnets 

The topic provides an introduction to IBM Cloud® Virtual Private Cloud subnet-integrated VMware® virtual machines architecture, as shown on the following diagram.

Zoom

![Architecture Overview of VMware on IBM Cloud VPC](https://cloud.ibm.com/docs-content/v1/content/5bdb37c0149145723c5437e8176651928565e722/vmwaresolutions/images/vpc-ryo-diagrams-aod-non-nsx-based.svg)

Figure 1. Architecture Overview of VMware on IBM Cloud VPC



 Overview to VMware virtual machine networking integration with VPC subnets 

The IBM Cloud bare metal server for VPC fully supports VMware vSphere® networking capabilities with standard and distributed vSwitches. Before setting up networks in the vSphere environment, it is important to understand the networking concepts between IBM Cloud bare metal server in IBM Cloud VPC and vSphere networking.

When you provision a bare metal server, a primary PCI interface is created by default for every server. This primary PCI interface automatically becomes the bare metal server uplink on vSwitch0 in ESXi™. The PCI interface IP address is in the vmk0 adapter, which is located in the management network port group on vSwitch0. The VLAN ID of the management network port group is automatically set to 0.

VLAN interfaces in IBM Cloud bare metal server can be used as virtual network adapters of the VMKernel adapters. Or also, as virtual machines (VMs) including management VMs, such as vCenter Server® and your VMs workload.

In this design, the standard vSwitches are migrated to distributed vSwitch in every host, and the default PCI interface is the uplink of the distributed vSwitch. When deploying VMkernel adapters in the ESXi™, you create a distributed port group for each System Traffic Type and VMK.
Preparing to create clusters 

Create a cluster in Red Hat® OpenShift® on IBM Cloud®.

After [getting started](https://cloud.ibm.com/docs/containers?topic=containers-getting-started), you might want to create a cluster that is customized to your use case and different public and private cloud environments. Consider the following steps to create the correct cluster each time.



1. [Prepare your account to create clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-clusterscluster_prepare). This step includes creating a billable account, setting up an API key with infrastructure permissions, making sure that you have Administrator access in IBM Cloud IAM, planning resource groups, and setting up account networking.
2. [Decide on your cluster setup](https://cloud.ibm.com/docs/openshift?topic=openshift-clustersprepare_cluster_level). This step includes planning cluster network and HA setup, estimating costs, and if applicable, allowing network traffic through a firewall.
3. Create your [VPC Gen2](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-create-vpc-gen2) or [classic](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-create-classic) cluster by following the steps in the IBM Cloud console or CLI.





 Preparing to create clusters at the account level 

Prepare your IBM Cloud account for IBM Cloud Kubernetes Service. After the account administrator makes these preparations, you might not need to change them each time that you create a cluster. However, each time that you create a cluster, you still want to verify that the current account-level state is what you need it to be.



1. [Create or upgrade your account to a billable account (IBM Cloud Pay-As-You-Go or Subscription)](https://cloud.ibm.com/registration).
2.
Give the VPC subnet a name and select the location where you want to create the cluster.
4. Attach a public gateway to your subnet so that you can access public endpoints from your cluster. This public gateway is used later on to access container images from Docker Hub.
5. Click Create virtual private cloud.



2. From the [IBM Cloud Kubernetes Service dashboard](https://cloud.ibm.com/kubernetes/clusters), click Create cluster.
3. Configure your VPC environment.



1. Select the Standard plan.
2. Select Kubernetes as your container platform and select the Kubernetes version 1.26 or later.
3. Select VPC infrastructure.
4. From the Virtual private cloud drop-down menu, select the VPC that you created earlier.



4. Configure the Location details for your cluster.



1. Select the Resource group that you want to create your cluster in. You can't change the resource group later.
2. Select the zones to create your cluster in. The zones are filtered based on the VPC that you selected, and include the subnets that you previously created.



5. Configure your Worker pool setup.



1. If you want a larger size for your worker nodes, click Change flavor. Otherwise, leave the default 4 vCPUs / 16 GB flavor selected.
2. Set how many worker nodes to create per zone, such as 2.



6. Review the rest of the cluster details, such as the service endpoint, cluster name, and tags.
7. In the Summary pane, review your order summary and then click Create.



The worker node can take a few minutes to provision, but you can see the progress in the Worker nodes tab. When the status reaches Ready, you can start working with your cluster by [deploying your first app](https://cloud.ibm.com/docs/containers?topic=containers-getting-starteddeploy-app)!





 Creating classic clusters in the IBM Cloud Kubernetes Service CLI
Looking for a fast way to create a cluster from the UI? Try out [Automating cluster creation with IBM Cloud Schematics templates](https://cloud.ibm.com/docs/openshift?topic=openshift-templates).





 Deciding on your cluster setup 

After you set up your account to create clusters, decide on the setup for your cluster. You must make these decisions every time that you create a cluster. Review the following decision tree image for more information, such as comparisons of Kubernetes and Red Hat OpenShift, and VPC and classic infrastructure.





 Next steps 

When the cluster is up and running, you can check out the following cluster administration tasks:



* If you created the cluster in a multizone capable zone, [spread worker nodes by adding a zone to your cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-add_workers).
* [Deploy an app in your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-deploy_appapp_cli)
* [Set up your own private registry in IBM Cloud to store and share Docker images with other users.](https://cloud.ibm.com/docs/Registry?topic=Registry-getting-started)
* [Set up the cluster autoscaler](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-scaling-classic-vpc) to automatically add or remove worker nodes from your worker pools based on your workload resource requests.
* Control who can create pods in your cluster with [pod security policies](https://cloud.ibm.com/docs/containers?topic=containers-psp).



Then, you can check out the following network configuration steps for your cluster setup:



* Classic clusters:



* Isolate networking workloads to edge worker nodes [in classic clusters without a gateway](https://cloud.ibm.com/docs/openshift?topic=openshift-edge).
For more ideas of what to do with your cluster, review the [Next steps](https://cloud.ibm.com/docs/containers?topic=containers-clustersnext_steps).





 Example commands to create classic clusters 

Classic cluster, shared virtual machine

ibmcloud ks cluster create classic --name my_cluster --zone dal10 --flavor b3c.4x16 --hardware shared --workers 3 --public-vlan <public_VLAN_ID> --private-vlan <private_VLAN_ID>

Classic cluster, bare metal

ibmcloud ks cluster create classic --name my_cluster --zone dal10 --flavor mb2c.4x32 --hardware dedicated --workers 3 --public-vlan <public_VLAN_ID> --private-vlan <private_VLAN_ID>

Classic cluster that uses private VLANs and the private cloud service endpoint only:

ibmcloud ks cluster create classic --name my_cluster --zone dal10 --flavor b3c.4x16 --hardware shared --workers 3 --private-vlan <private_VLAN_ID> --private-only --private-service-endpoint

For a classic multizone cluster, after you created the cluster in a [multizone metro](https://cloud.ibm.com/docs/containers?topic=containers-regions-and-zoneszones-mz), [add zones](https://cloud.ibm.com/docs/containers?topic=containers-add_workersadd_zone):

ibmcloud ks zone add classic --zone <zone> --cluster <cluster_name_or_ID> --worker-pool <pool_name> --private-vlan <private_VLAN_ID> --public-vlan <public_VLAN_ID>





 Creating a single-zone classic cluster with Terraform 

Terraform on IBM Cloud enables predictable and consistent provisioning of IBM Cloud platform infrastructure and resources, including classic clusters. To create a classic cluster with Terraform, you first create a Terraform configuration file that declares the type of cluster resource you want to create.
ibmcloud login -r <region> [-g <resource_group>] [--sso]
2. [Create a VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-vpc-resources-with-cli-and-api&interface=clicreate-a-vpc-cli) in the same region where you want to create the cluster.



To create a VPC subnet, follow these steps.



1. Get the ID of the VPC where you want to create the subnet.

ibmcloud ks vpcs
2. Create the subnet. For more information about the options in this command, see the [CLI reference](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-vpc-resources-with-cli-and-api&interface=clicreate-a-subnet-cli).

ibmcloud is subnet-create <subnet_name> <vpc_id> --zone <vpc_zone> --ipv4-address-count <number_of_ip_address>



* VPC subnets provide IP addresses for your worker nodes and load balancer services in the cluster, so [create a VPC subnet with enough IP addresses](https://cloud.ibm.com/docs/containers?topic=containers-vpc-subnetsvpc_basics_subnets), such as 256. You can't change the number of IPs that a VPC subnet has later.
* If you are running Kubernetes 1.20 or earlier , don't use the following reserved ranges: 172.16.0.0/16, 172.18.0.0/16, 172.19.0.0/16, and 172.20.0.0/16.



3. Check whether you have a public gateway in the zones where you want to create a cluster. Within one VPC, you can create only one public gateway per zone, but that public gateway can be attached to multiple subnets within the zone.

ibmcloud is public-gateways

Example output
Use the IBM Cloud CLI to create a VPC subnet for your cluster and optionally attach a public gateway to the subnet.

Before you begin



1. In your command line, log in to your IBM Cloud account and target the IBM Cloud region and resource group where you want to create your VPC cluster. For supported regions, see [Creating a VPC in a different region](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-a-vpc-in-a-different-region). The cluster's resource group can differ from the VPC resource group. Enter your IBM Cloud credentials when prompted. If you have a federated ID, use the --sso option to log in.

ibmcloud login -r <region> [-g <resource_group>] [--sso]
2. [Create a VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-vpc-resources-with-cli-and-api&interface=clicreate-a-vpc-cli) in the same region where you want to create the cluster.



To create a VPC subnet, follow these steps.



1. Get the ID of the VPC where you want to create the subnet.

ibmcloud oc vpcs
2. Create the subnet. For more information about the options in this command, see the [CLI reference](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-vpc-resources-with-cli-and-api&interface=clicreate-a-subnet-cli).

ibmcloud is subnet-create <subnet_name> <vpc_id> --zone <vpc_zone> --ipv4-address-count <number_of_ip_address>