vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Supported persistent storage solutions, such as IBM Cloud File Storage for Classic, IBM Cloud Block Storage, or IBM Cloud® Object Storage are integrated as Kubernetes flex drivers and can be set up by using [Helm charts](https://cloud.ibm.com/docs/containers?topic=containers-helm). The Helm chart automatically sets up Kubernetes storage classes, the storage provider, and the storage driver in your cluster. You can use the storage classes to provision persistent storage by using persistent volume claims (PVCs). For more information, see [Planning highly available persistent storage](https://cloud.ibm.com/docs/containers?topic=containers-storage-plan).

To secure your cluster network or connect to an on-prem data center, you can configure one of the following options:



* [strongSwan IPSec VPN Service](https://cloud.ibm.com/docs/containers?topic=containers-vpnvpn-setup)
* [IBM Cloud® Direct Link](https://cloud.ibm.com/docs/dl?topic=dl-get-started-with-ibm-cloud-dl)
* [Virtual Router Appliance (VRA)](https://cloud.ibm.com/docs/containers?topic=containers-vpnvyatta)
* [Fortigate Security Appliance (FSA)](https://cloud.ibm.com/docs/vmwaresolutions/services?topic=vmwaresolutions-fsa_considerations)







 IBM Cloud VPC infrastructure services 

With IBM Cloud Kubernetes Service, you can create a standard cluster in a [Virtual Private Cloud (VPC)](https://cloud.ibm.com/docs/vpc?topic=vpc-getting-started). A VPC gives you the security of a private cloud environment with the dynamic scalability of a public cloud.

Before you can create a VPC cluster, you must have a VPC and at least one VPC subnet that you provision by using the IBM Cloud console, CLI, or API. You manage these resources in the VPC dashboard directly.
The Block Storage for VPC add-on automatically retries mounting the storage to your apps. Wait a few more minutes for the storage to mount to your app.

kubectl describe deployment <deployment_name>

Example output

...
Volumes:
myvol:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: mypvc
ReadOnly: false







 Using an existing Block Storage for VPC instance 

If you have an existing physical Block Storage for VPC device that you want to use in your cluster, you can manually create the PV and PVC to statically provision the storage.

You can attach a volume to one worker node only. Make sure that the volume is in the same zone as the worker node for the attachment to succeed.



1. Determine the volume that you want to attach to a worker node in your VPC cluster. Note the volume ID.

ibmcloud is volumes
2. List the details of your volume. Note the Size, Zone, and IOPS. These values are used to create your PV.

ibmcloud is volume <volume_id>
3. Retrieve a list of worker nodes in your VPC cluster. Note the Zone of the worker node that is in the same zone as your storage volume.

ibmcloud ks worker ls -c <cluster_name>
4. Optional: If you provisioned your physical Block Storage for VPC instance by using a retain storage class, the PV and the physical storage is not removed when you remove the PVC. To use your physical Block Storage for VPC device in your cluster, you must remove the existing PV first.



1. List the PVs in your cluster and look for the PV that belongs to your Block Storage for VPC device. The PV is in a released state.

kubectl get pv
2. Remove the PV.

kubectl delete pv <pv_name>



5. Create a configuration file for your PV. Include the ID, Size, Zone, and IOPS that you retrieved earlier.
* Worker node communication to the Kubernetes master is established over the [VPC virtual private endpoint (VPE)](https://cloud.ibm.com/docs/containers?topic=containers-vpc-subnetsvpc_basics_vpe). If the public cloud service endpoint is also enabled, worker-to-master traffic is established half over the public endpoint and half over the VPE for protection from potential outages of the public or private network.



To secure communication over public and private cloud service endpoints or VPEs, IBM Cloud Kubernetes Service automatically sets up a Konnectivity connection between the Kubernetes master and worker nodes when the cluster is created. Worker nodes securely talk to the master through TLS certificates, and the master talks to workers through the Konnectivity connection.





 User-to-master communication in VPC clusters 

You can allow authorized cluster users to communicate with the Kubernetes master by enabling the public and private cloud service endpoints, or the private cloud service endpoint only.



* Public and private cloud service endpoints: By default, all calls to the master that are initiated by authorized cluster users are routed through the public cloud service endpoint. If authorized cluster users are in your VPC network or are connected through a [VPC VPN connection](https://cloud.ibm.com/docs/containers?topic=containers-vpc-vpnaas), the master is privately accessible through the private cloud service endpoint.
* Private cloud service endpoint only: To access the master through the private cloud service endpoint, authorized cluster users must either be in your VPC network or connected through a [VPC VPN connection](https://cloud.ibm.com/docs/containers?topic=containers-vpc-vpnaas).



You can secure access to your private cloud service endpoint by creating a subnet allowlist. Only authorized requests to your cluster master that originate from subnets in the allowlist are permitted through the cluster's private cloud service endpoint. Use of the allowlist function helps prevent unauthorized scanning activities.
In the output, note the instance ID.

ibmcloud resource service-instance-create myvpc-cos cloud-object-storage standard global
4. Create a cluster in your VPC in the same zone as the subnet. The following command creates a version 4.11 cluster in Dallas with the minimum configuration of 2 worker nodes that have at least 4 cores and 16 GB memory so that default Red Hat OpenShift components can deploy. By default, your cluster is created with a public and a private cloud service endpoint. You can use the public cloud service endpoint to access the Kubernetes master, such as to run oc commands, from your local machine. Your worker nodes communicate with the master on the private cloud service endpoint. For the purposes of this tutorial, do not specify the --disable-public-service-endpoint option. For more information about the command options, see the [cluster create vpc-gen2 CLI reference docs](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clicli_cluster-create-vpc-gen2).

ibmcloud oc cluster create vpc-gen2 --name myvpc-cluster --zone us-south-1 --version 4.11_openshift --flavor bx2.4x16 --workers 2 [--operating-system (REDHAT_7_64|REDHAT_8_64)] --vpc-id <vpc_ID> --subnet-id <vpc_subnet_ID> --cos-instance <COS_CRN>
5. List your cluster details. Review the cluster State, check the Ingress Subdomain, and note the Master URL. Your cluster creation might take some time to complete. After the cluster state shows Normal, the cluster network and Ingress components take about 10 more minutes to deploy and update the cluster domain that you use for the Red Hat OpenShift web console and other routes.
Virtual Private Cloud

List all VPCs in the targeted resource group. If no resource group is targeted, then all VPCs in the account are listed.

ibmcloud ks vpcs [--provider vpc-gen2] [--output json] [-q]

Minimum required permissions:



* Viewer platform access role for IBM Cloud Kubernetes Service



Command options:

--provider vpc-gen2
: The infrastructure provider type ID for the VPC worker node machine. vpc-gen2 for VPC Generation 2 compute is supported.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.

Example:

ibmcloud ks vpcs





 flavor command 

Each flavor includes the amount of virtual CPU, memory, and disk space for each worker node in the cluster.

By default, the secondary storage disk directory where all container data is stored, is encrypted with LUKS encryption. If the disable-disk-encrypt option is included during cluster creation, then the host's container runtime data is not encrypted. [Learn more about the encryption](https://cloud.ibm.com/docs/containers?topic=containers-securityencrypted_disk).

You can provision your worker node as a virtual machine on shared or dedicated hardware, or for classic clusters only, as a physical machine on bare metal. [Learn more about your flavor options](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodesplanning_worker_nodes).



 flavor get command 

Virtual Private Cloud

Classic infrastructure

Get the information of a flavor for a zone and provider.

ibmcloud ks flavor get --flavor FLAVOR --provider PROVIDER --zone ZONE [--output OUTPUT] [-q]

Minimum required permissions
: None

Command options:

--flavor FLAVOR
: The flavor you want to get information for. Flavors determine how much virtual CPU, memory, and disk space is available to each worker node.

--provider PROVIDER
See the [CLI reference](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clicli_worker_pool_create_vpc_gen2) for command details.

ibmcloud ks worker-pool create vpc-gen2 --name NAME --cluster CLUSTER --flavor FLAVOR --operating-system UBUNTU_20_64 --size-per-zone WORKERS-PER-ZONE --label LABEL --label LABEL
2. Verify that the worker pool is created.

ibmcloud ks worker-pool ls --cluster CLUSTER
3. Prepare to add a zone to your worker pool. When you add a zone, the number of worker nodes you specified with the --size-per-zone option are added to the zone. These worker nodes run the Ubuntu 20 operating system.

Classic and VPC clusters: If possible, use the same Classic VLAN or VPC subnet for the new zone as you are using for the Ubuntu 18 worker pool.

If you need to use a different Classic VLAN or VPC Subnet for these new nodes, note that switching your worker nodes to a different Classic VLAN or VPC subnet can have significant effects your workload and cluster functionality. For example, you might experience the following. - In Classic clusters: Classic LoadBalancers might not work because the LoadBalancer IPs are specific to a single VLAN, and traffic can only be routed to the LoadBalancer if there is a worker node in the cluster on that VLAN that the LoadBalancer IP can be placed on. - In Classic or VPC Clusters: Problems with the network connections to and from workers in the new Classic VLANs or VPC subnets if you have any Security Groups, Network ACLs, Firewall/Gateway rules, Custom routing, coreDNS configurations, and so on, that are specific to the old VLANs, VPC subnets, or cluster worker IP addresses or subnets.
4. Add the zone to your worker pool that you retrieved earlier. When you add a zone, the worker nodes that are defined in your worker pool are provisioned in the zone and considered for future workload scheduling.



* Classic clusters:
* [Communication with resources in other VPCs](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-worker-services-onprem-other-vpcs)
* [Communication with IBM Cloud classic resources](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-worker-services-onprem-classic)



* [External communication to apps that run on worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-external-workers)
* [Private traffic to cluster apps](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-worker-services-onprem-apps-private)
* [Public traffic to cluster apps](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-worker-services-onprem-apps-public)
* [Example scenarios for VPC cluster network setups](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-scenarios)



* [Scenario: Run internet-facing app workloads in a VPC cluster](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-no-pgw)
* [Worker-to-worker communication](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-no-pgw-worker)
* [Worker-to-master and user-to-master communication](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-no-pgw-master)
* [Worker communication to other services or networks](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-no-pgw-services)
For example, on a vSphere Cluster that consists of multiple bare metal server hosts, and you configured a distributed vSwitch, or the hosts are using the default standard switch. In this case, you can create a port group with VLAN ID 100 and add it to the specific vSwitch or vSwitches. The traffic between vNICs of two VMs that are attached to Port Group 100 is controlled by the vSwitch.

The setting has the following consequences:



* Security Group rules that control traffic between the network interfaces in Port Group VLAN ID 100 are not applied.
* If you need Security Group to control traffic between the two VMs, you must use separate VLAN IDs for the VLAN interfaces to force traffic to traverse through VPC.
* If you use separate VLAN IDs for VMs, you must deploy multiple port groups with mapping VLAN IDs in standard distributed vSwitches.



For more information about security groups, see [Security in your VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-security-in-your-vpc). For more information about security groups with IBM Cloud bare metal server, see [Introduction to IBM Cloud bare metal server networking](https://cloud.ibm.com/docs/vpc?topic=vpc-bare-metal-servers-network).







 Public connectivity with VMware VMs on VPC subnet 

Bare Metal Server for VPC provides full support for VPC public networking features. External connectivity can be achieved either by using a Public Gateway that is attached to a VPC subnet, or by using a floating IP address that is attached to a PCI or VLAN interface of bare metal server. Public gateway uses source network address translation (SNAT) and a floating IP uses destination network address translation (DNAT). These functionalities are identical to VPC virtual servers.

The following diagram depicts how VMware workloads that are attached to VPC subnets can use these two options, and shows the key differences between them. Each VM has a VLAN interface, and the external connectivity depends how the VLAN interface is provisioned.

Zoom

!
If you created your own custom image, edit the value to the name of your custom image. 
 management_node_count You can have up to three management nodes in the cluster. If you want failover support that is provided by LSF, you need to specify the value to be larger than one. In this case, when the primary management node is down, one of the candidate management nodes become the primary and your cluster remains functional without interruption. 
 region Edit the region where you want your cluster to be created. To get a full list of regions, see [Creating a VPC in a different region](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-a-vpc-in-a-different-region). 
 resource_group The resource group name from your IBM Cloud account where the VPC resources are deployed. 
 vpc_name You can use an existing VPC in which the cluster resources are provisioned. If no value given, then a new VPC is provisioned for the cluster. 
 vpn_enabled You can deploy a VPN gateway for VPC in the cluster. By default, the value is set to false. 
 vpn_peer_address The peer public IP address to which the VPN is connected. 
 vpn_peer_cidrs Comma-separated list of peer CIDRs (for example, 192.168.0.0/24) to which the VPN is connected. 
 vpn_preshared_key The pre-shared key for the VPN. 
 zone Edit the zone based on the selected region. To get a full list of zones within a region, see [Get zones by using the CLI](https://cloud.ibm.com/docs/vpc?topic=vpc-creating-a-vpc-in-a-different-regionget-zones-using-the-cli). 







 Parameters for auto scaling 

You can set the following parameters for auto scaling:



* worker_node_min_count: The minimum number of worker nodes that are provisioned at the time the cluster is created and remain running regardless of job demands in the cluster.
VPC clusters: Why does the VPC load balancer for Ingress controller only route to one zone? 

Virtual Private Cloud

 What’s happening 

You create a multizone VPC cluster. However, when you run ibmcloud is load-balancers to find the VPC load balancer that exposes the Ingress controller, the VPC subnet for only one zone in your cluster is listed instead of the subnets for all zones in your cluster. In the output, look for the VPC load balancer Name that starts with kube-crtmgr-<cluster_ID>.

ID Name Family Subnets Is public Provision status Operating status Resource group
r006-d044af9b-92bf-4047-8f77-a7b86efcb923 kube-bsaucubd07dhl66e4tgg-1f4f408ce6d2485499bcbdec0fa2d306 Application mysubnet-us-south-3 true active online default

 Why it’s happening 

When you create a Red Hat OpenShift cluster on VPC infrastructure in the CLI, the cluster is initially created with worker nodes in one zone only.

You then make the cluster multizone by manually adding zones to your worker pools with the ibmcloud oc zone add vpc-gen2 command. Currently, when you add zones to your cluster, the Ingress controller is not updated with the VPC subnets for the new zones, and does not route requests to apps in the new zones.

 How to fix it 

Restart the Ingress controller so that a new VPC load balancer is created, which registers the Ingress controller behind a hostname and forwards traffic to the Ingress controller. Then, update your Ingress subdomain to use the new VPC load balancer hostname.



1. Run the following command to find the VPC load balancer that exposes the Ingress controller. In the output, look for the VPC load balancer name that starts with kube-crtmgr-<cluster_ID>.

ibmcloud is load-balancers

Example output

ID Name Family Subnets Is public Provision status Operating status Resource group
Enabling Flow Logs for VPC cluster components 

Virtual Private Cloud

You can configure IBM Cloud Flow Logs for VPC to gather information about the traffic entering or leaving your VPC cluster worker nodes. Flow logs are stored in an IBM Cloud Object Storage instance and can be used for troubleshooting purposes, adhering to compliance regulations, and more. For more information about Flow Logs for VPC, see [Flow logs use cases](https://cloud.ibm.com/docs/vpc?topic=vpc-flow-logs&interface=uiflow-logs-use-cases).

When you use Flow Logs for VPC with a Red Hat OpenShift VPC cluster, you can enable flow logs at the VPC level, or at the VPC subnet or VPC load balancer level. You cannot specify which worker nodes to gather flow logs for. However, you can review the flow log output to identify information that is specific to the worker nodes you want to investigate.



 Configuring a flow log collector 

To configure flow logs at the VPC, VPC subnet, or VPC load balancer level, see [Creating a flow log collector](https://cloud.ibm.com/docs/vpc?topic=vpc-ordering-flow-log-collector) in the VPC documentation.

To enable flow logs, you must have an IBM Cloud Object Storage instance with a single-region bucket that is in the same region as the VPC resource you are monitoring.





 Viewing worker node flow logs 

Your Flow Logs for VPC gathers information from the VPC, VPC subnet, or VPC load balancer level. However, you can use the flow logs to gather information that is specific to your worker nodes. Separate flow log files are created for ingress and egress traffic.



1. In the CLI, find the ibm-cloud.kubernetes.io/instance-id label value for the worker node.

oc describe node <worker_node_ip> | grep instance-id

Example output.
(https://cloud.ibm.com/docs/openshift?topic=openshift-securityonprem-network-setup)
* [Network segmentation and privacy for VPC clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-securitynetwork_segmentation_vpc)
* [What network traffic is allowed for my VPC cluster by default?](https://cloud.ibm.com/docs/openshift?topic=openshift-securityvpc-network-traffic-default)
* [What is network segmentation and how can I set it up for a VPC cluster?](https://cloud.ibm.com/docs/openshift?topic=openshift-securitynetwork-segment-what-is)
* [What else can I do to reduce the surface for external attacks for VPC clusters?](https://cloud.ibm.com/docs/openshift?topic=openshift-securityvpc-external-what-else)
* [Securely expose apps with routes](https://cloud.ibm.com/docs/openshift?topic=openshift-securityexpose-apps-with-routes)
* [Securely expose apps with LoadBalancer and Ingress services](https://cloud.ibm.com/docs/openshift?topic=openshift-securitynetwork_lb_ingress)
* [Can I use security groups to manage my cluster's network traffic?](https://cloud.ibm.com/docs/openshift?topic=openshift-securitycan-i-use-security-groups)
* [How can I do TLS termination with LoadBalancer and Ingress services?](https://cloud.ibm.com/docs/openshift?topic=openshift-securitytls-termination-lb)



* [Persistent storage](https://cloud.ibm.com/docs/openshift?topic=openshift-securitystorage)
* kubernetes.io/os
* node.kubernetes.io/instance-type
* topology.kubernetes.io/region
* topology.kubernetes.io/zone



service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets
: Annotation to specify one or more subnets that the VPC ALB service deploys to. If specified, this annotation takes precedence over the service.kubernetes.io/ibm-load-balancer-cloud-provider-zone annotation. Note that you can specify a different subnet in the same VPC than the subnets that your cluster is attached to. In this case, even though the VPC ALB deploys to a different subnet in the same VPC, the VPC ALB can still route traffic to your worker nodes on the cluster subnets. To see subnets in all resource groups, run ibmcloud ks subnets --provider vpc-gen2 --vpc-id <vpc> --zone <zone>.

service.kubernetes.io/ibm-load-balancer-cloud-provider-zone
: Annotation to specify a VPC zone that your cluster is attached to. When you specify a zone in this annotation, two processes occur.



1. The VPC ALB is deployed to the same subnet in that zone that your worker nodes are connected to.
2. Only worker nodes in your cluster in this zone are configured to receive traffic from the VPC ALB.



: To see zones, run ibmcloud ks zone ls --provider vpc-gen2.
: To place the load balancer in a specific zone, you must specify this annotation when you create the load balancer. If you later change this annotation to a different zone, the load balancer itself is not moved to the new zone. However, the load balancer is reconfigured to send traffic to only worker nodes in the new zone.