vania-janet/multiturn-rag-retrieval-data huggingface dataset:
If you want to privately expose your apps instead, you must first create a private Ingress controller, and then create private routes.
* VPC clusters with private cloud service endpoint only: Your cluster is created with a private Ingress controller by default. The Ingress controller assigns privately accessible routes for your apps and listens on the private host network interface. Only clients that are connected to your private VPC network can access apps that are exposed by a private route. If you want to publicly expose your apps instead, you must first create a public Ingress controller, and then create public routes.



If you have a multizone cluster, one high-availability Ingress controller is deployed to your cluster, and one Ingress controller service is created in each zone. Two worker nodes are required per zone so that the two replicas of the Ingress controller can be deployed and updated correctly. Note that the Ingress controller service in the first zone where you have workers nodes is always named router-default, and Ingress controller services in zones that you subsequently add to your cluster have names such as router-dal12.



* To see the Ingress controller services in each zone of your cluster, run oc get svc -n openshift-ingress.
* To see the Ingress controller subdomain for your cluster and the IP addresses for the Ingress controller service in each zone, run ibmcloud oc nlb-dns ls -c <cluster_name_or_ID> and look for the subdomain formatted like <cluster_name>-<random_hash>-0000.<region>.containers.appdomain.cloud.



In your VPC infrastructure dashboard, the VPC load balancer reports as healthy only the two worker nodes that run the Ingress controller replica pods, because these worker nodes are configured as the listeners for the VPC load balancer. Even though only the listener worker nodes are reported as healthy, the listeners' backend pool of worker nodes is kept up-to-date by Red Hat OpenShift on IBM Cloud so that all worker nodes in your cluster can still receive requests from the VPC load balancer.



 Traffic flow in a classic single-zone cluster 

The following diagram shows how a router directs network traffic from the internet to an app in a single-zone, classic cluster.
Allow inbound traffic from all sources to the listener port on a public load balancer. None 
 [VPC access control lists (ACLs)](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-acls) (Not recommended) VPC subnet The default ACL for the VPC, allow-all-network-acl-<VPC_ID>, allows all traffic to and from your subnets. Changing the default ACL is not recommended; instead, use security groups. Control inbound and outbound traffic to and from the cluster subnet that you attach the ACL to. Rules allow or deny traffic to or from an IP range with specified protocols and ports. can't be used to control traffic between the clusters that share the same VPC subnets. Instead, you can [create Calico policies](https://cloud.ibm.com/docs/openshift?topic=openshift-network_policiesisolate_workers) to isolate your clusters on the private network. 
 [Kubernetes network policies](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-kube-policies) Worker node host endpoint None Control traffic within the cluster at the pod level by using pod and namespace labels. Protect pods from internal network traffic, such as isolating app microservices from each other within a namespace or across namespaces. None 



VPC Load Balancer also supports security groups. For more information, see [Integrating an IBM Cloud Application Load Balancer for VPC with security groups](https://cloud.ibm.com/docs/vpc?topic=vpc-alb-integration-with-security-groups).





 Access control lists (ACLs) or security groups? 

Although you can use either VPC ACLs or VPC security groups to control inbound traffic to and outbound traffic from your cluster, security groups are easier to implement. You can simplify your security setup by adding rules to only the default security group for your cluster, and leaving the default ACL for your VPC as-is.
Persistent VPC NLB names have a format kube-<cluster_ID>-<kubernetes_lb_service_UID>. To see your cluster ID, run ibmcloud ks cluster get --cluster <cluster_name>. To see the Kubernetes LoadBalancer service UID, run kubectl get svc myloadbalancer -o yaml and look for the metadata.uid field in the output. The dashes (-) are removed from the Kubernetes LoadBalancer service UID in the VPC NLB name.

ibmcloud is load-balancers

In the following example CLI output, the VPC NLB that is named kube-bh077ne10vqpekt0domg-046e0f754d624dca8b287a033d55f96e is created for the Kubernetes LoadBalancer service:

ID Name Created Host Name Is Public Listeners Operating Status Pools Private IPs Provision Status Public IPs Subnets Resource Group
06496f64-a689-4693-ba23-320959b7b677 kube-bh077ne10vqpekt0domg-046e0f754d624dca8b287a033d55f96e 8 minutes ago 1234abcd-us-south.lb.appdomain.cloud no 95482dcf-6b9b-4c6a-be54-04d3c46cf017 online 717f2122-5431-403c-b21d-630a12fc3a5a 10.XXX.XXX.XXX active - c6540331-1c1c-40f4-9c35-aa42a98fe0d9 00809211b934565df546a95f86160f62
8. From your connection to the VPC private network, access the IP address of the Kubernetes LoadBalancer service that you found in step 6 and your app port in the format <external_IP>:<app_port>.
9.
In your service's allowlist, you must add the external IP addresses of the public gateways on your cluster's VPC subnets.

If you want to permit egress from your allowlist-protected services to your cluster, you must add your worker nodes' private IP addresses or your cluster's VPC subnet CIDRs in your service's allowlist. Note that because worker nodes in VPC clusters have only private IP addresses, connections into the VPC cluster worker nodes can only originate from systems that are connected to your IBM Cloud private network.

Before you begin



1. [Access your Red Hat OpenShift cluster](https://cloud.ibm.com/docs/openshift?topic=openshift-access_cluster).
2. Install the infrastructure-service CLI plug-in. The prefix for running VPC infrastructure commands is ibmcloud is.

ibmcloud plugin install infrastructure-service





 Allowing ingress from a cluster to another service 

To permit ingress from your cluster to another service, modify that service's allowlist or your on-premises allowlist.



1. Get the Worker Zones and VPCs that your cluster is created in.

ibmcloud oc cluster get -c <cluster>

Example output

...
Worker Zones: us-south-1, us-south-2, us-south-3
Ingress Subdomain: vpc-prod.us-south.containers.appdomain.cloud
Ingress Secret: vpc-prod
Creator: -
Public Service Endpoint URL: https://c2.us-south.containers.cloud.ibm.com:20267
Private Service Endpoint URL: https://c2.private.us-south.containers.cloud.ibm.com:20267
Pull Secrets: enabled in the default namespace
VPCs: ff537d43-a5a4-4b65-9627-17eddfa5237b
...
2.
However, you might need to also provide limited public egress from your worker nodes to a public endpoint, and want to ensure that this public egress is controlled and isolated in your cluster. For example, you might need your app pods to access an IBM Cloud service that does not support private cloud service endpoints, and must be accessed over the public network.

Zoom

![Network setup for a cluster that allows limited, secure public access.](https://cloud.ibm.com/docs-content/v1/content/4d497aff9e27b92fbbc2ee4e343a9ec4549cb1d7/containers/images/cs_org_ov_vpc.png)

Figure 1. Network setup for a VPC cluster that allows limited, secure public access



 Worker-to-worker communication 

To achieve this setup in, for example, a multizone cluster that has worker nodes in two zones, you create a VPC subnet in one zone that has no public gateway attached, and a VPC subnet in another zone that does have a public gateway attached. Then, you create a VPC cluster that uses these VPC subnets and zones.





 Worker-to-master and user-to-master communication 

When you create the cluster you can choose to allow worker-to-master and user-to-master communication over the public and private networks, or over the private network only.



* Public and private cloud service endpoints: Communication between worker nodes and master is established over the private network through the private cloud service endpoint. By default, all calls to the master that are initiated by authorized cluster users are routed through the public cloud service endpoint.
* Private service endpoint only: Communication to master from both worker nodes and cluster users is established over the private network through the private cloud service endpoint. Cluster users must either be in your VPC network or connect through a [VPC VPN connection](https://cloud.ibm.com/docs/containers?topic=containers-vpc-vpnaas).







 Worker communication to other services or networks 

After the cluster is created, you create a worker pool that is deployed only to the zone where subnet has an attached public gateway.
In a classic cluster, list the network load balancer (NLB) IP addresses that are registered to DNS subdomains. In a VPC cluster, list the VPC load balancer hostnames that are registered to DNS subdomains.

ibmcloud oc nlb-dns ls --cluster CLUSTER [--output json] [-q]

Minimum required permissions
: Editor platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.



 Example nlb-dns ls command 

ibmcloud oc nlb-dns ls --cluster mycluster







 ibmcloud oc nlb-dns monitor configure 

Classic infrastructure

Configure and optionally enable a health check monitor for an existing NLB subdomain in a cluster. When you enable a monitor for your subdomain, the monitor health checks the NLB IP in each zone and keeps the DNS lookup results updated based on these health checks.

You can use this command to create and enable a health check monitor, or to update the settings for an existing health check monitor. To create a new monitor, include the --enable option and the options for all settings that you want to configure.

To update an existing monitor, you must include all the options for the settings that you want, including existing settings.
Allow inbound traffic from all sources to the listener port on a public load balancer. None 
 [VPC access control lists (ACLs)](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-acls) (Not recommended) VPC subnet The default ACL for the VPC, allow-all-network-acl-<VPC_ID>, allows all traffic to and from your subnets. Changing the default ACL is not recommended; instead, use security groups. Control inbound and outbound traffic to and from the cluster subnet that you attach the ACL to. Rules allow or deny traffic to or from an IP range with specified protocols and ports. can't be used to control traffic between the clusters that share the same VPC subnets. Instead, you can [create Calico policies](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesisolate_workers) to isolate your clusters on the private network. 
 [Kubernetes network policies](https://cloud.ibm.com/docs/containers?topic=containers-vpc-kube-policies) Worker node host endpoint None Control traffic within the cluster at the pod level by using pod and namespace labels. Protect pods from internal network traffic, such as isolating app microservices from each other within a namespace or across namespaces. None 



VPC Load Balancer also supports security groups. For more information, see [Integrating an IBM Cloud Application Load Balancer for VPC with security groups](https://cloud.ibm.com/docs/vpc?topic=vpc-alb-integration-with-security-groups).





 Access control lists (ACLs) or security groups? 

Although you can use either VPC ACLs or VPC security groups to control inbound traffic to and outbound traffic from your cluster, security groups are easier to implement. You can simplify your security setup by adding rules to only the default security group for your cluster, and leaving the default ACL for your VPC as-is.
To set up a secure connection between your cluster and an on-premises network, [configure and deploy the strongSwan IPSec VPN service](https://cloud.ibm.com/docs/containers?topic=containers-vpnvpn-setup) directly in a pod in your cluster. Yes 
 Transit Gateway Use IBM Cloud Transit Gateway to manage access between your VPCs. Transit Gateway instances can be configured to route between VPCs that are in the same region (local routing) or VPCs that are in different regions (global routing). Additionally, you can use Transit Gateway to manage access between your VPCs in multiple regions to resources in your IBM Cloud classic infrastructure. To get started, see the [Transit Gateway documentation](https://cloud.ibm.com/docs/transit-gateway?topic=transit-gateway-getting-started). Yes 







 Logging and monitoring services 



Table 4. Logging and monitoring services

 Service Description Classic VPC 

 CoScale Monitor worker nodes, containers, replica sets, replication controllers, and services with [CoScale](https://newrelic.com/blog/nerd-life/coscale). Yes Yes 
 Datadog Monitor your cluster and view infrastructure and application performance metrics with [Datadog](https://www.datadoghq.com/). Yes Yes 
 IBM Cloud Activity Tracker Monitor the administrative activity that is made in your cluster by analyzing logs through Grafana. For more information about the service, see the [Activity Tracker](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-getting-started) documentation. For more information about the types of events that you can track, see [Activity Tracker events](https://cloud.ibm.com/docs/containers?topic=containers-at_events). Yes Yes 
 IBM Log Analysis Add log management capabilities to your cluster by deploying a Log Analysis agent to your worker nodes to manage logs from your pod containers.
Instead, you can [remove the security group from the VPC ALB](https://cloud.ibm.com/docs/vpc-infrastructure-cli-plugin?topic=vpc-infrastructure-cli-plugin-vpc-reference&interface=clisecurity-group-target-remove) and [replace it with a security group](https://cloud.ibm.com/docs/vpc-infrastructure-cli-plugin?topic=vpc-infrastructure-cli-plugin-vpc-reference&interface=clisecurity-group-target-add) that you create and manage.

Note that a public gateway is not required on your subnets to allow inbound network traffic from the internet to LoadBalancer services or ALBs. Public gateways are required only to allow worker nodes to make outbound requests to public endpoints. For more information, see [Planning public external load balancing](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planningpublic_access).





 Example scenarios for VPC cluster network setups 

Now that you understand the basics of cluster networking, check out some example scenarios in which various VPC cluster network setups can meet your workload needs.



 Scenario: Run internet-facing app workloads in a VPC cluster 

In this scenario, you run workloads in a VPC cluster that are accessible to requests from the Internet. Public access is controlled by security groups so that end users can access your apps while unwanted public requests to your apps are denied. Additionally, your workers have automatic access to any IBM Cloud services that support private cloud service endpoints.

Zoom

![Network setup for a VPC cluster that runs internet-facing app workloads.](https://cloud.ibm.com/docs-content/v1/content/4d497aff9e27b92fbbc2ee4e343a9ec4549cb1d7/containers/images/vpc_no_pgw.png)

Figure 1. Network setup for a VPC cluster that runs internet-facing app workloads





 Worker-to-worker communication
: In VPC clusters, you can now [enable the PROXY protocol](https://cloud.ibm.com/docs/containers?topic=containers-comm-ingress-annotationspreserve_source_ip_vpc) for all load balancers that expose Ingress ALBs in your cluster. The PROXY protocol enables load balancers to pass client connection information that is contained in headers on the client request, including the client IP address, the proxy server IP address, and both port numbers, to ALBs.

Helm version 2 unsupported
: Removed all steps for using Helm v2, which is unsupported. [Migrate to Helm v3 today](https://helm.sh/docs/topics/v2_v3_migration/) for several advantages over Helm v2, such as the removal of the Helm server, Tiller.

Istio add-on
: [Version 1.8 of the Istio managed add-on](https://cloud.ibm.com/docs/containers?topic=containers-istio-changelogv18) is released.





 7 December 2020 

Key Protect enhancements
: For clusters that run 1.18.8_1525 or later, your cluster now [integrates more features from Key Protect](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkms-keyprotect-features) when you enable Key Protect as the KMS provider for the cluster. When you enable this integration, a Reader[service-to-service authorization policy](https://cloud.ibm.com/docs/account?topic=account-serviceauth) between IBM Cloud Kubernetes Service and Key Protect is automatically created for your cluster, if the policy does not already exist. If you have a cluster that runs an earlier version, [update your cluster](https://cloud.ibm.com/docs/containers?topic=containers-update) and then [re-enable KMS encryption](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkeyprotect) to register your cluster with Key Protect again.

Worker node versions
However, you can [remove the security group](https://cloud.ibm.com/docs/vpc-infrastructure-cli-plugin?topic=vpc-infrastructure-cli-plugin-vpc-reference&interface=clisecurity-group-target-remove) from your ALB and [replace it](https://cloud.ibm.com/docs/vpc-infrastructure-cli-plugin?topic=vpc-infrastructure-cli-plugin-vpc-reference&interface=clisecurity-group-target-add) with one that you create and manage. If you do so, you must make sure that the following traffic rules are still covered.



 Inbound rules 



Required inbound rules for VPC ALB security groups

 Rule purpose Protocol Port or Value Source 

 If you use your own security group for the LBaaS for Ingress and you expose any applications in your cluster with Ingress, you must allow TCP protocol to ports 443 or 80, or both. You can allow access for all clients, or you can allow only certain source IPs or subnets to access these applications. Note that this requirement also applies if you want to use the Openshift Console, which is exposed by the cluster's default Ingress instance and router. To allow access for the Openshift Console, you must allow TCP protocol to port 443 from all sources, or from specific source IP addresses or subnets. TCP 80 or 443 All sources, or specific source IP addresses or subnets 
 If you use your own security group for any LBaaS instances that you use to expose other applications, then within that security group you must allow any intended sources to access the appropriate ports and protocols. TCP, UDP, or both LBaaS ports Any sources, or specific source IPs or subnets 







 Outbound rules 



Required outbound rules for VPE and VPC ALB security groups

 Rule purpose Protocol Port or Value Destination 

 Allow the ALB to send traffic to the cluster workers on the TCP NodePort range TCP 30000 - 32767 Any 
 Allow the ALB to send traffic to the cluster workers on the UDP NodePort range UDP 30000 - 32767 Any
3. Check that your cluster has public connectivity so that the networking components can talk to the master as they deploy.



* VPC clusters with public and private cloud service endpoints enabled: [Ensure that a public gateway is enabled on each subnet](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-subnetscreate_vpc_subnet) that your cluster is attached to. Public gateway are required for default components such as the web console and OperatorHub to use a secure, public connection to complete actions such as pulling images from remote, private registries. Note that if only the private service endpoint is enabled for your cluster, no public gateway is required because the private cloud service endpoint is used by default to access OpenShift components such as the OpenShift web console or OperatorHub.
* Classic clusters:



* In the output of Step 2, check that your cluster has a Public Service Endpoint URL. If your cluster does not have a public cloud service endpoint, [enable it](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_network_clusterset-up-public-se).
* Check that at least some worker nodes in your cluster have a Public IP address. If no worker node does, you must [set up public VLANs for at least one worker pool](https://cloud.ibm.com/docs/openshift?topic=openshift-cs_network_clusterchange-vlans).

ibmcloud oc workers -c <cluster_name_or_ID>





4. In the output of Step 2, check that the Ingress Subdomain is available. The Ingress components in your cluster must provision before the Ingress controller components can be created. If the Ingress Subdomain and Ingress Secret are not available, see [Why does no Ingress subdomain exist after cluster creation?](https://cloud.ibm.com/docs/openshift?topic=openshift-ingress_subdomain).
5.
SMC supports combination of single smc_zone as ["us-east-1"] or double smc_zone as ["au-syd-3","eu-de-1"] or triple smc_zone as ["us-east-1","ca-tor-3","jp-tok-3"]. list(string) n/a 
 remote_allowed_ips Comma-separated list of IP addresses that can access the Symphony Multi Cluster instance through an SSH interface. For security purposes, provide the public IP addresses assigned to the devices that are authorized to establish SSH connections (for example, "169.45.117.34"). To fetch the IP address of the device, use [https://ipv4.icanhazip.com/](https://ipv4.icanhazip.com/). list(string) n/a 



Parameters that have default values that you might or might not need to update:



 Name Description Type Default 

 cluster_prefix value The specific cluster prefix for your Lone cluster. string hpcc-smc 
 cluster_id ID of the cluster used by Symphony Multi Cluster for configuration of resources. The ID must be up to 39 alphanumeric characters, including the underscore (_), the hyphen (-), and the period (.). Other special characters and spaces are not allowed. Do not use the name of any host or user as the name of your cluster. You cannot change it after installation. string HPCMultiCluster 
 bastion_host_instance_type Specify the virtual server instance profile type name to be used to create the bastion node for the Symphony Multi Cluster. For more information, see [VPC Profiles](https://cloud.ibm.com/docs/vpc?topic=vpc-profiles). string bx2-2x8 
 cluster_prefix Prefix that is used to name the Symphony Multi cluster and IBM Cloud resources that are provisioned to build the Symphony Multi cluster instance. You cannot create more than one instance of the Symphony Multi cluster with the same name. Make sure that the name is unique.