vania-janet/multiturn-rag-retrieval-data huggingface dataset:
If your cluster is created on VPC infrastructure and you enabled only the private cloud service endpoint during cluster creation, your cluster is created with only a private router by default. To publicly expose your apps, you must first create a public IngressController resource and configure that with a subdomain. The Ingress operator automatically creates and configures a new public Ingress controller based on the IngressController, which you can use to create public routes for your apps.

Note that even though you create an IngressController resource in the following steps, the IngressController is only required to create and configure the necessary Ingress controller for you. After the Ingress controller is created, you use the Ingress controller directly to create routes.



1. Prepare the domain that you want to use for your Ingress controller.



* Custom domain: To register a custom domain, work with your Domain Name Service (DNS) provider or [IBM Cloud DNS](https://cloud.ibm.com/docs/dns?topic=dns-getting-started). If you want to use the same subdomain for multiple services in your cluster, you can register a wildcard subdomain, such as .example.com.
* IBM-provided domain:



1. List the existing subdomains in your cluster. In the Subdomain column of the output, copy the subdomain that has the highest i00<n> value.

ibmcloud oc nlb-dns ls --cluster <cluster_name_or_id>

In this example output, the mycluster-a1b2cdef345678g9hi012j3kl4567890-i002.us-south.containers.appdomain.cloud subdomain has the highest i00<n> value of i002.

Subdomain Load Balancer Hostname Health Monitor SSL Cert Status SSL Cert Secret Name
* To assign access to individual users or users in an access group, ensure that the user or group has been assigned at least one [IBM Cloud IAM platform access role](https://cloud.ibm.com/docs/containers?topic=containers-usersadd_users) at the IBM Cloud Kubernetes Service service level.



To create custom RBAC permissions,



1. Create the role or cluster role with the access that you want to assign.



1. Create a .yaml file to define the role or cluster role.

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: my_role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
- apiGroups: ["apps", "extensions"]
resources: ["daemonsets", "deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]



Table 3. Understanding the YAML parameters

 Parameter Description 

 kind Use Role to grant access to resources within a specific namespace. Use ClusterRole to grant access to cluster-wide resources such as worker nodes, or to namespace-scoped resources such as pods in all namespaces. 
 apiVersion <br><br> * For clusters that run Kubernetes 1.8 or later, use rbac.authorization.k8s.io/v1.<br> * For earlier versions, use apiVersion: rbac.authorization.k8s.io/v1beta1.<br><br><br> 
 metadata.namespace For kind Role only: Specify the Kubernetes namespace to which access is granted. 
 metadata.name Name the role or cluster role.
These components collectively provide a console for vSphere operations, the ability to automate management of the cloud platform, centralized log collection, and network visibility, analysis, and optimization.
* Veeam provides continuous backup of the management stack for protection against disasters and rapid restoration to known good states if corruption of any management stack component were to occur. Veeam can also provide backup services for the workload cluster. The single site deployment can use the Veeam bare metal option to provide a suitable backup repository.







 Gateway cluster 

The FortiGate physical 10 Gb HA appliance is the only point of entry into the IBM Cloud for VMware Regulated Workloads. The gateway cluster with the vSRX HA appliance or a suitable bring your own gateway appliance is an alternative to the physical FortiGate appliance. The gateway cluster uses local storage only (no vSAN, no NFS) and each host is running a virtual appliance node. The gateway cluster requires only two ESXi hosts. The nodes are configured as a highly available cluster. The cluster is sized to support 10 Gb networking.





 Workload cluster 

The sizing of four ESXi servers for the workload cluster is based on the required compute resources, memory, and storage needs of the customer applications. The minimum host count of four is a vSAN requirement. Each workload cluster added is provisioned on new VLANs in the same data center location to separate traffic of different workload clusters with the firewall, either physical or an appliance that is running on the gateway cluster.

vSAN is the storage provider and is fully contained within the workload environment. Therefore, all regulated data is isolated from external access and threats.

NSX-T provides a highly secure and flexible software-defined network to support the application requirements. NSX-T management is external to the workload cluster thus ensuring that network and security changes are not possible by anyone other than the designated administrators. All north-south network access in the workload cluster is done through private and secure connections by using IPsec or IBM Direct Link.

Add more workload clusters to deliver dedicated resources for more applications or to satisfy business or compliance requirements.





 Option for consolidated workload and management cluster
: The [cluster autoscaler](https://cloud.ibm.com/docs/openshift?topic=openshift-cluster-scaling-classic-vpc) is available as a managed add-on. The cluster autoscaler Helm chart is deprecated. Migrate your autoscaled worker pools to use the add-on.

New! Community Kubernetes Ingress support
: The Ingress ALBs in your version 3.11 cluster can now run the Kubernetes Ingress image, which is built on the community Kubernetes project's implementation of the NGINX Ingress controller. To use the Kubernetes Ingress image, you create your Ingress resources and ConfigMaps according to the Kubernetes Ingress format, including community Kubernetes Ingress annotations instead of custom Red Hat OpenShift on IBM Cloud annotations.

New! Default IBM Cloud® Certificate Manager instances
: An IBM Cloud Certificate Manager service instance is now created by default for all new and existing standard clusters. The Certificate Manager service instance, which is named in the format kube-crtmgr-<cluster_ID>, stores the TLS certificate for your cluster's default Ingress subdomain. You can also upload your own TLS certificates for custom Ingress domains to this Certificate Manager instance and use the new [ibmcloud oc ingress secret commands](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_ingress_secret_create) to create secrets for these certificates in your cluster. To ensure that a Certificate Manager instance is automatically created for your new or existing cluster, verify that the API key for the region and resource group that the cluster is created in has the correct Certificate Manager permissions.





 21 August 2020 

Versions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--hardware HARDWARE
: Optional: The level of hardware isolation for your worker node. Use dedicated so that available physical resources are dedicated to you only, or shared to allow physical resources to be shared with other IBM customers. The default is shared. For bare metal flavors, specify dedicated.

--flavor FLAVOR
: Choose a machine type, or flavor, for your worker nodes. You can deploy your worker nodes as virtual machines on shared or dedicated hardware, or as physical machines on bare metal. Available physical and virtual machines types vary by the zone in which you deploy the cluster. For more information, see the documentation for the ibmcloud oc flavors (machine-types)[command](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_machine_types).

--workers NUMBER
: An integer that represents the number of worker nodes to create in the cluster.

--private-vlan PRIVATE_VLAN
: Required: The private VLAN that was specified when the cluster was created. Private VLAN routers always begin with bcr (back-end router) and public VLAN routers always begin with fcr (front-end router). When you create a cluster and specify the public and private VLANs, the number and letter combination after those prefixes must match.

--public-vlan PUBLIC_VLAN
: Optional: The public VLAN that was specified when the cluster was created. If you want your worker nodes to exist on a private VLAN only, don't provide a public VLAN ID. Private VLAN routers always begin with bcr (back-end router) and public VLAN routers always begin with fcr (front-end router). When you create a cluster and specify the public and private VLANs, the number and letter combination after those prefixes must match.
Cluster autoscaler
: The cluster autoscaler is available for private network-only clusters. To get started, update to the latest Helm chart version.

New DevOps tutorial
: Learn how to set up your own DevOps toolchain and configure continuous delivery pipeline stages to deploy your containerized app that is stored in GitHub to a cluster in IBM Cloud Kubernetes Service.





 14 October 2019 

CLI change log
: Updated the IBM Cloud Kubernetes Service CLI plug-in change log page for the [release of version 0.4.38](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_changelog).

Calico MTU
: Added [steps](https://cloud.ibm.com/docs/containers?topic=containers-kernelcalico-mtu) for changing the Calico plug-in maximum transmission unit (MTU) to meet the network throughput requirements of your environment.

Creating DNS subdomains for VPC load balancers and private Ingress ALBs
: Added steps for [registering a VPC load balancer hostname with a DNS subdomain](https://cloud.ibm.com/docs/containers?topic=containers-vpc-lbaasvpc_lb_dns) and for [exposing apps to a private network](https://cloud.ibm.com/docs/containers?topic=containers-managed-ingress-setup) in VPC clusters.

Let's Encrypt rate limits for Ingress
: Added troubleshooting steps for when no subdomain or secret is generated for the Ingress ALB when you create or delete clusters of the same name.

Version change logs
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
details ClusterIP 172.21.19.104 <none> 9080/TCP 2m
kubernetes ClusterIP 172.21.0.1 <none> 443/TCP 1d
productpage ClusterIP 172.21.168.196 <none> 9080/TCP 2m
ratings ClusterIP 172.21.11.131 <none> 9080/TCP 2m
reviews ClusterIP 172.21.117.164 <none> 9080/TCP 2m

kubectl get pods -n bookinfo

NAME READY STATUS RESTARTS AGE
details-v1-6865b9b99d-7v9h8 2/2 Running 0 2m
productpage-v1-f8c8fb8-tbsz9 2/2 Running 0 2m
ratings-v1-77f657f55d-png6j 2/2 Running 0 2m
reviews-v1-6b7f6db5c5-fdmbq 2/2 Running 0 2m
reviews-v2-7ff5966b99-zflkv 2/2 Running 0 2m
reviews-v3-5df889bcff-nlmjp 2/2 Running 0 2m
4. Create a YAML file called bookinfo-custom-gateway.yaml to define Gateway and VirtualService resources. Note that the Gateway resource specifies custom-ingressgateway for the name of the custom ingress load balancer that you previously created, and the VirtualService resource specifies bookinfo-gateway for the name of the Gateway resource.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: bookinfo-gateway
spec:
selector:
istio: custom-ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- ""
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
Cluster health image v1.2.18 v1.2.20 Updated universal base image (UBI) to the latest 8.4 version to resolve CVEs. Updated to use Go version 1.16.10. 
 CoreDNS 1.8.4 1.8.6 See the [CoreDNS release notes](https://coredns.io/2021/10/07/coredns-1.8.6-release/). 
 Gateway-enabled cluster controller 1567 1586 Updated Alpine base image to the latest 3.14 version to resolve CVEs. Updated to use Go version 1.16.10. 
 GPU device plug-in and installer 7fd867d c9bfc8c Updated universal base image (UBI) to the 8.5-204 version to resolve CVEs. Updated to use Go version 1.16.10. 
 IBM Calico extension 864 900 Updated universal base image (UBI) to the 8.5-204 version to resolve CVEs. Updated to use Go version 1.16.10. 
 IBM Cloud Controller Manager v1.20.12-3 v1.20.13-4 Updated to support the Kubernetes 1.20.13 release and to use calicoctl version 3.17.6. 
 IBM Cloud File Storage for Classic plug-in and monitor 401 402 Updated universal base image (UBI) to the 8.5-204 version to resolve CVEs. Updated to use Go version 1.16.10. 
 IBM Cloud RBAC Operator 4ca5637 3430e03 Updated universal base image (UBI) to the latest 8.5 version to resolve CVEs. Updated to use Go version 1.16.10. 
 Key Management Service provider v2.3.10 v2.3.12 Updated universal base image (UBI) to the latest 8.4 version to resolve CVEs. Updated to use Go version 1.16.10. 
 Kubernetes v1.20.12 v1.20.13 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.20.13).
* Replace <profile-id>, <access_token>, <cluster_crn>, and <cluster_name> with the values that you previously retrieved.
* For <ns>, enter the namespace in your cluster. You can list namespaces by logging in to the cluster and running kubectl get ns. The Kubernetes namespace that you enter does not have to exist already. Any future namespace with this name can establish trust.

curl -L -X POST 'https://iam.cloud.ibm.com/v1/profiles/<profile-id>/links' -H 'Accept: application/json' -H 'Authorization: Bearer <access_token>' -H 'Content-Type: application/json' --data-raw '{
"name": "<link_name>",
"cr_type": "IKS_SA",
"link": {
"crn": "<cluster_crn>",
"namespace": "<ns>",
"name": "<cluster_name>"
}
}'





3. [Assign the trusted profile to an access group](https://cloud.ibm.com/docs/account?topic=account-groupsaccess_ag_api) with access policies to the IBM Cloud services that you want your apps to have access to.



Your clusters can now establish trust with IBM Cloud IAM. Next, your developers can [configure their application pods to exchange the identity token for an IAM token to authenticate requests to IBM Cloud services](https://cloud.ibm.com/docs/containers?topic=containers-pod-iam-identityiam-identity-pod).





 Creating an IAM trusted profile for your cluster in the UI 

As an account administrator, create a trusted profile in IBM Cloud Identity and Access Management. For more information, see the [IAM documentation](https://cloud.ibm.com/docs/account?topic=account-create-trusted-profile&interface=ui).
Cluster health image v1.2.21 v1.3.3 Updated golang.org/x/crypto to v0.0.0-20220214200702-86341886e292. Adds fix for [CVE-2021-43565](https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-43565). Adds Golang dependency updates. 
 Gateway-enabled cluster controller 1586 1653 Updated to use Go version 1.17.7 and updated Go modules to fix CVEs. 
 GPU device plug-in and installer eefc4ae d7daae6 Updated GPU images to resolve CVEs. 
 IBM Calico extension 923 929 Updated universal base image (UBI) to the 8.5-230 version to resolve CVEs. Updated to use Go version 1.16.13. 
 IBM Cloud Controller Manager v1.20.15-1 v1.20.15-4 Adds changes to the renovate rules. 
 IBM Cloud File Storage for Classic plug-in and monitor 404 405 Fix for [CVE-2021-3538](https://nvd.nist.gov/vuln/detail/CVE-2021-3538) and adds dependency updates. 
 Key Management Service provider v2.3.13 v2.4.3 Updated golang.org/x/crypto to v0.0.0-20220214200702-86341886e292. Adds fix for [CVE-2021-43565](https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-43565). Adds Golang dependency updates. 







 Change log for worker node fix pack 1.20.15_1572, released 28 February 2022 



Changes since version 1.20.15_1570

 Component Previous Current Description
Cluster health image v1.2.21 v1.2.23 Updated golang.org/x/crypto to v0.0.0-20220214200702-86341886e292. Adds fix for [CVE-2021-43565](https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-43565). Adds Golang dependency updates. 
 Gateway-enabled cluster controller 1586 1653 Updated to use Go version 1.17.7 and updated Go modules to fix CVEs. 
 IBM Calico extension 923 929 Updated universal base image (UBI) to version 8.5-230 to resolve CVEs. Updated to use Go version 1.16.13. 
 IBM Cloud Controller Manager v1.19.16-6 v1.19.16-9 Adds changes to the renovate rules. 
 IBM Cloud File Storage for Classic plug-in and monitor 404 405 Fix for [CVE-2021-3538](https://nvd.nist.gov/vuln/detail/CVE-2021-3538). Adds dependency updates. 
 Key Management Service provider v2.3.13 v2.4.3 Updated golang.org/x/crypto to v0.0.0-20220214200702-86341886e292. Adds fix for [CVE-2021-43565](https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-43565). Adds Golang dependency updates. 







 Change log for worker node fix pack 1.19.16_1579, released 28 February 2022 



Changes since version 1.19.16_1577

 Component Previous Current Description
You can improve the security of your cluster and control public traffic apps by modifying the default VPC security group for your cluster. Security groups consist of rules that define which inbound traffic is permitted for your worker nodes. For example, you can use inbound rules to control incoming public traffic to your apps through the VPC load balancer, and outbound rules to control outgoing requests from your apps through the public gateway.

Ready to get started with a cluster for this scenario? After you plan your [high availability](https://cloud.ibm.com/docs/containers?topic=containers-ha_clusters) and [worker node](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodes) setups, see [Creating VPC clusters](https://cloud.ibm.com/docs/containers?topic=containers-clusters).







 Extend your on-premises data center to a VPC cluster 

In this scenario, you run workloads in a VPC cluster. However, you want these workloads to be accessible only to services, databases, or other resources in your private networks in an on-premises data center. Your cluster workloads might need to access a few other IBM Cloud services that support communication over the private network.

Zoom

![Network setup for a VPC cluster that extends an on-prem data center.](https://cloud.ibm.com/docs-content/v1/content/4d497aff9e27b92fbbc2ee4e343a9ec4549cb1d7/containers/images/vpc_extend.png)

Figure 1.Network setup for a VPC cluster that extends an on-prem data center



 Worker-to-worker communication 

To achieve this setup, you create VPC subnets in each zone where you want to deploy worker nodes. No public gateways are required for these subnets. Then, you create a VPC cluster that uses these VPC subnets.

Note that you might have subnet conflicts between the default ranges for workers nodes, pods, and services, and the subnets in your on-premises networks.
Cluster health image v1.2.21 v1.3.3 Updated golang.org/x/crypto to v0.0.0-20220214200702-86341886e292. Adds fix for [CVE-2021-43565](https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-43565). Golang dependency updates. 
 Gateway-enabled cluster controller 1586 1653 Updated to use Go version 1.17.7 and updated Go modules to fix CVEs. 
 IBM Calico extension 923 929 Updated universal base image (UBI) to the 8.5-230 version to resolve CVEs. Updated to use Go version 1.16.13. 
 IBM Cloud Block Storage for Classic driver and plug-in v2.1.6 v2.1.7 Fix for [CVE-2021-3538](https://nvd.nist.gov/vuln/detail/CVE-2021-3538). 
 IBM Cloud Controller Manager v1.20.15-1 v1.20.15-4 Adds changes to the renovate rules. 
 IBM Cloud File Storage for Classic plug-in and monitor 404 405 Adds fix for [CVE-2021-3538](https://nvd.nist.gov/vuln/detail/CVE-2021-3538) and adds dependency updates. 
 Key Management Service provider v2.3.13 v2.4.3 Updated golang.org/x/crypto to v0.0.0-20220214200702-86341886e292. Adds fix for [CVE-2021-43565](https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021-43565). Adds Golang dependency updates. 
 Red Hat OpenShift 4.7.40 4.7.43 For more information, see the [change logs](https://docs.openshift.com/container-platform/4.7/release_notes/ocp-4-7-release-notes.htmlocp-4-7-43).
If you disable this integration and want to enable it later, see [Getting started with Activity Tracker](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-getting-startedgs_step2).



: 

* Logging: You can use Log Analysis to manage operating system logs, application logs, and platform logs. If you want to enable this integration later, see [Logging for clusters](https://cloud.ibm.com/docs/containers?topic=containers-health&interface=ui).



: 

* Monitoring: The monitoring service integration allows operational visibility into the performance and health of your applications, services, and platforms. If you disable this integration and want to enable it later, see [Monitoring cluster health](https://cloud.ibm.com/docs/containers?topic=containers-health-monitor&interface=ui).







 Creating a standard classic cluster in the CLI 

Create your single zone or multizone classic cluster by using the IBM Cloud CLI.



* Ensure that you complete the prerequisites to [prepare your account](https://cloud.ibm.com/docs/containers?topic=containers-clusters&interface=ui) and decide on your cluster setup.
* Install the IBM Cloud CLI and the [IBM Cloud Kubernetes Service plug-in](https://cloud.ibm.com/docs/containers?topic=containers-cli-install).





1. Log in to the IBM Cloud CLI. Enter your IBM Cloud credentials when prompted. If you are logging in with a federated ID, use ibmcloud login --sso.

ibmcloud login [--sso]
2. If you have multiple IBM Cloud accounts, select the account where you want to create your cluster.
3. To create clusters in a resource group other than default, target that resource group. A cluster can be created in only one resource group, and after the cluster is created, you can't change its resource group.
Allow TCP Any 80 Each [control plane CIDR for the region where your cluster is located](https://github.com/IBM-Cloud/kube-samples/tree/master/control-plane-ips). Any After 5 
 Deny all other traffic that does not match the previous rules. Deny ALL Any <br><br> * <br><br><br> Any <br><br> * <br><br><br> Set to bottom 



* Alternatively, to allow the inbound traffic for ALB health checks, you can create a single inbound rule and outbound rule to allow all incoming and outgoing traffic on port 80.
7. In the Attach subnets section, choose the name of the subnet for which you created this ACL.
8. Click Create access control list.
9. Multizone clusters: Repeat steps 2 - 8 to create an ACL for each subnet that your cluster is attached to.





 Creating ACLs with the CLI 

For each subnet that your cluster is attached to, use the IBM Cloud CLI to create a custom ACL with rules that limit inbound and outbound network traffic to only communication that is necessary for the cluster to function. Note that these steps outline the minimum ACL rules that are required to allow a cluster to deploy and to have basic function; they aren't intended to cover all use cases.

Looking for a simpler security setup? Leave the default ACL for your VPC as-is, and modify the [default security group](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-security-group) instead.

Before you begin



1. Install the infrastructure-service plug-in. The prefix for running commands is ibmcloud is.

ibmcloud plugin install infrastructure-service
2. Target the region that your VPC is in.

ibmcloud target -r <region>



To create an ACL for each subnet that your cluster is attached to,



1. List your VPC subnets. For each subnet that your cluster is attached to, get the ID and Subnet CIDR.