vania-janet/multiturn-rag-retrieval-data huggingface dataset:
You can connect multiple VPCs to classic infrastructure, such as using IBM Cloud Transit Gateway to manage access between your VPCs in multiple regions to resources in your IBM Cloud classic infrastructure.









 External communication to apps that run on worker nodes 

Allow private or public traffic requests from outside the cluster to your apps that run on worker nodes.





 Private traffic to cluster apps 

When you deploy an app in your cluster, you might want to make the app accessible to only users and services that are on the same private network as your cluster. Private load balancing is ideal for making your app available to requests from outside the cluster without exposing the app to the general public. You can also use private load balancing to test access, request routing, and other configurations for your app before you later expose your app to the public with public network services.

To allow private network traffic requests from outside the cluster to your apps, you can use private Kubernetes networking services, such as creating [LoadBalancer services](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-lbaas). For example, when you create a Kubernetes LoadBalancer service in your cluster, a load balancer for VPC is automatically created in your VPC outside of your cluster. The VPC load balancer is multizonal and routes requests for your app through the private NodePorts that are automatically opened on your worker nodes. For VPC ALBs, a [security group](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-security-group), named in the form of kube-<vpc-id>, is automatically attached to your ALB instance.

The kube-<vpc-id> security group that is attached to your VPC ALB is the same security group used for the private VPE gateway that communicates with the Kubernetes master. Do not modify this security group, as doing so might result in disruptions to the network connectivity between the cluster and the Kubernetes master.
To remove a relation mention later, you can left-click the relation type and then press the Delete key or click X next to the relation type.
9. Click Save at any time to save your work.







 What to do next 

After you finish annotating all entity mentions, relation mentions, and coreferences in the document, as applicable, change the document status from In Progress to Completed, click Save, and then close the document.

After you finish annotating all documents and mark them Completed, the status of the annotation set changes to Submitted. That is how project managers know that they can start to evaluate the documents for inter-annotator agreement, and reject them or accept them and promote them to ground truth.







 Related information 

[Creating dictionaries](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-dictionaries)

[Troubleshooting the ground truth editor](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-ts_user_contactingsupport)

[Establishing a type system](https://cloud.ibm.com/docs/watson-knowledge-studio-data?topic=watson-knowledge-studio-data-typesystem)
* etcd data store: Stores all Kubernetes resources of a cluster, such as Services, Deployments, and Pods. Kubernetes ConfigMaps and Secrets are app data that is stored as key value pairs so that they can be used by an app that runs in a pod. Data in etcd is stored on the local disk of the Red Hat OpenShift master and is backed up to IBM Cloud Object Storage. Data is encrypted during transit to IBM Cloud Object Storage and at rest. You can choose to enable encryption for your etcd data on the local disk of your Red Hat OpenShift master by [enabling IBM Key Protect encryption](https://cloud.ibm.com/docs/openshift?topic=openshift-encryptionkeyprotect) for your cluster. When etcd data is sent to a pod, data is encrypted via TLS to ensure data protection and integrity.
* openshift-api: Serves as the main entry point for all cluster management requests from the worker node to the Red Hat OpenShift master. The API server validates and processes requests that change the state of cluster resources, such as pods or services, and stores this state in the etcd data store.
* openshift-controller: Watches for newly created pods and decides where to deploy them based on capacity, performance needs, policy constraints, anti-affinity specifications, and workload requirements. If no worker node can be found that matches the requirements, the pod is not deployed in the cluster. The controller also watches the state of cluster resources, such as replica sets. When the state of a resource changes, for example if a pod in a replica set goes down, the controller manager initiates correcting actions to achieve the required state.
* cloud-controller-manager: The cloud controller manager manages cloud provider-specific components such as the IBM Cloud load balancer.
* Konnectivity: Red Hat OpenShift on IBM Cloud-specific component to provide secured network connectivity for all Red Hat OpenShift master to worker node communication. The Konnectivity server works with the Konnectivity agent to securely connect the master to the worker node.
: The [Kubernetes scheduler](https://github.com/openshift/cluster-kube-scheduler-operator) watches for newly created pods and decides where to deploy them based on capacity, performance needs, policy constraints, anti-affinity specifications, and workload requirements. If no worker node can be found that matches the requirements, the pod is not deployed in the cluster.

manifests-bootstrapper
: The manifests-boot-strapper job sets up the master with the required certificates to join as the master node of the cluster.

oauth-openshift
: The built-in OAuth server is automatically set up to integrate with IBM Cloud Identity and Access Management (IAM). You can't add other supported identity providers to the cluster. For more information about how to authenticate with the cluster via IAM, see [Accessing Red Hat OpenShift clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-access_cluster).

openshift-apiserver, openshift-apiserver-operator, kube-apiserver
: The API server is the main entry point for all cluster management requests from the worker node to the master. The API server validates and processes requests that change the state of Kubernetes objects, such as pods or services, and Red Hat OpenShift objects, such as projects or users. Then, the API server stores this state in the etcd data store.

konnectivity-server, konnectivity-operator
: The Konnectivity server works with the Konnectivity agent to securely connect the master to the worker node. This connection supports apiserver proxy calls to your pods and services; oc exec, attach, and logs calls to the kubelet; and mutating and validating webhooks.

Admission controllers
: Admission controllers are implemented for specific features in Red Hat OpenShift on IBM Cloud clusters. With admission controllers, you can set up policies in your cluster that determine whether a particular action in the cluster is allowed or not.
To remove a relation mention later, you can left-click the relation type and then press the Delete key or click X next to the relation type.
9. Click Save at any time to save your work.







 What to do next 

After you finish annotating all entity mentions, relation mentions, and coreferences in the document, as applicable, change the document status from In Progress to Completed, click Save, and then close the document.

After you finish annotating all documents and mark them Completed, the status of the annotation set changes to Submitted. That is how project managers know that they can start to evaluate the documents for inter-annotator agreement, and reject them or accept them and promote them to ground truth.







 Related information 



* [Creating dictionaries](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-dictionaries)
* [Troubleshooting the ground truth editor](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-ts_user_contactingsupport)
* [Establishing a type system](https://cloud.ibm.com/docs/watson-knowledge-studio?topic=watson-knowledge-studio-typesystem)
Regular maintenance and automation tooling accesses the masters of Satellite-enabled IBM Cloud service clusters in your location through the default openshift-api-<cluster_ID> Link endpoint. This endpoint allows the Red Hat OpenShift on IBM Cloud API to communicate with the master for the service cluster. For example, if you create a Red Hat OpenShift cluster to run applications in your location, all version updates for that cluster's master are automatically applied through the default openshift-api-<cluster_ID> Link endpoint for that cluster.

Updates to Satellite-enabled IBM Cloud services that are running on hosts in your location are initiated by the IBM Cloud team for that service. When a change is ready to be rolled out to a service, the Satellite-enabled IBM Cloud service team uses [Satellite Config](https://cloud.ibm.com/docs/satellite?topic=satellite-setup-clusters-satconfig) to upload a new version of the service to the subscription that the Satellite-enabled IBM Cloud service cluster is included in. To apply the service update through Satellite Config, the control plane automation for the IBM Cloud Satellite API server, which exists in IBM Cloud, deploys the update to the master of the Satellite-enabled IBM Cloud service cluster in your location through the default openshift-api-<cluster_ID> Link endpoint. The cluster master then applies the updates to the worker nodes in the cluster.





 What access do IBM SREs have to my location control plane, including the masters of Satellite-enabled IBM Cloud service clusters? 

The default satellite-healthcheck-<location_ID> Link endpoint allows the Satellite control plane master to check the health of your location's control plane cluster, and alerts IBM Site Reliability Engineers (SREs) when manual intervention is required.



* To manually resolve issues with your Satellite location infrastructure management, such as host assignment or attachment, IBM SREs use tooling to access the IBM Cloud Satellite API server. The Satellite API server communicates with the Satellite control plane master in IBM Cloud.
Download and add the kubeconfig configuration file for your cluster to your existing kubeconfig in /.kube/config or the last file in the KUBECONFIG environment variable.

ibmcloud ks cluster config -c <cluster_name_or_ID> --endpoint private
3. Verify that kubectl commands run properly and that the Kubernetes context is set to your cluster.

kubectl config current-context

Example output

<cluster_name>/<cluster_ID>







 Accessing classic clusters through the private cloud service endpoint 

The Kubernetes master is accessible through the private cloud service endpoint if authorized cluster users are in your IBM Cloud private network or are connected to the private network such as through a [classic VPN connection](https://cloud.ibm.com/docs/iaas-vpn?topic=iaas-vpn-getting-started) or [IBM Cloud Direct Link](https://cloud.ibm.com/docs/dl?topic=dl-get-started-with-ibm-cloud-dl). However, communication with the Kubernetes master over the private cloud service endpoint must go through the 166.X.X.X IP address range, which is not routable from a classic VPN connection or through IBM Cloud Direct Link. You can expose the private cloud service endpoint of the master for your cluster users by using a private network load balancer (NLB). The private NLB exposes the private cloud service endpoint of the master as an internal 10.X.X.X IP address range that users can access with the VPN or IBM Cloud Direct Link connection. If you enable only the private cloud service endpoint, you can use the Kubernetes dashboard or temporarily enable the public cloud service endpoint to create the private NLB.



1. Get the private cloud service endpoint URL and port for your cluster.

ibmcloud ks cluster get -c <cluster_name_or_ID>

In this example output, the Private Service Endpoint URL is https://c1.private.us-east.containers.cloud.ibm.com:25073.

NAME: setest
If the public cloud service endpoint is also enabled, worker-to-master traffic is established half over the public endpoint and half over the VPE for protection from potential outages of the public or private network.



To secure communication over public and private cloud service endpoints or VPEs, Red Hat OpenShift on IBM Cloud automatically sets up a Konnectivity connection between the Kubernetes master and worker nodes when the cluster is created. Worker nodes securely talk to the master through TLS certificates, and the master talks to workers through the Konnectivity connection.





 User-to-master communication in VPC clusters 

You can allow authorized cluster users to communicate with the Kubernetes master by enabling the public and private cloud service endpoints, or the private cloud service endpoint only.



* Public and private cloud service endpoints: By default, all calls to the master that are initiated by authorized cluster users are routed through the public cloud service endpoint. If authorized cluster users are in your VPC network or are connected through a [VPC VPN connection](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-vpnaas), the master is privately accessible through the private cloud service endpoint.
* Private cloud service endpoint only: To access the master through the private cloud service endpoint, authorized cluster users must either be in your VPC network or connected through a [VPC VPN connection](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-vpnaas).



You can secure access to your private cloud service endpoint by creating a subnet allowlist. Only authorized requests to your cluster master that originate from subnets in the allowlist are permitted through the cluster's private cloud service endpoint. Use of the allowlist function helps prevent unauthorized scanning activities. For more information, see [Creating an allowlist for the private cloud service endpoint](https://cloud.ibm.com/docs/containers?topic=containers-access_clusterprivate-se-allowlist).





 Worker communication to other services or networks 

Allow your worker nodes to securely communicate with other IBM Cloud services, on-premises networks, other VPCs, and IBM Cloud classic infrastructure resources.
Change log for master fix pack 4.9.17_1525_openshift and worker node fix pack 4.9.17_1523_openshift, released 9 February 2022 

The following table shows the changes that are in the master fix pack 4.9.17_1525_openshift and worker node fix pack 4.9.17_1523_openshift. Master patch updates are applied automatically. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 4.8.26_1542_openshift master and 4.8.28_1543_openshift worker node.

 Component Previous Current Description 

 Calico v3.20.3 v3.21.4 See the [Calico release notes](https://docs.tigera.io/archive). 
 Calico Operator v1.20.5 v1.23.5 See the [Calico Operator release notes](https://github.com/tigera/operator/releases/tag/v1.23.5). 
 Cluster health image v1.2.21 v1.3.0 Added new health check for cluster autoscaler add-on. 
 etcd N/A N/A Updated to support metrics collection for OpenShift. 
 IBM Cloud Controller Manager v1.21.9-1 v1.22.6-1 Updated to support the Kubernetes 1.22.6 release and to use Go version 1.16.12. In addition, the code for this component is now [open source](https://github.com/IBM-Cloud/cloud-provider-ibm). 
 Red Hat OpenShift (master) 4.8.26 4.9.17 See the [OpenShift release notes](https://docs.openshift.com/container-platform/4.9/release_notes/ocp-4-9-release-notes.htmlocp-4-9-17).
: Kubernetes version [1.18.9_1528](https://cloud.ibm.com/docs/containers?topic=containers-118_changelog1189_1528), [1.17.12_1540](https://cloud.ibm.com/docs/containers?topic=containers-117_changelog11712_1540), and [1.16.15_1547](https://cloud.ibm.com/docs/containers?topic=containers-116_changelog11615_1547).

Versions
: Worker node fix pack update change log documentation is available.
: Kubernetes version [1.18.8_1527](https://cloud.ibm.com/docs/containers?topic=containers-118_changelog1188_1527), [1.17.11_1539](https://cloud.ibm.com/docs/containers?topic=containers-117_changelog11711_1539), [1.16.14_1546](https://cloud.ibm.com/docs/containers?topic=containers-116_changelog11614_1546), and [1.15.12_1552](https://cloud.ibm.com/docs/containers?topic=containers-115_changelog11512_1552).

Istio add-on
: Versions [1.7.1](https://cloud.ibm.com/docs/containers?topic=containers-istio-changelog171) and [1.6.9](https://cloud.ibm.com/docs/containers?topic=containers-istio-changelog169) of the Istio managed add-on are released.

VPC load balancer
: In version 1.18 and later VPC clusters, added the service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: "proxy-protocol" annotation to preserve the source IP address of requests to apps in your cluster.





 4 September 2020 

New! CIS Kubernetes Benchmark
IBM Cloud Activity Tracker events 

You can view, manage, and audit user-initiated activities in your IBM Cloud® Kubernetes Service community Kubernetes or Red Hat OpenShift cluster by using the IBM Cloud® Activity Tracker service.

IBM Cloud Kubernetes Service automatically generates cluster management events and forwards these event logs to IBM Cloud Activity Tracker. To access these logs, you must [provision an instance of IBM Cloud Activity Tracker](https://cloud.ibm.com/docs/activity-tracker?topic=activity-tracker-getting-started).



 Cluster events 

The following list of cluster events are sent to IBM Cloud Activity Tracker.



Cluster events

 Action Description 

 containers-kubernetes.cluster.config The Kubernetes configuration file (kubeconfig) for a cluster is requested. Depending on the request, the kubeconfig might contain administrator or network certificates and secrets to access the cluster. 
 containers-kubernetes.cluster.create The creation of a cluster is requested. This event is sent for any type of cluster, such as clusters that are created in different infrastructure providers. 
 containers-kubernetes.cluster.list Clusters are listed. The list might be filtered by details such as the infrastructure provider. 
 containers-kubernetes.cluster.delete A cluster is deleted. 
 containers-kubernetes.cluster.get Details for a cluster are returned. 
 containers-kubernetes.cluster-apikey-owner.get The API key owner for the region and resource group that the cluster is in is returned. 
 containers-kubernetes.cluster-audit-webhook.delete A cluster audit webhook for forwarding master API server audit logs is removed. 
 containers-kubernetes.cluster-audit-webhook.get Audit webhooks for a cluster are listed. 
 containers-kubernetes.cluster-audit-webhook.update A cluster audit webhook for forwarding master API server audit logs is updated. 
 containers-kubernetes.cluster-ca-certificate.create A certificate authority (CA) for a cluster is created.
To re-create the VPC load balancer for your Kubernetes LoadBalancer service, restart the Kubernetes master by running ibmcloud ks cluster master refresh --cluster <cluster_name_or_id>.



If you want to remove the load balancing setup for an app in your VPC cluster, delete the Kubernetes LoadBalancer service by running kubectl delete svc <kubernetes_lb_service_name>. The VPC load balancer that is associated with the Kubernetes LoadBalancer service is automatically deleted from your VPC.
3. If the load balancer exists, [view the VPC security groups that are attached to it](https://cloud.ibm.com/docs/containers?topic=containers-vpc-security-group). If you have made any modifications to the kube-<vpc-id> security group, which is automatically attached to the load balancer, set the original rules back in the security group.



* If the VPC load balancer is listed and you have not modified any attached security groups, it might not be responsive for the following reasons:



* Its DNS entry might still be registering. When a VPC load balancer is created, the hostname is registered through a public DNS. Sometimes, it can take several minutes for this DNS entry to be replicated to the specific DNS that your client is using. You can either wait for the hostname to be registered in your DNS, or access the VPC load balancer directly by using one of its IP addresses. To find the VPC load balancer IP addresses, run ibmcloud is lb <LB_ID> and look for the Public IPs field.
* If after several minutes you can't reach the load balancer, it might be offline due to provisioning or connection issues. [Open an IBM Cloud support case](https://cloud.ibm.com/unifiedsupport/cases/add). For the type, select Technical. For the category, select Network in the VPC section. In the description, include your cluster ID and the VPC load balancer ID.
To re-create the VPC load balancer for your Kubernetes LoadBalancer service, restart the Kubernetes master by running ibmcloud oc cluster master refresh --cluster <cluster_name_or_id>.



If you want to remove the load balancing setup for an app in your VPC cluster, delete the Kubernetes LoadBalancer service by running oc delete svc <kubernetes_lb_service_name>. The VPC load balancer that is associated with the Kubernetes LoadBalancer service is automatically deleted from your VPC.
3. If the load balancer exists, [view the VPC security groups that are attached to it](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-security-group). If you have made any modifications to the kube-<vpc-id> security group, which is automatically attached to the load balancer, set the original rules back in the security group.



* If the VPC load balancer is listed and you have not modified any attached security groups, it might not be responsive for the following reasons:



* Its DNS entry might still be registering. When a VPC load balancer is created, the hostname is registered through a public DNS. Sometimes, it can take several minutes for this DNS entry to be replicated to the specific DNS that your client is using. You can either wait for the hostname to be registered in your DNS, or access the VPC load balancer directly by using one of its IP addresses. To find the VPC load balancer IP addresses, run ibmcloud is lb <LB_ID> and look for the Public IPs field.
* If after several minutes you can't reach the load balancer, it might be offline due to provisioning or connection issues. [Open an IBM Cloud support case](https://cloud.ibm.com/unifiedsupport/cases/add). For the type, select Technical. For the category, select Network in the VPC section. In the description, include your cluster ID and the VPC load balancer ID.