vania-janet/multiturn-rag-retrieval-data huggingface dataset:
* Hyper Protect Crypto Services: See [Rotating root keys manually](https://cloud.ibm.com/docs/hs-crypto?topic=hs-crypto-rotate-keys).









 Verifying secret encryption 

After you enable a KMS provider in your IBM Cloud Kubernetes Service cluster, you can verify that your cluster secrets are encrypted by disabling the root key. When you disable the root key, the cluster can no longer decrypt the secrets and becomes unusable, which signifies that your secrets were encrypted.

Before you begin



* Make sure that you have the IBM Cloud IAM Administrator platform and Manager service access role for the cluster.



To verify secret encryption by disabling a root key



1. [Enable KMS encryption in your cluster](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkeyprotect). To check that KMS encryption is enabled, verify that the Key Management Service status is set to enabled in the output of the following command.

ibmcloud ks cluster get -c <cluster_name_or_ID>
2. [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)
3. Verify that you can list the secrets in your cluster.

kubectl get secrets --all-namespaces
4. In your KMS instance, [disable the root key](https://cloud.ibm.com/docs/key-protect?topic=key-protect-disable-keys) that is used to encrypt your cluster. If you encrypted your cluster with a KMS and CRK from a different account, the CRK can only be disabled from the account where it is located.
5. Wait for the cluster to detect the change to your root key.
6. Try to list your secrets. You get a timeout error because you can no longer connect to your cluster.
Operator or site reliability engineer [Administrator platform access role for a cluster, region, or resource group](https://cloud.ibm.com/docs/containers?topic=containers-iam-platform-access-roles), [Reader service access role for a cluster or region](https://cloud.ibm.com/docs/containers?topic=containers-iam-service-access-roles) or [Manager service access role for all cluster namespaces](https://cloud.ibm.com/docs/containers?topic=containers-iam-service-access-roles) to be able to use kubectl top nodes,pods commands. 







 Where can I find a list of security bulletins that affect my cluster? 

If vulnerabilities are found in Kubernetes, Kubernetes releases CVEs in security bulletins to inform users and to describe the actions that users must take to remediate the vulnerability. Kubernetes security bulletins that affect IBM Cloud Kubernetes Service users or the IBM Cloud platform are published in the [IBM Cloud security bulletin](https://cloud.ibm.com/status?component=containers-kubernetes&selected=security).

Some CVEs require the latest patch update for a version that you can install as part of the regular [cluster update process](https://cloud.ibm.com/docs/containers?topic=containers-updateupdate) in IBM Cloud Kubernetes Service. Make sure to apply security patches in time to protect your cluster from malicious attacks. For more information about what is included in a security patch, refer to the [version change log](https://cloud.ibm.com/docs/containers?topic=containers-cs_versionscs_versions_available).





 Does the service offer support for bare metal and GPU? 

Certain VPC worker node flavors offer GPU support. For more information, see the [VPC flavors](https://cloud.ibm.com/docs/containers?topic=containers-vpc-flavors).

Yes, you can provision your worker node as a single-tenant physical bare metal server.
Minimum required permissions
: Operator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--version MAJOR.MINOR.PATCH
: Optional: The Kubernetes version of the cluster. If you don't specify a version, the Kubernetes master is updated to the default API version. To see available versions, run [ibmcloud oc versions](https://cloud.ibm.com/docs/openshift?topic=openshift-kubernetes-service-clics_versions_command).

--force-update
: Optional: Attempt the update even if the change is greater than two minor versions from the worker node version.

-f
: Optional: Force the command to run with no user prompts.

-q
: Optional: Do not show the message of the day or update reminders.



 Example cluster master update command 

ibmcloud oc cluster master update --cluster my_cluster







 ibmcloud oc cluster pull-secret apply 

Virtual Private Cloud

Classic infrastructure

Make an IBM Cloud IAM service ID for the cluster, create a policy for the service ID that assigns the Reader service access role in IBM Cloud Container Registry, and then create an API key for the service ID. The API key is then stored in a Kubernetes image pull secret so that you can pull images from your IBM Cloud Container Registry namespaces for containers that are in the default Red Hat OpenShift project. This process happens automatically when you create a cluster. If you got an error during the cluster creation process or have an existing cluster, you can use this command to apply the process again.

This API key method replaces the previous method of authorizing a cluster to access IBM Cloud Container Registry by automatically creating a [token](https://www.ibm.com/cloud/blog/announcements/announcing-end-of-ibm-cloud-container-registry-support-for-uaa-tokens) and storing the token in an image pull secret.
You can [install your own admission controllers in the cluster](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/admission-webhooks) or choose an optional admission controller, such as [Portieris](https://cloud.ibm.com/docs/openshift?topic=openshift-imagesportieris-image-sec). With Portieris, you can block container deployments from unsigned images.

If you manually installed admission controllers and you don't want to use them anymore, make sure to remove them entirely. If admission controllers are not entirely removed, they might block all actions that you want to perform on the cluster.





 What else can I do to secure my API server? 

You can restrict connections to the master nodes by enabling the private cloud service endpoint, and creating a subnet allowlist. This combination provides the greatest degree of isolation.. Note that your options for service endpoints vary based on your cluster's infrastructure provider. For more information about service endpoints, see worker-to-master and user-to-master communication in [classic clusters](https://cloud.ibm.com/docs/containers?topic=containers-plan_basicsworkeruser-master) and [VPC clusters](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basicsvpc-workeruser-master).

If you enable the private cloud service endpoint, you can create a subnet allowlist. Only authorized requests to your cluster master that originate from subnets in the allowlist are permitted through the cluster's private cloud service endpoint. For more information, see [Creating an allowlist for the private cloud service endpoint](https://cloud.ibm.com/docs/containers?topic=containers-access_clusterprivate-se-allowlist).





 Rotating CA certificates in your cluster 

Revoke existing certificate authority (CA) certificates in your cluster and issue new CA certificates.

By default, certificate authority (CA) certificates are administered to secure access to various components of your cluster, such as the master API server.
Verify that you are connected to the private network through a [VPN](https://cloud.ibm.com/docs/iaas-vpn?topic=iaas-vpn-getting-started) or [IBM Cloud Direct Link](https://cloud.ibm.com/docs/dl?topic=dl-get-started-with-ibm-cloud-dl) connection.
6. Download and add the kubeconfig configuration file for your cluster to your existing kubeconfig in /.kube/config or the last file in the KUBECONFIG environment variable.

ibmcloud ks cluster config -c <cluster_name_or_ID> --endpoint private
7. Verify that kubectl commands run properly and that the Kubernetes context is set to your cluster.

kubectl config current-context

Example output

<cluster_name>/<cluster_ID>







 Creating an allowlist for the private cloud service endpoint 

Control access to your private cloud service endpoint by creating a subnet allowlist.

After you [grant users access to your cluster through IBM Cloud IAM](https://cloud.ibm.com/docs/containers?topic=containers-userschecking-perms), you can add a secondary layer of security by creating an allowlist for the private cloud service endpoint. Only authorized requests to your cluster master that originate from subnets in the allowlist are permitted through the cluster's private cloud service endpoint.

If you want to allow requests from a different VPC than the one your cluster is in, you must include the cloud service endpoint for that VPC in the allowlist.

For example, to access your cluster's private cloud service endpoint, you must connect to your IBM Cloud classic network or your VPC network through a VPN or IBM Cloud Direct Link. You can add the subnet for the VPN or Direct Link tunnel so that only authorized users in your organization can access the private cloud service endpoint from that subnet.

A private cloud service endpoint allowlist can also help prevent users from accessing your cluster after their authorization is revoked.
Verify that the oc commands run properly with your cluster through the private cloud service endpoint by checking the version.

oc version

Example output

Client Version: 4.5.0-0.okd-2020-09-04-180756
Server Version: 4.5.35
Kubernetes Version: v1.18.3+cdb0358







 Creating an allowlist for the private cloud service endpoint 

Control access to your private cloud service endpoint by creating a subnet allowlist.

After you [grant users access to your cluster through IBM Cloud IAM](https://cloud.ibm.com/docs/openshift?topic=openshift-userschecking-perms), you can add a secondary layer of security by creating an allowlist for the private cloud service endpoint. Only authorized requests to your cluster master that originate from subnets in the allowlist are permitted through the cluster's private cloud service endpoint.

If you want to allow requests from a different VPC than the one your cluster is in, you must include the cloud service endpoint for that VPC in the allowlist.

For example, to access your cluster's private cloud service endpoint, you must connect to your IBM Cloud classic network or your VPC network through a VPN or IBM Cloud Direct Link. You can add the subnet for the VPN or Direct Link tunnel so that only authorized users in your organization can access the private cloud service endpoint from that subnet.

A private cloud service endpoint allowlist can also help prevent users from accessing your cluster after their authorization is revoked. When a user leaves your organization, you remove their IBM Cloud IAM permissions that grant them access to the cluster. However, the user might have copied the API key that contains a functional ID's credentials, which contain the necessary IAM permissions for your cluster. That user can still use those credentials and the private cloud service endpoint address to access your cluster from a different subnet, such as from a different IBM Cloud account. If you create an allowlist that includes only the subnets for your VPN tunnel in your organization's IBM Cloud account, the user's attempted access from another IBM Cloud account is denied.
: Viewer platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.



 Example api-key info command 

ibmcloud oc api-key info --cluster my_cluster







 ibmcloud oc api-key reset 

Virtual Private Cloud

Classic infrastructure

Create an IBM Cloud IAM API key that impersonates the user's permissions to authenticate requests for all clusters in the current resource group and region. For more information, see [Understanding how the API key works](https://cloud.ibm.com/docs/openshift?topic=openshift-access-credsapi_key_about).

If you use the Block Storage for VPC or cluster autoscaler add-ons in your cluster, you must re-create the add-on controller pods after you reset your API key. For more information, see [Block Storage for VPC PVC creation fails after API key reset](https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-block-api-key-reset-ts) and [Autoscaling fails after API key reset](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-storage-ca-apikey-reset).

Before you use this command, make sure that the user who runs this command has the required [IBM Cloud Kubernetes Service and IBM Cloud infrastructure permissions](https://cloud.ibm.com/docs/openshift?topic=openshift-access_referencecluster_create_permissions). Target the resource group and region that you want to set the API key for. When the API key is reset, the previous API key that was used, if any, for the region and resource group is now obsolete. You can then delete the old API key from your list of API keys.
: Viewer platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

--c, --cluster CLUSTER
: Required. The name or ID of the cluster.

--output json
: Optional: Prints the command output in JSON format.

-q
: Optional: Do not show the message of the day or update reminders.

--show-deleted
: Optional. Add this option to include instances that were unregistered from the cluster.

Example:

ibmcloud ks ingress instance ls --cluster my-cluster --show-deleted





 ibmcloud ks ingress instance register 

Virtual Private Cloud

Classic infrastructure

Register a Secrets Manager instance to a cluster.

ibmcloud ks ingress instance register --cluster CLUSTER --crn CRN [--is-default] [--secret-group GROUP] [-q]

Minimum required permissions
: Administrator platform access role for the cluster in IBM Cloud Kubernetes Service

Command options:

--c, --cluster CLUSTER
: Required. The name or ID of the cluster.

--crn
: Required. The CRN of the IBM Cloud Secret Manager instance.

--is-default
: Optional. Include this option to also set the registered instance as the default Secrets Manager instance where all Ingress subdomain certificates are stored. If another instance is already set as default, it is removed. Note that you must manually update any certificates or secrets to upload them to the new default instance. Otherwise, they are uploaded at the next scheduled update time for the secret.

-q
: Optional: Do not show the message of the day or update reminders.

--secret-group GROUP
: Secret group ID of the IBM Cloud Secret Manager instance where the secrets are persisted. To get a secret group ID, see the [Secrets Manager CLI reference](https://cloud.ibm.com/docs/secrets-manager-cli-plugin?topic=secrets-manager-cli-plugin-secrets-manager-clisecrets-manager-cli-secret-groups-command).
[Create a Key Protect service instance](https://cloud.ibm.com/docs/key-protect?topic=key-protect-provisionprovision).
3. [Create a Key Protect root key](https://cloud.ibm.com/docs/key-protect?topic=key-protect-create-root-keyscreate-root-keys).
4. Get the following values for your Key Protect instance:



* [The service instance ID](https://cloud.ibm.com/docs/key-protect?topic=key-protect-retrieve-instance-ID).
* [The API key for the service instance ID](https://cloud.ibm.com/docs/account?topic=account-serviceidapikeyscreate_service_key).
* [The service endpoint URL](https://cloud.ibm.com/docs/key-protect?topic=key-protect-regionsservice-endpoints).



5. Create a Kubernetes secret named keyprotect-config that contains the values that you retrieved. The Image Key Synchronizer add-on uses the environment variables in this secret to authenticate with your Key Protect instance.

apiVersion: v1
kind: Secret
metadata:
name: keyprotect-config
namespace: image-key-synchronizer
type: Opaque
stringData:
config.json: 
{
"keyprotect-url":"<service_endpoint>",
"instance-id": "<service_instance_ID>",
"apikey": "<service_instance_ID_API_key>"
}





To deploy containers that use encrypted images:



1. Enable the Image Key Synchronizer add-on.

ibmcloud oc cluster addon enable image-key-synchronizer -c <cluster_name_or_ID>
2.
secret-key: xxx
kp-root-key-crn: <keyprotect-root-key-crn in base64 encoded format>
kind: Secret
metadata:
name: <cos-write-access>
type: ibm/ibmc-s3fs
2. Apply the secret to your cluster.

kubectl apply -f <secret_name>



api-key
: Enter the API key that you retrieved from your IBM Cloud Object Storage service credentials earlier. If you want to use HMAC authentication, specify the access-key and secret-key instead.

access-key
: Enter the access key ID that you retrieved from your IBM Cloud Object Storage service credentials earlier. If you want to use OAuth2 authentication, specify the api-key instead.

secret-key
: Enter the secret access key that you retrieved from your IBM Cloud Object Storage service credentials earlier. If you want to use OAuth2 authentication, specify the api-key instead.

service-instance-id
: Enter the GUID of your IBM Cloud Object Storage service instance that you retrieved earlier.

kp-root-key-crn
: Enter the base64 encoded Key Protect root key CRN to use Key Protect encryption.
5. Get the secrets in your cluster and verify the output.

oc get secret

Example output

NAME TYPE DATA AGE
cos-write-access ibm/ibmc-s3fs 2 7d19h
default-au-icr-io kubernetes.io/dockerconfigjson 1 55d
default-de-icr-io kubernetes.io/dockerconfigjson 1 55d
...
6.
: Authorization: Your IBM Cloud IAM access token (bearer <iam_token>).
5. Review the IBM Cloud Kubernetes Service API documentation]https_containers.cloud.ibm.com_global_swagger-global-api] ! ! ! to find a list of supported APIs.<-- </ol> -->When you use the API for automation, be sure to rely on the responses from the API, not files within those responses. For example, the Kubernetes configuration file for your cluster context is subject to change, so don't build automation based on specific contents of this file when you use the GET /v1/clusters/{idOrName}/config call.<-- </section "id="section-cs_api" "> --><-- <section "id="section-kube_api" "> --> Working with your cluster by using the Kubernetes API You can use the Kubernetes API]] to interact with your cluster in IBM Cloud Kubernetes Service. For authentication details, see IBM Cloud IAM issuer details for RBAC users]] ! .The following instructions require public network access in your cluster to connect to the public cloud service endpoint of your Kubernetes master.<-- <ol> -->1. Follow the steps in Automating cluster deployments with the API]] !!!! !!!! !!! to retrieve your IBM Cloud IAM access token, refresh token, the ID of the cluster where you want to run Kubernetes API requests, and the IBM Cloud Kubernetes Service region where your cluster is located.2. Retrieve an IBM Cloud IAM delegated refresh token.
POST https://iam.cloud.ibm.com/identity/token

Header
: <-- <ul> -->

* Content-Type: application/x-www-form-urlencoded
* Authorization: Basic Yng6Yng= where Yng6Yng= equals the URL-encoded authorization for the username bx and the password bx.
* cache-control: no-cache
ibmcloud oc cluster config --cluster CLUSTER [--admin] [--endpoint ENDPOINT_TYPE] [--network] [--skip-rbac] [-q] [--yaml]

Minimum required permissions
: Viewer or Reader IBM Cloud IAM service access role for the cluster in IBM Cloud Kubernetes Service. Further, if you have only a platform access role or only a service access role, additional constraints apply.



* Platform: If you have only a platform access role, you can perform this command, but you need a [service access role](https://cloud.ibm.com/docs/openshift?topic=openshift-userschecking-perms) or a [custom RBAC policy](https://cloud.ibm.com/docs/openshift?topic=openshift-access-overviewrole-binding) to perform Kubernetes actions in the cluster.
* Service: If you have only a service access role, you can perform this command. However, your cluster admin must give you the cluster name, ID, and master URL because you can't run the ibmcloud oc cluster ls command or open the IBM Cloud Kubernetes Service console to view clusters. After you receive the cluster name and ID, you can open the Red Hat OpenShift web console by opening your browser to <master_URL>/console.



Command options:

-c, --cluster CLUSTER
: Required: The name or ID of the cluster.

--admin
: Optional: Download the TLS certificates and permission files for the Super User role. You can use the certs to automate tasks in a cluster without having to reauthenticate. The files are downloaded to <user_home_directory>/.bluemix/plugins/kubernetes-service/clusters/<cluster_name>-admin.

--endpoint ENDPOINT_TYPE
: Optional: Specify the type of endpoint to use to connect to the cluster. If you don't specify this option, the default service endpoint for your cluster is used.
* Both service endpoint URLs: You can access your cluster either through the [public](https://cloud.ibm.com/docs/containers?topic=containers-access_clusteraccess_public_se) or the [private](https://cloud.ibm.com/docs/containers?topic=containers-access_clusteraccess_private_se) service endpoint.



6. You can also access your VPC cluster through the [Virtual Private Endpoint](https://cloud.ibm.com/docs/containers?topic=containers-access_clustervpc_vpe).







 Accessing clusters through the public cloud service endpoint 

To work with your cluster, set the cluster that you created as the context for a CLI session to run kubectl commands.

If you want to use the IBM Cloud console instead, you can run CLI commands directly from your web browser in the [IBM Cloud Shell](https://cloud.ibm.com/docs/cloud-shell?topic=cloud-shell-shell-ui).



1. Set the cluster that you created as the context for this session. Complete these configuration steps every time that you work with your cluster.



1. Download and add the kubeconfig configuration file for your cluster to your existing kubeconfig in /.kube/config or the last file in the KUBECONFIG environment variable.

ibmcloud ks cluster config -c <cluster_name_or_ID>
2. Verify that kubectl commands run properly and that the Kubernetes context is set to your cluster.

kubectl config current-context

Example output

<cluster_name>/<cluster_ID>



2. Launch your Kubernetes dashboard with the default port 8001.



1. Set the proxy with the default port number.

kubectl proxy

Starting to serve on 127.0.0.1:8001
2. Open the following URL in a web browser to see the Kubernetes dashboard.