vania-janet/multiturn-rag-retrieval-data huggingface dataset:
After that, you bind the database to the Kubernetes cluster through the private endpoint with the cluster service bind command.

ibmcloud ks cluster service bind <your_cluster_name> <resource_group> <your_database_deployment> --key <your-private-key>

Verify - Verify that the Kubernetes secret was created in your cluster namespace. Running the following command, you get the API key for accessing the instance of your deployment that is provisioned in your account.

kubectl get secrets --namespace=default

More information on binding services is found in the [Kubernetes Service documentation](https://cloud.ibm.com/docs/containers?topic=containers-service-bindingbind-services).





 Configuring in your Kubernetes app 

When you bind your application to Kubernetes Service, it creates an environment variable from the cluster's secrets. Your deployment's connection information lives in BINDING as a JSON object. Load and parse the JSON object into your application to retrieve the information your application's driver needs to make a connection to the database.

The [Connection Strings](https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-connection-stringsconnection-string-breakdown) page contains a reference of the JSON fields.

For more information, see the [Kubernetes Service docs](https://cloud.ibm.com/docs/containers?topic=containers-service-bindingreference_secret).
You must deploy a broad set of security tools for Kubernetes, such as IBM Cloud Security and Compliance Center Workload Protection. As part of a broader Kubernetes security strategy, KSPM allows teams to validate the security of Kubernetes configurations to find and remediate mistakes that can trigger a breach. By running continuous, automatic scans of Kubernetes configurations, admins can mitigate one of the most common areas of attack, human error. At the same time, you can automate compliance in even the most complex Kubernetes clusters.

For more information, see [About KSPM](https://cloud.ibm.com/docs/workload-protection?topic=workload-protection-kspm).





 Network policies for containerized workloads 

When clients provision a cluster, they can choose a networking setup so that certain cluster components can communicate with each other and with networks or services outside of the cluster. Examples of [networking configurations](https://cloud.ibm.com/docs/containers?topic=containers-plan_vpc_basics) are:



* Worker-to-worker
* Worker-to-master
* User-to-master
* Worker communication to other services or networks
* External communication to apps that run on worker nodes.



Clusters on Virtual Private Clouds (VPCs) run on secure, isolated virtual networks where traffic flow to and from worker nodes is controlled by applying rules defined through security groups (SGs). Traffic flow to and from the cluster subnets is controlled by ACLs. Clients can define container network policies to restrict egress and ingress traffic and communication between applications.

With the [Network Security Policy tool](https://cloud.ibm.com/docs/workload-protection?topic=workload-protection-netsec_policy) that is available through the IBM Cloud Security and Compliance Center Workload Protection service, clients can generate and fine-tune Kubernetes network policies based on the traffic allowed or denied, and track ingress and egress communication. For example, you can configure a "least-privilege" policy to protect your workloads or view existing network policies that have been applied to you workloads.
Learn more about [securing your personal information](https://cloud.ibm.com/docs/containers?topic=containers-securitypi) when you work with Kubernetes resources.
6. Make the app accessible by exposing the deployment as a NodePort service. Because your VPC worker nodes are connected to a private subnet only, the NodePort is assigned only a private IP address and is not exposed on the public network. Other services that run on the private network can access your app by using the private IP address of the NodePort service.

kubectl expose deployment/hello-world-deployment --type=NodePort --name=hello-world-service --port=8080 --target-port=8080

Example output

service/hello-world-service exposed



Table 1. Information about the command options.

 Parameter Description 

 expose Expose a Kubernetes resource, such as a deployment, as a Kubernetes service so that users can access the resource by using the IP address of the service. 
 deployment/<hello-world-deployment> The resource type and the name of the resource to expose with this service. 
 --name=<hello-world-service> The name of the service. 
 --type=NodePort The service type to create. In this lesson, you create a NodePort service. In the following lesson, you create a LoadBalancer service. 
 --port=<8080> The port on which the service listens for external network traffic. 
 --target-port=<8080> The port that your app listens on and to which the service directs incoming network traffic. In this example, the target-port is the same as the port, but other apps that you create might use a different port. 



7. Now that all the deployment work is done, you can test your app from within the cluster. Get the details to form the private IP address that you can use to access your app.



1. Get information about the service to see which NodePort was assigned. The NodePorts are randomly assigned when they are generated with the expose command, but within 30000-32767. In this example, the NodePort is 30872.
* [Stack overflow issue](https://stackoverflow.com/questions/70962800/error-post-http-localhost-api-v1-namespaces-kube-system-configmaps-dial-tc)



Based on previous experience following are the two workaround solution to fix.

Solution 1

If you are using provider authentication to Kuberentes something like

provider "kubernetes" {
config_path = data.ibm_container_cluster_config.cluster_config.config_file_path
}

Change the template as follows.

provider "kubernetes" {
host = data.ibm_container_cluster_config.cluster_config.host
client_certificate = data.ibm_container_cluster_config.cluster_config.admin_certificate
client_key = data.ibm_container_cluster_config.cluster_config.admin_key
cluster_ca_certificate = data.ibm_container_cluster_config.cluster_config.ca_certificate
}

Solution 2

You can [remove the state](https://cloud.ibm.com/docs/schematics?topic=schematics-schematics-cli-referenceschematics-wks_staterm) related to cluster config from the statefile by using IBM Cloud Schematics CLI command and retrigger the Terraform plan or Terraform apply command.

ibmcloud schematics workspace state rm --id --address <enter the workspace ID and the address of the resource to mark as taint>
containers-kubernetes.storage-volume.delete A volume is deleted. 
 containers-kubernetes.storage-volume.list Volumes in the IBM Cloud account or as filtered by provider are retrieved. 
 containers-kubernetes.storage-volume.read A volume is retrieved 
 containers-kubernetes.storage-volume.update A volume is updated. 
 containers-kubernetes.storage-attachment.create A volume attachment is created. 
 containers-kubernetes.storage-attachment.delete A volume attachment is deleted. 
 containers-kubernetes.storage-attachment.list Volume attachments are retrieved. 
 containers-kubernetes.storage-attachment.read A volume attachment is retrieved. 
 containers-kubernetes.storage-odf.install The OpenShift Data Foundation add-on is enabled. 
 containers-kubernetes.storage-odf.uninstall The OpenShift Data Foundation add-on is disabled or the cluster is deleted. 







 Worker node and worker pool events 

The following list of worker node and worker pool events are sent to IBM Cloud Activity Tracker.



Worker node and worker pool events

 Action Description 

 containers-kubernetes.cluster-worker.add A worker node is added to the cluster. Note: Adding stand-alone worker nodes is deprecated. 
 containers-kubernetes.cluster-worker.list The worker nodes for a cluster are listed. 
 containers-kubernetes.cluster-worker-pool.create A worker pool is created in the cluster. 
 containers-kubernetes.cluster-worker-pool.delete A worker pool is deleted from a cluster. 
 containers-kubernetes.cluster-worker-pool.get The details of a worker pool in the cluster are returned. 
 containers-kubernetes.cluster-worker-pool.list The worker pools for a cluster are listed. 
 containers-kubernetes.cluster-worker-pool.rebalance A worker pool is rebalanced. 
 containers-kubernetes.cluster-worker-pool.resize A worker pool is resized, to add or decrease the number of worker nodes in the pool.
kubectl get daemonsets -n ibm-observe

Example output

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
sysdig-agent 9 9 9 9 9 <none> 14m

The number of daemon set instances that are deployed equals the number of worker nodes in your cluster.
3. Review the ConfigMap that was created for your Monitoring agent.

kubectl describe configmap -n ibm-observe



4. Access the metrics for your pods and cluster from the Monitoring dashboard.



1. From the [Kubernetes clusters console](https://cloud.ibm.com/kubernetes/clusters), select the cluster that you configured.
2. On the cluster Overview page, click Launch. The Monitoring dashboard opens.
3. Review the pod and cluster metrics that the Monitoring agent collected from your cluster. It might take a few minutes for your first metrics to show.



5. Review how you can work with the [Monitoring dashboard](https://cloud.ibm.com/docs/monitoring?topic=monitoring-panels) to further analyze your metrics.







 Viewing cluster states 

Review the state of a Kubernetes cluster to get information about the availability and capacity of the cluster, and potential problems that might occur.

To view information about a specific cluster, such as its zones, service endpoint URLs, Ingress subdomain, version, and owner, use the ibmcloud ks cluster get --cluster <cluster_name_or_ID>[command](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_get). Include the --show-resources option to view more cluster resources such as add-ons for storage pods or subnet VLANs for public and private IPs.

You can review information about the overall cluster, the IBM-managed master, and your worker nodes. To troubleshoot your cluster and worker nodes, see [Troubleshooting clusters](https://cloud.ibm.com/docs/containers?topic=containers-debug_clusters).
Why do pods remain in pending state? 

Virtual Private Cloud

Classic infrastructure

 What’s happening 

When you run kubectl get pods, you can see pods that remain in a Pending state.

 Why it’s happening 

If you just created the Kubernetes cluster, the worker nodes might still be configuring.

If this cluster is an existing one:



* You might not have enough capacity in your cluster to deploy the pod.
* The pod might have exceeded a resource request or limit.



 How to fix it 

This task requires the IBM Cloud IAM [Administrator platform access role](https://cloud.ibm.com/docs/containers?topic=containers-userschecking-perms) for the cluster and the [Manager service access role](https://cloud.ibm.com/docs/containers?topic=containers-userschecking-perms) for all namespaces.

If you just created the Kubernetes cluster, run the following command and wait for the worker nodes to initialize.

kubectl get nodes

If this cluster is an existing one, check your cluster capacity.



1. Set the proxy with the default port number.

kubectl proxy
2. Open the Kubernetes dashboard.

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
3. Check if you have enough capacity in your cluster to deploy your pod.
4. If you don't have enough capacity in your cluster, resize your worker pool to add more nodes.



1. Review the current sizes and flavors of your worker pools to decide which one to resize.

ibmcloud ks worker-pool ls
2. Resize your worker pools to add more nodes to each zone that the pool spans.

ibmcloud ks worker-pool resize --worker-pool <worker_pool> --cluster <cluster_name_or_ID> --size-per-zone <workers_per_zone>



5. Optional: Check your pod resource requests.



1.
Konnectivity agent v0.0.21e_231_iks v0.0.23_245_iks See the [Konnectivity release notes](https://github.com/kubernetes-sigs/apiserver-network-proxy/releases/tag/v0.0.23). 
 Konnectivity server v0.0.21e_231_iks v0.0.23_245_iks See the [Konnectivity release notes](https://github.com/kubernetes-sigs/apiserver-network-proxy/releases/tag/v0.0.23). 
 Kubernetes v1.21.4 v1.21.5 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.21.5). 
 Kubernetes API server auditing configuration N/A N/A Updated to support verbose[Kubernetes API server auditing](https://cloud.ibm.com/docs/containers?topic=containers-health-auditaudit-api-server). 
 Kubernetes NodeLocal DNS cache N/A N/A Increased memory resource requests from 5Mi to 8Mi to better align with normal resource utilization. 
 Load balancer and load balancer monitor for IBM Cloud Provider 1510 1547 Updated image for [CVE-2021-3711](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-3711) and [CVE-2021-3712](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-3712).
containers-kubernetes.cluster.operate containers-kubernetes.cluster.update 
 POST/v2/satellite/removeController Remove an IBM Cloud Satellite Location. containers-kubernetes.cluster.create containers-kubernetes.cluster.delete 







 Storage 

Review the following storage API methods, their corresponding actions in IBM Cloud IAM, and the events that are sent to IBM Cloud Activity Tracker for IBM Cloud Kubernetes Service.



Storage API methods, IAM actions, and Activity Tracker events.

 API Method Description IAM action for the API Activity Tracker event 

 GET/v2/storage/getAttachment Get details of a storage attachment. containers-kubernetes.cluster.read containers-kubernetes.containers-kubernetes.storage.attachment.read 
 GET/v2/storage/getAttachments List storage attachments containers-kubernetes.cluster.read containers-kubernetes.containers-kubernetes.storage.attachment.read 
 GET/v2/storage/getVolume Get the details of a storage volume. containers-kubernetes.cluster.read containers-kubernetes.containers-kubernetes.storage.volume.read 
 GET/v2/storage/getVolumes List storage volumes for a cluster or for the account if no cluster is provided. containers-kubernetes.cluster.read containers-kubernetes.containers-kubernetes.storage.volume.read 
 POST/v2/storage/createAttachment Attach a volume to a worker node. containers-kubernetes.cluster.update containers-kubernetes.containers-kubernetes.storage.attachment.create 
 POST/v2/storage/deleteAttachment Detach a volume from a worker node. containers-kubernetes.cluster.update containers-kubernetes.containers-kubernetes.storage.attachment.delete 







 Worker nodes and worker pools 

Review the following worker node and worker pool API methods, their corresponding actions in IBM Cloud IAM, and the events that are sent to IBM Cloud Activity Tracker for IBM Cloud Kubernetes Service.
[Rewrite scripts that call IBM Cloud Kubernetes Service plug-in commands](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cliscript).



* [ibmcloud ks script update](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cliscript_update)







 ibmcloud ks security-group 

[Reset or sync security group settings](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clisecurity_group)



* [ibmcloud ks security-group reset](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clisecurity_group_reset)
* [ibmcloud ks security-group sync](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clisecurity_group_sync)







 ibmcloud ks storage 

Beta[View and modify storage resources](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_storage).



* storage attachment: View and modify storage volume attachments of worker nodes in your cluster.



* [ibmcloud ks storage attachment create](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_storage_att_cr)
* [ibmcloud ks storage attachment get](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_storage_att_get)
* [ibmcloud ks storage attachment ls](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_storage_att_ls)
Kubernetes network policies can also help you enforce workload isolation between namespaces by controlling how pods and services in different namespaces can communicate. For clusters that run Kubernetes 1.21 and later, the service account tokens that pods use to communicate with the Kubernetes API server are time-limited, automatically refreshed, scoped to a particular audience of users (the pod), and invalidated after the pod is deleted. To continue communicating with the API server, you must design your apps to read the refreshed token value on a regular basis, such as every minute. For more information, see [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md).





 Managing access and monitoring app health 

After you deploy your app, you can control who can access the app, and monitor the health and performance of the app.



 How can I control who has access to my app deployments? 

The account and cluster administrators can control access on many different levels: the cluster, Kubernetes namespace, pod, and container.

With IBM Cloud IAM, you can assign permissions to individual users, groups, or service accounts at the cluster-instance level. You can scope cluster access down further by restricting users to particular namespaces within the cluster. For more information, see [Assigning cluster access](https://cloud.ibm.com/docs/containers?topic=containers-usersusers).

To control access at the pod level, you can configure [pod security policies (PSPs)](https://cloud.ibm.com/docs/containers?topic=containers-psp).

Within the app deployment YAML, you can set the security context for a pod or container. For more information, review the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).

Want to control access at the application level?
Every cluster on your Kubernetes cluster on IBM Cloud comes with predefined, default storage class that is used to provision persistent storage on IBM Cloud. When you deploy a blockchain node to that cluster by using the IBM Blockchain Platform console or the APIs, the node uses this default storage class to dynamically provision the amount of storage that you specify on IBM Cloud.

For IBM Cloud Kubernetes service, the default storage class is Gold File Storage backed by [Endurance File Storage](https://cloud.ibm.com/docs/FileStorage). See [Deciding on the File Storage configuration](https://cloud.ibm.com/docs/containers?topic=containers-kube_conceptscustomized_storageclass) for more details. OpenShift clusters default to Gold[Block Storage](https://cloud.ibm.com/docs/openshift?topic=openshift-block_storageblock_predefined_storageclass). If you need to change your default storage class, see [Ordering File Storage with pre-defined IOPS Tiers (Endurance)](https://cloud.ibm.com/docs/containers?topic=containers-file_storagefile-topology) and then follow instructions to [configure a custom storage class](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-v2-deploy-iksibp-console-storage-custom).



 Provision persistent storage 

If you are linking to a Red Hat OpenShift cluster in IBM Cloud, review the topic on [Planning highly available persistent storage](https://cloud.ibm.com/docs/openshift?topic=openshift-storage_planningchoose_storage_solution). If you are linking to an IBM Cloud Kubernetes Service cluster, you can choose from several [Kubernetes storage options](https://cloud.ibm.com/docs/containers?topic=containers-storage_planningchoose_storage_solution) and decide on the storage type that best fits your use case. In both cases, you need to provision persistent storage.
Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"csi-block-pvc-good","namespace":"default"},"spec":{...
volume.beta.kubernetes.io/storage-provisioner: vpc.block.csi.ibm.io
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 10Gi
Access Modes:
VolumeMode: Filesystem
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ExternalProvisioning 9s (x3 over 18s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "vpc.block.csi.ibm.io" or manually created by system administrator
Mounted By: <none>
7. Create a deployment configuration file for your app and mount the PVC to your app.

apiVersion: apps/v1
kind: Deployment
metadata:
name: <deployment_name>
labels:
app: <deployment_label>
spec:
selector:
matchLabels:
app: <app_name>
template:
metadata:
labels:
app: <app_name>
spec:
containers:
- image: <image_name>
name: <container_name>
volumeMounts:
- name: <volume_name>
mountPath: /<file_path>
volumes:
- name: <volume_name>
persistentVolumeClaim:
claimName: <pvc_name>
Show more

labels.app
: In the metadata section, enter a label for the deployment.

matchLabels.app and labels.app
: In the spec selector and template metadata sections, enter a label for your app.
--operating-system SYSTEM
: Optional. The operating system of the worker nodes you want to provision in your cluster. For a list of available operating systems by cluster version, see [IBM Cloud Kubernetes Service version information](https://cloud.ibm.com/docs/containers?topic=containers-cs_versions).

--version MAJOR.MINOR.PATCH
: Optional: The Kubernetes version for the cluster master node. When the version is not specified, the cluster is created with the default of supported Kubernetes versions. To see available versions, run ibmcloud ks versions.

--no-subnet
: By default, a public and a private portable subnet are created on the VLAN associated with the cluster. Include the --no-subnet option to avoid creating subnets with the cluster. You can [create](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_subnet_create) or [add](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_subnet_add) subnets to a cluster later.

--sm-group GROUP
: The secret group ID of the Secrets Manager instance where your secrets are persisted. To get a secret group ID, see the [Secrets Manager CLI reference](https://cloud.ibm.com/docs/secrets-manager-cli-plugin?topic=secrets-manager-cli-plugin-secrets-manager-clisecrets-manager-secret-groups-cli).

--sm-instance INSTANCE
: The CRN of the Secrets Manager instance. To get the CRN of an instance, run [ibmcloud ks ingress instance ls --cluster CLUSTER](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_ingress_instance_ls).

--private-vlan PRIVATE_VLAN