vania-janet/multiturn-rag-retrieval-data huggingface dataset:
By defining topologykey: failure-domain.beta.kubernetes.io/zone, the Kubernetes scheduler can't schedule a pod on a worker node if the worker node is in the same zone as a pod that has the app: nginx label. For each stateful set pod, two PVCs are created as defined in the volumeClaimTemplates section, but the creation of the block storage instances is delayed until a stateful set pod that uses the storage is scheduled. This setup is referred to as [topology-aware volume scheduling](https://kubernetes.io/blog/2018/10/11/topology-aware-volume-provisioning-in-kubernetes/).

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ibmc-block-bronze-delayed
parameters:
billingType: hourly
classVersion: "2"
fsType: ext4
iopsPerGB: "2"
sizeRange: '[20-12000]Gi'
type: Endurance
provisioner: ibm.io/ibmc-block
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx"
replicas: 3
podManagementPolicy: "Parallel"
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
containers-kubernetes.cluster-audit-webhook.delete A cluster audit webhook for forwarding master API server audit logs is removed. 
 containers-kubernetes.cluster-audit-webhook.get Audit webhooks for a cluster are listed. 
 containers-kubernetes.cluster-audit-webhook.update A cluster audit webhook for forwarding master API server audit logs is updated. 
 containers-kubernetes.cluster-ca-certificate.create A certificate authority (CA) for a cluster is created. 
 containers-kubernetes.cluster-ca-certificate.get A certificate authority (CA) for a cluster is retrieved. 
 containers-kubernetes.cluster-ca-certificate.rotate The certificate authority (CA) for a cluster is rotated. 
 containers-kubernetes.image-security.enable Image security is enabled. 
 containers-kubernetes.image-security.disable Image security is disabled. 
 containers-kubernetes.cluster-key-state.update The root key in the key management service (KMS) provider that the cluster uses is updated, such as a root key being enabled, disabled, or rotated in a key management service (KMS) provider. 
 containers-kubernetes.cluster-kms.enable A key management service (KMS) provider is enabled for a cluster. 
 containers-kubernetes.cluster-master.refresh A cluster master refresh is requested. 
 containers-kubernetes.cluster-master.update A cluster master update is requested. 
 containers-kubernetes.cluster-private-service-endpoint.disable The private cloud service endpoint for a cluster is disabled. 
 containers-kubernetes.cluster-private-service-endpoint.enable The private cloud service endpoint for a cluster is enabled. 
 containers-kubernetes.cluster-public-service-endpoint.disable The public cloud service endpoint for a cluster is disabled. 
 containers-kubernetes.cluster-public-service-endpoint.enable The public cloud service endpoint for a cluster is enabled.
Kubernetes v1.9.3 v1.9.7 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.9.7) This release addresses [CVE-2017-1002101](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-1002101) and [CVE-2017-1002102](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-1002102) vulnerabilities. <br>Note: Now secret, configMap, downwardAPI, and projected volumes are mounted as read-only. Previously, apps could write data to these volumes, but the system could automatically revert the data. If your apps rely on the previous insecure behavior, modify them accordingly. 
 Kubernetes configuration N/A N/A Added admissionregistration.k8s.io/v1alpha1=true to the --runtime-config option for the cluster's Kubernetes API server. 
 IBM Cloud Provider v1.9.3-71 v1.9.7-102 NodePort and LoadBalancer services now support [preserving the client source IP](https://cloud.ibm.com/docs/containers?topic=containers-loadbalancerlb_source_ip) by setting service.spec.externalTrafficPolicy to Local. 
 General N/A N/A Fix [edge node](https://cloud.ibm.com/docs/containers?topic=containers-edgeedge) toleration setup for older clusters.
Resilient and secure multi-region Kubernetes clusters with IBM Cloud Internet Services 

This tutorial may incur costs. Use the [Cost Estimator](https://cloud.ibm.com/estimator/review) to generate a cost estimate based on your projected usage.

Users are less likely to experience downtime when an application is designed with resiliency in mind. When implementing a solution with Kubernetes Service, you benefit from built-in capabilities, like load balancing and isolation, increased resiliency against potential failures with hosts, networks, or apps. By creating multiple clusters and if an outage occurs with one cluster, users can still access an app that is also deployed in another cluster. With multiple clusters in different locations, users can also access the closest cluster and reduce network latency. For additional resiliency, you have the option to also select the multi-zone clusters, meaning your nodes are deployed across multiple zones within a location.

This tutorial highlights how Cloud Internet Services (CIS), a uniform platform to configure and manage the Domain Name System (DNS), Global Load Balancing (GLB), Web Application Firewall (WAF), and protection against Distributed Denial of Service (DDoS) for internet applications, can be integrated with Kubernetes clusters to support this scenario and to deliver a secure and resilient solution across multiple locations.



 Objectives 



* Deploy an application on multiple Kubernetes clusters in different locations.
* Distribute traffic across multiple clusters with a Global Load Balancer.
* Route users to the closest cluster.
* Protect your application from security threats.
* Increase application performance with caching.



Zoom

![Architecture](https://cloud.ibm.com/docs-content/v1/content/f84e2238288b7f0355715a4c355609ce7ac48fb0/solution-tutorials/images/solution32-multi-region-k8s-cis/Architecture.svg)

Figure 1. Architecture diagram of the tutorial



1. The developer builds a Docker image for the application.
2. The image is pushed to a Container Registry.
3.
containers-kubernetes.cluster-lb-hostname.delete The VPC load balancer hostname is removed from the DNS record for an existing NLB subdomain. 
 containers-kubernetes.cluster-lb-hostname.update The DNS record for an NLB subdomain in a VPC cluster is updated by replacing the load balancer hostname. 
 containers-kubernetes.cluster-nlb-dns.update A DNS record in a classic cluster is updated by adding an NLB IP address. 
 containers-kubernetes.cluster-nlb-dns-monitor.update The health check monitor for an NLB subdomain is enabled or disabled. 







 Private service endpoint allowlist events 

The following table lists the actions related to access control lists (ACLs) and the generation of events for clusters that use a private cloud service endpoint allowlist.



ACL events

 Action Description 

 containers-kubernetes.network.acl.delete The private cloud service endpoint allowlist feature for a cluster is disabled. 
 containers-kubernetes.network.acl.get The subnet allowlist for the private cloud service endpoint of a cluster is requested. 
 containers-kubernetes.network.acl.update The private cloud service endpoint allowlist feature for a cluster is enabled, subnets are added to the allowlist, or subnets are removed from the allowlist. 







 Satellite events 

See the [IBM Cloud Satellite documentation](https://cloud.ibm.com/docs/satellite?topic=satellite-at_events).





 Storage events 

The following table lists the actions related to storage resources and the generation of events.



Storage events

 Action Description 

 containers-kubernetes.storage-volume.delete A volume is deleted. 
 containers-kubernetes.storage-volume.list Volumes in the IBM Cloud account or as filtered by provider are retrieved. 
 containers-kubernetes.storage-volume.read A volume is retrieved 
 containers-kubernetes.storage-volume.update A volume is updated. 
 containers-kubernetes.storage-attachment.create A volume attachment is created.
ibmcloud is load-balancers

In the following example CLI output, the VPC NLB that is named kube-bh077ne10vqpekt0domg-046e0f754d624dca8b287a033d55f96e is created for the Kubernetes LoadBalancer service:

ID Name Created Host Name Is Public Listeners Operating Status Pools Private IPs Provision Status Public IPs Subnets Resource Group
06496f64-a689-4693-ba23-320959b7b677 kube-bh077ne10vqpekt0domg-046e0f754d624dca8b287a033d55f96e 8 minutes ago 1234abcd-us-south.lb.appdomain.cloud yes 95482dcf-6b9b-4c6a-be54-04d3c46cf017 online 717f2122-5431-403c-b21d-630a12fc3a5a 10.241.0.7 active 169.63.99.184 c6540331-1c1c-40f4-9c35-aa42a98fe0d9 00809211b934565df546a95f86160f62
6. Access the IP address of the Kubernetes LoadBalancer service that you found in step 4 and your app port in the format <external_IP>:<app_port>.
7. Optional: Repeat these steps to deploy a public VPC NLB in each zone where you want to expose your app. Then, you can register the external IP addresses of the VPC NLB in each zone with one [DNS subdomain](https://cloud.ibm.com/docs/containers?topic=containers-vpc-lbaasvpc_nlb_dns).



Do not delete the subnets that you attached to your cluster during cluster creation or when you add worker nodes in a zone.
* UBI Linux: The Fabric Docker images use [Red Hat UBI images](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/building_running_and_managing_containers/indexhow_are_ubi_images_different/), which is a smaller, lighter, and more secure version of Linux.







 Keys and cluster access information 



Table 3. Types of keys and information used to access the customer Kubernetes cluster

 Type Description Storage Access 

 Customer Kubernetes cluster access information In order for the service components to access and manage the components that are deployed into this cluster, the service components generate a Kubernetes secret for cluster access. Each Kubernetes secret is tied to that specific customer cluster. After generation, these secrets never leave the IBM Kubernetes cluster where the service components are running and are deleted whenever a customer deletes the associated cluster. Accessed only by the service component code and only in response to customer requests to deploy or manage components via the blockchain console. Developers of the service do not have access to the information in the secrets. 







 Membership Service Providers (MSPs) 

Organizations in a blockchain network are represented by [MSP](https://cloud.ibm.com/docs/blockchain?topic=blockchain-glossaryglossary-msp) definitions. You can use the blockchain console to add a new organization to the network by creating a new MSP definition in the Organizations tab. If you are the admin of an ordering service, you can use the console to add that organization MSP to a consortium using the Ordering service tab. Finally, if you are an administrator of a channel, you can add that organization to an existing channel so the organization can transact on the network using the Channels tab (this task might require the signature of other organizations). See the topic on [Join the consortium hosted by the ordering service](https://cloud.ibm.com/docs/blockchain?topic=blockchain-ibp-console-build-networkibp-console-build-network-add-org) for detailed steps.





 Storage
container-service[kubernetes-service] 1.0.372 false
cra 0.1.11 false
doi 0.3.3 false
schematics 1.7.2 false
cloud-functions[wsk/functions/fn] 1.0.58 false

 ibmcloud dev --version
ibmcloud dev version 2.10.1

 java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
Eclipse OpenJ9 VM AdoptOpenJDK-11.0.11+9 (build openj9-0.26.0, JRE 11 Linux amd64-64-Bit Compressed References 20210421_975 (JIT enabled, AOT enabled)
OpenJ9 - b4cc246d9
OMR - 162e6f729
JCL - 7796c80419 based on jdk-11.0.11+9)

 ant -version
Apache Ant(TM) version 1.10.12 compiled on October 13 2021

 mvn -version
Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
Maven home: /opt/IBM/maven
Java version: 11.0.11, vendor: AdoptOpenJDK, runtime: /usr/local/openjdk-11
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.18-200.fc35.x86_64", arch: "amd64", family: "unix"

 gradle -version

Welcome to Gradle 7.4!

Here are the highlights of this release:
- Aggregated test and JaCoCo reports
- Marking additional test source directories as tests in IntelliJ
- Support for Adoptium JDKs in Java toolchains
Resilient and secure multi-region Kubernetes clusters with IBM Cloud Internet Services 

This tutorial may incur costs. Use the [Cost Estimator](https://cloud.ibm.com/estimator/review) to generate a cost estimate based on your projected usage.

Users are less likely to experience downtime when an application is designed with resiliency in mind. When implementing a solution with Kubernetes Service, you benefit from built-in capabilities, like load balancing and isolation, increased resiliency against potential failures with hosts, networks, or apps. By creating multiple clusters and if an outage occurs with one cluster, users can still access an app that is also deployed in another cluster. With multiple clusters in different locations, users can also access the closest cluster and reduce network latency. For additional resiliency, you have the option to also select the multi-zone clusters, meaning your nodes are deployed across multiple zones within a location.

This tutorial highlights how Cloud Internet Services (CIS), a uniform platform to configure and manage the Domain Name System (DNS), Global Load Balancing (GLB), Web Application Firewall (WAF), and protection against Distributed Denial of Service (DDoS) for internet applications, can be integrated with Kubernetes clusters to support this scenario and to deliver a secure and resilient solution across multiple locations.



 Objectives 



* Deploy an application on multiple Kubernetes clusters in different locations.
* Distribute traffic across multiple clusters with a Global Load Balancer.
* Route users to the closest cluster.
* Protect your application from security threats.
* Increase application performance with caching.



Zoom

![Architecture](https://cloud.ibm.com/docs-content/v1/content/cb1eb27836421578019401fa7556779109430b29/cis/includes/solution-tutorials/includes/solution-tutorials/images/solution32-multi-region-k8s-cis/Architecture.svg)

Figure 1. Architecture diagram of the tutorial



1. The developer builds a Docker image for the application.
2.
Key Management Service provider v1.0.0 v2.0.2 Updated image for [CVE-2020-15586](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15586). 
 Kubernetes v1.18.6 v1.18.8 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.18.8). 
 Kubernetes add-on resizer 1.8.7 1.8.11 See the [Kubernetes add-on resizer release notes](https://github.com/kubernetes/autoscaler/releases/tag/addon-resizer-1.8.11). 
 Kubernetes configuration N/A N/A The Kubernetes API server audit policy configuration is updated to include auditing of all API groups except apiregistration.k8s.io and coordination.k8s.io. 
 Kubernetes Metrics Server v0.3.6 v0.3.7 See the [Kubernetes Metrics Server release notes](https://github.com/kubernetes-sigs/metrics-server/releases/tag/v0.3.7). 
 Kubernetes NodeLocal DNS cache configuration N/A N/A Increased the pod termination grace period. 







 Change log for worker node fix pack 1.18.8_1525, released 17 August 2020 

The following table shows the changes that are in the worker node fix pack 1.18.8_1525. Worker node patch updates can be applied by updating, reloading (in classic infrastructure), or replacing (in VPC infrastructure) the worker node.



Changes since version 1.18.6_1523

 Component Previous Current Description
Make the application accessible within the cluster by creating a service:

kubectl expose deployment/hello-world-deployment --type=ClusterIP --port=80 --name=hello-world-service --target-port=3000

It returns message like service "hello-world-service" exposed. To see the services:

kubectl get services
4. Run the application in the cluster with two replicas:

kubectl scale deployment hello-world-deployment --replicas=2
5. You can check the status of the deployment with the following command:

kubectl get pods







 Get the Ingress Subdomain assigned to the cluster 

When a Kubernetes cluster is created, it gets assigned an Ingress subdomain (for example, my-us-cluster.us-south.containers.appdomain.cloud) and a public Application Load Balancer IP address.



1. Retrieve the Ingress subdomain of the cluster:

ibmcloud ks cluster get --cluster $MYCLUSTER

Look for the Ingress Subdomain value.
2. Make note of this information for a later step.



This tutorial uses the Ingress Subdomain to configure the Global Load Balancer. You could also replace the Ingress Subdomain with the public Application Load Balancer, ALB of the cluster. An <IngressSubdomain> looks something like my-us-cluster-e7f2ca73139645ddf61a8702003a483a-0000.us-south.containers.appdomain.cloud





 Configure the Ingress for your DNS subdomain 

It will be required to have your own DNS domain name and a global load balancer subdomain will be created below: <glb_name>.<your_domain_name>. Something like hello-world-service.example.com <glb_name> = hello-world-service and <your_domain_name> = example.com



1. Create the file glb-ingress.yaml and replace the placeholders with their respective values:
: [Satellite Config](https://cloud.ibm.com/docs/satellite?topic=satellite-setup-clusters-satconfig) is a collection of configurations, versions, and subscriptions that you use to automatically deploy Kubernetes resources to groups of clusters that are registered with the Satellite Config component. However, access to Satellite Config does not give a user access to the clusters that run the Kubernetes resources of the configuration. You can scope access to the following Satellite Config resources.



* Configurations, where you upload the version of the configuration file for the Kubernetes resources that you want to deploy. You cannot scope a policy to a particular configuration.
* Subscriptions, which you use to use to specify the cluster group where you want to deploy the Kubernetes resource definition that you added as a version to your configuration. You cannot scope a policy to a particular configuration.
* Clusters or cluster groups, which are Red Hat OpenShift on IBM Cloud that are registered with Satellite Config and can be subscribed to configurations.
* Resources, which are Kubernetes resources such as pods or services that are described in a Satellite Config and run in a subscribed cluster. Certain roles permit access to view and manage Kubernetes resource through Satellite Config, but you cannot scope an access policy to a particular resource.







 Link 

Review details about the Satellite Link IAM resource type, which includes actions for endpoints and sources.

Name of the resource type
: UI: Link
: API or CLI: link

Type of role that you can assign for the resource in IAM
: Platform access Viewer, Operator, Editor, and Administrator roles
: Custom Satellite Link Administrator and Satellite Link Source Access Controller service access roles

What you can scope an access policy for the resource to
: Account
: Resource group
: Particular instances of the resource

Description
: [Link endpoints](https://cloud.ibm.com/docs/satellite?topic=satellite-link-location-cloud) connect services, servers, or apps that run in your Satellite location with an endpoint that runs in IBM Cloud.
Resilient and secure multi-region Kubernetes clusters with IBM Cloud Internet Services 

This tutorial may incur costs. Use the [Cost Estimator](https://cloud.ibm.com/estimator/review) to generate a cost estimate based on your projected usage.

Users are less likely to experience downtime when an application is designed with resiliency in mind. When implementing a solution with Kubernetes Service, you benefit from built-in capabilities, like load balancing and isolation, increased resiliency against potential failures with hosts, networks, or apps. By creating multiple clusters and if an outage occurs with one cluster, users can still access an app that is also deployed in another cluster. With multiple clusters in different locations, users can also access the closest cluster and reduce network latency. For additional resiliency, you have the option to also select the multi-zone clusters, meaning your nodes are deployed across multiple zones within a location.

This tutorial highlights how Cloud Internet Services (CIS), a uniform platform to configure and manage the Domain Name System (DNS), Global Load Balancing (GLB), Web Application Firewall (WAF), and protection against Distributed Denial of Service (DDoS) for internet applications, can be integrated with Kubernetes clusters to support this scenario and to deliver a secure and resilient solution across multiple locations.



 Objectives 



* Deploy an application on multiple Kubernetes clusters in different locations.
* Distribute traffic across multiple clusters with a Global Load Balancer.
* Route users to the closest cluster.
* Protect your application from security threats.
* Increase application performance with caching.



Zoom

![Architecture](https://cloud.ibm.com/docs-content/v1/content/cbb80bf851fb762a838129cba09d5674f8bfffda/Registry/includes/solution-tutorials/includes/solution-tutorials/images/solution32-multi-region-k8s-cis/Architecture.svg)

Figure 1. Architecture diagram of the tutorial



1. The developer builds a Docker image for the application.
2.
You can define [Kubernetes resource requests and resource limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for your containers or pods to limit the amount of CPU and memory that they can consume. If no limits for CPU and memory are set, and the container is busy, the container uses all the resources that are available. This high consumption of resources might affect other containers on the worker node that don't have enough resources to properly start or run, and puts your worker node at risk for denial-of-service attacks.

Enforce policy-driven authentication
: You can add an Ingress annotation to your deployments to control access to your services and APIs. By using App ID and declarative security, you can ensure user authentication and token validation.







 Storing personal information 

You are responsible for ensuring the security of your personal information in Kubernetes resources and container images. Personal information includes your name, address, phone number, email address, or other information that might identify, contact, or locate you, your customers, or anyone else.

Use a Kubernetes secret to store personal information
: Store personal information only in Kubernetes resources that are designed to hold personal information. For example, don't use your name in the name of a Kubernetes namespace, deployment, service, or config map. For proper protection and encryption, store personal information in [secrets](https://kubernetes.io/docs/concepts/configuration/secret/) instead.
: For centralized management of all your secrets across clusters and injection at application runtime, try [IBM Cloud Secrets Manager](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-tutorial-kubernetes-secrets).

Use a Kubernetes imagePullSecret to store image registry credentials
: Do not store personal information in container images or registry namespaces.