vania-janet/multiturn-rag-retrieval-data huggingface dataset:
This setup is referred to as [topology-aware volume scheduling](https://kubernetes.io/blog/2018/10/11/topology-aware-volume-provisioning-in-kubernetes/).

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ibmc-file-bronze-delayed
parameters:
billingType: hourly
classVersion: "2"
iopsPerGB: "2"
sizeRange: '[20-12000]Gi'
type: Endurance
provisioner: ibm.io/ibmc-file
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
serviceName: "nginx"
replicas: 3
podManagementPolicy: "Parallel"
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx
topologyKey: failure-domain.beta.kubernetes.io/zone
containers:
- name: nginx
image: k8s.gcr.io/nginx-slim:0.8
ports:
- containerPort: 80
name: web
volumeMounts:
For more information, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/run-application/configure-pdb/).

apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
name: wasliberty
spec:
maxUnavailable: 1
selector:
matchLabels:
app: wasliberty





 Exposing the app service 

You can create a service that exposes your app. In the spec section, make sure to match the port and label values with the ones that you used in the deployment. The service exposes objects that match the label, such as app: wasliberty in the following example.



* By default, a service uses [ClusterIP](https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-intro/), which makes the service accessible only within the cluster but not outside the cluster.
* You can create a NodePort, load balancer, or Ingress service to expose the app publicly. These services have two IPs, one external and one internal. When traffic is received on the external IP, it is forwarded to the internal cluster IP. Then, from the internal cluster IP, the traffic is routed to the container IP of the app.
* The example uses NodePort to expose the service outside the cluster. For more information about how to set up external access, see [Choosing a NodePort, load balancer, or Ingress service](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planningexternal).



apiVersion: v1
kind: Service
metadata:
name: wasliberty
labels:
app: wasliberty
spec:
ports:
- port: 9080
selector:
app: wasliberty
type: NodePort

If you have a requirement to deploy hostNetwork pods to listen on specific ports or to use a hostPort to expose your app pods on a specific port on the worker node, use a port in the 11000-11200 range.
Get the Kubernetes LoadBalancer service definition and save the output as a yaml file called my-lb.yaml.

kubectl describe service my-load-balancer -o yaml
5. Delete the Kubernetes LoadBalancer service. This also deletes the corresponding VPC load balancer.

kubectl delete service my-load-balancer
6. Update the Kubernetes LoadBalancer service definition file with the subnet or zone changes you want to implement. Do not change the name of the LoadBalancer service. For details on specifying subnets or zones for network load balancers, see [Setting up a Network Load Balancer for VPC](https://cloud.ibm.com/docs/containers?topic=containers-vpc-lbaassetup_vpc_nlb). For details on specifying subnets or zones for application load balancers, see [Setting up an Application Load Balancer for VPC](https://cloud.ibm.com/docs/containers?topic=containers-vpc-lbaassetup_vpc_ks_vpc_lb).
7. Apply the new LoadBalancer definition file.

kubectl apply -f my-lb.yaml
8. Verify that the Kubernetes LoadBalancer service is recreated successfully in your cluster. When the service is created, the LoadBalancer Ingress field is populated with either an external IP address for NLBs or a hostname for ALBs. Note that it takes a few minutes to provision the load balancer and you may see a pending status until it is fully provisioned.

kubectl describe service my-load-balancer

Example output.

NAME: my-load-balancer
Namespace: default
Labels: <none>
Annotations: service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: nlb
Selector: app=echo-server
Type: LoadBalancer
IP: 172.X.XXX.XX
When you create a Kubernetes LoadBalancer service in your cluster, a load balancer for VPC is automatically created in your VPC outside of your cluster. The load balancer is multizonal and routes requests for your app through the private NodePorts that are automatically opened on your worker nodes. The following diagram illustrates how a user accesses an app's service through the load balancer, even though your worker node is connected to only a private subnet.

Zoom

![VPC load balancing for a cluster.](https://cloud.ibm.com/docs-content/v1/content/4d497aff9e27b92fbbc2ee4e343a9ec4549cb1d7/containers//images/vpc-alb-mz.svg)

Figure 1. VPC load balancing for a cluster



1. Create a Kubernetes LoadBalancer service in your cluster to publicly expose the hello world app.

kubectl expose deployment/hello-world-deployment --type=LoadBalancer --name=hw-lb-svc --port=8080 --target-port=8080

Example output

service "hw-lb-svc" exposed



Table 2. Information about the command options.

 Parameter Description 

 expose Expose a Kubernetes resource, such as a deployment, as a Kubernetes service so that users can access the resource by using the IP address of the service. 
 deployment/<hello-world-deployment> The resource type and the name of the resource to expose with this service. 
 --name=<hello-world-service> The name of the service. 
 --type=LoadBalancer The Kubernetes service type to create. In this lesson, you create a LoadBalancer service. 
 --port=<8080> The port on which the service listens for external network traffic. 
 --target-port=<8080> The port that your app listens on and to which the service directs incoming network traffic. In this example, the target-port is the same as the port, but other apps that you create might use a different port.
Kubernetes configuration N/A N/A The Kubernetes API server audit policy configuration is updated to include logging metadata for cluster-admin requests and logging the request body of workload create, update, and patch requests. 
 Kubernetes DNS autoscaler 1.2.0 1.3.0 See the [Kubernetes DNS autoscaler release notes](https://github.com/kubernetes-sigs/cluster-proportional-autoscaler/releases/tag/1.3.0). 
 OpenVPN client 2.4.6-r3-IKS-8 2.4.6-r3-IKS-13 Updated image for [CVE-2018-0734](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-0734) and [CVE-2018-5407](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-5407). Added CriticalAddonsOnly toleration to the vpn deployment in the kube-system namespace. Additionally, the pod configuration is now obtained from a secret instead of from a ConfigMap. 
 OpenVPN server 2.4.6-r3-IKS-8 2.4.6-r3-IKS-13 Updated image for [CVE-2018-0734](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-0734) and [CVE-2018-5407](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-5407). 
 systemd 230 229 Security patch for [CVE-2018-16864](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-16864).
Annotations: oc.kubernetes.io/last-applied-configuration={"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"name":"nginx","namespace":"default"},"spec":{"podManagementPolicy":"Par..."
Replicas: 3 desired | 3 total
Pods Status: 0 Running / 3 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=nginx
billingType=hourly
region=us-south
zone=dal10
...

A stateful set is fully deployed when the number of replicas that you find in the Replicas section of your CLI output equals the number of Running pods in the Pods Status section. If a stateful set is not fully deployed yet, wait until the deployment is finished before you proceed.
3. Create a configuration file for your stateful set and the service that you use to expose the stateful set. The following example shows how to deploy NGINX as a stateful set with three replicas. For each replica, a 20 gigabyte block storage device is provisioned based on the specifications that are defined in the ibmc-block-retain-bronze storage class. All storage devices are provisioned in the dal10 zone. Because block storage can't be accessed from other zones, all replicas of the stateful set are also deployed onto worker nodes that are located in dal10.

apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nginx
spec:
serviceName: "nginx"
replicas: 3
podManagementPolicy: Parallel
selector:
matchLabels:
app: nginx
Fixed VPC load balancer error message to use the current IBM Cloud Kubernetes Service plug-in CLI syntax. 
 Kubernetes Both v1.15.10 v1.15.11 See the [Kubernetes release notes](https://github.com/kubernetes/kubernetes/releases/tag/v1.15.11). 
 Ubuntu 18.04 packages Worker N/A N/A Updated worker node images with package updates for [CVE-2019-13734](https://nvd.nist.gov/vuln/detail/CVE-2019-13734), [CVE-2019-13750](https://nvd.nist.gov/vuln/detail/CVE-2019-13750), [CVE-2019-13751](https://nvd.nist.gov/vuln/detail/CVE-2019-13751), [CVE-2019-13752](https://nvd.nist.gov/vuln/detail/CVE-2019-13752), [CVE-2019-13753](https://nvd.nist.gov/vuln/detail/CVE-2019-13753), [CVE-2019-19923](https://nvd.nist.gov/vuln/detail/CVE-2019-19923), [CVE-2019-19924](https://nvd.nist.gov/vuln/detail/CVE-2019-19924), [CVE-2019-19925](https://nvd.nist.gov/vuln/detail/CVE-2019-19925), [CVE-2019-19926](https://nvd.nist.gov/vuln/detail/CVE-2019-19926), [CVE-2019-19959](https://nvd.nist.gov/vuln/detail/CVE-2019-19959), and [CVE-2019-20218](https://nvd.nist.gov/vuln/detail/CVE-2019-20218).
container-service[kubernetes-service/ks] 1.0.489 false
cra 1.1.2 false
doi 0.4.1 false
schematics[sch] 1.12.7 true
secrets-manager 0.1.25 true

 ibmcloud dev --version
ibmcloud dev version 2.17.0

 java -version
openjdk version "17.0.5" 2022-10-18
IBM Semeru Runtime Open Edition 17.0.5.0 (build 17.0.5+8)
Eclipse OpenJ9 VM 17.0.5.0 (build openj9-0.35.0, JRE 17 Linux amd64-64-Bit Compressed References 20221018_325 (JIT enabled, AOT enabled)
OpenJ9 - e04a7f6c1
OMR - 85a21674f
JCL - 32d2c409a33 based on jdk-17.0.5+8)

 ant -version
Apache Ant(TM) version 1.10.13 compiled on January 4 2023

 mvn -version
Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
Maven home: /opt/IBM/maven
Java version: 17.0.5, vendor: IBM Corporation, runtime: /usr/local/jdk17
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.49-linuxkit", arch: "amd64", family: "unix"

 gradle -version

Welcome to Gradle 7.6!

Here are the highlights of this release:
- Added support for Java 19.
- Introduced --rerun flag for individual task rerun.
- Improved dependency block for test suites to be strongly typed.
- Added a pluggable system for Java toolchains provisioning.
* [ibmcloud ks cluster master pod-security policy get](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clicluster-master-pod-security-policy-get)
* [ibmcloud ks cluster master pod-security set](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clicluster-master-pod-security-set)
* [ibmcloud ks cluster master pod-security unset](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clicluster-master-pod-security-unset)
* [ibmcloud ks cluster master private-service-endpoint allowlist](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_master_pse_allowlist)
* [ibmcloud ks cluster master private-service-endpoint disable](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_master_pse_disable)
* [ibmcloud ks cluster master private-service-endpoint enable](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_master_pse_enable)
* [ibmcloud ks cluster master public-service-endpoint disable](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_master_pub_se_disable)
* [ibmcloud ks cluster master public-service-endpoint enable](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_cluster_master_pub_se_enable)
* [ibmcloud ks cluster master refresh](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_apiserver_refresh)
* [Code Engine API](https://cloud.ibm.com/apidocs/codeengine)
* [Kubernetes REST API](https://kubernetes.io/docs/reference/using-api/api-overview/)
* [Kubernetes API concepts](https://kubernetes.io/docs/reference/using-api/api-concepts/)
* [API client libraries](https://kubernetes.io/docs/reference/api-client-libraries)
* [kubectl command](https://kubernetes.io/docs/reference/cli-reference)







 Custom resource definition (CRD) 

The following sections list the custom resource definition methods to use with Code Engine.



 Batch CRD methods 



Batch CRDs for Code Engine

 Group Version Kind 

 codeengine.cloud.ibm.com v1beta1 JobDefinition 
 codeengine.cloud.ibm.com v1beta1 JobRun 



After you retrieve the Kubernetes configuration, you can view Batch CRD details by using the following methods.



1. Use kubectl explain --api-version='codeengine.cloud.ibm.com/v1beta1' <Kind>.
2. [Download Swagger or OpenAPI specification of CRDs](https://kubernetes.io/docs/concepts/overview/kubernetes-api/).



Note that you cannot delete a job run without also deleting any associated pods. Any attempt to delete with the propagationPolicy=Orphan option is rejected.





 Serving CRD methods 



Serving CRDs for Code Engine

 Group Version Kind 

 serving.knative.dev v1 Configuration 
 serving.knative.dev v1 Revision 
 serving.knative.dev v1 Route 
 serving.knative.dev v1 Service
volume.beta.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs
spec:
accessModes: [ "ReadOnlyMany" ]
storageClassName: "ibmc-s3fs-standard-perf-cross-region"
resources:
requests:
storage: 1Gi
Show more



name
: Enter a name for your stateful set. The name that you enter is used to create the name for your PVC in the format: <volume_name>-< statefulset_name>-<replica_number>.

serviceName
: Enter the name of the service that you want to use to expose your stateful set.

replicas
: Enter the number of replicas for your stateful set.

matchLabels
: In the spec selector match labels section, enter all labels that you want to include in your stateful set and your PVC. Labels that you include in the volumeClaimTemplates of your stateful set are not recognized by Kubernetes. Instead, you must define these labels in the spec.selector.matchLabels and spec.template.metadata.labels section of your stateful set YAML. To make sure that all your stateful set replicas are into the load balancing of your service, include the same label that you used in the spec.selector section of your service YAML.

labels
: In the spec metadata labels section, enter the same labels that you added to the spec.selector.matchLabels section of your stateful set YAML.

terminationGracePeriodSeconds
: Enter the number of seconds to give the kubelet to safely terminate the pod that runs your stateful set replica. For more information, see [Delete Pods](https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/delete-pods).

VolumeClaimTemplates.name
: In the spec volume claim templates metadata section, enter a name for your volume. Use the same name that you defined in the spec.containers.volumeMount.name section.
Kubernetes Network Policies 

You can generate Kubernetes Network Policies (KNPs) using the IBM Cloud® Security and Compliance Center Workload Protection network security policy tool and check a topology map for defined policies and make updates.

There are four steps to generate KNPs using the IBM Cloud Security and Compliance Center Workload Protection network security policy tool:



* [Setting the scope.](https://cloud.ibm.com/docs/workload-protection?topic=workload-protection-netsec_policyknp_scope)
* [Reviewing ingress and egress and customizing communications as required.](https://cloud.ibm.com/docs/workload-protection?topic=workload-protection-netsec_policyknp_coms)
* [Reviewing the topology map.](https://cloud.ibm.com/docs/workload-protection?topic=workload-protection-netsec_policyknp_map)
* [Generating the policy and downloading the policy file.](https://cloud.ibm.com/docs/workload-protection?topic=workload-protection-netsec_policyknp_file)





 Setting the scope 

The scope selects the Kubernetes entity and timeframe for aggregate communications.

Communications are aggregated using Kubernetes metadata. This avoids including entities that are not relevant to the policy. For example, if pod A under deployment A communicates multiple times with pod B under deployment B, only one entry will appear in the UI. Or, if pod A1 and pod A2 are both under deployment A, and both communicate with pod B, deployment A will represent all its pods.

To set the scope, do the following:



1. Open the [Workload Protection UI](https://cloud.ibm.com/docs/workload-protection?topic=workload-protection-launch).
2. Click the Network icon !
Kubernetes secrets are encrypted by default which makes secrets a safer and a more flexible option to store sensitive data than to store this data verbatim in a pod definition or in a container image.

One way of using secrets in Kubernetes is by doing something like this:



1. Create a file called cloud-secrets.txt and store the service credentials of any cloud service inside it.

{
"url": "<SERVICE_URL>",
"api_key": <API_Key>
}
2. Then, create a Kubernetes secret by running a command below and verify that the secret is created by using kubectl get secrets after running the command below:

kubectl create secret generic cloud-service-secret --from-file=cloud-secrets.txt=./cloud-secrets.txt









 Containerize your app 

To containerize your app, you must create a container image.

An image is created from a [Dockerfile](https://docs.docker.com/engine/reference/builder/usage), which is a file that contains instructions and commands to build the image. A Dockerfile might reference build artifacts in its instructions that are stored separately, such as an app, the app's configuration, and its dependencies.

To build your own Dockerfile for your existing app, you might use the following commands:



* FROM - Choose a parent image to define the container runtime.
* ADD/COPY - Copy the content of a directory into the container.
* WORKDIR - Set the working directory inside the container.
* RUN - Install software packages that the apps needs during runtime.
* EXPOSE - Make a port available outside the container.
* ENV NAME - Define environment variables.
* CMD - Define commands that run when the container launches.



Images are typically stored in a registry that can either be accessible by the public (public registry) or set up with limited access for a small group of users (private registry). Public registries, such as Docker Hub, can be used to get started with Docker and Kubernetes to create your first containerized app in a cluster.
container-service[kubernetes-service] 1.0.353 false
cra 0.1.11 false
doi 0.3.3 false
schematics 1.7.0 false
cloud-functions[wsk/functions/fn] 1.0.58 false

 ibmcloud dev --version
ibmcloud dev version 2.10.0

 java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
Eclipse OpenJ9 VM AdoptOpenJDK-11.0.11+9 (build openj9-0.26.0, JRE 11 Linux amd64-64-Bit Compressed References 20210421_975 (JIT enabled, AOT enabled)
OpenJ9 - b4cc246d9
OMR - 162e6f729
JCL - 7796c80419 based on jdk-11.0.11+9)

 ant -version
Apache Ant(TM) version 1.10.12 compiled on October 13 2021

 mvn -version
Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
Maven home: /opt/IBM/maven
Java version: 11.0.11, vendor: AdoptOpenJDK, runtime: /usr/local/openjdk-11
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.10.76-linuxkit", arch: "amd64", family: "unix"

 gradle -version

Welcome to Gradle 7.3.3!

Here are the highlights of this release:
- Easily declare new test suites in Java projects
- Support for Java 17
- Support for Scala 3
* [16 May 2019](https://cloud.ibm.com/docs/containers?topic=containers-containers-relnotescontainers-may1619)



* CLI reference
* Unsupported: Kubernetes version 1.10



* [15 May 2019](https://cloud.ibm.com/docs/containers?topic=containers-containers-relnotescontainers-may1519)



* Default Kubernetes version
* Service limits



* [13 May 2019](https://cloud.ibm.com/docs/containers?topic=containers-containers-relnotescontainers-may1319)



* Version change logs
* Worker node flavors



* [8 May 2019](https://cloud.ibm.com/docs/containers?topic=containers-containers-relnotescontainers-may0819)



* API
* Cloud Object Storage
* Kubernetes strategy
* Kubernetes version 1.14
* Reference topics



* [7 May 2019](https://cloud.ibm.com/docs/containers?topic=containers-containers-relnotescontainers-may0719)



* Cluster DNS provider
* Edge nodes
* Free clusters
* New! Integrations
* New! Kubernetes version 1.14
* Deprecated Kubernetes version 1.11
* Permissions
* Worker pools
* Reference topics



* [1 May 2019](https://cloud.ibm.com/docs/containers?topic=containers-containers-relnotescontainers-may0119)



* Assigning infrastructure access











 Tutorials library for Kubernetes Service 

[Tutorials library for Kubernetes Service](https://cloud.ibm.com/docs?tab=tutorials&tags=containers&page=1&pageSize=20)





 Creating a cluster in your Virtual Private Cloud (VPC)
Controlling traffic between pods with Kubernetes policies 

Virtual Private Cloud

You can use Kubernetes policies to control network traffic between pods in your cluster and to isolate app microservices from each other within a namespace or across namespaces.

Level of application: Worker node host endpoint

Default behavior: No Kubernetes network policies exist by default in your cluster. By default, any pod has access to any other pod in the cluster. Additionally, any pod has access to any services that are exposed by the pod network, such as a metrics service, the cluster DNS, the API server, or any services that you manually create in your cluster.

Use case: Kubernetes network policies specify how pods can communicate with other pods and with external endpoints. Both incoming and outgoing network traffic can be allowed or blocked based on protocol, port, and source or destination IP addresses. Traffic can also be filtered based on pod and namespace labels. When Kubernetes network policies are applied, they are automatically converted into Calico network policies. The Calico network plug-in in your cluster enforces these policies by setting up Linux Iptables rules on the worker nodes. Iptables rules serve as a firewall for the worker node to define the characteristics that the network traffic must meet to be forwarded to the targeted resource.

If most or all pods don't require access to specific pods or services, and you want to ensure that pods by default can't access those pods or services, you can create a Kubernetes network policy to block ingress traffic to those pods or services. For example, any pod can access the metrics endpoints on the CoreDNS pods. To block unnecessary access, you can apply a policy such as the following, which allows all ingress to TCP and UDP port 53 so that pods can access the CoreDNS functionality. However, it blocks all other ingress, such as any attempts to gather metrics from the CoreDNS pods, except from pods or services in namespaces that have the coredns-metrics-policy: allow label, or from pods in the kube-system namespace that have the coredns-metrics-policy: allow label.

apiVersion: networking.k8s.io/v1