vania-janet/multiturn-rag-retrieval-data huggingface dataset:
NodePort You can access an app through a NodePort only if you are connected to your private VPC network, such as through a VPN connection. To access an app from the internet, you must use a VPC load balancer or Ingress service instead. 
 Pod network VPC access control lists (ACLs) filter incoming and outgoing traffic for your cluster at the subnet level, and security groups filter incoming and outgoing traffic for your cluster at the worker nodes level. To control traffic within the cluster at the pod-to-pod level, you can't use VPC security groups or ACLs. Instead, use [Calico](https://cloud.ibm.com/docs/containers?topic=containers-network_policies) and [Kubernetes network policies](https://cloud.ibm.com/docs/containers?topic=containers-vpc-kube-policies), which can control the pod-level network traffic that uses IP in IP encapsulation. 
 strongSwan VPN service The strongSwan service is not supported. To connect your cluster to resources in an on-premises network or another VPC, see [Using VPN with your VPC](https://cloud.ibm.com/docs/vpc?topic=vpc-vpn-onprem-example). 
 Subnets <br><br> * See [VPC networking limitations](https://cloud.ibm.com/docs/containers?topic=containers-vpc-subnetsvpc_basics_limitations).<br> * Do not delete the subnets that you attach to your cluster during cluster creation or when you add worker nodes in a zone. If you delete a VPC subnet that your cluster used, any load balancers that use IP addresses from the subnet might experience issues, and you might be unable to create new load balancers.<br><br><br> 
 VPC load balancer See [VPC load balancer limitations](https://cloud.ibm.com/docs/containers?topic=containers-vpc-lbaaslbaas_limitations). 







 Storage
All private traffic is allowed by default by the allow-all-private-default policy. Any other inbound network traffic from the internet to your worker nodes that isn't specified in the default policies gets blocked. The default policies don't affect pod to pod traffic.

Don't remove the default policies from your cluster, because they are recreated on the next cluster master refresh or master update. If you want to further restrict traffic, apply lower order Calico policies to block the traffic. Be sure you fully understand what you are blocking, and that the cluster components do not need the traffic you want to block.

Review the following default Calico host policies that are automatically applied to your cluster.



Default Calico host policies for each cluster

 Calico policy Description 

 allow-all-outbound Allows all outbound traffic on the public network. 
 allow-all-private-default Allows all inbound and outbound traffic on the private network. 
 allow-bigfix-port Allows incoming traffic on port 52311 to the BigFix app to allow necessary worker node updates. 
 allow-icmp Allows incoming ICMP packets (pings). 
 allow-node-port-dnat Allows incoming network load balancer (NLB), Ingress application load balancer (ALB), and NodePort service traffic to the pods that those services are exposing. Note: You don't need to specify the exposed ports because Kubernetes uses destination network address translation (DNAT) to forward the service requests to the correct pods. That forwarding takes place before the host endpoint policies are applied in Iptables. 
 allow-sys-mgmt Allows incoming connections for specific IBM Cloud infrastructure systems that are used to manage the worker nodes. 
 allow-vrrp Allows VRRP packets, which monitor and move virtual IP addresses between worker nodes. 



Default Kubernetes policies that limits access to the Kubernetes Dashboard are also created. Kubernetes policies don't apply to the host endpoint, but at the pod-level instead, and to all classic and VPC clusters.



Default Kubernetes policies for each cluster

 Kubernetes policy Description
If you created other priority classes in your cluster, you can check to make sure that they don't set a globalDefault by running kubectl describe priorityclass <name>. 
 description Optional: Tell users why to use this priority class. Enclose the string in quotations (""). 



4. Create the priority class in your cluster.

kubectl apply -f filepath/priorityclass.yaml
5. Verify that the priority class is created.

kubectl get priorityclasses



Great! You created a priority class. Let your team know about the priority class and which priority class, if any, that they must use for their pod deployments.





 Assigning priority to your pods 

Assign a priority class to your pod spec to set the pod's priority within your IBM Cloud Kubernetes Service cluster.

Before you begin:



* [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster)
* Ensure that you have the [Writer or Manager IBM Cloud IAM service access role](https://cloud.ibm.com/docs/containers?topic=containers-userschecking-perms) in the namespace that you want to deploy the pods to.
* [Understand how priority scheduling works](https://cloud.ibm.com/docs/containers?topic=containers-pod_prioritypriority_scheduling), as priority can preempt existing pods and affect how your cluster's resources are consumed.



Complete the following steps to check the importance of other deployed pods so that you can choose the correct priority class for your pods in relation to what already is deployed.



1. View the priority classes that other pods in the namespace use.

kubectl get pods -n <namespace> -o custom-columns=NAME:.metadata.name,PRIORITY:.spec.priorityClassName
2. Get the details of the priority class and note the value number. Pods with higher numbers are prioritized before pods with lower numbers.
Use / to request the default audio format, audio/ogg;codecs=opus. For more information, see [Using audio formats](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-audio-formats).

The Ogg audio format is not supported with the Safari browser. If you are using the the Text to Speech service with the Safari browser, you must specify a different format in which you want the service to return the audio.

timings (optional string[ ])
: Specifies that the service is to return word timing information for all strings of the input text. The service returns the start and end time of each token of the input. Specify words as the lone element of the array to request word timings. Specify an empty array or omit the parameter to receive no word timings. For more information, see [Generating word timings](https://cloud.ibm.com/docs/text-to-speech?topic=text-to-speech-timing). Not supported for Japanese input text.

The following snippet of JavaScript code passes a simple "Hello world" message as the input text and requests the default format for the audio. The calls are included in the onOpen() function that is defined for the client to ensure that they are sent only after the connection is established.

function onOpen(evt) {
var message = {
text: 'Hello world',
accept: '/'
};
websocket.send(JSON.stringify(message));
}

The service responds to this message by sending a text message that confirms the format of the audio response. The following response confirms the default audio format.

{
'binary_streams': [
{
content_type: 'audio/ogg;codecs=opus'
}
]
}





 Receive a response 

After it confirms the audio format, the service sends the synthesized audio as a binary stream of data in the indicated format.
* [Configuring pod security admission](https://cloud.ibm.com/docs/openshift?topic=openshift-pod-security-admissionpod-security-configure)
* [Additional resources](https://cloud.ibm.com/docs/openshift?topic=openshift-pod-security-admissionpod-sec-additional-resources)







 Securing the cluster network 



 Controlling traffic in Classic clusters 

[Using Calico network policies to control traffic on Classic clusters](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutorialpolicy_tutorial)



* [Objectives](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutorialpolicies_objectives)
* [Audience](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutorialpolicies_audience)
* [Prerequisites](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutorialpolicies_prereqs)
* [Deploy an app and expose it by using an NLB](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutoriallesson1)
* [Block all incoming traffic to all node ports](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutoriallesson2)
* [Allow incoming traffic from a specific IP to the NLB](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutoriallesson3)
* [Deny incoming traffic from specific IPs to the NLB](https://cloud.ibm.com/docs/openshift?topic=openshift-policy_tutoriallesson4)
Kubernetes network policies can also help you enforce workload isolation between namespaces by controlling how pods and services in different namespaces can communicate.

How can I control pod permissions?
: By default, every cluster enables the [Kubernetes pod security policy admission controller](https://kubernetes.io/docs/concepts/security/pod-security-policy/) that you can use to define what requirements a pod must meet to get deployed in a namespace. With pod security policies, you can control the usage of privileged containers, root namespaces, host networking and ports, volume types, host file systems, and Linux permissions such as read-only or group IDs. For more information, see [Configuring pod security policies](https://cloud.ibm.com/docs/containers?topic=containers-psp).

What else can I do to protect my container?
: Limit the number of privileged containers. Containers run as a separate Linux process on the compute host that is isolated from other processes. Although users have root access inside the container, the permissions of this user are limited outside the container to protect other Linux processes, the host file system, and host devices. Some apps require access to the host file system or advanced permissions to run properly. You can run containers in privileged mode to allow the container the same access as the processes running on the compute host.
: Keep in mind that privileged containers can cause huge damage to the cluster and the underlying compute host if they become compromised. Try to limit the number of containers that run in privileged mode and consider changing the configuration for your app so that the app can run without advanced permissions.

If you want to block privileged containers from running in your cluster, consider setting up custom [pod security policies](https://cloud.ibm.com/docs/containers?topic=containers-pspcustomize_psp).

Apply OS security settings to pods
When you grant the supplemental group ID read and write access to the file storage, any non-root user that belongs to the group ID, including your pod, is granted access to the file storage.

You can use one of the provided [gid storage classes](https://cloud.ibm.com/docs/openshift?topic=openshift-file_storagefile_storageclass_reference) or create a custom storage class to define your own supplemental group ID.

Allocating a supplemental group ID for a non-root user of a file storage device is supported for single zone clusters only, and can't be used in multizone clusters.



1. Select one of the [provided gid storage classes](https://cloud.ibm.com/docs/openshift?topic=openshift-file_storagefile_storageclass_reference) to assign the default group ID 65531 to your non-root user that you want to read and write to your file storage. If you want to assign a custom group ID, create a YAML file for a customized storage class. In your customized storage class YAML file, include the gidAllocate: "true" parameter and define the group ID in the gidFixed parameter.

Example storage classes for assigning the default group ID 65531.



* ibmc-file-bronze-gid
* ibmc-file-silver-gid
* ibmc-file-gold-gid



Example customized storage class to specify a different group ID.

apiVersion: storage.k8s.io/v1beta1
kind: StorageClass
metadata:
name: ibmc-file-bronze-gid-custom
labels:
kubernetes.io/cluster-service: "true"
provisioner: ibm.io/ibmc-file
parameters:
type: "Endurance"
iopsPerGB: "2"
sizeRange: "[1-12000]Gi"
mountOptions: nfsvers=4.1,hard
billingType: "hourly"
reclaimPolicy: "Delete"
classVersion: "2"
gidAllocate: "true"
For a more detailed explanation of different features that you can add to your deployment, see [Making your app deployment YAML file](https://cloud.ibm.com/docs/containers?topic=containers-appapp_yaml).





 What type of Kubernetes objects can I make for my app? 

When you prepare your app YAML file, you have many options to increase the app's availability, performance, and security. For example, instead of a single pod, you can use a Kubernetes controller object to manage your workload, such as a replica set, job, or daemon set. For more information about pods and controllers, view the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/pods/). A deployment that manages a replica set of pods is a common use case for an app.

For example, a kind: Deployment object is a good choice to deploy an app pod because with it, you can specify a replica set for more availability for your pods.

The following table describes why you might create different types of Kubernetes workload objects.



Types of Kubernetes workload objects that you can create.

 Object Description 

 [Pod](https://kubernetes.io/docs/concepts/workloads/pods/) A pod is the smallest deployable unit for your workloads, and can hold a single or multiple containers. Similar to containers, pods are disposable and are often used for unit testing of app features. To avoid downtime for your app, consider deploying pods with a Kubernetes controller, such as a deployment. A deployment helps you to manage multiple pods, replicas, pod scaling, rollouts, and more. 
 [ReplicaSet](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/) A replica set makes sure that multiple replicas of your pod are running, and reschedules a pod if the pod goes down. You might create a replica set to test how pod scheduling works, but to manage app updates, rollouts, and scaling, create a deployment instead.
5.2.8 IBM Cloud Kubernetes Service can optionally configure [pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.9 IBM Cloud Kubernetes Service can optionally configure [pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.3.2 IBM Cloud Kubernetes Service has a set of [default Calico and Kubernetes network policies defined](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesdefault_policy) and [additional network policies can optionally be added](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesadding_network_policies). 
 5.4.2 IBM Cloud Kubernetes Service can optionally [enable a Kubernetes Key Management Service (KMS) provider](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkms). 
 5.5.1 IBM Cloud Kubernetes Service can optionally [enable image security enforcement](https://cloud.ibm.com/docs/containers?topic=containers-imagesportieris-image-sec). 
 5.6.2 IBM Cloud Kubernetes Service does not annotate all pods with [seccomp profiles](https://kubernetes.io/docs/concepts/security/pod-security-policy/seccomp).
5.2.8 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.9 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.3.2 IBM Cloud Kubernetes Service has a set of [Default Calico and Kubernetes network policies defined](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesdefault_policy) and [Additional network policies can optionally be added](https://cloud.ibm.com/docs/containers?topic=containers-network_policiesadding_network_policies). 
 5.4.2 IBM Cloud Kubernetes Service can optionally [Enable a Kubernetes Key Management Service (KMS) provider](https://cloud.ibm.com/docs/containers?topic=containers-encryptionkms). 
 5.5.1 IBM Cloud Kubernetes Service can optionally [Enable image security enforcement](https://cloud.ibm.com/docs/containers?topic=containers-imagesportieris-image-sec). 
 5.6.2 IBM Cloud Kubernetes Service does not annotate all pods with [seccomp profiles](https://kubernetes.io/docs/concepts/security/pod-security-policy/seccomp).
Use deployments and replica sets to deploy your app and its dependencies
: A deployment is a Kubernetes resource that you can use to declare all the components of your app and its dependencies. With deployments, you don't have to write down all the steps and instead can focus on your app. When you deploy more than one pod, a replica set is automatically created for your deployments that monitors the pods and assures that the specified number of pods is up and running. When a pod goes down, the replica set replaces the unresponsive pod with a new one. You can use a deployment to define update strategies for your app, including the number of pods that you want to add during a rolling update and the number of pods that can be unavailable at a time. When you perform a rolling update, the deployment checks whether the revision is working and stops the rollout when failures are detected. With deployments, you can concurrently deploy multiple revisions with different options. For example, you can test a deployment first before you decide to push it to production. By using Deployments, you can track any deployed revisions. You can use this history to roll back to a previous version if you encounter that your updates are not working as expected.

Include enough replicas for your app's workload, plus two
: To make your app even more highly available and more resilient to failure, consider including extra replicas than the minimum to handle the expected workload. Extra replicas can handle the workload in case a pod crashes and the replica set did not yet recover the crashed pod. For protection against two simultaneous failures, include two extra replicas. This setup is an N+2 pattern, where N is the number of replicas to handle the incoming workload and +2 is an extra two replicas. As long as your cluster has enough space, you can have as many pods as you want.

Spread pods across multiple nodes (anti-affinity)
: When you create your deployment, each pod can be deployed to the same worker node. This is known as affinity, or colocation. To protect your app against worker node failure, you can configure your deployment to spread your pods across multiple worker nodes by using the podAntiAffinity option with your standard clusters.
5.6.3 Apply Security Context to Your Pods and Containers Not Scored 2 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-126cis-benchmark-remediations-126) Shared 
 5.6.4 The default namespace should not be used Scored 2 [Fail](https://cloud.ibm.com/docs/containers?topic=containers-cis-benchmark-126cis-benchmark-remediations-126) Shared 









 IBM remediations and explanations 

Review information from IBM on the CIS Benchmark results.



Explanation and remediation

 Section Remediation/Explanation 

 1.2.1 IBM Cloud Kubernetes Service utilizes RBAC for cluster protection, but allows anonymous discovery, which is considered reasonable per [CIS Kubernetes Benchmark](https://www.cisecurity.org/benchmark/kubernetes/). 
 1.2.10 IBM Cloud Kubernetes Service does not enable the [EventRateLimit](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/eventratelimit) admission controller since it is a Kubernetes alpha feature. 
 1.2.12 IBM Cloud Kubernetes Service does not enable the [AlwaysPullImages](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/alwayspullimages) admission controller since it overrides a container's imagePullPolicy and may impact performance. 
 1.2.13 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/).
2. Three pods with high, medium, and low priority are pending scheduling. The Red Hat OpenShift controller finds an available worker node, but the worker node has only enough resources to support the high and medium priority pods. The low-priority pod is not scheduled and it remains in pending.
3. Two pods with high and medium priority are pending scheduling. A third pod with low priority exists on an available worker node. However, the worker node does not have enough resources to schedule any of the pending pods. The Red Hat OpenShift controller preempts, or removes, the low-priority pod, which returns the pod to a pending state. Then, the Red Hat OpenShift controller tries to schedule the high priority pod. However, the worker node does not have enough resources to schedule the high priority pod, and instead, the Red Hat OpenShift controller schedules the medium priority pod.



For more information, see the Kubernetes documentation about [pod priority and preemption](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/).

Can I disable the pod priority admission controller?
: No. If you don't want to use pod priority, don't set a globalDefault or include a priority class in your pod deployments. Every pod defaults to zero, except the cluster-critical pods that IBM deploys with the [default priority classes](https://cloud.ibm.com/docs/openshift?topic=openshift-pod_prioritydefault_priority_class). Because pod priority is relative, this basic setup ensures that the cluster-critical pods are prioritized for resources, and schedules any other pods by following the existing scheduling policies that you have in place.



 Understanding default priority classes 

Your Red Hat® OpenShift® on IBM Cloud® clusters come with some priority classes by default.

Do not modify the default classes, which are used to properly manage your cluster.
Temporarily disable the gateway by changing the corresponding istio-ingressgateway-public-<1|2|3>-enabled setting to "false". In this example ConfigMap, to move the ibm-cloud-provider-ip pod for the gateway that exists in dal10, change istio-ingressgateway-public-1-enabled to "false".

apiVersion: v1
data:
istio-ingressgateway-zone-1: dal10
istio-ingressgateway-zone-2: dal12
istio-ingressgateway-public-1-enabled: "false"
istio-ingressgateway-public-2-enabled: "true"
istio-monitoring: "true"
kind: ConfigMap
...
4. Save and close the configuration file.
5. Verify that the gateway's load balancer pod is deleted. Note that it might take up to 30 minutes for the changes to complete.

kubectl get service -n istio-system -o wide
6. Open the managed-istio-custom ConfigMap resource.

kubectl edit cm managed-istio-custom -n ibm-operators
7. Re-enable the gateway by changing the istio-ingressgateway-public-<1|2|3>-enabled setting back to "true".
8. Save and close the configuration file. A new load balancer service for the gateway (the istio-ingressgateway pod) is created, and a new ibm-cloud-provider-ip pod for that load balancer is deployed to a worker node in the same zone as the istio-ingressgateway pod.
9. To verify that the pods now exist in the same zone, follow the steps in the Why it’s happening section.