vania-janet/multiturn-rag-retrieval-data huggingface dataset:
Note that after you lock the data store in the following steps, existing pods continue to run, but any pods that are created remain in the ContainerCreating state and can't start until you unlock the data store. This data store lock ensures that the IPAM records are not modified while you release IP addresses. For more information, see the [Calico open source documentation](https://docs.tigera.io/calico/latest/reference/calicoctl/ipam/check).



1. Lock the data store for the Calico IPAM records.

calicoctl datastore migrate lock
2. Save the IPAM check results.

calicoctl ipam check -o report.json
3. Release unused IP addresses. This process can run for up to 20 minutes depending on how many IP addresses must be released.

calicoctl ipam release --from-report=report.json
4. Unlock the data store.

calicoctl datastore migrate unlock
5. Verify that all unused IP addresses are released.

calicoctl ipam check

Example output

Check complete; found 0 problems.



5. Optional: To verify that the data store was successfully unlocked and that IP addresses are now available for assignment, create a pod and check that it starts correctly.



1. For example, create a simple NGINX pod.

kubectl run test --image=nginx --generator=run-pod/v1
2. Verify that the pod has an IP address and is running successfully.

kubectl get po test
3. Delete the test pod.

kubectl delete pod test



6. Continue to the next section to check for unused IP address blocks.







 Step 2: Releasing IP address blocks 

Next, check for and clean up entire blocks of IP addresses that are assigned to a worker node but are not used by that worker node.

Occasionally, when a worker node is assigned a second or third block of IP addresses, entire blocks of IP addresses that the worker node previously used might later be completely unused.
5.2.5 Red Hat OpenShift on IBM Cloud can optionally configure [Red Hat OpenShiftta.keyword.redhat_openshift_notm}} security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.6 Red Hat OpenShift on IBM Cloud can optionally configure [Red Hat OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.7 Red Hat OpenShift on IBM Cloud can optionally configure [Red Hat OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/).
openshift-storage.noobaa.io openshift-storage.noobaa.io/obc Delete Immediate false 6m32s
2. Create a PVC that refers to the storage class that you want to use. Save and edit the following PVC configuration file to refer to the storage class that you want to use. If you enabled encryption with Hyper Protect Crypto Services, you can use the storage class ocs-storagecluster-ceph-rbd-encrypted which supports encryption. Example PVC for using the ocs-storagecluster-cephfs storage class.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: odf-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: ocs-storagecluster-cephfs
3. Create the PVC in your cluster.

oc create -f <my-pvc.yaml>
4. Create a YAML configuration file for a pod that mounts the PVC that you created. The following example creates an nginx pod that writes the current date and time to a test.txt file.

apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: app
image: nginx
command: ["/bin/sh"]
args: ["-c", "while true; do echo $(date -u) >> /test/test.txt; sleep 600; done"]
volumeMounts:
- name: persistent-storage
mountPath: /test
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: odf-pvc
Show more
5. Create the pod in your cluster.

oc apply -f pod.yaml
6. To verify that the pod is deployed, wait for your app to get into a Running status.

oc get pods

Example output
5.6.3 Apply security context to your pods and containers. Not Scored 2 [Fail](https://cloud.ibm.com/docs/openshift?topic=openshift-cis-benchmark-49ibm-remediations-and-explanations-49) Shared 
 5.6.4 The default namespace should not be used. Scored 2 Pass Shared 









 IBM remediations and explanations 

Review information from IBM on the CIS Benchmark results.



Explanation and remediation

 Section Remediation/Explanation 

 1.2.1 Red Hat OpenShift on IBM Cloud utilizes RBAC for cluster protection, but allows anonymous discovery, which is considered reasonable per [CIS Kubernetes Benchmark](https://www.cisecurity.org/benchmark/kubernetes/). 
 1.2.10 Red Hat OpenShift on IBM Cloud does not enable the [EventRateLimit](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/eventratelimit) admission controller since it is a Kubernetes alpha feature. 
 1.2.12 Red Hat OpenShift on IBM Cloud does not enable the [AlwaysPullImages](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/alwayspullimages) admission controller since it overrides a container's imagePullPolicy and may impact performance. 
 1.2.13 Red Hat OpenShift on IBM Cloud supports [pod security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) which are similar to Kubernetes pod security policies. 
 1.2.16 Red Hat OpenShift on IBM Cloud supports [pod security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) which are similar to Kubernetes pod security policies.
Pods that are managed by a daemon set are automatically scheduled when a worker node is added to a cluster. Typical use cases include log collectors, such as logstash or prometheus, that collect logs from every worker node to provide insight into the health of a cluster or an app. 
 [Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) A job ensures that one or more pods run successfully to completion. You might use a job for queues or batch jobs to support parallel processing of separate but related work items, such as specific frames to render, emails to send, and files to convert. To schedule a job to run at certain times, use a [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/). 







 What if I want my app configuration to use variables? How do I add these variables to the YAML? 

To add variable information to your deployments instead of hardcoding the data into the YAML file, you can use a Kubernetes [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) or [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) object.

To consume a ConfigMap or secret, you need to mount it to the pod. The ConfigMap or secret is combined with the pod just before the pod is run. You can reuse a deployment spec and image across many apps, but then swap out the customized configmaps and secrets. Secrets in particular can take up a lot of storage on the local node, so plan accordingly.

Both resources define key-value pairs, but you use them for different situations.

Configmap
: Provide non-sensitive configuration information for workloads that are specified in a deployment. You can use configmaps in three main ways.



* File system: You can mount an entire file or a set of variables to a pod.
5.2.2 IBM Cloud Kubernetes Service can optionally configure [pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.3 IBM Cloud Kubernetes Service can optionally configure [pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.4 IBM Cloud Kubernetes Service can optionally configure [pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.5 IBM Cloud Kubernetes Service can optionally configure [pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.6 IBM Cloud Kubernetes Service can optionally configure [pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/).
5.2.2 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.3 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.4 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.5 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.6 IBM Cloud Kubernetes Service can optionally configure [Pod security admission](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission), which is similar to the unsupported [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/).
5.2.1 Red Hat OpenShift on IBM Cloud can optionally configure [OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.2 Red Hat OpenShift on IBM Cloud can optionally configure [OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.3 Red Hat OpenShift on IBM Cloud can optionally configure [OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/).
name: app
spec:
containers:
- name: app
image: nginx
command: ["/bin/sh"]
args: ["-c", "while true; do echo $(date -u) >> /test/test.txt; sleep 5; done"]
volumeMounts:
- name: persistent-storage
mountPath: /test
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: netapp-pvc
5. Create the pod in your cluster.

oc apply -f pod.yaml
6. Verify that the pod is deployed. Note that it might take a few minutes for your app to get into a Running state.

oc get pods

Example output

NAME READY STATUS RESTARTS AGE
app 1/1 Running 0 50s
7. Verify that the app can write to your ONTAP-NAS instance.



1. Log in to your pod.

oc exec app -it bash
2. Display the contents of the test.txt file to confirm that your app can write data to your persistent storage.

cat /test/test.txt

Example output

Wed May 19 13:28:31 UTC 2021
Wed May 19 13:28:37 UTC 2021
Wed May 19 13:28:42 UTC 2021
Wed May 19 13:28:47 UTC 2021
3. Exit the pod.

exit









 Removing NetApp ONTAP-NAS storage from your apps 

Before you remove your storage configuration, remove the app pods and PVCs that are using your NetApp storage.



1. List your PVCs and note the name of the PVC and the corresponding PV that you want to remove.

oc get pvc
2. List all the pods that currently mount the PVC that you want to delete. If no pods are returned, you do not have any pods that currently use your PVC.
You can also use Cloud Shell to preview apps that are running in Kubernetes, such as a Prometheus dashboard or your typical web app. Because apps that are running in a pod in a Kubernetes cluster use an internal port, you need to expose the pod's port either by creating a service or setting up port forwarding.

If you create a service, you need to expose the service publicly, set up authentication to secure access, and set up an application load balancer (ALB) for the service. For more information, see [Connecting applications with services](https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/).

If you're doing app development and testing, port forwarding is a simpler way to be able to quickly preview apps in your cluster. It requires much less up-front configuration, but you also have to reconfigure it whenever the pod restarts. The following steps walk you through setting up port forwarding and previewing an app that's running in a IBM Cloud Kubernetes Service cluster.



1. In a Cloud Shell session, set the cluster where your app is running as the context. This enables you to use kubectl commands to work with your cluster. For more information, see [Configuring the CLI to run kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_installcs_cli_configure).
2. Find the name of the pod where your app is running.

kubectl get pods

The pod name is listed under the NAME column, for example my-pod-123abc.

NAME READY STATUS RESTARTS AGE
my-pod-123abc 1/1 Running 0 9d
3. Find the port in the pod that you want to expose by running the [kubectl describe](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commandsdescribe) command on the pod.

kubectl describe pod my-pod-123abc
If you want to prevent host networking pods from sending and receiving data over the VPN, you can set the following options in your values.yaml deployment file: local.subnet: 172.30.0.0/16 and enablePodSNAT: false. These configuration settings expose all the Kubernetes pods over the VPN connection to the remote network. However, only the pods that are located in the specified secure namespace are reachable over the VPN.



Before you begin



* [Deploy the strongSwan Helm chart](https://cloud.ibm.com/docs/openshift?topic=openshift-vpnvpn_configure) and [ensure that VPN connectivity is working correctly](https://cloud.ibm.com/docs/openshift?topic=openshift-vpnvpn_test).
* [Install and configure the Calico CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-network_policiescli_install).



To limit VPN traffic to a certain namespace,



1. Create a Calico global network policy named allow-non-vpn-outbound.yaml. This policy allows all namespaces to continue to send outbound traffic to all destinations, except to the remote subnet that the strongSwan VPN accesses. Replace <remote.subnet> with the remote.subnet that you specified in the Helm values.yaml configuration file. To specify multiple remote subnets, see the [Calico documentation](https://docs.tigera.io/calico/latest/reference/resources/globalnetworkpolicy).

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
name: allow-non-vpn-outbound
spec:
selector: has(projectcalico.org/namespace)
egress:
- action: Allow
destination:
notNets:
- <remote.subnet>
order: 900
types:
- Egress
2. Apply the policy.
5.2.7 Red Hat OpenShift on IBM Cloud can optionally configure [Red Hat OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.8 Red Hat OpenShift on IBM Cloud can optionally configure [Red Hat OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/). 
 5.2.9 Red Hat OpenShift on IBM Cloud can optionally configure [Red Hat OpenShift security context constraints](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_scc) and [Kubernetes pod security admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) which are similar to the deprecated [Kubernetes pod security policies](https://kubernetes.io/docs/concepts/security/pod-security-policy/).
Then, you also [tainted those edge nodes](https://cloud.ibm.com/docs/containers?topic=containers-edgeedge_workloads) to prevent any other workloads from running on edge nodes. However, you didn't add an edge node affinity rule and toleration to your app deployment. Your app pods can't be scheduled on the same tainted nodes as the service pods, and no traffic reaches the back-end service for your app.
* Custom taints: You used custom taints on several nodes so that only app pods with that taint toleration can deploy to those nodes. You added affinity rules and tolerations to the deployments of your app and load balancer service so that their pods deploy to only those nodes. However, ibm-cloud-provider-ipkeepalived pods that are automatically created in the ibm-system namespace ensure that the load balancer and the app pods are always scheduled onto the same worker node. These keepalived pods don't have the tolerations for the custom taints that you used. They can't be scheduled on the same tainted nodes that your app pods are running on, and no traffic reaches the back-end service for your app.



 How to fix it 

Resolve the issue by choosing one of the following options:

Edge node taints: To ensure that your load balancer and app pods deploy to tainted edge nodes, [add edge node affinity rules and tolerations to your app deployment](https://cloud.ibm.com/docs/containers?topic=containers-loadbalancerlb_edge_nodes). Load balancer pods have these affinity rules and tolerations by default.

Custom taints: Remove custom taints that the keepalived pods don't have tolerations for. Instead, you can [label worker nodes as edge nodes, and then taint those edge nodes](https://cloud.ibm.com/docs/containers?topic=containers-edge).

If you complete one of the previous options but the keepalived pods are still not scheduled, you can get more information about the keepalived pods:



1. Get the keepalived pods.

kubectl get pods -n ibm-system
Use the config option to opt out of only the configuration update actions as in the cloudant scenario above.

Here are sample opt-out specifications in a manifest.yml file for the dashDB and cloudant scenarios.

env:
services_autoconfig_excludes: dashDB=all

env:
services_autoconfig_excludes: cloudant=config

env:
services_autoconfig_excludes: cloudant=config dashDB=all

Here are examples of how to set the services_autoconfig_excludes environment variable for the app myapp by using the command-line interface.

ibmcloud cf set-env myapp services_autoconfig_excludes cloudant=config

ibmcloud cf set-env myapp services_autoconfig_excludes "cloudant=config dashDB=all"

To find the label for a service in VCAP_SERVICES issue a command like the following example:

ibmcloud cf env myapp

The output includes text similar to the following, where you can see the label field with value elephantsql:

"elephantsql": [ ! ! ! ! ! ! ! ! ! !
Additionally, when you remove or replace a worker node, the cleanup of the IP address block by the Calico IPAM might fail due to the lack of a worker node for the calico-kube-controllers to temporarily run on, or due to problems with the CNI plug-in or containerd runtime when the worker node is removed or replaced.

Ensuring that IP blocks are free is especially important for all classic clusters that run Kubernetes version 1.19 or later and all VPC clusters. In these clusters, the strictAffinity Calico setting is set to true, which forces a worker node to use IP addresses that are from its assigned IP blocks only, as opposed to using IP addresses from blocks that are assigned to other worker nodes. Over time, the blocks that are assigned to nodes but that are unused can accumulate until no more IP address blocks can be assigned to worker nodes.



1. Follow the steps to [release individual IP addresses](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-app-container-startindividual-ips).
2. Choose whether to lock the data store for the Calico IPAM records.



* If you lock the data store, existing pods continue to run, but any pods that are created remain in the ContainerCreating state and can't start until you unlock the data store. This data store lock ensures that pods can't be created after you check for unused blocks, but before you release the blocks.
* If you don't lock the data store, you must immediately verify that no new pods used IP addresses from a released block that you deleted.



calicoctl datastore migrate lock
3. List the Calico IPAM records. In the output, look for blocks that have 0 IPS IN USE, which indicates that the block is not used by its assigned worker node.

calicoctl ipam show --show-blocks

In this example output, the 172.24.10.64/26 block has no IP addresses in use.

...
Block 172.24.10.64/26 64 0 (0%) 64 (100%) 
...
4.