vania-janet/multiturn-rag-retrieval-data huggingface dataset:
You can use this history to roll back to a previous version if you encounter that your updates are not working as expected.

Include enough replicas for your app's workload, plus two
: To make your app even more highly available and more resilient to failure, consider including extra replicas than the minimum to handle the expected workload. Extra replicas can handle the workload in case a pod crashes and the replica set did not yet recover the crashed pod. For protection against two simultaneous failures, include two extra replicas. This setup is an N+2 pattern, where N is the number of replicas to handle the incoming workload and +2 is an extra two replicas. As long as your cluster has enough space, you can have as many pods as you want.

Spread pods across multiple nodes (anti-affinity)
: When you create your deployment, each pod can be deployed to the same worker node. This is known as affinity, or colocation. To protect your app against worker node failure, you can configure your deployment to spread your pods across multiple worker nodes by using the podAntiAffinity option with your standard clusters. You can define two types of pod anti-affinity: preferred or required. For more information, see the Kubernetes documentation on [Assigning Pods to Nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/).

For an example of affinity in an app deployment, see [Making your app deployment YAML file](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift_appsapp_yaml).

Distribute pods across multiple zones or regions
: To protect your app from a zone failure, you can create multiple clusters in separate zones or add zones to a worker pool in a multizone cluster.
* [Step 1: Verifying the IBM Cloud catalog information](https://cloud.ibm.com/docs/openshift?topic=openshift-debug-portworxpx-verify-catalog)
* [Step 2: Verifying the cluster setup](https://cloud.ibm.com/docs/openshift?topic=openshift-debug-portworxpx-verify-cluster)
* [Step 3: Reach out to Portworx and IBM](https://cloud.ibm.com/docs/openshift?topic=openshift-debug-portworxpx-support)



[Why does encryption fail with an invalid KMS endpoint?](https://cloud.ibm.com/docs/openshift?topic=openshift-px-kms-endpointpx-kms-endpoint)





 OpenShift Data Foundation 

[Debugging OpenShift Data Foundation failures](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs)



* [Checking whether the pod that mounts your storage instance is successfully deployed](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs_deploy)
* [Restarting your app pod](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs_restart)
* [Verifying that the storage driver and plug-in pods show a status of Running](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs_driver_plugin)
(response, statusCode, error) -> Void in

print(response)

}

// with this

push.registerWithDeviceToken(deviceToken: "<apns-device-token>") {
response, statusCode, error in

print(response?.id ?? "")
}
Show more
* If you are using user ID for registration

// Replace this

push.registerWithDeviceToken(deviceToken:("<apns-device-token>", WithUserId: "<your-user-id>") {
(response, statusCode, error) -> Void in

print(response)
}

// with this

push.registerWithDeviceToken(deviceToken: "<apns-device-token>",
withUserId: "<your-user-id>") {
response, statusCode, error in

print(response)
}
Show more



4. Change the unregister API call

// Replace this

push.unregisterDevice(completionHandler: {
(response, statusCode, error) -> Void in

print(response)
}

// with this

push.unregisterDevice { response, statusCode, error in
print(response)
}
5. Optionally, if you are using tags:



* Change the tag subscriptions as follows:

// Replace this

push.subscribeToTags(tagsArray: ["<tag_name>"], completionHandler: { (response, statusCode, error) -> Void in

/...../
}

// with this
push.subscribeToTags(tagName: "<tag_name>") {
response, statusCode, error in

/...../
});
* Changes to the get all tag subscription for the device

// Replace this
- name: persistent-storage
persistentVolumeClaim:
claimName: ocs-pvc
Show more
4. Create the pod in your cluster.

oc apply -f pod.yaml
5. Verify that the pod is deployed. Note that it might take a few minutes for your app to get into a Running state.

oc get pods

Example output

NAME READY STATUS RESTARTS AGE
app 1/1 Running 0 2m58s
6. Verify that the app can write data.



1. Log in to your pod.

oc exec <app-pod-name> -it bash
2. Display the contents of the test.txt file to confirm that your app can write data to your persistent storage.

cat /test/test.txt

Example output

Tue Mar 2 20:09:19 UTC 2021
Tue Mar 2 20:09:25 UTC 2021
Tue Mar 2 20:09:31 UTC 2021
Tue Mar 2 20:09:36 UTC 2021
Tue Mar 2 20:09:42 UTC 2021
Tue Mar 2 20:09:47 UTC 2021
3. Exit the pod.

exit









 Scaling your ODF configuration by attaching raw disks 

To scale your ODF configuration by adding disks to your worker nodes, increase the num-of-osd parameter value and specify the new worker node names with the worker-nodes parameter.

In the following example, 3 worker nodes are added to the configuration that was created previously. You can scale your configuration by adding updating the command parameters as follows:



* --name - Create a configuration with a new name.
* --template-name - Use the same parameter value as in your existing configuration.
* --template-version - Use the same parameter value as in your existing configuration.
* osd-device-path - Specify all previous osd-device-path values from your existing configuration and the device paths from the worker nodes that you have added to your cluster.
: ServiceLBNodePortControl=false
: PodSecurity=false

1.22
: ServiceLBNodePortControl=false
: CustomCPUCFSQuotaPeriod=true
: IPv6DualStack=false

1.21
: ServiceLoadBalancerClass=true
: CustomCPUCFSQuotaPeriod=true
: IPv6DualStack=false

1.20
: AllowInsecureBackendProxy=false
: CustomCPUCFSQuotaPeriod=true

1.19
: RuntimeClass=false
: CustomCPUCFSQuotaPeriod=true
: AllowInsecureBackendProxy=false
: SCTPSupport=false
: ServiceAppProtocol=false

1.18
: RuntimeClass=false
: CustomCPUCFSQuotaPeriod=true
: AllowInsecureBackendProxy=false
Add capacity to your cluster such as by [resizing worker pools](https://cloud.ibm.com/docs/containers?topic=containers-add_workers) so that the resources can be fulfilled.







 Fixing pod replacement by higher priority pods 

To see if your pod is being replaced by higher priority pods:



1. Get the name of your pod.

kubectl get pods
2. Describe your pod YAML.

kubectl get pod <pod_name> -o yaml
3. Check the priorityClassName field.



1. If there is no priorityClassName field value, then your pod has the globalDefault priority class. If your cluster admin did not set a globalDefault priority class, then the default is zero (0), or the lowest priority. Any pod with a higher priority class can preempt, or remove, your pod.
2. If there is a priorityClassName field value, get the priority class.

kubectl get priorityclass <priority_class_name> -o yaml
3. Note the value field to check your pod's priority.



4. List existing priority classes in the cluster.

kubectl get priorityclasses
5. For each priority class, get the YAML file and note the value field.

kubectl get priorityclass <priority_class_name> -o yaml
6. Compare your pod's priority class value with the other priority class values to see if it is higher or lower in priority.
7. Repeat steps 1 to 3 for other pods in the cluster, to check what priority class they are using. If those other pods' priority class is higher than your pod, your pod is not provisioned unless there is enough resources for your pod and every pod with higher priority.
8. Contact your cluster admin to add more capacity to your cluster and confirm that the correct priority classes are assigned.
UNMutableNotificationContent)
ENPushRichPushNotificationOptions.didReceive(request, withContentHandler: contentHandler)
}







 Step 8: Send notifications to the iOS device 

Use the [Send Notification API](https://cloud.ibm.com/apidocs/event-notifications/event-notificationssend-notifications) to send the push notification for the iOS device. You can use the [Node](mailto:https://github.com/IBM/event-notifications-node-admin-sdk) or [Go](https://github.com/IBM/event-notifications-go-admin-sdksend-notifications) admin SDK instead of calling the API directly.

Zoom

![Send notifications](https://cloud.ibm.com/docs-content/v1/content/4c9a07fe500838e5e55e71e73131ef20b44a6c66/event-notifications/images/en-send-notifications.png)

Figure 27. Send notifications

Zoom

![Receive notifications](https://cloud.ibm.com/docs-content/v1/content/4c9a07fe500838e5e55e71e73131ef20b44a6c66/event-notifications/images/en-receive-push.png)

Figure 28. Receive notifications
oc delete pod <dns_operator_pod> -n openshift-dns-operator
7. Wait for the DNS pod to restart. Run get pods with the --watch option to to verify that the pod is deployed.

oc get pods -n openshift-dns --watch
8. Verify that the DNS Operator pod is running.

oc get pods -n openshift-dns-operator
9. Get the ConfigMap YAML and verify that the the bufsize is 512.

oc get configmap -n openshift-dns dns-default -o yaml
10. After a few minutes, verify that the DNS resolution works. If you see an nslookup error, retry the nslookup.

/ nslookup XXX.XXX.databases.appdomain.cloud
Server: XXX.XX.X.XX
Address: XXX.XX.X.XX:XX

Non-authoritative answer:
XXX.XXX.databases.appdomain.cloud canonical name = icd-prod-us-south-db-lm0sr.us-south.containers.appdomain.cloud
icd-prod-us-south-db-lm0sr.us-south.containers.appdomain.cloud canonical name = icd-prod-us-south-db-lm0sr.XXX.akadns.net

Non-authoritative answer:
XXX.XXX.databases.appdomain.cloud canonical name = icd-prod-us-south-db-lm0sr.us-south.containers.appdomain.cloud
icd-prod-us-south-db-lm0sr.us-south.containers.appdomain.cloud canonical name = icd-prod-us-south-db-lm0sr.XXX.akadns.net
Name: icd-prod-us-south-db-lm0sr.XXX.akadns.net
Address: XXX.XX.XXX.XX
Why am I seeing egress connection failures from pods? 

Virtual Private Cloud Gen 2

Egress connections from pods in your Red Hat OpenShift on IBM Cloud version 4.11 are failing.

If you have a LoadBalancer in your cluster of type NLB with the annotation service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: "nlb" and are running Red Hat OpenShift on IBM Cloud version 4.11 earlier, then your pod-network pod egress connections might infrequently chose one of the NodePorts that your NLB is using, which creates a conflict and causes that egress connection to fail.

Update the pod natPortRange in Calico to 32768 - 65535 so it doesn't conflict with the NodePort range of 30000 - 32767.

Run the following command to patch the Calico natPortRange. This change takes effect immediately.

calicoctl patch felixconfiguration default --patch '{"spec":{"natPortRange": "32768:65535"}}'

Check whether the egress connections for pods are successful. If the issue persists, contact support. Open a [support case](https://cloud.ibm.com/docs/get-support?topic=get-support-using-avatar). In the case details, be sure to include any relevant log files, error messages, or command outputs.
* [Review namespace permissions](https://kubernetes.io/docs/tasks/configure-pod-container/migrate-from-psp/review-namespace-permissions)







 Step 3: Simplify and standardize PSPs 

Clean up your Pod Security configuration by following the steps in the external Kubernetes documentation. Do not modify or delete any of the following PSPs: ibm-privileged-psp, ibm-anyuid-psp, ibm-anyuid-hostpath-psp, ibm-anyuid-hostaccess-psp and ibm-restricted-psp.



* [Simplify & standardize PodSecurityPolicies](https://kubernetes.io/docs/tasks/configure-pod-container/migrate-from-psp/simplify-psps)







 Step 4: Update the namespaces in your cluster 

Update the namespaces in your cluster by following the steps in the external Kubernetes documentation. These steps must be performed on every namespace in the cluster that is not managed by IBM Cloud. Note the exceptions included in this section.

Do not delete or modify the Pod Security labels for the following namespaces that are managed by IBM Cloud: kube-system, ibm-system, ibm-operators.

In step 3.d. Bypass PodSecurity Policy of the external documentation linked in this step, do not create the suggested privileged PSP. If you create this additional PSP, it must be deleted before you upgrade to version 1.25, as detailed in the [upgrade requirements](https://cloud.ibm.com/docs/containers?topic=containers-pod-security-admission-migrationpsa-upgrade-reqs). Instead, use the following command to create a RoleBinding to the ibm-privileged-psp-user cluster role: kubectl create -n $NAMESPACE rolebinding disable-psp --clusterrole ibm-privileged-psp-user --group system:serviceaccounts:$NAMESPACE.
Debugging Block Storage for Classic failures 

Review the options to debug Block Storage for Classic and find the root causes of any failures.



 Checking whether the pod that mounts your storage instance is successfully deployed 

Follow the steps to review any error messages related to pod deployment.



1. List the pods in your cluster. A pod is successfully deployed if the pod shows a status of Running.

kubectl get pods
2. Get the details of your pod and review any error messages that are displayed in the Events section of your CLI output.

kubectl describe pod <pod_name>
3. Retrieve the logs for your pod and review any error messages.

kubectl logs <pod_name>
4. [Review the Block Storage for Classic troubleshooting documentation for steps to resolve common errors](https://cloud.ibm.com/docs/containers?topic=containers-cs_sitemapsitemap_block_storage).







 Restarting your app pod 

Some issues can be resolved by restarting and redeploying your pods. Follow the steps to redeploy a specific pod.



1. If your pod is part of a deployment, delete the pod and let the deployment rebuild it. If your pod is not part of a deployment, delete the pod and reapply your pod configuration file.



1. Delete the pod.

kubectl delete pod <pod_name>

Example output

pod "nginx" deleted
2. Reapply the configuration file to redeploy the pod.

kubectl apply -f <app.yaml>

Example output

pod/nginx created



2. If restarting your pod does not resolve the issue, [reload your worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-clics_worker_reload).
3. Verify that you use the latest IBM Cloud and IBM Cloud Kubernetes Service plug-in version.

ibmcloud update

ibmcloud plugin repo-plugins
* In spec.template.spec, add serviceAccount: <sa_name>.
* In spec.template.spec.containers, add securityContext: privileged: true.



Example

apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp_deployment
labels:
app: myapp
spec:
...
template:
...
spec:
serviceAccount: <sa_name>
containers:
- securityContext:
privileged: true
...
6. Deploy your app configuration file.

oc apply -f <filepath/deployment.yaml>
7. Verify that the pod is in a Running status. If your pod shows an error status or is stuck in one status for a long time, describe the pod and review the Events section to start troubleshooting your deployment.

oc get pods









 Understanding Kubernetes objects for apps 

With Kubernetes, you declare many types of objects in YAML configuration files such as pods, deployments, and jobs. These objects describe things like what containerized apps are running, what resources they use, and what policies manage their behavior for restarting, updating, replicating, and more. For more information, see the Kubernetes docs for [Configuration best practices](https://kubernetes.io/docs/concepts/configuration/overview/).



 I thought that I needed to put my app in a container. Now what's all this stuff about pods? 

A [pod](https://kubernetes.io/docs/concepts/workloads/pods/) is the smallest deployable unit that Kubernetes can manage. You put your container (or a group of containers) into a pod and use the pod configuration file to tell the pod how to run the container and share resources with other pods. All containers that you put into a pod run in a shared context, which means that they share the virtual or physical machine.

What to put in a container
If the port is not accessible, the service might not be listening on the correct port or the app might have issues. Update the configuration file for the service with the correct port and redeploy or investigate potential issues with the app.

curl localhost: <port>



6. Verify that the service is linked correctly to the pods.



1. Get the name of a pod.

oc get pods
2. Log in to a container.

oc exec -it <pod_name> -- /bin/bash
3. Curl the cluster IP address and port of the service.

curl <cluster_IP>:<port>
4. If the IP address and port are not accessible, look at the endpoints for the service.



* If no endpoints are listed, then the selector for the service does not match the pods. For example, your app deployment might have the label app=foo, but the service might have the selector run=foo.
* If endpoints are listed, then look at the target port field on the service and make sure that the target port is the same as what is being used for the pods. For example, your app might listen on port 9080, but the service might listen on port 80.





7. For Ingress services, verify that the service is accessible from within the cluster.



1. Get the name of a pod.

oc get pods
2. Log in to a container.

oc exec -it <pod_name> -- /bin/bash
3. Curl the URL specified for the Ingress service. If the URL is not accessible, check for a firewall issue between the cluster and the external endpoint.

curl <host_name>.<domain>
------------------------------------------------------------

Build time: 2021-05-14 12:02:31 UTC
Revision: 1ef1b260d39daacbf9357f9d8594a8a743e2152e

Kotlin: 1.4.31
Groovy: 3.0.7
Ant: Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM: 11.0.10 (Eclipse OpenJ9 openj9-0.24.0)
OS: Linux 5.10.42 amd64

 oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO
error: server took too long to respond with version information.

 zip
Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
This is Zip 3.0 (July 5th 2008), by Info-ZIP.

 unzip
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.

 git --version
git version 2.17.1

 curl
curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3

 wget
GNU Wget 1.19.4 built on linux-gnu.

 openssl version
OpenSSL 1.1.1 11 Sep 2018

 make
GNU Make 4.1

 docker
* If another application in a cluster with constrained resources is in a pending state, and is prioritized higher than the blockchain component, the blockchain pod cannot restart because the resources are consumed by the pending application.
* If the blockchain component needs to restart, but is on a worker node that has been "cordoned", meaning no new components can be brought to that node, it cannot restart. A blockchain pod becomes unschedulable when the Kubernetes scheduler is unable to find a node that can accommodate the resource requirements of the pod.
* If there is a timeout between the kubelet, the node agent that runs on each node, and the Kubernetes master node, a pod restart can fail.
* The pod restart itself can time out when its worker node restarts and takes too long to mount the persistent volume claim (PVC).



 How to fix it 

To enable the blockchain pod to restart successfully, you need to address the underlying cluster issues:



* Ensure there are adequate resources available on worker nodes in the cluster. If not, deploy a new worker node with increased resources.
* After you resolve the cluster issues, delete the failing blockchain pod. When Kubernetes restarts the pod, it will attempt to move the pod workload to another node with sufficient resources.
* If the timeout between a kubelet and the master node cannot be resolved, open a support ticket with your cluster provider.







 What is the proper way to clean up a failed node deployment? 

 What’s happening 

Sometimes a node can fail to deploy, for example, due to lack of resources in your Kubernetes cluster. After you understand the cause of the node deployment failure, you need to clean up the failed node in your cluster.

 How to fix it 

Do not attempt to use Kubernetes commands to remove the node. Instead, it is extremely important that you use the IBM Blockchain Platform console or the APIs to remove the failed node to ensure that the associated metadata and storage are also cleaned up.





 How can I view my smart contract container logs? 

 What’s happening 

You may need to view your smart contract, or chaincode, container logs to debug a smart contract issue.
appConfigClient.RegisterConfigurationUpdateListener(func () {
// add your code
// To find the effect of any configuration changes, you can call the feature or property related methods

// feature, err := appConfigClient.GetFeature("json-feature")
// newValue := feature.GetCurrentValue(entityID, entityAttributes)
})





 Fetch latest data 

appConfigClient.FetchConfigurations()
If your classic cluster has both private and public service endpoints enabled, but you don't have both VRF and Service Endpoint enabled in your account, don't update to 1.21. For more information, see [After upgrading my classic cluster to version 1.21, I'm finding connectivity issues](https://cloud.ibm.com/docs/containers?topic=containers-ts-network-classic121).



 Update before master 

The following table shows the actions that you must take before you update the Kubernetes master.



Changes to make before you update the master to Kubernetes 1.21

 Type Description 

 Unsupported: Kubernetes REST API export query parameter removed The export query parameter is removed from the Kubernetes REST API and now returns a 400 error status response. If you use this query parameter, remove it from your API requests. 
 Unsupported: Kubernetes external IP services The Kubernetes [DenyServiceExternalIPs admission controller](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/denyserviceexternalips) is enabled, which prevents creating or updating Kubernetes external IP services. If you use Kubernetes external IP services, migrate to a supported service type. For more information, see the [IBM security bulletin](https://www.ibm.com/support/pages/node/6428013). 
 Kubernetes snapshot CRDs IBM Cloud Kubernetes Service installs Kubernetes snapshot custom resource definition (CRD) version v1beta1. If you use other Kubernetes snapshot CRD versions v1 or v1alpha1, you must change the version to v1beta1. To check the currently installed version of your snapshot CRDs, run grep snapshot.storage.k8s.io <<(kubectl get apiservices).
* [Restarting your app pod](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs_restart)
* [Verifying that the storage driver and plug-in pods show a status of Running](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs_driver_plugin)
* [Checking and updating the oc CLI version](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs_cli)
* [Debugging your ODF resources](https://cloud.ibm.com/docs/openshift?topic=openshift-debug_storage_ocsdebug_storage_ocs_cluster)



[Why is the status of my OpenShift Data Foundation storage cluster stuck at Failed to reconcile.](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-ocs-roks-debugts-ocs-roks-debug)



* [ODF device set creation fails due to PVC names exceeding the Kubernetes character limit](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-ocs-roks-debugocs-ts-sc-character-limit)



[Why are the ODF pods stuck at Pending?](https://cloud.ibm.com/docs/openshift?topic=openshift-ts-ocs-pods-pending-statusts-ocs-pods-pending-status)

[Why is the status of my OpenShift Data Foundation storage cluster stuck at Progressing?](https://cloud.ibm.com/docs/openshift?topic=openshift-ocs-ts-error-progressingocs-ts-error-progressing)