Chapter 3. Commonly required logs for troubleshooting Some of the commonly used logs for troubleshooting OpenShift Data Foundation are listed, along with the commands to generate them. Generating logs for a specific pod: Generating logs for Ceph or OpenShift Data Foundation cluster: Important Currently, the rook-ceph-operator logs do not provide any information about the failure and this acts as a limitation in troubleshooting issues, see Enabling and disabling debug logs for rook-ceph-operator . Generating logs for plugin pods like cephfs or rbd to detect any problem in the PVC mount of the app-pod: To generate logs for all the containers in the CSI pod: Generating logs for cephfs or rbd provisioner pods to detect problems if PVC is not in BOUND state: To generate logs for all the containers in the CSI pod: Generating OpenShift Data Foundation logs using cluster-info command: When using Local Storage Operator, generating logs can be done using cluster-info command: Check the OpenShift Data Foundation operator logs and events. To check the operator logs : <ocs-operator> To check the operator events : Get the OpenShift Data Foundation operator version and channel. Example output : Example output : Confirm that the installplan is created. Verify the image of the components post updating OpenShift Data Foundation. Check the node on which the pod of the component you want to verify the image is running. For Example : Example output: dell-r440-12.gsslab.pnq2.redhat.com is the node-name . Check the image ID. <node-name> Is the name of the node on which the pod of the component you want to verify the image is running. For Example : Take a note of the IMAGEID and map it to the Digest ID on the Rook Ceph Operator page. Additional resources Using must-gather 3.1. Adjusting verbosity level of logs The amount of space consumed by debugging logs can become a significant issue. Red Hat OpenShift Data Foundation offers a method to adjust, and therefore control, the amount of storage to be consumed by debugging logs. In order to adjust the verbosity levels of debugging logs, you can tune the log levels of the containers responsible for container storage interface (CSI) operations. In the container's yaml file, adjust the following parameters to set the logging levels: CSI_LOG_LEVEL - defaults to 5 CSI_SIDECAR_LOG_LEVEL - defaults to 1 The supported values are 0 through 5 . Use 0 for general useful logs, and 5 for trace level verbosity.

Commands:
oc logs <pod-name> -n <namespace>
oc logs rook-ceph-operator-<ID> -n openshift-storage
oc logs csi-cephfsplugin-<ID> -n openshift-storage -c csi-cephfsplugin
oc logs csi-rbdplugin-<ID> -n openshift-storage -c csi-rbdplugin
oc logs csi-cephfsplugin-<ID> -n openshift-storage --all-containers
oc logs csi-rbdplugin-<ID> -n openshift-storage --all-containers
oc logs csi-cephfsplugin-provisioner-<ID> -n openshift-storage -c csi-cephfsplugin
oc logs csi-rbdplugin-provisioner-<ID> -n openshift-storage -c csi-rbdplugin
oc logs csi-cephfsplugin-provisioner-<ID> -n openshift-storage --all-containers
oc logs csi-rbdplugin-provisioner-<ID> -n openshift-storage --all-containers
oc cluster-info dump -n openshift-storage --output-directory=<directory-name>
oc cluster-info dump -n openshift-local-storage --output-directory=<directory-name>
oc logs <ocs-operator> -n openshift-storage
oc get pods -n openshift-storage | grep -i "ocs-operator" | awk '{print USD1}'
oc get events --sort-by=metadata.creationTimestamp -n openshift-storage
oc get csv -n openshift-storage
NAME DISPLAY VERSION REPLACES PHASE mcg-operator.v4.14.0 NooBaa Operator 4.14.0 Succeeded ocs-operator.v4.14.0 OpenShift Container Storage 4.14.0 Succeeded odf-csi-addons-operator.v4.14.0 CSI Addons 4.14.0 Succeeded odf-operator.v4.14.0 OpenShift Data Foundation 4.14.0 Succeeded
oc get subs -n openshift-storage
NAME PACKAGE SOURCE CHANNEL mcg-operator-stable-4.14-redhat-operators-openshift-marketplace mcg-operator redhat-operators stable-4.14 ocs-operator-stable-4.14-redhat-operators-openshift-marketplace ocs-operator redhat-operators stable-4.14 odf-csi-addons-operator odf-csi-addons-operator redhat-operators stable-4.14 odf-operator odf-operator redhat-operators stable-4.14
oc get installplan -n openshift-storage
oc get pods -o wide | grep <component-name>
oc get pods -o wide | grep rook-ceph-operator
rook-ceph-operator-566cc677fd-bjqnb 1/1 Running 20 4h6m 10.128.2.5 rook-ceph-operator-566cc677fd-bjqnb 1/1 Running 20 4h6m 10.128.2.5 dell-r440-12.gsslab.pnq2.redhat.com <none> <none> <none> <none>
oc debug node/<node name>
chroot /host
crictl images | grep <component>
crictl images | grep rook-ceph