Chapter 4. Enabling and disabling encryption in-transit post deployment You can enable encryption in-transit for the existing clusters after the deployment of clusters both in internal and external modes. 4.1. Enabling encryption in-transit after deployment in internal mode Prerequisites OpenShift Data Foundation is deployed and a storage cluster is created. Procedure Patch the storagecluster to add encryption enabled as true to the storage cluster spec: Check the configurations. Wait for around 10 minutes for ceph daemons to restart and then check the pods. Remount existing volumes. Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume." 4.2. Disabling encryption in-transit after deployment in internal mode Prerequisites OpenShift Data Foundation is deployed and a storage cluster is created. Encryption in-transit is enabled. Procedure Patch the storagecluster to update encryption enabled as false in the storage cluster spec: Check the configurations. Wait for around 10 minutes for ceph daemons to restart and then check the pods. Remount existing volumes. Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume." 4.3. Enabling encryption in-transit after deployment in external mode Prerequisites OpenShift Data Foundation is deployed and a storage cluster is created. Procedure Patch the storagecluster to add encryption enabled as true the storage cluster spec: Check the connection settings in the CR. 4.3.1. Applying encryption in-transit on Red Hat Ceph Storage cluster Procedure Apply Encryption in-transit settings. Check the settings. Restart all Ceph daemons. Wait for the restarting of all the daemons. 4.3.2. Remount existing volumes. Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume. 4.4. Disabling encryption in-transit after deployment in external mode Prerequisites OpenShift Data Foundation is deployed and a storage cluster is created. Encryption in-transit is enabled for the external mode cluster. Procedure Removing encryption in-transit settings from Red Hat Ceph Storage cluster Remove and check encryption in-transit configurations. Restart all Ceph daemons. Patching the CR Patch the storagecluster to update encryption enabled as false in the storage cluster spec: Check the configurations. Remount existing volumes Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume.

Commands:
oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": true}}} }]' storagecluster.ocs.openshift.io/ocs-storagecluster patched
oc get storagecluster ocs-storagecluster -n openshift-storage -o yaml | yq '.spec.network' connections: encryption: enabled: true
oc get pods -n openshift-storage | grep rook-ceph rook-ceph-crashcollector-ip-10-0-2-111.ec2.internal-796ffcm9kn9 1/1 Running 0 5m11s rook-ceph-crashcollector-ip-10-0-27-61.ec2.internal-854b4d8sk5z 1/1 Running 0 5m9s rook-ceph-crashcollector-ip-10-0-33-53.ec2.internal-589d9f4f8vx 1/1 Running 0 5m7s rook-ceph-exporter-ip-10-0-2-111.ec2.internal-6d48cdc5fd-2tmsl 1/1 Running 0 5m9s rook-ceph-exporter-ip-10-0-27-61.ec2.internal-546c66c7cc-9lnpz 1/1 Running 0 5m7s rook-ceph-exporter-ip-10-0-33-53.ec2.internal-b5555994c-x8mzz 1/1 Running 0 5m5s rook-ceph-mds-ocs-storagecluster-cephfilesystem-a-7bd754f6vwps2 2/2 Running 0 4m56s rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-6cc5cc647c78m 2/2 Running 0 4m30s rook-ceph-mgr-a-6f8467578d-f8279 3/3 Running 0 3m40s rook-ceph-mgr-b-66754d99cf-9q58g 3/3 Running 0 3m27s rook-ceph-mon-a-75bc5dd655-tvdqf 2/2 Running 0 4m7s rook-ceph-mon-b-6b6d4d9b4c-tjbpz 2/2 Running 0 4m55s rook-ceph-mon-c-7456bb5f67-rtwpj 2/2 Running 0 4m32s rook-ceph-operator-7b5b9cdb9b-tvmb6 1/1 Running 0 45m rook-ceph-osd-0-b78dd99f6-n4wbm 2/2 Running 0 3m3s rook-ceph-osd-1-5887bf6d8d-2sncc 2/2 Running 0 2m39s rook-ceph-osd-2-784b59c4c8-44phh 2/2 Running 0 2m14s rook-ceph-osd-prepare-a075cf185c9b2e5d92ec3f7769565e38-ztrms 0/1 Completed 0 42m rook-ceph-osd-prepare-b4b48dc5e3bef99ab377e2a255a9142a-mvgnd 0/1 Completed 0 42m rook-ceph-osd-prepare-fae2ea2ad4aacbf62010ae5b60b87f57-6t9l5 0/1 Completed 0 42m
oc get storagecluster -n openshift-storage NAME AGE PHASE EXTERNAL CREATED AT VERSION ocs-storagecluster 27m Ready 2024-11-06T16:15:26Z 4.18.0
~ USD oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": false}}} }]' storagecluster.ocs.openshift.io/ocs-storagecluster patched
oc get storagecluster ocs-storagecluster -n openshift-storage -o yaml | yq '.spec.network' connections: encryption: enabled: false
oc get pods -n openshift-storage | grep rook-ceph rook-ceph-crashcollector-ip-10-0-2-111.ec2.internal-796ffcm9kn9 1/1 Running 0 5m11s rook-ceph-crashcollector-ip-10-0-27-61.ec2.internal-854b4d8sk5z 1/1 Running 0 5m9s rook-ceph-crashcollector-ip-10-0-33-53.ec2.internal-589d9f4f8vx 1/1 Running 0 5m7s rook-ceph-exporter-ip-10-0-2-111.ec2.internal-6d48cdc5fd-2tmsl 1/1 Running 0 5m9s rook-ceph-exporter-ip-10-0-27-61.ec2.internal-546c66c7cc-9lnpz 1/1 Running 0 5m7s rook-ceph-exporter-ip-10-0-33-53.ec2.internal-b5555994c-x8mzz 1/1 Running 0 5m5s rook-ceph-mds-ocs-storagecluster-cephfilesystem-a-7bd754f6vwps2 2/2 Running 0 4m56s rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-6cc5cc647c78m 2/2 Running 0 4m30s rook-ceph-mgr-a-6f8467578d-f8279 3/3 Running 0 3m40s rook-ceph-mgr-b-66754d99cf-9q58g 3/3 Running 0 3m27s rook-ceph-mon-a-75bc5dd655-tvdqf 2/2 Running 0 4m7s rook-ceph-mon-b-6b6d4d9b4c-tjbpz 2/2 Running 0 4m55s rook-ceph-mon-c-7456bb5f67-rtwpj 2/2 Running 0 4m32s rook-ceph-operator-7b5b9cdb9b-tvmb6 1/1 Running 0 45m rook-ceph-osd-0-b78dd99f6-n4wbm 2/2 Running 0 3m3s rook-ceph-osd-1-5887bf6d8d-2sncc 2/2 Running 0 2m39s rook-ceph-osd-2-784b59c4c8-44phh 2/2 Running 0 2m14s rook-ceph-osd-prepare-a075cf185c9b2e5d92ec3f7769565e38-ztrms 0/1 Completed 0 42m rook-ceph-osd-prepare-b4b48dc5e3bef99ab377e2a255a9142a-mvgnd 0/1 Completed 0 42m rook-ceph-osd-prepare-fae2ea2ad4aacbf62010ae5b60b87f57-6t9l5 0/1 Completed 0 42m
oc get storagecluster -n openshift-storage NAME AGE PHASE EXTERNAL CREATED AT VERSION ocs-storagecluster 27m Ready 2024-11-06T16:15:26Z 4.18.0
oc patch storagecluster ocs-external-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": true}}} }]' storagecluster.ocs.openshift.io/ocs-external-storagecluster patched
get storagecluster NAME AGE PHASE EXTERNAL CREATED AT VERSION ocs-external-storagecluster 9h Ready true 2024-11-06T20:48:03Z 4.18.0
oc get storagecluster ocs-external-storagecluster -o yaml | yq '.spec.network.connections' encryption: enabled: true
root@ceph-client ~]# ceph config set global ms_client_mode secure ceph config set global ms_cluster_mode secure ceph config set global ms_service_mode secure ceph config set global rbd_default_map_options ms_mode=secure
ceph config dump | grep ms_ ceph config dump | grep ms_ global basic ms_client_mode secure * global basic ms_cluster_mode secure * global basic ms_service_mode secure * global advanced rbd_default_map_options ms_mode=secure *
ceph orch ls --format plain | tail -n +2 | awk '{print USD1}' | xargs -I {} ceph orch restart {} Scheduled to restart alertmanager.osd-0 on host 'osd-0' Scheduled to restart ceph-exporter.osd-0 on host 'osd-0' Scheduled to restart ceph-exporter.osd-2 on host 'osd-2' Scheduled to restart ceph-exporter.osd-3 on host 'osd-3' Scheduled to restart ceph-exporter.osd-1 on host 'osd-1' Scheduled to restart crash.osd-0 on host 'osd-0' Scheduled to restart crash.osd-2 on host 'osd-2' Scheduled to restart crash.osd-3 on host 'osd-3' Scheduled to restart crash.osd-1 on host 'osd-1' Scheduled to restart grafana.osd-0 on host 'osd-0' Scheduled to restart mds.fsvol001.osd-0.lpciqk on host 'osd-0' Scheduled to restart mds.fsvol001.osd-2.wocnxz on host 'osd-2' Scheduled to restart mgr.osd-0.dtkyni on host 'osd-0' Scheduled to restart mgr.osd-2.kqcxwu on host 'osd-2' Scheduled to restart mon.osd-2 on host 'osd-2' Scheduled to restart mon.osd-3 on host 'osd-3' Scheduled to restart mon.osd-1 on host 'osd-1' Scheduled to restart node-exporter.osd-0 on host 'osd-0' Scheduled to restart node-exporter.osd-2 on host 'osd-2' Scheduled to restart node-exporter.osd-3 on host 'osd-3' Scheduled to restart node-exporter.osd-1 on host 'osd-1' Scheduled to restart osd.1 on host 'osd-0' Scheduled to restart osd.4 on host 'osd-0' Scheduled to restart osd.0 on host 'osd-2' Scheduled to restart osd.5 on host 'osd-2' Scheduled to restart osd.2 on host 'osd-3' Scheduled to restart osd.6 on host 'osd-3' Scheduled to restart osd.3 on host 'osd-1' Scheduled to restart osd.7 on host 'osd-1' Scheduled to restart prometheus.osd-0 on host 'osd-0' Scheduled to restart rgw.rgw.ssl.osd-1.smzpfj on host 'osd-1'
ceph config rm global ms_client_mode ceph config rm global ms_cluster_mode ceph config rm global ms_service_mode ceph config rm global rbd_default_map_options ceph config dump | grep ms_
ceph orch ls --format plain | tail -n +2 | awk '{print USD1}' | xargs -I {} ceph orch restart {} Scheduled to restart alertmanager.osd-0 on host 'osd-0' Scheduled to restart ceph-exporter.osd-0 on host 'osd-0' Scheduled to restart ceph-exporter.osd-2 on host 'osd-2' Scheduled to restart ceph-exporter.osd-3 on host 'osd-3' Scheduled to restart ceph-exporter.osd-1 on host 'osd-1' Scheduled to restart crash.osd-0 on host 'osd-0' Scheduled to restart crash.osd-2 on host 'osd-2' Scheduled to restart crash.osd-3 on host 'osd-3' Scheduled to restart crash.osd-1 on host 'osd-1' Scheduled to restart grafana.osd-0 on host 'osd-0' Scheduled to restart mds.fsvol001.osd-0.lpciqk on host 'osd-0' Scheduled to restart mds.fsvol001.osd-2.wocnxz on host 'osd-2' Scheduled to restart mgr.osd-0.dtkyni on host 'osd-0' Scheduled to restart mgr.osd-2.kqcxwu on host 'osd-2' Scheduled to restart mon.osd-2 on host 'osd-2' Scheduled to restart mon.osd-3 on host 'osd-3' Scheduled to restart mon.osd-1 on host 'osd-1' Scheduled to restart node-exporter.osd-0 on host 'osd-0' Scheduled to restart node-exporter.osd-2 on host 'osd-2' Scheduled to restart node-exporter.osd-3 on host 'osd-3' Scheduled to restart node-exporter.osd-1 on host 'osd-1' Scheduled to restart osd.1 on host 'osd-0' Scheduled to restart osd.4 on host 'osd-0' Scheduled to restart osd.0 on host 'osd-2' Scheduled to restart osd.5 on host 'osd-2' Scheduled to restart osd.2 on host 'osd-3' Scheduled to restart osd.6 on host 'osd-3' Scheduled to restart osd.3 on host 'osd-1' Scheduled to restart osd.7 on host 'osd-1' Scheduled to restart prometheus.osd-0 on host 'osd-0' Scheduled to restart rgw.rgw.ssl.osd-1.smzpfj on host 'osd-1'
ceph orch ps NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID alertmanager.osd-0 osd-0 *:9093,9094 running (116s) 9s ago 10h 19.5M - 0.26.0 7dbf12091920 4694a72d4bbd ceph-exporter.osd-0 osd-0 running (19s) 9s ago 10h 7310k - 18.2.1-229.el9cp 3fd804e38f5b 49bdc7d99471 ceph-exporter.osd-1 osd-1 running (97s) 26s ago 10h 7285k - 18.2.1-229.el9cp 3fd804e38f5b 7000d59d23b4 ceph-exporter.osd-2 osd-2 running (76s) 26s ago 10h 7306k - 18.2.1-229.el9cp 3fd804e38f5b 3907515cc352 ceph-exporter.osd-3 osd-3 running (49s) 26s ago 10h 6971k - 18.2.1-229.el9cp 3fd804e38f5b 3f3952490780 crash.osd-0 osd-0 running (17s) 9s ago 10h 6878k - 18.2.1-229.el9cp 3fd804e38f5b 38e041fb86e3 crash.osd-1 osd-1 running (96s) 26s ago 10h 6895k - 18.2.1-229.el9cp 3fd804e38f5b 21ce3ef7d896 crash.osd-2 osd-2 running (74s) 26s ago 10h 6899k - 18.2.1-229.el9cp 3fd804e38f5b 210ca9c8d928 crash.osd-3 osd-3 running (47s) 26s ago 10h 6899k - 18.2.1-229.el9cp 3fd804e38f5b 710d42d9d138 grafana.osd-0 osd-0 *:3000 running (114s) 9s ago 10h 72.9M - 10.4.0-pre f142b583a1b1 3dc5e2248e95 mds.fsvol001.osd-0.qjntcu osd-0 running (99s) 9s ago 10h 17.5M - 18.2.1-229.el9cp 3fd804e38f5b 50efa881c04b mds.fsvol001.osd-2.qneujv osd-2 running (51s) 26s ago 10h 15.3M - 18.2.1-229.el9cp 3fd804e38f5b a306f2d2d676 mgr.osd-0.zukgyq osd-0 *:9283,8765,8443 running (21s) 9s ago 10h 442M - 18.2.1-229.el9cp 3fd804e38f5b 8ef9b728675e mgr.osd-1.jqfyal osd-1 *:8443,9283,8765 running (92s) 26s ago 10h 480M - 18.2.1-229.el9cp 3fd804e38f5b 1ab52db89bfd mon.osd-1 osd-1 running (90s) 26s ago 10h 41.7M 2048M 18.2.1-229.el9cp 3fd804e38f5b 88d1fe1e10ac mon.osd-2 osd-2 running (72s) 26s ago 10h 31.1M 2048M 18.2.1-229.el9cp 3fd804e38f5b 02f57d3bb44f mon.osd-3 osd-3 running (45s) 26s ago 10h 24.0M 2048M 18.2.1-229.el9cp 3fd804e38f5b 5e3783f2b4fa node-exporter.osd-0 osd-0 *:9100 running (15s) 9s ago 10h 7843k - 1.7.0 8c904aa522d0 2dae2127349b node-exporter.osd-1 osd-1 *:9100 running (94s) 26s ago 10h 11.2M - 1.7.0 8c904aa522d0 010c3fcd55cd node-exporter.osd-2 osd-2 *:9100 running (69s) 26s ago 10h 17.2M - 1.7.0 8c904aa522d0 436f2d513f31 node-exporter.osd-3 osd-3 *:9100 running (41s) 26s ago 10h 12.4M - 1.7.0 8c904aa522d0 5579f0d494b8 osd.0 osd-0 running (109s) 9s ago 10h 126M 4096M 18.2.1-229.el9cp 3fd804e38f5b 997076cd39d4 osd.1 osd-1 running (85s) 26s ago 10h 139M 4096M 18.2.1-229.el9cp 3fd804e38f5b 08b720f0587d osd.2 osd-2 running (65s) 26s ago 10h 143M 4096M 18.2.1-229.el9cp 3fd804e38f5b 104ad4227163 osd.3 osd-3 running (36s) 26s ago 10h 94.5M 1435M 18.2.1-229.el9cp 3fd804e38f5b db8b265d9f43 osd.4 osd-0 running (104s) 9s ago 10h 164M 4096M 18.2.1-229.el9cp 3fd804e38f5b 50dcbbf7e012 osd.5 osd-1 running (80s) 26s ago 10h 131M 4096M 18.2.1-229.el9cp 3fd804e38f5b 63b21fe970b5 osd.6 osd-3 running (32s) 26s ago 10h 243M 1435M 18.2.1-229.el9cp 3fd804e38f5b 26c7ba208489 osd.7 osd-2 running (61s) 26s ago 10h 130M 4096M 18.2.1-229.el9cp 3fd804e38f5b 871a2b75e64f prometheus.osd-0 osd-0 *:9095 running (12s) 9s ago 10h 44.6M - 2.48.0 58069186198d e49a064d2478 rgw.rgw.ssl.osd-1.bsmbgd osd-1 *:80 running (78s) 26s ago 10h 75.4M - 18.2.1-229.el9cp 3fd804e38f5b d03c9f7ae4a4
oc patch storagecluster ocs-external-storagecluster -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": false}}} }]' storagecluster.ocs.openshift.io/ocs-external-storagecluster patched
oc get storagecluster NAME AGE PHASE EXTERNAL CREATED AT VERSION ocs-external-storagecluster 12h Ready true 2024-11-06T20:48:03Z 4.18.0
oc get storagecluster ocs-external-storagecluster -o yaml | yq '.spec.network.connections' encryption: enabled: false