Chapter 3. Deploy OpenShift Data Foundation using IBM FlashSystem OpenShift Data Foundation can use IBM FlashSystem storage available for consumption through OpenShift Container Platform clusters. You need to install the OpenShift Data Foundation operator and then create an OpenShift Data Foundation cluster for IBM FlashSystem storage. 3.1. Installing Red Hat OpenShift Data Foundation Operator You can install Red Hat OpenShift Data Foundation Operator using the Red Hat OpenShift Container Platform Operator Hub. Prerequisites Access to an OpenShift Container Platform cluster using an account with cluster-admin and operator installation permissions. For additional resource requirements, see the Planning your deployment guide. Important When you need to override the cluster-wide default node selector for OpenShift Data Foundation, you can use the following command to specify a blank node selector for the openshift-storage namespace (create openshift-storage namespace in this case): Procedure Log in to the OpenShift Web Console. Click Operators OperatorHub . Scroll or type OpenShift Data Foundation into the Filter by keyword box to find the OpenShift Data Foundation Operator. Click Install . Set the following options on the Install Operator page: Update Channel as stable-4.16 . Installation Mode as A specific namespace on the cluster . Installed Namespace as Operator recommended namespace openshift-storage . If Namespace openshift-storage does not exist, it is created during the operator installation. Select Approval Strategy as Automatic or Manual . If you select Automatic updates, then the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention. If you select Manual updates, then the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator to a newer version. Ensure that the Enable option is selected for the Console plugin . Click Install . Verification steps After the operator is successfully installed, a pop-up with a message, Web console update is available appears on the user interface. Click Refresh web console from this pop-up for the console changes to reflect. In the Web Console: Navigate to Installed Operators and verify that the OpenShift Data Foundation Operator shows a green tick indicating successful installation. Navigate to Storage and verify if the Data Foundation dashboard is available. 3.2. Creating an OpenShift Data Foundation Cluster for external IBM FlashSystem storage You need to create a new OpenShift Data Foundation cluster after you install the OpenShift Data Foundation operator on the OpenShift Container Platform. Prerequisites A valid Red Hat OpenShift Data Foundation Advanced subscription. For more information, see the knowledgebase article on OpenShift Data Foundation subscriptions . For Red Hat Enterprise Linux(R) operating system, ensure that there is iSCSI connectivity and then configure Linux multipath devices on the host. For Red Hat Enterprise Linux CoreOS or when the packages are already installed, configure Linux multipath devices on the host. Ensure to configure each worker with storage connectivity according to your storage system instructions. For the latest supported FlashSystem storage systems and versions, see IBM ODF FlashSystem driver documentation . Procedure In the OpenShift Web Console, click Operators Installed Operators to view all the installed operators. Ensure that the Project selected is openshift-storage . Click OpenShift Data Foundation and then click Create StorageSystem . In the Backing storage page, select the following options: Select Full deployment for the Deployment type option. Select Connect an external storage platform from the available options. Select IBM FlashSystem Storage from the Storage platform list. Click . In the Create storage class page, provide the following information: Enter a name for the storage class. When creating block storage persistent volumes, select the storage class <storage_class_name> for best performance. The storage class allows direct I/O path to the FlashSystem. Enter the following details of IBM FlashSystem connection: IP address User name Password Pool name Select thick or thin for the Volume mode . Click . In the Capacity and nodes page, provide the necessary details: Select a value for Requested capacity. The available options are 0.5 TiB , 2 TiB , and 4 TiB . The requested capacity is dynamically allocated on the infrastructure storage class. Select at least three nodes in three different zones. It is recommended to start with at least 14 CPUs and 34 GiB of RAM per node. If the nodes selected do not match the OpenShift Data Foundation cluster requirement of an aggregated 30 CPUs and 72 GiB of RAM, a minimal cluster will be deployed. For minimum starting node requirements, see the Resource requirements section in the Planning guide. Click . Optional: In the Security and network page, provide the necessary details: To enable encryption, select Enable data encryption for block and file storage . Choose any one or both Encryption level: Cluster-wide encryption to encrypt the entire cluster (block and file). StorageClass encryption to create encrypted persistent volume (block only) using encryption enabled storage class. Select the Connect to an external key management service checkbox. This is optional for cluster-wide encryption. Key Management Service Provider is set to Vault by default. Enter Vault Service Name, host Address of Vault server ('https://<hostname or ip>'), Port number, and Token. Expand Advanced Settings to enter additional settings and certificate details based on your Vault configuration: Enter the Key Value secret path in the Backend Path that is dedicated and unique to OpenShift Data Foundation. Optional: Enter TLS Server Name and Vault Enterprise Namespace. Provide CA Certificate, Client Certificate, and Client Private Key by uploading the respective PEM encoded certificate file. Click Save . Select Default (SDN) if you are using a single network or Custom (Multus) if you are using multiple network interfaces. Select a Public Network Interface from the dropdown. Select a Cluster Network Interface from the dropdown. NOTE: If you are using only one additional network interface, select the single NetworkAttachementDefinition , that is, ocs-public-cluster for the Public Network Interface, and leave the Cluster Network Interface blank. Click . To enable in-transit encryption, select In-transit encryption . Select a Network . Click . In the Review and create page, review if all the details are correct: To modify any configuration settings, click Back to go back to the configuration page. Click Create StorageSystem . Verification Steps Verifying the state of the pods Click Workloads Pods from the left pane of the OpenShift Web Console. Select openshift-storage from the Project drop-down list. Note If the Show default projects option is disabled, use the toggle button to list all the default projects. Table 3.1. Pods corresponding to OpenShift Data Foundation components Component Corresponding pods OpenShift Data Foundation Operator ocs-operator-* (1 pod on any worker node) ocs-metrics-exporter-* (1 pod on any worker node) odf-operator-controller-manager-* (1 pod on any worker node) odf-console-* (1 pod on any worker node) csi-addons-controller-manager-* (1 pod on any worker node) ibm-storage-odf-operator ibm-storage-odf-operator-* (2 pods on any worker nodes) ibm-odf-console-* ibm-flashsystem-storage ibm-flashsystem-storage-* (1 pod on any worker node) rook-ceph Operator rook-ceph-operator-* (1 pod on any worker node) Multicloud Object Gateway noobaa-operator-* (1 pod on any worker node) noobaa-core-* (1 pod on any worker node) noobaa-db-pg-* (1 pod on any worker node) noobaa-endpoint-* (1 pod on any worker node) CSI ibm-block-csi-* (1 pod on any worker node) Verifying that the OpenShift Data Foundation cluster is healthy In the Web Console, click Storage Data Foundation . In the Status card of the Overview tab, verify that the Storage System has a green tick mark. In the Details card, verify that the cluster information is displayed. For more information on the health of the OpenShift Data Foundation cluster using the Block and File dashboard, see Monitoring OpenShift Data Foundation . Verifying that the Multicloud Object Gateway is healthy In the Web Console, click Storage Data Foundation . In the Status card of the Overview tab, click Storage System and then click the storage system link from the pop up. In the Status card of the Object tab, verify that both Object Service and Data Resiliency have a green tick. In the Details card, verify that the MCG information is displayed. For more information on the health of the OpenShift Data Foundation cluster using the object dashboard, see Monitoring OpenShift Data Foundation . Verifying that IBM FlashSystem is connected and the storage cluster is ready Run the following command to verify if the OpenShift Data Foundation cluster is connected to the external IBM FlashSystem. Verifying the StorageSystem of the storage Run the following command to verify the storageSystem of IBM FlashSystem storage cluster. Verifying the subscription of the IBM operator Run the following command to verify the subscription: Verifying the CSVs Run the following command to verify that the CSVs are in the succeeded state. Verifying the IBM operator and CSI pods Run the following command to verify the IBM operator and CSI pods:

Commands:
oc annotate namespace openshift-storage openshift.io/node-selector=
oc get flashsystemclusters.odf.ibm.com NAME AGE PHASE CREATED AT ibm-flashsystemcluster 35s 2021-09-23T07:44:52Z
oc get storagesystems.odf.openshift.io NAME STORAGE-SYSTEM-KIND STORAGE-SYSTEM-NAME ibm-flashsystemcluster-storagesystem flashsystemcluster.odf.ibm.com/v1alpha1 ibm-flashsystemcluster ocs-storagecluster-storagesystem storagecluster.ocs.openshift.io/v1 ocs-storagecluster
oc get subscriptions.operators.coreos.com NAME PACKAGE SOURCE CHANNEL ibm-block-csi-operator-stable-certified-operators-openshift-marketplace ibm-block-csi-operator certified-operators stable ibm-storage-odf-operator ibm-storage-odf-operator odf-catalogsource stable-v1 noobaa-operator-alpha-odf-catalogsource-openshift-storage noobaa-operator odf-catalogsource alpha ocs-operator-alpha-odf-catalogsource-openshift-storage ocs-operator odf-catalogsource alpha odf-operator odf-operator odf-catalogsource alpha
oc get csv NAME DISPLAY VERSION REPLACES PHASE ibm-block-csi-operator.v1.6.0 Operator for IBM block storage CSI driver 1.6.0 ibm-block-csi-operator.v1.5.0 Succeeded ibm-storage-odf-operator.v0.2.1 IBM Storage ODF operator 0.2.1 Installing noobaa-operator.v5.9.0 NooBaa Operator 5.9.0 Succeeded ocs-operator.v4.15.0 OpenShift Container Storage 4.15.0 Succeeded odf-operator.v4.15.0 OpenShift Data Foundation 4.15.0 Succeeded
oc get pods NAME READY STATUS RESTARTS AGE 5cb2b16ec2b11bf63dbe691d44a63535dc026bb5315d5075dc6c398b3c58l94 0/1 Completed 0 10m 7c806f6568f85cf10d72508261a2535c220429b54dbcf87349b9b4b9838fctg 0/1 Completed 0 8m47s c4b05566c04876677a22d39fc9c02512401d0962109610e85c8fb900d3jd7k2 0/1 Completed 0 10m c5d1376974666727b02bf25b3a4828241612186744ef417a668b4bc1759rzts 0/1 Completed 0 10m ibm-block-csi-operator-7b656d6cc8-bqnwp 1/1 Running 0 8m3s ibm-odf-console-97cb7c84c-r52dq 0/1 ContainerCreating 0 8m4s ibm-storage-odf-operator-57b8bc47df-mgkc7 1/2 ImagePullBackOff 0 94s noobaa-operator-7698579d56-x2zqs 1/1 Running 0 9m37s ocs-metrics-exporter-94b57d764-zq2g2 1/1 Running 0 9m32s ocs-operator-5d96d778f6-vxlq5 1/1 Running 0 9m33s odf-catalogsource-j7q72 1/1 Running 0 10m odf-console-8987868cd-m7v29 1/1 Running 0 9m35s odf-operator-controller-manager-5dbf785564-rwsgq 2/2 Running 0 9m35s rook-ceph-operator-68b4b976d8-dlc6w 1/1 Running 0 9m32s