Chapter 24. OSD migration from FileStore to BlueStore After you complete and verify the upgrade process, you must migrate your FileStore OSDs to BlueStore. You must complete the migration one node at a time. The following procedure uses ceph-ansible to complete the migration. This procedure only applies if the Ceph cluster is deployed by director. 24.1. Checking that your cluster runs FileStore and therefore requires migration Procedure Log in as the heat-admin user on a node with Ceph MON containers, such as Controller nodes or standalone Ceph MON nodes. For example, in a standard overcloud deployment, overcloud-controller-1 uses Ceph MON containers. Query the Ceph cluster to see what driver is in use by the OSDs: If any line returns "objectstore": "filestore" , that node requires OSD migration. Warning The migration time can vary depending on the size of your cluster. If you have a very large cluster, the migration time is proportional to the number of OSDs in that cluster and the amount of data stored. Ensure that you complete the migration as soon as possible so that your environment is not in a mixed architecture scenario, which can impact performance. Warning Because managing FileStore-based OSDs with Red Hat Ceph Storage (RHCS) 4 versions of ceph-ansible is not supported, complete the migration before you run any stack updates. 24.2. Migrating OSDs from FileStore to BlueStore To migrate from FileStore to BlueStore, director uses Ansible to delete and recreate all OSDs on the node. Director performs a capacity check before the migration process. Finally, director redploys the OSDs with the BlueStore back end. Note You can expedite the migration from FileStore to BlueStore by increasing the throttling on backfill and recovery operations. For more information about throttling, see the Red Hat Knowledgebase article Ceph: Throttling down or up backfill and recovery and rebalance . Before beginning the throttling procedure, contact the Red Hat Ceph Storage team to ensure that the procedure is safe to use in your environment. Prerequisites A functional and running Red Hat Ceph Storage (RHCS) 4 cluster. You can check the cluster by entering the following command in a ceph MON container on a Controller or Standalone Ceph MON node: Procedure Ensure that osd_objectstore in the CephAnsibleDisksConfig parameter does not default to filestore . If the osd_objectstore parameter is present in any of your custom heat environment files, you must define the value bluestore explicitly or remove it: Note If you have any specific FileStore configuration with, for example, journals, ensure that you update the configuration accordingly. For more information about advanced configurations, see Mapping the Ceph Storage Node Disk Layout in the Deploying an overcloud with containerized Red Hat Ceph guide. Log in to the undercloud as the stack user. Enter the openstack overcloud external-upgrade run command with the ceph_fstobs tag. Replace <NODE_NAME> with the name of the Ceph OSD node you want to upgrade. You can use the openstack server list command to find the node name. Log in to a node that has Ceph MON services and query the Ceph cluster to check the status of the OSDs of the node you have upgraded. Before you can start the migration of the OSD node, you must ensure that the one you have upgraded is successfully migrated: Replace <NODE_NAME> with the name of the node that was migrated. If the result shows that the OSDs use BlueStore, its migration is successful. Optional: To view additional details about a specific OSD, enter the following command: Replace <ID> with the ID of the OSD you want to query. Before you can start the migration process on the node, you must wait for the cluster to synchronize. Before you can start the migration process on the node, you must wait for the cluster rebalancing process to complete. To follow the status, run the following command: Replace <NODE_NAME> with the name of the node that was migrated. Repeat the migration process for each node in the storage cluster. For more information about migration from FileStore to BlueStore, see BlueStore in the Red Hat Ceph Storage Administration Guide . 24.3. Verifying your FileStore to BlueStore migration You can check the status of an OSD to ensure that you have successfully completed the migration. Procedure Log in as the heat-admin user to a ceph-mon container that is hosted on one of the Controller nodes. Query the Ceph cluster: If the configuration shows that all the OSDs across the cluster use BlueStore, the migration is successful. Important A recommended best practice is to run an idempotent stack update to ensure that the configuration definition and the actual configuration match. The stack update duration varies depending on the size of your system, so to reduce downtime you can plan to complete the migration during a maintenance window.

Commands:
[heat-admin@overcloud-controller-1 ~]USD sudo -i podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -f json osd metadata" | jq -c 'sort_by(.hostname) | .[] | ["host", .hostname, "osd_id", .id, "objectstore", .osd_objectstore]'
podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -s"
parameter_defaults: CephAnsibleDisksConfig: devices: - /dev/sdb - /dev/sdc - /dev/sdd osd_scenario: lvm osd_objectstore: bluestore
[stack@undercloud ~] USD openstack overcloud external-upgrade run --tags ceph_fstobs -e ceph_ansible_limit=<NODE_NAME> | tee oc-fstobs.log
[heat-admin@overcloud-controller-1 ~]USD sudo -i podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -f json osd metadata" | jq -c '.[] | select(.hostname == "<NODE_NAME>") | ["host", .hostname, "osd_id", .id, "objectstore", .osd_objectstore]' exit
podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph osd metadata <ID>"
podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -s"
Review the command output and ensure that the health of the cluster is `HEALTH_OK` and the PGs are in the `active+clean` state.
[heat-admin@overcloud-controller-0 ~]USD sudo podman exec ceph-mon-<NODE_NAME> ceph -w
[heat-admin@overcloud-controller-1 ~]USD sudo -i podman exec -it ceph-mon-overcloud-controller-1 sh -c "ceph -f json osd metadata" | jq -c 'sort_by(.hostname) | .[] | ["host", .hostname, "osd_id", .id, "objectstore", .osd_objectstore]'