Chapter 3. Deploying Ceph services on dedicated nodes By default, director deploys the Ceph MON and Ceph MDS services on the Controller nodes. This is suitable for small deployments. However, with larger deployments, Red Hat recommends that you deploy the Ceph MON and Ceph MDS services on dedicated nodes to improve the performance of your Ceph cluster. Create a custom role for services that you want to isolate on dedicated nodes. Note For more information about custom roles, see Creating a New Role in the Advanced Overcloud Customization guide. Director uses the following file as a default reference for all overcloud roles: /usr/share/openstack-tripleo-heat-templates/roles_data.yaml 3.1. Creating a custom roles file To create a custom role file, complete the following steps: Procedure Make a copy of the roles_data.yaml file in /home/stack/templates/ so that you can add custom roles: Include the new custom role file in the openstack overcloud deploy command. 3.2. Creating a custom role and flavor for the Ceph MON service Create a custom role CephMon and flavor ceph-mon for the Ceph MON role. Prerequisites You must already have a copy of the default roles data file. For more information, see Chapter 3, Deploying Ceph services on dedicated nodes . Procedure Open the /home/stack/templates/roles_data_custom.yaml file. Remove the service entry for the Ceph MON service, OS::TripleO::Services::CephMon , from the Controller role. Add the OS::TripleO::Services::CephClient service to the Controller role: At the end of the roles_data_custom.yaml file, add a custom CephMon role that contains the Ceph MON service and all the other required node services: Enter the openstack flavor create command to define a new flavor named ceph-mon for the CephMon role: Note For more information about this command, enter: openstack flavor create --help . Map this flavor to a new profile, also named ceph-mon : Note For more information about this command, enter openstack flavor set --help . Tag nodes into the new ceph-mon profile: Add the following configuration to the node-info.yaml file to associate the ceph-mon flavor with the CephMon role: For more information about tagging nodes, see Section 2.4, "Manually tagging nodes into profiles" . For more information about custom role profiles, see Tagging Nodes Into Profiles . 3.3. Creating a custom role and flavor for the Ceph MDS service Complete the following steps to create a custom role CephMDS and flavor ceph-mds for the Ceph MDS role. You must already have a copy of the default roles data file as described in Chapter 3, Deploying Ceph services on dedicated nodes . Procedure Open the /home/stack/templates/roles_data_custom.yaml file. Remove the service entry for the Ceph MDS service, OS::TripleO::Services::CephMds , from the Controller role: 1 Comment out this line. In the step, you add this service to the new custom role. At the end of the roles_data_custom.yaml file, add a custom CephMDS role that contains the Ceph MDS service and all the other required node services: 1 The Ceph MDS service requires the admin keyring, which you can set with either the Ceph MON or Ceph Client service. If you deploy Ceph MDS on a dedicated node without the Ceph MON service, you must also include the Ceph Client service in the new CephMDS role. Enter the openstack flavor create command to define a new flavor named ceph-mds for this role: Note For more information about this command, enter openstack flavor create --help . Map the new ceph-mds flavor to a new profile, also named ceph-mds : Note For more information about this command, enter openstack flavor set --help . Tag nodes into the new ceph-mds profile: For more information about tagging nodes, see Section 2.4, "Manually tagging nodes into profiles" . For more information about custom role profiles, see Tagging Nodes Into Profiles .

Commands:
cp /usr/share/openstack-tripleo-heat-templates/roles_data.yaml /home/stack/templates/roles_data_custom.yaml
[...] - name: Controller # the 'primary' role goes first CountDefault: 1 ServicesDefault: - OS::TripleO::Services::CACerts - OS::TripleO::Services::CephMds - OS::TripleO::Services::CephClient - OS::TripleO::Services::CephExternal - OS::TripleO::Services::CephRbdMirror - OS::TripleO::Services::CephRgw - OS::TripleO::Services::CinderApi [...]
- name: CephMon ServicesDefault: # Common Services - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned # Role-Specific Services - OS::TripleO::Services::CephMon
openstack flavor create --id auto --ram 6144 --disk 40 --vcpus 4 ceph-mon
openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="ceph-mon" ceph-mon
openstack baremetal node set --property capabilities=' profile:ceph-mon ,boot_option:local' UUID
parameter_defaults: OvercloudCephMonFlavor: CephMon CephMonCount: 3
[...] - name: Controller # the 'primary' role goes first CountDefault: 1 ServicesDefault: - OS::TripleO::Services::CACerts # - OS::TripleO::Services::CephMds 1 - OS::TripleO::Services::CephMon - OS::TripleO::Services::CephExternal - OS::TripleO::Services::CephRbdMirror - OS::TripleO::Services::CephRgw - OS::TripleO::Services::CinderApi [...]
- name: CephMDS ServicesDefault: # Common Services - OS::TripleO::Services::AuditD - OS::TripleO::Services::CACerts - OS::TripleO::Services::CertmongerUser - OS::TripleO::Services::Collectd - OS::TripleO::Services::Docker - OS::TripleO::Services::FluentdClient - OS::TripleO::Services::Kernel - OS::TripleO::Services::Ntp - OS::TripleO::Services::ContainersLogrotateCrond - OS::TripleO::Services::SensuClient - OS::TripleO::Services::Snmp - OS::TripleO::Services::Timezone - OS::TripleO::Services::TripleoFirewall - OS::TripleO::Services::TripleoPackages - OS::TripleO::Services::Tuned # Role-Specific Services - OS::TripleO::Services::CephMds - OS::TripleO::Services::CephClient 1
openstack flavor create --id auto --ram 6144 --disk 40 --vcpus 4 ceph-mds
openstack flavor set --property "cpu_arch"="x86_64" --property "capabilities:boot_option"="local" --property "capabilities:profile"="ceph-mds" ceph-mds
openstack baremetal node set --property capabilities=' profile:ceph-mds ,boot_option:local' UUID