Chapter 2. Management of services using the Ceph Orchestrator As a storage administrator, after installing the Red Hat Ceph Storage cluster, you can monitor and manage the services in a storage cluster using the Ceph Orchestrator. A service is a group of daemons that are configured together. This section covers the following administrative information: Checking service status . Checking daemon status . Placement specification of the Ceph Orchestrator . Deploying the Ceph daemons using the command line interface . Deploying the Ceph daemons on a subset of hosts using the command line interface . Service specification of the Ceph Orchestrator . Deploying the Ceph daemons using the service specification . 2.1. Checking service status You can check the following status of the services of the Red Hat Ceph Storage cluster using the ceph orch ls command: Print a list of services. Locate the service whose status you want to check. Print the status of the service. Note If the services are applied with the ceph orch apply command while bootstrapping, changing the service specification file is complicated. Instead, you can use the --export option with the ceph orch ls command to export the running specification, update the yaml file, and re-apply the service. Prerequisites A running Red Hat Ceph Storage cluster. Log into the cephadm shell. Procedure Print a list of services: Syntax The format can be plain, json , json-pretty , yaml , xml-pretty , or xml . Example Check the status of a particular service or a daemon: Syntax Example Export the service specification: Example This exports the file in the .yaml file format. This file can be used with the ceph orch apply -i command for retrieving the service specification of a single service. 2.2. Checking daemon status A daemon is a systemd unit that is running and is part of the service. You can check the following status of the daemons of the Red Hat Ceph Storage cluster using the ceph orch ps command: Print a list of all the daemons. Query the status of the target daemon. Prerequisites A running Red Hat Ceph Storage cluster. Log into the cephadm shell. Procedure Print a list of daemons: Syntax Example Check the status of a particular service instance: Syntax Example 2.3. Placement specification of the Ceph Orchestrator You can use the Ceph Orchestrator to deploy osds , mons , mgrs , mds and rgw , and iSCSI services. Red Hat recommends deploying services using placement specifications. You need to know where and how many daemons have to be deployed to deploy a service using the Ceph Orchestrator. Placement specifications can either be passed as command line arguments or as a service specification in a yaml file. There are two ways of deploying the services using the placement specification: Using the placement specification directly in the command line interface. For example, if you want to deploy three monitors on the hosts, running the following command deploys three monitors on host01 , host02 , and host03 . Example Using the placement specification in the YAML file. For example, if you want to deploy node-exporter on all the hosts, then you can specify the following in the yaml file. Example 2.4. Deploying the Ceph daemons using the command line interface Using the Ceph Orchestrator, you can deploy the daemons such as Ceph Manager, Ceph Monitors, Ceph OSDs, monitoring stack, and others using the ceph orch command. Placement specification is passed as --placement argument with the Orchestrator commands. Prerequisites A running Red Hat Ceph Storage cluster. Hosts are added to the storage cluster. Procedure Log into the Cephadm shell: Example Use one of the following methods to deploy the daemons on the hosts: Method 1 : Specify the number of daemons and the host names: Syntax Example Method 2 : Add the labels to the hosts and then deploy the daemons using the labels: Add the labels to the hosts: Syntax Example Deploy the daemons with labels: Syntax Example Method 3 : Add the labels to the hosts and deploy using the --placement argument: Add the labels to the hosts: Syntax Example Deploy the daemons using the label placement specification: Syntax Example Verification List the service: Example List the hosts, daemons, and processes: Syntax Example Additional Resources See the Adding hosts using the Ceph Orchestrator section in the Red Hat Ceph Storage Operations Guide . 2.5. Deploying the Ceph daemons on a subset of hosts using the command line interface You can use the --placement option to deploy daemons on a subset of hosts. You can specify the number of daemons in the placement specification with the name of the hosts to deploy the daemons. Prerequisites A running Red Hat Ceph Storage cluster. Hosts are added to the cluster. Procedure Log into the Cephadm shell: Example List the hosts on which you want to deploy the Ceph daemons: Example Deploy the daemons: Syntax Example In this example, the mgr daemons are deployed only on two hosts. Verification List the hosts: Example Additional Resources See the Listing hosts using the Ceph Orchestrator section in the Red Hat Ceph Storage Operations Guide . 2.6. Service specification of the Ceph Orchestrator A service specification is a data structure to specify the service attributes and configuration settings that is used to deploy the Ceph service. The following is an example of the multi-document YAML file, cluster.yaml , for specifying service specifications: Example The following list are the parameters where the properties of a service specification are defined as follows: service_type : The type of service: Ceph services like mon, crash, mds, mgr, osd, rbd, or rbd-mirror. Ceph gateway like nfs or rgw. Monitoring stack like Alertmanager, Prometheus, Grafana or Node-exporter. Container for custom containers. service_id : A unique name of the service. placement : This is used to define where and how to deploy the daemons. unmanaged : If set to true , the Orchestrator will neither deploy nor remove any daemon associated with this service. Stateless service of Orchestrators A stateless service is a service that does not need information of the state to be available. For example, to start an rgw service, additional information is not needed to start or run the service. The rgw service does not create information about this state in order to provide the functionality. Regardless of when the rgw service starts, the state is the same. 2.7. Deploying the Ceph daemons using the service specification Using the Ceph Orchestrator, you can deploy daemons such as ceph Manager, Ceph Monitors, Ceph OSDs, monitoring stack, and others using the service specification in a YAML file. Prerequisites A running Red Hat Ceph Storage cluster. Root-level access to all the nodes. Procedure Create the yaml file: Example This file can be configured in two different ways: Edit the file to include the host details in placement specification: Syntax Example Edit the file to include the label details in placement specification: Syntax Example Optional: You can also use extra container arguments in the service specification files such as CPUs, CA certificates, and other files while deploying services: Example Mount the YAML file under a directory in the container: Example Navigate to the directory: Example Deploy the Ceph daemons using service specification: Syntax Example Verification List the service: Example List the hosts, daemons, and processes: Syntax Example Additional Resources See the Listing hosts using the Ceph Orchestrator section in the Red Hat Ceph Storage Operations Guide .

Commands:
ceph orch ls [--service_type SERVICE_TYPE ] [--service_name SERVICE_NAME ] [--export] [--format FORMAT ] [--refresh]
ceph orch ls
ceph orch ls [--service_type SERVICE_TYPE ] [--service_name SERVICE_NAME ] [--refresh]
ceph orch ls --service-type mds ceph orch ls --service-name rgw.realm.myzone
ceph orch ls --service-type mgr --export > mgr.yaml ceph orch ls --export > cluster.yaml
ceph orch ps [--daemon-type DAEMON_TYPE ] [--service_name SERVICE_NAME ] [--daemon_id DAEMON_ID ] [--format FORMAT ] [--refresh]
ceph orch ps
ceph orch ls [--daemon-type DAEMON_TYPE ] [--daemon_id DAEMON_ID ] [--refresh]
ceph orch ps --daemon_type osd --daemon_id 0
ceph orch apply mon --placement="3 host01 host02 host03"
service_type: node-exporter placement: host_pattern: '*'
cephadm shell
ceph orch apply SERVICE_NAME --placement=" NUMBER_OF_DAEMONS HOST_NAME_1 HOST_NAME_2 HOST_NAME_3 "
ceph orch apply mon --placement="3 host01 host02 host03"
ceph orch host label add HOSTNAME_1 LABEL
ceph orch host label add host01 mon
ceph orch apply DAEMON_NAME label: LABEL
ceph orch apply mon label:mon
ceph orch host label add HOSTNAME_1 LABEL
ceph orch host label add host01 mon
ceph orch apply DAEMON_NAME --placement="label: LABEL "
ceph orch apply mon --placement="label:mon"
ceph orch ls
ceph orch ps --daemon_type= DAEMON_NAME ceph orch ps --service_name= SERVICE_NAME
ceph orch ps --daemon_type=mon ceph orch ps --service_name=mon
cephadm shell
ceph orch host ls
ceph orch apply SERVICE_NAME --placement=" NUMBER_OF_DAEMONS HOST_NAME_1 _HOST_NAME_2 HOST_NAME_3 "
ceph orch apply mgr --placement="2 host01 host02 host03"
ceph orch host ls
service_type: mon placement: host_pattern: "mon*" --- service_type: mgr placement: host_pattern: "mgr*" --- service_type: osd service_id: default_drive_group placement: host_pattern: "osd*" data_devices: all: true
touch mon.yaml
service_type: SERVICE_NAME placement: hosts: - HOST_NAME_1 - HOST_NAME_2
service_type: mon placement: hosts: - host01 - host02 - host03
service_type: SERVICE_NAME placement: label: " LABEL_1 "
service_type: mon placement: label: "mon"
extra_container_args: - "-v" - "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro" - "--security-opt" - "label=disable" - "cpus=2"
cephadm shell --mount mon.yaml:/var/lib/ceph/mon/mon.yaml
cd /var/lib/ceph/mon/
ceph orch apply -i FILE_NAME .yaml
ceph orch apply -i mon.yaml
ceph orch ls
ceph orch ps --daemon_type= DAEMON_NAME
ceph orch ps --daemon_type=mon