Appendix C. Ceph Monitor configuration options The following are Ceph monitor configuration options that can be set up during deployment. mon_initial_members Description The IDs of initial monitors in a cluster during startup. If specified, Ceph requires an odd number of monitors to form an initial quorum (for example, 3). Type String Default None mon_force_quorum_join Description Force monitor to join quorum even if it has been previously removed from the map Type Boolean Default False mon_dns_srv_name Description The service name used for querying the DNS for the monitor hosts/addresses. Type String Default ceph-mon fsid Description The cluster ID. One per cluster. Type UUID Required Yes. Default N/A. May be generated by a deployment tool if not specified. mon_data Description The monitor's data location. Type String Default /var/lib/ceph/mon/USDcluster-USDid mon_data_size_warn Description Ceph issues a HEALTH_WARN status in the cluster log when the monitor's data store reaches this threshold. The default value is 15GB. Type Integer Default 15*1024*1024*1024*` mon_data_avail_warn Description Ceph issues a HEALTH_WARN status in cluster log when the available disk space of the monitor's data store is lower than or equal to this percentage. Type Integer Default 30 mon_data_avail_crit Description Ceph issues a HEALTH_ERR status in cluster log when the available disk space of the monitor's data store is lower or equal to this percentage. Type Integer Default 5 mon_warn_on_cache_pools_without_hit_sets Description Ceph issues a HEALTH_WARN status in cluster log if a cache pool does not have the hit_set_type paramater set. Type Boolean Default True mon_warn_on_crush_straw_calc_version_zero Description Ceph issues a HEALTH_WARN status in the cluster log if the CRUSH's straw_calc_version is zero. See CRUSH tunables for details. Type Boolean Default True mon_warn_on_legacy_crush_tunables Description Ceph issues a HEALTH_WARN status in the cluster log if CRUSH tunables are too old (older than mon_min_crush_required_version ). Type Boolean Default True mon_crush_min_required_version Description This setting defines the minimum tunable profile version required by the cluster. Type String Default firefly mon_warn_on_osd_down_out_interval_zero Description Ceph issues a HEALTH_WARN status in the cluster log if the mon_osd_down_out_interval setting is zero, because the Leader behaves in a similar manner when the noout flag is set. Administrators find it easier to troubleshoot a cluster by setting the noout flag. Ceph issues the warning to ensure administrators know that the setting is zero. Type Boolean Default True mon_cache_target_full_warn_ratio Description Ceph issues a warning when between the ratio of cache_target_full and target_max_object . Type Float Default 0.66 mon_health_data_update_interval Description How often (in seconds) a monitor in the quorum shares its health status with its peers. A negative number disables health updates. Type Float Default 60 mon_health_to_clog Description This setting enable Ceph to send a health summary to the cluster log periodically. Type Boolean Default True mon_health_detail_to_clog Description This setting enable Ceph to send a health details to the cluster log periodically. Type Boolean Default True mon_op_complaint_time Description Number of seconds after which the Ceph Monitor operation is considered blocked after no updates. Type Integer Default 30 mon_health_to_clog_tick_interval Description How often (in seconds) the monitor sends a health summary to the cluster log. A non-positive number disables it. If the current health summary is empty or identical to the last time, the monitor will not send the status to the cluster log. Type Integer Default 3600 mon_health_to_clog_interval Description How often (in seconds) the monitor sends a health summary to the cluster log. A non-positive number disables it. The monitor will always send the summary to cluster log. Type Integer Default 60 mon_osd_full_ratio Description The percentage of disk space used before an OSD is considered full . Type Float: Default .95 mon_osd_nearfull_ratio Description The percentage of disk space used before an OSD is considered nearfull . Type Float Default .85 mon_sync_trim_timeout Description, Type Double Default 30.0 mon_sync_heartbeat_timeout Description, Type Double Default 30.0 mon_sync_heartbeat_interval Description, Type Double Default 5.0 mon_sync_backoff_timeout Description, Type Double Default 30.0 mon_sync_timeout Description Number of seconds the monitor will wait for the update message from its sync provider before it gives up and bootstraps again. Type Double Default 30.0 mon_sync_max_retries Description, Type Integer Default 5 mon_sync_max_payload_size Description The maximum size for a sync payload (in bytes). Type 32-bit Integer Default 1045676 paxos_max_join_drift Description The maximum Paxos iterations before we must first sync the monitor data stores. When a monitor finds that its peer is too far ahead of it, it will first sync with data stores before moving on. Type Integer Default 10 paxos_stash_full_interval Description How often (in commits) to stash a full copy of the PaxosService state. Current this setting only affects mds , mon , auth and mgr PaxosServices. Type Integer Default 25 paxos_propose_interval Description Gather updates for this time interval before proposing a map update. Type Double Default 1.0 paxos_min Description The minimum number of paxos states to keep around Type Integer Default 500 paxos_min_wait Description The minimum amount of time to gather updates after a period of inactivity. Type Double Default 0.05 paxos_trim_min Description Number of extra proposals tolerated before trimming Type Integer Default 250 paxos_trim_max Description The maximum number of extra proposals to trim at a time Type Integer Default 500 paxos_service_trim_min Description The minimum amount of versions to trigger a trim (0 disables it) Type Integer Default 250 paxos_service_trim_max Description The maximum amount of versions to trim during a single proposal (0 disables it) Type Integer Default 500 mon_max_log_epochs Description The maximum amount of log epochs to trim during a single proposal Type Integer Default 500 mon_max_pgmap_epochs Description The maximum amount of pgmap epochs to trim during a single proposal Type Integer Default 500 mon_mds_force_trim_to Description Force monitor to trim mdsmaps to this point (0 disables it. dangerous, use with care) Type Integer Default 0 mon_osd_force_trim_to Description Force monitor to trim osdmaps to this point, even if there is PGs not clean at the specified epoch (0 disables it. dangerous, use with care) Type Integer Default 0 mon_osd_cache_size Description The size of osdmaps cache, not to rely on underlying store's cache Type Integer Default 10 mon_election_timeout Description On election proposer, maximum waiting time for all ACKs in seconds. Type Float Default 5 mon_lease Description The length (in seconds) of the lease on the monitor's versions. Type Float Default 5 mon_lease_renew_interval_factor Description mon lease * mon lease renew interval factor will be the interval for the Leader to renew the other monitor's leases. The factor should be less than 1.0 . Type Float Default 0.6 mon_lease_ack_timeout_factor Description The Leader will wait mon lease * mon lease ack timeout factor for the Providers to acknowledge the lease extension. Type Float Default 2.0 mon_accept_timeout_factor Description The Leader will wait mon lease * mon accept timeout factor for the Requesters to accept a Paxos update. It is also used during the Paxos recovery phase for similar purposes. Type Float Default 2.0 mon_min_osdmap_epochs Description Minimum number of OSD map epochs to keep at all times. Type 32-bit Integer Default 500 mon_max_pgmap_epochs Description Maximum number of PG map epochs the monitor should keep. Type 32-bit Integer Default 500 mon_max_log_epochs Description Maximum number of Log epochs the monitor should keep. Type 32-bit Integer Default 500 clock_offset Description How much to offset the system clock. See Clock.cc for details. Type Double Default 0 mon_tick_interval Description A monitor's tick interval in seconds. Type 32-bit Integer Default 5 mon_clock_drift_allowed Description The clock drift in seconds allowed between monitors. Type Float Default .050 mon_clock_drift_warn_backoff Description Exponential backoff for clock drift warnings. Type Float Default 5 mon_timecheck_interval Description The time check interval (clock drift check) in seconds for the leader. Type Float Default 300.0 mon_timecheck_skew_interval Description The time check interval (clock drift check) in seconds when in the presence of a skew in seconds for the Leader. Type Float Default 30.0 mon_max_osd Description The maximum number of OSDs allowed in the cluster. Type 32-bit Integer Default 10000 mon_globalid_prealloc Description The number of global IDs to pre-allocate for clients and daemons in the cluster. Type 32-bit Integer Default 100 mon_sync_fs_threshold Description Synchronize with the filesystem when writing the specified number of objects. Set it to 0 to disable it. Type 32-bit Integer Default 5 mon_subscribe_interval Description The refresh interval, in seconds, for subscriptions. The subscription mechanism enables obtaining the cluster maps and log information. Type Double Default 300 mon_stat_smooth_intervals Description Ceph will smooth statistics over the last N PG maps. Type Integer Default 2 mon_probe_timeout Description Number of seconds the monitor will wait to find peers before bootstrapping. Type Double Default 2.0 mon_daemon_bytes Description The message memory cap for metadata server and OSD messages (in bytes). Type 64-bit Integer Unsigned Default 400ul << 20 mon_max_log_entries_per_event Description The maximum number of log entries per event. Type Integer Default 4096 mon_osd_prime_pg_temp Description Enables or disable priming the PGMap with the OSDs when an out OSD comes back into the cluster. With the true setting the clients will continue to use the OSDs until the newly in OSDs as that PG peered. Type Boolean Default true mon_osd_prime_pg_temp_max_time Description How much time in seconds the monitor should spend trying to prime the PGMap when an out OSD comes back into the cluster. Type Float Default 0.5 mon_osd_prime_pg_temp_max_time_estimate Description Maximum estimate of time spent on each PG before we prime all PGs in parallel. Type Float Default 0.25 mon_osd_allow_primary_affinity Description allow primary_affinity to be set in the osdmap. Type Boolean Default False mon_osd_pool_ec_fast_read Description Whether turn on fast read on the pool or not. It will be used as the default setting of newly created erasure pools if fast_read is not specified at create time. Type Boolean Default False mon_mds_skip_sanity Description Skip safety assertions on FSMap, in case of bugs where we want to continue anyway. Monitor terminates if the FSMap sanity check fails, but we can disable it by enabling this option. Type Boolean Default False mon_max_mdsmap_epochs Description The maximum amount of mdsmap epochs to trim during a single proposal. Type Integer Default 500 mon_config_key_max_entry_size Description The maximum size of config-key entry (in bytes). Type Integer Default 4096 mon_scrub_interval Description How often, in seconds, the monitor scrub its store by comparing the stored checksums with the computed ones of all the stored keys. Type Integer Default 3600*24 mon_scrub_max_keys Description The maximum number of keys to scrub each time. Type Integer Default 100 mon_compact_on_start Description Compact the database used as Ceph Monitor store on ceph-mon start. A manual compaction helps to shrink the monitor database and improve the performance of it if the regular compaction fails to work. Type Boolean Default False mon_compact_on_bootstrap Description Compact the database used as Ceph Monitor store on on bootstrap. Monitor starts probing each other for creating a quorum after bootstrap. If it times out before joining the quorum, it will start over and bootstrap itself again. Type Boolean Default False mon_compact_on_trim Description Compact a certain prefix (including paxos) when we trim its old states. Type Boolean Default True mon_cpu_threads Description Number of threads for performing CPU intensive work on monitor. Type Boolean Default True mon_osd_mapping_pgs_per_chunk Description We calculate the mapping from placement group to OSDs in chunks. This option specifies the number of placement groups per chunk. Type Integer Default 4096 mon_osd_max_split_count Description Largest number of PGs per "involved" OSD to let split create. When we increase the pg_num of a pool, the placement groups will be splitted on all OSDs serving that pool. We want to avoid extreme multipliers on PG splits. Type Integer Default 300 rados_mon_op_timeout Description Number of seconds to wait for a response from the monitor before returning an error from a rados operation. 0 means at limit, or no wait time. Type Double Default 0 Additional Resources Pool Values CRUSH tunables

Commands: