ClickHouse Keeper (clickhouse-keeper)

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/guides/sre/keeper/index.md)

Not supported in ClickHouse Cloud

Note

This page isn't applicable to [ClickHouse Cloud](https://clickhouse.com/cloud). The procedure documented here is automated in ClickHouse Cloud services.

ClickHouse Keeper provides the coordination system for data [replication](/docs/engines/table-engines/mergetree-family/replication) and [distributed DDL](/docs/sql-reference/distributed-ddl) queries execution. ClickHouse Keeper is compatible with ZooKeeper.

### Implementation details[​](#implementation-details "Direct link to Implementation details")

ZooKeeper is one of the first well-known open-source coordination systems. It's implemented in Java, and has a simple and powerful data model. ZooKeeper's coordination algorithm, ZooKeeper Atomic Broadcast (ZAB), doesn't provide linearizability guarantees for reads, because each ZooKeeper node serves reads locally. Unlike ZooKeeper, ClickHouse Keeper is written in C++ and uses the [RAFT algorithm](https://raft.github.io/) [implementation](https://github.com/eBay/NuRaft). This algorithm allows linearizability for reads and writes, and has several open-source implementations in different languages.

By default, ClickHouse Keeper provides the same guarantees as ZooKeeper: linearizable writes and non-linearizable reads. It has a compatible client-server protocol, so any standard ZooKeeper client can be used to interact with ClickHouse Keeper. Snapshots and logs have an incompatible format with ZooKeeper, but the `clickhouse-keeper-converter` tool enables the conversion of ZooKeeper data to ClickHouse Keeper snapshots. The interserver protocol in ClickHouse Keeper is also incompatible with ZooKeeper so a mixed ZooKeeper / ClickHouse Keeper cluster is impossible.

ClickHouse Keeper supports Access Control Lists (ACLs) the same way as [ZooKeeper](https://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html#sc_ZooKeeperAccessControl) does. ClickHouse Keeper supports the same set of permissions and has the identical built-in schemes: `world`, `auth` and `digest`. The digest authentication scheme uses the pair `username:password`, the password is encoded in Base64.

Note

External integrations aren't supported.

### Configuration[​](#configuration "Direct link to Configuration")

ClickHouse Keeper can be used as a standalone replacement for ZooKeeper or as an internal part of the ClickHouse server. In both cases the configuration is almost the same `.xml` file.

#### Keeper configuration settings[​](#keeper-configuration-settings "Direct link to Keeper configuration settings")

The main ClickHouse Keeper configuration tag is `<keeper_server>` and has the following parameters:

|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Parameter Description Default|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `tcp_port` Port for a client to connect. `2181`| `tcp_port_secure` Secure port for an SSL connection between client and keeper-server. -|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `server_id` Unique server id, each participant of the ClickHouse Keeper cluster must have a unique number (1, 2, 3...). -|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `log_storage_path` Path to coordination logs, just like ZooKeeper it's best to store logs on non-busy nodes. -|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `snapshot_storage_path` Path to coordination snapshots. -|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `enable_reconfiguration` Enable dynamic cluster reconfiguration via [`reconfig`](#reconfiguration). `False`| `max_memory_usage_soft_limit` Soft limit in bytes of keeper max memory usage. `max_memory_usage_soft_limit_ratio` \* `physical_memory_amount`| `max_memory_usage_soft_limit_ratio` If `max_memory_usage_soft_limit` isn't set or set to zero, we use this value to define the default soft limit. `0.9`| `cgroups_memory_observer_wait_time` If `max_memory_usage_soft_limit` isn't set or is set to `0`, we use this interval to observe the amount of physical memory. Once the memory amount changes, we will recalculate Keeper's memory soft limit by `max_memory_usage_soft_limit_ratio`. `15`| `http_control` Configuration of [HTTP control](#http-control) interface. -|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `digest_enabled` Enable real-time data consistency check `True`| `create_snapshot_on_exit` Create a snapshot during shutdown -|  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `hostname_checks_enabled` Enable sanity hostname checks for cluster configuration (e.g. if localhost is used with remote endpoints) `True`| `four_letter_word_white_list` White list of 4lw commands. `conf, cons, crst, envi, ruok, srst, srvr, stat, wchs, dirs, mntr, isro, rcvr, apiv, csnp, lgif, rqld, ydld`| `enable_ipv6` Enable IPv6 `True` | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

Other common parameters are inherited from the ClickHouse server config (`listen_host`, `logger`, and so on).

#### Internal coordination settings[​](#internal-coordination-settings "Direct link to Internal coordination settings")

Internal coordination settings are located in the `<keeper_server>.<coordination_settings>` section and have the following parameters: