Project: ClickHouse (App Definition and Development / Database)
File: keepermap.md

slug: /en/engines/table-engines/special/keeper-map sidebar_position: 150 sidebar_label: KeeperMap This engine allows you to use Keeper/ZooKeeper cluster as consistent key-value store with linearizable writes and sequentially consistent reads. To enable KeeperMap storage engine, you need to define a ZooKeeper path where the tables will be stored using `<keepermappath_prefix>` config. For example: ```xml <clickhouse> <keepermappathprefix>/keepermaptables</keepermappathprefix> </clickhouse> ``` where path can be any other valid ZooKeeper path. ``` sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] ( name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], ... ) ENGINE = KeeperMap(rootpath, [keyslimit]) PRIMARY KEY(primarykeyname) ``` Engine parameters: `rootpath` - ZooKeeper path where the `tablename` will be stored. This path should not contain the prefix defined by `<keepermappathprefix>` config because the prefix will be automatically appended to the `rootpath`. Additionally, format of `auxiliaryzookeperclustername:/some/path` is also supported where `auxiliaryzookepercluster` is a ZooKeeper cluster defined inside `<auxiliaryzookeepers>` config. By default, ZooKeeper cluster defined inside `<zookeeper>` config is used. `keys_limit` - number of keys allowed inside the table. This limit is a soft limit and it can be possible that more keys will end up in the table for some edge cases. `primarykeyname` any column name in the column list. `primary key` must be specified, it supports only one column in the primary key. The primary key will be serialized in binary as a `node name` inside ZooKeeper. columns other than the primary key will be serialized to binary in corresponding order and stored as a value of the resulting node defined by the serialized key. queries with key `equals` or `in` filtering will be optimized to multi keys lookup from `Keeper`, otherwise all values will be fetched. Example: ``` sql CREATE TABLE keepermaptable ( `key` String, `v1` UInt32, `v2` String, `v3` Float32 ) ENGINE = KeeperMap(/keepermaptable, 4) PRIMARY KEY key ``` with ```xml <clickhouse> <keepermappathprefix>/keepermaptables</keepermappathprefix> </clickhouse> ``` Each value, which is binary serialization of `(v1, v2, v3)`, will be stored inside `/keepermaptables/keepermaptable/data/serialized_key` in `Keeper`. Additionally, number of keys will have a soft limit of 4 for the number of keys. If multiple tables are created on the same ZooKeeper path, the values are persisted until there exists at least 1 table using it. As a result, it is possible to use `ON CLUSTER` clause when creating the table and sharing the data from multiple ClickHouse instances. Of course, it's possible to manually run `CREATE TABLE` with same path on unrelated ClickHouse instances to have same data sharing effect. When new rows are inserted into `KeeperMap`, if the key does not exist, a new entry for the key is created. If the key exists, and setting `keepermapstrict_mode` is set to `true`, an exception is thrown, otherwise, the value for the key is overwritten. Example: ```sql INSERT INTO keepermaptable VALUES ('some key', 1, 'value', 3.2); ``` Rows can be deleted using `DELETE` query or `TRUNCATE`. If the key exists, and setting `keepermapstrict_mode` is set to `true`, fetching and deleting data will succeed only if it can be executed atomically. ```sql DELETE FROM keepermaptable WHERE key LIKE 'some%' AND v1 > 1; ``` ```sql ALTER TABLE keepermaptable DELETE WHERE key LIKE 'some%' AND v1 > 1; ``` ```sql TRUNCATE TABLE keepermaptable; ``` Values can be updated using `ALTER TABLE` query. Primary key cannot be updated. If setting `keepermapstrict_mode` is set to `true`, fetching and updating data will succeed only if it's executed atomically. ```sql ALTER TABLE keepermaptable UPDATE v1 = v1 * 10 + 2 WHERE key LIKE 'some%' AND v3 > 3.1; ``` Blog: