Integrate Google Cloud Storage with ClickHouse

### Handling replication[​](#handling-replication "Direct link to Handling replication")

Replication with GCS disks can be accomplished by using the `ReplicatedMergeTree` table engine. See the [replicating a single shard across two GCP regions using GCS](#gcs-multi-region) guide for details.

### Learn more[​](#learn-more "Direct link to Learn more")

The [Cloud Storage XML API](https://cloud.google.com/storage/docs/xml-api/overview) is interoperable with some tools and libraries that work with services such as Amazon Simple Storage Service (Amazon S3).

For further information on tuning threads, see [Optimizing for Performance](/docs/integrations/s3#s3-optimizing-performance).

## Using Google Cloud Storage (GCS)[​](#gcs-multi-region "Direct link to Using Google Cloud Storage (GCS)")

Tip

Object storage is used by default in ClickHouse Cloud, you don't need to follow this procedure if you're running in ClickHouse Cloud.

### Plan the deployment[​](#plan-the-deployment "Direct link to Plan the deployment")

This tutorial is written to describe a replicated ClickHouse deployment running in Google Cloud and using Google Cloud Storage (GCS) as the ClickHouse storage disk "type".

In the tutorial, you will deploy ClickHouse server nodes in Google Cloud Engine VMs, each with an associated GCS bucket for storage. Replication is coordinated by a set of ClickHouse Keeper nodes, also deployed as VMs.

Sample requirements for high availability:

- Two ClickHouse server nodes, in two GCP regions
- Two GCS buckets, deployed in the same regions as the two ClickHouse server nodes
- Three ClickHouse Keeper nodes, two of them are deployed in the same regions as the ClickHouse server nodes. The third can be in the same region as one of the first two Keeper nodes, but in a different availability zone.

ClickHouse Keeper requires two nodes to function, hence a requirement for three nodes for high availability.

### Prepare virtual machines[​](#prepare-vms "Direct link to Prepare virtual machines")

Deploy five VMS in three regions:

|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Region ClickHouse Server Bucket ClickHouse Keeper|  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1 `chnode1` `bucket_regionname` `keepernode1`| 2 `chnode2` `bucket_regionname` `keepernode2`| 3 `*` `keepernode3` | | | | | | | | | | | | | | | |

`*` This can be a different availability zone in the same region as 1 or 2.

#### Deploy ClickHouse[​](#deploy-clickhouse "Direct link to Deploy ClickHouse")

Deploy ClickHouse on two hosts, in the sample configurations these are named `chnode1`, `chnode2`.

Place `chnode1` in one GCP region, and `chnode2` in a second. In this guide `us-east1` and `us-east4` are used for the compute engine VMs, and also for GCS buckets.

Note

Don't start `clickhouse server` until after it is configured. Just install it.

Refer to the [installation instructions](/docs/install) when performing the deployment steps on the ClickHouse server nodes.

#### Deploy ClickHouse Keeper[​](#deploy-clickhouse-keeper "Direct link to Deploy ClickHouse Keeper")

Deploy ClickHouse Keeper on three hosts, in the sample configurations these are named `keepernode1`, `keepernode2`, and `keepernode3`. `keepernode1` can be deployed in the same region as `chnode1`, `keepernode2` with `chnode2`, and `keepernode3` in either region, but in a different availability zone from the ClickHouse node in that region.

Refer to the [installation instructions](/docs/install) when performing the deployment steps on the ClickHouse Keeper nodes.

### Create two buckets[​](#create-two-buckets "Direct link to Create two buckets")

The two ClickHouse servers will be located in different regions for high availability. Each will have a GCS bucket in the same region.

In **Cloud Storage > Buckets** choose **CREATE BUCKET**. For this tutorial two buckets are created, one in each of `us-east1` and `us-east4`. The buckets are single region, standard storage class, and not public. When prompted, enable public access prevention. Don't create folders, they will be created when ClickHouse writes to the storage.

If you need step-by-step instructions to create buckets and an HMAC key, then expand **Create GCS buckets and an HMAC key** and follow along:

Create GCS buckets and an HMAC key

### ch\_bucket\_us\_east1[​](#ch_bucket_us_east1 "Direct link to ch_bucket_us_east1")

### ch\_bucket\_us\_east4[​](#ch_bucket_us_east4 "Direct link to ch_bucket_us_east4")

### Generate an access key[​](#generate-an-access-key "Direct link to Generate an access key")

### Create a service account HMAC key and secret[​](#create-a-service-account-hmac-key-and-secret "Direct link to Create a service account HMAC key and secret")

Open **Cloud Storage > Settings > Interoperability** and either choose an existing **Access key**, or **CREATE A KEY FOR A SERVICE ACCOUNT**. This guide covers the path for creating a new key for a new service account.

### Add a new service account[​](#add-a-new-service-account "Direct link to Add a new service account")

If this is a project with no existing service account, **CREATE NEW ACCOUNT**.

There are three steps to creating the service account, in the first step give the account a meaningful name, ID, and description.

In the Interoperability settings dialog the IAM role **Storage Object Admin** role is recommended; select that role in step two.

Step three is optional and not used in this guide. You may allow users to have these privileges based on your policies.

The service account HMAC key will be displayed. Save this information, as it will be used in the ClickHouse configuration.

### Configure ClickHouse Keeper[​](#configure-clickhouse-keeper "Direct link to Configure ClickHouse Keeper")

All of the ClickHouse Keeper nodes have the same configuration file except for the `server_id` line (first highlighted line below). Modify the file with the hostnames for your ClickHouse Keeper servers, and on each of the servers set the `server_id` to match the appropriate `server` entry in the `raft_configuration`. Since this example has `server_id` set to `3`, we have highlighted the matching lines in the `raft_configuration`.

- Edit the file with your hostnames, and make sure that they resolve from the ClickHouse server nodes and the Keeper nodes
- Copy the file into place (`/etc/clickhouse-keeper/keeper_config.xml` on each of the Keeper servers
- Edit the `server_id` on each machine, based on its entry number in the `raft_configuration`

```
<clickhouse>
    <logger>
        <level>trace</level>
        <log>/var/log/clickhouse-keeper/clickhouse-keeper.log</log>
        <errorlog>/var/log/clickhouse-keeper/clickhouse-keeper.err.log</errorlog>
        <size>1000M</size>
        <count>3</count>
    </logger>
    <listen_host>0.0.0.0</listen_host>
    <keeper_server>
        <tcp_port>9181</tcp_port>
<!--highlight-next-line-->
        <server_id>3</server_id>
        <log_storage_path>/var/lib/clickhouse/coordination/log</log_storage_path>
        <snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path>

        <coordination_settings>
            <operation_timeout_ms>10000</operation_timeout_ms>
            <session_timeout_ms>30000</session_timeout_ms>
            <raft_logs_level>warning</raft_logs_level>
        </coordination_settings>