Integrating S3 with ClickHouse

#### Deploy ClickHouse[​](#deploy-clickhouse "Direct link to Deploy ClickHouse")

Deploy ClickHouse on two hosts, in the sample configurations these are named `chnode1`, `chnode2`.

Place `chnode1` in one AWS region, and `chnode2` in a second.

#### Deploy ClickHouse Keeper[​](#deploy-clickhouse-keeper "Direct link to Deploy ClickHouse Keeper")

Deploy ClickHouse Keeper on three hosts, in the sample configurations these are named `keepernode1`, `keepernode2`, and `keepernode3`. `keepernode1` can be deployed in the same region as `chnode1`, `keepernode2` with `chnode2`, and `keepernode3` in either region but a different availability zone from the ClickHouse node in that region.

Refer to the [installation instructions](/docs/install) when performing the deployment steps on the ClickHouse Keeper nodes.

### Create S3 buckets[​](#create-s3-buckets "Direct link to Create S3 buckets")

Create two S3 buckets, one in each of the regions that you have placed `chnode1` and `chnode2`.

If you need step-by-step instructions to create buckets and an IAM role, then expand **Create S3 buckets and an IAM role** and follow along:

Create S3 buckets and an IAM user

This article demonstrates the basics of how to configure an AWS IAM user, create an S3 bucket and configure ClickHouse to use the bucket as an S3 disk.
You should work with your security team to determine the permissions to be used, and consider these as a starting point.

### Create an AWS IAM user[​](#create-an-aws-iam-user "Direct link to Create an AWS IAM user")

In the following steps you'll be creating a service account user (not a login user).

1. Log into the AWS IAM Management Console.
2. In the `Users` menu, select `Create user`

![AWS IAM Management Console - Adding a new user](/docs/assets/ideal-img/s3-1.24fd6a6.1024.png)

3. Enter the username and set the credential type to `Access key - Programmatic access` and select `Next: Permissions`

![Setting user name and access type for IAM user](/docs/assets/ideal-img/s3-2.da82511.984.png)

4. Don't add the user to any group; select `Next: Tags`

![Skipping group assignment for IAM user](/docs/assets/ideal-img/s3-3.3a8c14f.999.png)

5. Unless you need to add any tags, select `Next: Review`

![Skipping tag assignment for IAM user](/docs/assets/ideal-img/s3-4.1f4128c.983.png)

6. Select `Create User`

Note

The warning message stating that the user has no permissions can be ignored; permissions will be granted on the bucket for the user in the next section

![Creating the IAM user with no permissions warning](/docs/assets/ideal-img/s3-5.cf6d3e8.987.png)

7. The user is now created; click on `show` and copy the access and secret keys.

Note

Save the keys somewhere else; this is the only time that the secret access key will be available.

![Viewing and copying the IAM user access keys](/docs/assets/ideal-img/s3-6.1f4ef0f.983.png)

8. Click close, then find the user in the users screen.

![Finding the newly created IAM user in the users list](/docs/assets/ideal-img/s3-7.2bedf82.837.png)

9. Copy the ARN (Amazon Resource Name) and save it for use when configuring the access policy for the bucket.

![Copying the ARN of the IAM user](/docs/assets/ideal-img/s3-8.ca1384b.595.png)

### Create an S3 bucket[​](#create-an-s3-bucket "Direct link to Create an S3 bucket")

1. In the S3 bucket section, select `Create bucket`

![Starting the S3 bucket creation process](/docs/assets/ideal-img/s3-9.84b7005.1024.png)

2. Enter a bucket name, leave other options default

Note

The bucket name must be unique across AWS, not just the organization, or it will emit an error.

3. Leave `Block all Public Access` enabled; public access isn't needed.

![Configuring the S3 bucket settings with public access blocked](/docs/assets/ideal-img/s3-a.7e48fa9.841.png)

4. Select `Create Bucket` at the bottom of the page

![Finalizing S3 bucket creation](/docs/assets/ideal-img/s3-b.a91049d.826.png)

5. Select the link, copy the ARN, and save it for use when configuring the access policy for the bucket.
6. Once the bucket has been created, find the new S3 bucket in the S3 bucket list and select the link

![Finding the newly created S3 bucket in the buckets list](/docs/assets/ideal-img/s3-c.ca50023.1024.png)

7. Select `Create folder`

![Creating a new folder in the S3 bucket](/docs/assets/ideal-img/s3-d.8919166.1024.png)

8. Enter a folder name that will be the target for the ClickHouse S3 disk and select `Create folder`

![Setting the folder name for ClickHouse S3 disk usage](/docs/assets/ideal-img/s3-e.58ec6d8.853.png)

9. The folder should now be visible on the bucket list

![Viewing the newly created folder in the S3 bucket](/docs/assets/ideal-img/s3-f.a230918.1024.png)

10. Select the checkbox for the new folder and click on `Copy URL` Save the URL copied to be used in the ClickHouse storage configuration in the next section.

![Copying the S3 folder URL for ClickHouse configuration](/docs/assets/ideal-img/s3-g.2b23300.1024.png)

11. Select the `Permissions` tab and click on the `Edit` button in the `Bucket Policy` section

![Accessing the S3 bucket policy configuration](/docs/assets/ideal-img/s3-h.5358c81.1024.png)

12. Add a bucket policy, example below:

```
{
  "Version" : "2012-10-17",
  "Id" : "Policy123456",
  "Statement" : [
    {
      "Sid" : "abc123",
      "Effect" : "Allow",
      "Principal" : {
        "AWS" : "arn:aws:iam::921234567898:user/mars-s3-user"
      },
      "Action" : "s3:*",
      "Resource" : [
        "arn:aws:s3:::mars-doc-test",
        "arn:aws:s3:::mars-doc-test/*"
      ]
    }
  ]
}
```

```
|Parameter | Description | Example Value |
|----------|-------------|----------------|
|Version | Version of the policy interpreter, leave as-is | 2012-10-17 |
|Sid | User-defined policy id | abc123 |
|Effect | Whether user requests will be allowed or denied | Allow |
|Principal | The accounts or user that will be allowed | arn:aws:iam::921234567898:user/mars-s3-user |
|Action | What operations are allowed on the bucket| s3:*|
|Resource | Which resources in the bucket will operations be allowed in | "arn:aws:s3:::mars-doc-test", "arn:aws:s3:::mars-doc-test/*" |
```

Note

You should work with your security team to determine the permissions to be used, consider these as a starting point.
For more information on Policies and settings, refer to AWS documentation:
<https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-policy-language-overview.html>

13. Save the policy configuration.

The configuration files will then be placed in `/etc/clickhouse-server/config.d/`. Here is a sample configuration file for one bucket, the other is similar with the three highlighted lines differing:

```
<clickhouse>
  <storage_configuration>
     <disks>
        <s3_disk>
           <type>s3</type>
        <!--highlight-start-->
           <endpoint>https://docs-clickhouse-s3.s3.us-east-2.amazonaws.com/clickhouses3/</endpoint>
           <access_key_id>ABCDEFGHIJKLMNOPQRST</access_key_id>
           <secret_access_key>Tjdm4kf5snfkj303nfljnev79wkjn2l3knr81007</secret_access_key>
        <!--highlight-end-->
           <metadata_path>/var/lib/clickhouse/disks/s3_disk/</metadata_path>
        </s3_disk>

        <s3_cache>
           <type>cache</type>
           <disk>s3_disk</disk>
           <path>/var/lib/clickhouse/disks/s3_cache/</path>
           <max_size>10Gi</max_size>
        </s3_cache>
     </disks>
        <policies>
            <s3_main>
                <volumes>
                    <main>
                        <disk>s3_disk</disk>
                    </main>
                </volumes>
            </s3_main>
    </policies>
   </storage_configuration>
</clickhouse>
```

Note

Many of the steps in this guide will ask you to place a configuration file in `/etc/clickhouse-server/config.d/`. This is the default location on Linux systems for configuration override files. When you put these files into that directory ClickHouse will use the content to override the default configuration. By placing these files in the override directory you will avoid losing your configuration during an upgrade.