Repository: getsentry/sentry

Issue:
No clickhouse Docker image for Mac Arm64
Request for an official image: https://github.com/ClickHouse/ClickHouse/issues/22222

Building Clickhouse is also broken for Apple M1 machines:
https://github.com/ClickHouse/ClickHouse/issues/27593


Discussion:
I've built & pushed an image from instructions I've found around and it doesn't quite work.

```shell
git clone https://github.com/ClickHouse/ClickHouse.git
cd ClickHouse/docker/server
git checkout da6ad09db6249015026baaf5903cefbd9aa6c4b7
docker build . --network host --build-arg single_binary_location_url="https://builds.clickhouse.tech/master/aarch64/clickhouse" -t northamerica-northeast2-docker.pkg.dev/sentry-dev-tooling/clickhouse-arm64/clickhouse-server:master-testing-arm
docker push northamerica-northeast2-docker.pkg.dev/sentry-dev-tooling/clickhouse-arm64/clickhouse-server:master-testing-arm
```

<img width="877" alt="image" src="https://user-images.githubusercontent.com/44410/135872287-05c71d42-13b4-46b9-9f00-2688de59a393.png">

I tried using the recommended [testing image](https://github.com/ClickHouse/ClickHouse/issues/22222#issuecomment-919967124) w/o success. It seems to be rebooting constantly. It could be due to upgrading major versions (`20.3.9.70` -> `21.3.15.4-lts-arm`):

```shell
Processing configuration file '/etc/clickhouse-server/config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/sentry.xml'.
Logging trace to /var/log/clickhouse-server/clickhouse-server.log
Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
Processing configuration file '/etc/clickhouse-server/config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/sentry.xml'.
Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'.
```

```diff
diff --git a/src/sentry/conf/server.py b/src/sentry/conf/server.py
index 1b0973e01c..e0dac2cfe4 100644
--- a/src/sentry/conf/server.py
+++ b/src/sentry/conf/server.py
@@ -1782,7 +1782,7 @@ SENTRY_DEVSERVICES = {
     ),
     "clickhouse": lambda settings, options: (
         {
-            "image": "yandex/clickhouse-server:20.3.9.70",
+            "image": "kolsys/clickhouse-server:21.3.15.4-lts-arm",
             "pull": True,
             "ports": {"9000/tcp": 9000, "9009/tcp": 9009, "8123/tcp": 8123},
             "ulimits": [{"name": "nofile", "soft": 262144, "hard": 262144}],
```
There's potentially some help here:
https://github.com/lspgn/clickhouse-arm/blob/main/.github/workflows/build.yml
https://clickhouse.com/docs/en/development/build-cross-arm/
Also related:
https://hub.docker.com/r/lunalabsltd/clickhouse-server/tags?page=1&ordering=last_updated
I tried `lunalabsltd/clickhouse-server:20.6.7.4-arm` and `lunalabsltd/clickhouse-server:20.6.3.28-arm` with the same output:

```
Processing configuration file '/etc/clickhouse-server/config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/sentry.xml'.
Include not found: clickhouse_remote_servers
Include not found: clickhouse_compression
Logging trace to /var/log/clickhouse-server/clickhouse-server.log
Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
Processing configuration file '/etc/clickhouse-server/users.xml'.
Include not found: networks
Saved preprocessed configuration to '/var/lib/clickhouse//preprocessed_configs/users.xml'.
```
For the curious, I can run clickhouse server by fetching the binary:

```shell
curl -Oclickhouse https://builds.clickhouse.com/master/macos-aarch64/clickhouse
chmod a+x ./clickhouse
./clickhouse server
```

I've also found this Gist (which I believe is used for the altinity/clickhouse-server image):
https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d

I run `docker build . --network host --build-arg single_binary_location="https://builds.clickhouse.com/master/macos-aarch64/clickhouse"`, however, that gave me this error:
```
#5 11.13 Reading package lists...
#5 11.51 Building dependency tree...
#5 11.60 Reading state information...
#5 11.61 E: Version '21.4.1.*' for 'clickhouse-common-static' was not found
#5 11.61 E: Version '21.4.1.*' for 'clickhouse-client' was not found
#5 11.61 E: Version '21.4.1.*' for 'clickhouse-server' was not found
------
executor failed running [/bin/sh -c groupadd -r clickhouse --gid=101     && useradd -r -g clickhouse --uid=101 --home-dir=/var/lib/clickhouse --shell=/bin/bash clickhouse     && apt-get update     && apt-get install --yes --no-install-recommends         apt-transport-https         ca-certificates         dirmngr         gnupg         locales         wget         tzdata     && mkdir -p /etc/apt/sources.list.d     && apt-key adv --keyserver keyserver.ubuntu.com --recv E0C56BD4     && echo $repository > /etc/apt/sources.list.d/clickhouse.list     && if [ -n "$deb_location_url" ]; then             echo "installing from custom url with deb packages: $deb_location_url"             rm -rf /tmp/clickhouse_debs             && mkdir -p /tmp/clickhouse_debs             && wget --progress=bar:force:noscroll "${deb_location_url}/clickhouse-common-static_${version}_amd64.deb" -P /tmp/clickhouse_debs             && wget --progress=bar:force:noscroll "${deb_location_url}/clickhouse-client_${version}_all.deb" -P /tmp/clickhouse_debs             && wget --progress=bar:force:noscroll "${deb_location_url}/clickhouse-server_${version}_all.deb" -P /tmp/clickhouse_debs             && dpkg -i /tmp/clickhouse_debs/*.deb ;        elif [ -n "$single_binary_location_url" ]; then             echo "installing from single binary url: $single_binary_location_url"             && rm -rf /tmp/clickhouse_binary             && mkdir -p /tmp/clickhouse_binary             && wget --progress=bar:force:noscroll "$single_binary_location_url" -O /tmp/clickhouse_binary/clickhouse             && chmod +x /tmp/clickhouse_binary/clickhouse             && /tmp/clickhouse_binary/clickhouse install --user "clickhouse" --group "clickhouse" ;        else            echo "installing from repository: $repository"            && apt-get update            && apt-get --yes -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" upgrade            && apt-get install --allow-unauthenticated --yes --no-install-recommends                 clickhouse-common-static=$version                 clickhouse-client=$version                 clickhouse-server=$version ;        fi     && clickhouse-local -q 'SELECT * FROM system.build_options'     && rm -rf         /var/lib/apt/lists/*         /var/cache/debconf         /tmp/*     && apt-get clean     && mkdir -p /var/lib/clickhouse /var/log/clickhouse-server /etc/clickhouse-server /etc/clickhouse-client     && chmod ugo+Xrw -R /var/lib/clickhouse /var/log/clickhouse-server /etc/clickhouse-server /etc/clickhouse-client]: exit code: 100
```

Resolving patch:
diff --git a/src/sentry/conf/server.py b/src/sentry/conf/server.py
--- a/src/sentry/conf/server.py
+++ b/src/sentry/conf/server.py
@@ -1802,11 +1802,17 @@ def build_cdc_postgres_init_db_volume(settings):
     ),
     "clickhouse": lambda settings, options: (
         {
-            "image": "yandex/clickhouse-server:20.3.9.70",
+            # altinity provides clickhouse support to other companies
+            # Official support: https://github.com/ClickHouse/ClickHouse/issues/22222
+            # This image is build with this script https://gist.github.com/filimonov/5f9732909ff66d5d0a65b8283382590d
+            "image": "yandex/clickhouse-server:20.3.9.70"
+            if not APPLE_ARM64
+            else "altinity/clickhouse-server:20.10.1.4844-testing-arm",
             "pull": True,
             "ports": {"9000/tcp": 9000, "9009/tcp": 9009, "8123/tcp": 8123},
             "ulimits": [{"name": "nofile", "soft": 262144, "hard": 262144}],
-            "environment": {"MAX_MEMORY_USAGE_RATIO": "0.3"},
+            # The arm image does not properly load the MAX_MEMORY_USAGE_RATIO
+            # from the environment in loc_config.xml, thus, hard-coding it there
             "volumes": {
                 "clickhouse_dist"
                 if settings.SENTRY_DISTRIBUTED_CLICKHOUSE_TABLES


Test patch: