Repository: SPS-By-The-Numbers/transcripts
Pull request #120: Make Dockerfile.search work on Cloud Run
Author: login: kzys
Created: 2025-05-21T02:31:19Z | Merged: 2025-05-22T05:00:01Z

This new image downloads Meilisearch's index files from Cloud Storage.


Diff:
diff --git a/.github/workflows/container.yml b/.github/workflows/container.yml
index cf57876..9ab1de7 100644
--- a/.github/workflows/container.yml
+++ b/.github/workflows/container.yml
@@ -31,3 +31,25 @@ jobs:
           push: ${{ github.event_name == 'push' }}
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
+
+  build-search:
+    name: Build search
+    runs-on: ubuntu-latest
+    steps:
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - uses: docker/metadata-action@v5
+        id: meta
+        with:
+          images: |
+            ghcr.io/${{ github.repository_owner }}/search
+      - uses: actions/checkout@v4
+      - uses: docker/build-push-action@v6
+        with:
+          file: Dockerfile.search
+          push: ${{ github.event_name == 'push' }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
diff --git a/.github/workflows/pull-request.yml b/.github/workflows/pull-request.yml
index 9809e45..b241d52 100644
--- a/.github/workflows/pull-request.yml
+++ b/.github/workflows/pull-request.yml
@@ -24,14 +24,3 @@ jobs:
           context: tools/process_new_vids
           push: false
           tags: transcribe-worker
-
-  build-search:
-    name: Build search
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - uses: docker/build-push-action@v6
-        with:
-          file: Dockerfile.search
-          push: false
-          tags: search
diff --git a/Dockerfile.search b/Dockerfile.search
index dbb2f6f..e989867 100644
--- a/Dockerfile.search
+++ b/Dockerfile.search
@@ -1,6 +1,16 @@
-FROM getmeili/meilisearch:latest
+FROM getmeili/meilisearch:v1.14
 
-ARG SEARCH_DB
+ARG GOOGLE_CLOUD_CLI_VERSION=522.0.0
+
+RUN <<EOF
+apk add --no-cache python3
+curl --silent https://storage.googleapis.com/cloud-sdk-release/google-cloud-cli-${GOOGLE_CLOUD_CLI_VERSION}-linux-x86_64.tar.gz | tar zxf - -C /opt
+/opt/google-cloud-sdk/install.sh --quiet
+EOF
+
+ENV PATH="/opt/google-cloud-sdk/bin:$PATH"
+ENV SEARCH_DIR=/search
+ENV SEARCH_DB=${SEARCH_DIR}/data.ms
 
 ENTRYPOINT ["tini", "--"]
 EXPOSE 7700
diff --git a/tools/search/start_meilisearch.sh b/tools/search/start_meilisearch.sh
index 0f54923..267f2be 100755
--- a/tools/search/start_meilisearch.sh
+++ b/tools/search/start_meilisearch.sh
@@ -1,6 +1,21 @@
-#! /bin/bash
+#! /bin/sh
 set -euo pipefail
 
-# TODO: download stuff from Google Cloud Storage to /meili_data/data.ms
+# gcp_key_path is for testing this image outside of Cloud Run.
+# On Cloud Run, permissions are granted via the service account of the job.
+gcp_key_path=/run/secrets/gcp.json
+if [[ -e "$gcp_key_path" ]]; then
+    gcloud auth activate-service-account "--key-file=$gcp_key_path"
+fi
 
-exec /bin/meilisearch --master-key=$(cat /run/secrets/meilisearch)
+# This key is for testing.
+# The key must be at least 16 bytes long.
+# https://www.meilisearch.com/docs/learn/security/basic_security#creating-the-master-key-in-a-self-hosted-instance
+MASTER_KEY='MEILI-9133dcf0-d6d9-4d24-a217-ae68fdb04475'
+
+# Download the index from Cloud Storage.
+mkdir -p "$SEARCH_DIR"
+gcloud storage cp --project "$GCP_PROJECT_ID" \
+    --recursive "gs://$GCP_BUCKET/data.ms" "$SEARCH_DIR"
+
+exec /bin/meilisearch "--db-path=$SEARCH_DB" "--master-key=$MASTER_KEY"


Reviews:
user: login: awong-dev
body: Nice!   Got 2 comments but ready to submit for sure.

This is exciting! Thank you!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T03:44:44Z
user: login: kzys
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T04:10:06Z
user: login: kzys
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T04:10:59Z
user: login: awong-dev
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T04:15:28Z

Review comments:
diff_hunk: @@ -31,3 +31,26 @@ jobs:
           push: ${{ github.event_name == 'push' }}
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
+
+  build-search:
+    name: Build search
+    runs-on: ubuntu-latest
+    steps:
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - uses: docker/metadata-action@v5
+        id: meta
+        with:
+          images: |
+            ghcr.io/${{ github.repository_owner }}/search
+      - uses: actions/checkout@v4
+      - uses: docker/build-push-action@v6
+        with:
+          file: Dockerfile.search
+          # push: ${{ github.event_name == 'push' }}
path: .github/workflows/container.yml
user: login: awong-dev
body: I'm not familiar enough with github actions... what does hard coding true do here vs your event above?
created_at: 2025-05-22T03:38:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,6 +1,16 @@
-FROM getmeili/meilisearch:latest
+FROM getmeili/meilisearch:v1.14
 
-ARG SEARCH_DB
+ARG GOOGLE_CLOUD_CLI_VERSION=522.0.0
+
+RUN <<EOF
+apk add --no-cache python3
+curl --silent https://storage.googleapis.com/cloud-sdk-release/google-cloud-cli-${GOOGLE_CLOUD_CLI_VERSION}-linux-x86_64.tar.gz | tar zxf - -C /
path: Dockerfile.search
user: login: awong-dev
body: untarrring into / always makes me a little OCD.

Can I suggest

```
mkdir /google-cloud-sdk
....| tar --strip-components=1 -C /google-cloud-sdk
```

that should be exactly the same as currently but robust to someone in the future adding random stuff into the root of the tar.

If could even untar into ~ or /workspace
created_at: 2025-05-22T03:39:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -31,3 +31,26 @@ jobs:
           push: ${{ github.event_name == 'push' }}
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
+
+  build-search:
+    name: Build search
+    runs-on: ubuntu-latest
+    steps:
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - uses: docker/metadata-action@v5
+        id: meta
+        with:
+          images: |
+            ghcr.io/${{ github.repository_owner }}/search
+      - uses: actions/checkout@v4
+      - uses: docker/build-push-action@v6
+        with:
+          file: Dockerfile.search
+          # push: ${{ github.event_name == 'push' }}
path: .github/workflows/container.yml
user: login: kzys
body: Actually I need to remove this one. Good catch.

I'd like to update the image only from the pushes to `main`.
created_at: 2025-05-22T04:10:06Z
author_association: COLLABORATOR
diff_hunk: @@ -1,6 +1,16 @@
-FROM getmeili/meilisearch:latest
+FROM getmeili/meilisearch:v1.14
 
-ARG SEARCH_DB
+ARG GOOGLE_CLOUD_CLI_VERSION=522.0.0
+
+RUN <<EOF
+apk add --no-cache python3
+curl --silent https://storage.googleapis.com/cloud-sdk-release/google-cloud-cli-${GOOGLE_CLOUD_CLI_VERSION}-linux-x86_64.tar.gz | tar zxf - -C /
path: Dockerfile.search
user: login: kzys
body: Sure. `/usr/local/google-cloud-sdk` or `/opt/google-cloud-sdk`? It doesn't have to be `/`.
created_at: 2025-05-22T04:10:59Z
author_association: COLLABORATOR
diff_hunk: @@ -1,6 +1,16 @@
-FROM getmeili/meilisearch:latest
+FROM getmeili/meilisearch:v1.14
 
-ARG SEARCH_DB
+ARG GOOGLE_CLOUD_CLI_VERSION=522.0.0
+
+RUN <<EOF
+apk add --no-cache python3
+curl --silent https://storage.googleapis.com/cloud-sdk-release/google-cloud-cli-${GOOGLE_CLOUD_CLI_VERSION}-linux-x86_64.tar.gz | tar zxf - -C /
path: Dockerfile.search
user: login: awong-dev
body: Let's do `/opt`? shorter to type.
created_at: 2025-05-22T04:15:28Z
author_association: CONTRIBUTOR

PR comments:
user: login: kzys
created_at: 2025-05-22T00:04:43Z
body: The staging environment is using the latest version of this image. Seems it is working!
author_association: COLLABORATOR
user: login: kzys
created_at: 2025-05-22T05:00:45Z
body: Merged the PR since you've approved already.
author_association: COLLABORATOR