Repository: loculus-project/loculus
Pull request #3924: fix preview bucket config secret
Author: login: fhennig
Labels: name: preview
description: Triggers a deployment to argocd
Created: 2025-04-01T14:41:29Z | Merged: 2025-04-01T15:53:38Z

I didn't use the correct scope before when sealing the secret, I hope now it should work 🤞 

preview URL: https://fs-fix-preview-sealed-sec.loculus.org


Diff:
diff --git a/kubernetes/README.md b/kubernetes/README.md
index 273e1affe8..76bec89b60 100644
--- a/kubernetes/README.md
+++ b/kubernetes/README.md
@@ -134,6 +134,18 @@ We do not currently support branch names containing characters that can't go in
 
 For preview instances this repo contains [sealed secrets](https://sealed-secrets.netlify.app/) that allow the loculus-bot to access the GitHub container registry and (separately) the GitHub repository. These are encrypted such that they can only be decrypted on our cluster but are cluster-wide so can be used in any namespace.
 
+### Adding a sealed secret
+
+Create a secret, for example like this:
+
+    kubectl create secret generic my-secret --from-literal=accessKey=<secret> --from-literal=secretKey=<secret> --dry-run=client -o yaml > my-secret.yaml
+
+This will create a `my-secret.yaml` file. Now, ensure that you have correctly configured kubectl to point to the preview cluster. Then seal your secret like this:
+
+    kubeseal --scope cluster-wide --format=yaml < my-secret.yaml > my-sealed-secret.yaml
+
+You now have a file `my-sealed-secret.yaml` with `spec.encryptedData` in it. You can now add this `encryptedData` to the `values.yaml` under `secrets.<yoursecretname>`. See `values_preview_server.yaml` for examples.
+
 ## Setting up kubeconfig locally to access the remote cluster
 
 To access the remote cluster without `ssh`ing to the containing machine, you need to set up your `kubeconfig` file.
diff --git a/kubernetes/loculus/templates/_config-processor.tpl b/kubernetes/loculus/templates/_config-processor.tpl
index 48dc87806f..cda2853199 100644
--- a/kubernetes/loculus/templates/_config-processor.tpl
+++ b/kubernetes/loculus/templates/_config-processor.tpl
@@ -52,6 +52,16 @@
         secretKeyRef:
           name: orcid
           key: orcidSecret
+    - name: LOCULUSSUB_s3accessKey
+      valueFrom:
+        secretKeyRef:
+          name: s3-bucket
+          key: accessKey
+    - name: LOCULUSSUB_s3secretKey
+      valueFrom:
+        secretKeyRef:
+          name: s3-bucket
+          key: secretKey
 {{- end }}
 
 
diff --git a/kubernetes/loculus/templates/loculus-backend.yaml b/kubernetes/loculus/templates/loculus-backend.yaml
index 8edac4d294..714ede8d7f 100644
--- a/kubernetes/loculus/templates/loculus-backend.yaml
+++ b/kubernetes/loculus/templates/loculus-backend.yaml
@@ -112,21 +112,6 @@ spec:
                secretKeyRef:
                  name: database
                  key: password
-            # TODO - comment back in once sealed secret works
-            # - name: S3_BUCKET_ACCESSKEY
-            #   valueFrom:
-            #     secretKeyRef:
-            #       name: s3-bucket
-            #       key: accessKey
-            - name: S3_BUCKET_ACCESSKEY
-              value: foo
-            # - name: S3_BUCKET_SECRETKEY
-            #   valueFrom:
-            #     secretKeyRef:
-            #       name: s3-bucket
-            #       key: secretKey
-            - name: S3_BUCKET_SECRETKEY
-              value: bar
           volumeMounts:
             - name: loculus-backend-config-processed
               mountPath: /config
diff --git a/kubernetes/loculus/values.schema.json b/kubernetes/loculus/values.schema.json
index 07b20a34fe..5449c1313b 100644
--- a/kubernetes/loculus/values.schema.json
+++ b/kubernetes/loculus/values.schema.json
@@ -1109,12 +1109,12 @@
               "type": "string",
               "description": "TODO"
             },
-            "accesskey": {
+            "accessKey": {
               "groups": ["s3"],
               "type": "string",
               "description": "TODO"
             },
-            "secretkey": {
+            "secretKey": {
               "groups": ["s3"],
               "type": "string",
               "description": "TODO"
diff --git a/kubernetes/loculus/values_preview_server.yaml b/kubernetes/loculus/values_preview_server.yaml
index 5a5ba33055..aba22dc111 100644
--- a/kubernetes/loculus/values_preview_server.yaml
+++ b/kubernetes/loculus/values_preview_server.yaml
@@ -32,8 +32,15 @@ secrets:
     type: sealedsecret
     clusterWide: "true"
     encryptedData:
-      accessKey: AgB+zRfY/q8/2waj00tHmdJAVUuNNM5/Cvi98FPCTppnyL1RZODZbEluJj0tB/xYk3EYgX0brV9Aheyjcwb1VORbPGn5ErQGPRI3J6dbeLMjUm3QlFb3kvmoeRz1ZLWvEHsaUYqrRodsyQUOdVXFEbBxbfrPRbsbSIH1rxa6BcsblgaiTPX3ARij5utvuppTd5ldYiKMNiz72sRVckyvC3J5vJNTCrLKJTX5VUm4/HBayp1dNy713ZM2U6nJyjGV3urd91Kdewaoyfb7w+N8sIjcMlupW6h6TZzsRIcMVQ0OauKmxKZJgAupdoGASHK0Jblbv7xGT5qHwuTfHduad0G5AyX8xWBnNXy3PxcC1Od+FxALXe8yQX8W74xx/j6ha3rTY0k68jW56Y3xl/A7CmfTz8pWuG21l7y3JkudkPJTN9c4eZIvUxZw3lFp9BYZHi/2EfPklSlvZv29DYi5fgoiw1M5ydpEQs5sRzQHlUUIU2S6bJYEBrR5m4+IMzpOFbuM/VC3iz0vivEq9SnS/X63K/8LkpHWs5Q93gBaf2eb54n9xP6YZamJlAzh6twHqNuIhZouZrt23BaC5jLdhCNMPidyHzYrXAXvXukgXy15veR2ah+4iW9scNha6bb8HWoHbrHhaTayLXEFgJ6QxXTXOc6SXBn1RnQ8ZYD7DJJhyp+3ralJmIYF/wIg/IFN4IWFa0BqzYoXujAMy1Lbtrj/OAaLtg==
-      secretKey: AgBa7UboMqoaYNlmxjF+oz88jb5F3gP9fjKmn6SkrOrOvqyKQpukkV6JwX94ytHrtYA51FRAFOPkUHYX5vDwXqLQNOGju8G427yEaVkDJmf6/6OTpLtDF272gVeieolIpaE28c8e8f7LtJv5zfuHSmZVLyw9BTrZpYKBJphZRuHDRIiGuJa+1wtVsZ4vEKVQsgWUlaEwCTLV4lLR2Tf3GSbAAvzNAL6s8WyiNOQVtETeEAGXkXyLP0LaCEwY8l8U/SAuA/Zdi2W4JYEWWDEzGsmswZ0d1b4i8ZBGwLiO23xSrrPk8mO9PL4iB6/A/BCJsqFtMuGueH6Q3DtWp+wg9GM0uhM+VIL1FVSLy4kSavN7bdtSrwD8LVUsxr2lw47YdbXlbOCQOlh1jgUu/IDqUd4YKvHmpdV+GpbWSB7MR0izd6JGya/s6TgcSMmHszwRXSG808ylXYyawTaT1mVfECzo3btuXQjPZ2lAA2FqCJ25ABNoLgEPZe6SZz9aTuTtLia08FiTs0nr31liyhLgPwaJh25I15Q0upCz17Y+6B3VwDMsDcZ4fDC1hQOg/gGSbvi0zeN3TI+gIWEaAu3T3WOyBYq8S8dNst+WgR20dLg+9uJAa+v3sVYdYqmpw6iDP2B4nSicoaIJLKKHXsT4yLYqFZOapw4KW0rUYUqyDopj/Uu/Xg+VatruWKMBtMQAcRwEeyXfdRWOWpsd5fQXFgnP06PA2ytoL0NAZSd4rimSl4bXo/5dD8e0
+      accessKey: AgB1vxhG5eZ+snpMLa1oO1mNyP9XloqCbTZkoFxCipKK4CONzzPUnZmkOSldicrDd/T+HJ0KayM5y/6yG3GkdxlQ6MRwLJfdS3+jYPkFVOY9T8W+s5n8D/A4UdncKumtSwABMQMfq8Vq2bmurMN4AjyEhoe4CavXy+ZL+OkOoNCyKmKF+Ev63crQgNYGStzReCrbss654rgflsvcBm+iUpidx1UjlN/IObgmtdvdr0v08Wz9o4NTRVaIIRi0S7JSpoAWyq3Eh2Q0Oo4cmY+wtrmsKp+dRvEOtRvZJBxSCXWG8+O6lem1571Su14NF8UyhSc15dAkTWJGEJnWuvJz9KzGm7wwSPY6unUgDIzhIFd391Un5HwIyqRu+4qWzhj93idpFAPMdAGJTGeYDoh57IKt70+n5VW21sCoHlbYwtThvlf2nlqxWGnEqqKbZReBVY4OtohQUd9m1NtrN+2CeN06CYfmXlNcNDWdm3UAGjeQ8Ovn4Uq/FQ9jCA4U+yYLNAcqtdkEMm12CWOiubyx3ltgc2tgR1DcCRtVlVpthnZtFkCm4hgZD3sfoofHSyOElT7CB/aOnyyDcij3VCEcr4qvwHLLNVEgwynfBzYZFP9HVdOMvybE7aZDpB2vsAI5sJR25V1Iql6dsEZhQ7DIz72EZ/7UDDD6ufcgF2/ZSc/vdCiIuJyq5GfAd8F5lJEOPaI7fWk17bOtMekYxQRjZSYONijU3g==
+      secretKey: AgAh6Rdr/nKiSdFaEzAnE8qq0Nm57z2K1ARbb8IbGHUtaqDHp1IBHZi1t5KzBySuL3PTRUYFMJ4AgwQ6ZxHxiUrfypOLVRMmik3s8BunDf3PeYTZQ4l3gdByhKQDtwsW72IxOdGEcyYVCo7Rn7u803+R9ZGW4Y05wLBJ6gHlFDxKfBHAe7y0NrKdyQ9BIYmbimZxcKYc0swBb/LSgT3HA/MWTXoswoEULFIMaM9d/y3roeogCGomfRkQbwed4AoiA7n1rEWHM2CtCqWRkt4eWhDpuu70em5SignHZB5I/F1flbqcrhXVmsDFlblT785LSnU8PtkTJGhO1Dw7oWDSG/ADFSHaiR9XU4sjGwafHX8l8RcDZ4GPyzSHC6ZpC7P2mAbYysf/QKlRMHVwTChiIJrSgDC//6h7TqialAu5aS9Eqx/B+q1W7SUOFzY7e2abzpXy89epMwYlWt+iLlv+Zreg512DQFkVd1cNQCjR0gVi2q3ZgbBUkXBym4r3Z3/RE2oNp4/s7RqYFefa2p5v7oJg9H8WfOAJivzmM3V7pkNj18ZPDAfLRB6KpUGpgQIbntoycyGxWEDOFmd8jYNwWEFotBvUnfLDpaPUGH0rMQLvGFhbb3ij6Q74njDZFTsViqu1PUeLfFyweG/8D0i/311IkH9EwJkq9KAhiEINhB93Jzk+sjpx2ZdiITyben7OkFdxuJvQ2s8IM1fB9neop3vlJfVKjq8YR5HAzdhmQb7J7LyJCHlNjsHO
+s3:
+  enabled: true
+  bucket:
+    endpoint: hel1.your-objectstorage.com
+    bucket: loculus-preview-private
+    accessKey: "[[s3accessKey]]"
+    secretKey: "[[s3secretKey]]"      
 reduceResourceRequest: true
 previewDocs: true
 robotsNoindexHeader: true


Reviews:
user: login: chaoran-chen
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-01T15:05:53Z
user: login: fhennig
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-01T15:07:44Z
user: login: chaoran-chen
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-01T15:22:06Z
user: login: chaoran-chen
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-01T15:23:29Z
user: login: chaoran-chen
body: Nice!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-01T15:23:49Z
user: login: fhennig
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-01T15:24:20Z
user: login: fhennig
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-01T15:34:18Z

Review comments:
diff_hunk: @@ -26,7 +26,10 @@ dataUseTerms:
     open: https://#TODO-MVP/open
     restricted: https://#TODO-MVP/restricted
 s3:
-  enabled: false
+  enabled: true
+  bucket:
+    endpoint: hel1.your-objectstorage.com
+    bucket: loculus-preview-private
path: kubernetes/loculus/values.yaml
user: login: chaoran-chen
body: I changed the schema to require accessKey and secretKey if the feature is enabled. Could that be causing issues?
created_at: 2025-04-01T15:05:53Z
author_association: MEMBER
diff_hunk: @@ -26,7 +26,10 @@ dataUseTerms:
     open: https://#TODO-MVP/open
     restricted: https://#TODO-MVP/restricted
 s3:
-  enabled: false
+  enabled: true
+  bucket:
+    endpoint: hel1.your-objectstorage.com
+    bucket: loculus-preview-private
path: kubernetes/loculus/values.yaml
user: login: fhennig
body: Ah probably also that. But also I think the config wouldn't have read the env vars anyways? Maybe with the latest two commits it will work. I set both of the settings, but also in such a way that the config preprocessor should handle the replacing.
created_at: 2025-04-01T15:07:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -134,6 +134,18 @@ We do not currently support branch names containing characters that can't go in
 
 For preview instances this repo contains [sealed secrets](https://sealed-secrets.netlify.app/) that allow the loculus-bot to access the GitHub container registry and (separately) the GitHub repository. These are encrypted such that they can only be decrypted on our cluster but are cluster-wide so can be used in any namespace.
 
+### Adding a sealed secret
+
+Create a secret, for example like this:
+
+    kubectl create secret generic my-secret --from-literal=accessKey=ACC3SK3Y --from-literal=secretKey=S3Cr37K3Y --dry-run=client -o yaml > my-secret.yaml
path: kubernetes/README.md
user: login: chaoran-chen
body: ```suggestion
    kubectl create secret generic my-secret --from-literal=accessKey=<secret> --from-literal=secretKey=<secret> --dry-run=client -o yaml > my-secret.yaml
```

(just my preference)
created_at: 2025-04-01T15:22:06Z
author_association: MEMBER
diff_hunk: @@ -26,7 +26,12 @@ dataUseTerms:
     open: https://#TODO-MVP/open
     restricted: https://#TODO-MVP/restricted
 s3:
-  enabled: false
+  enabled: true
+  bucket:
+    endpoint: hel1.your-objectstorage.com
+    bucket: loculus-preview-private
+    accessKey: "[[s3accessKey]]"
+    secretKey: "[[s3secretKey]]"
path: kubernetes/loculus/values.yaml
user: login: chaoran-chen
body: Can we move this to the preview values? For the default values.yaml, we should disable it.
created_at: 2025-04-01T15:23:29Z
author_association: MEMBER
diff_hunk: @@ -26,7 +26,12 @@ dataUseTerms:
     open: https://#TODO-MVP/open
     restricted: https://#TODO-MVP/restricted
 s3:
-  enabled: false
+  enabled: true
+  bucket:
+    endpoint: hel1.your-objectstorage.com
+    bucket: loculus-preview-private
+    accessKey: "[[s3accessKey]]"
+    secretKey: "[[s3secretKey]]"
path: kubernetes/loculus/values.yaml
user: login: fhennig
body: good idea.
created_at: 2025-04-01T15:24:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -26,7 +26,12 @@ dataUseTerms:
     open: https://#TODO-MVP/open
     restricted: https://#TODO-MVP/restricted
 s3:
-  enabled: false
+  enabled: true
+  bucket:
+    endpoint: hel1.your-objectstorage.com
+    bucket: loculus-preview-private
+    accessKey: "[[s3accessKey]]"
+    secretKey: "[[s3secretKey]]"
path: kubernetes/loculus/values.yaml
user: login: fhennig
body: https://github.com/loculus-project/loculus/pull/3924/commits/603703496178b0dd7879e12469e0e1c717b61427
created_at: 2025-04-01T15:34:18Z
author_association: CONTRIBUTOR

PR comments:
user: login: fhennig
created_at: 2025-04-01T14:55:58Z
body: I looked at the cluster, and this time the secret was unsealed! It worked. Now something else seems to be not working.
author_association: CONTRIBUTOR
user: login: fhennig
created_at: 2025-04-01T15:15:52Z
body: It's working! I have requested a URL and uploaded a file to the bucket!
author_association: CONTRIBUTOR