Repository: kubeflow/manifests
Pull request #3051: Integrate profiles with seaweedfs
Author: login: pschoen-itsc
Labels: name: lgtm
name: approved
name: size/XL
Created: 2025-03-14T15:56:33Z | Merged: 2025-04-27T10:21:00Z

# Pull Request Template for Kubeflow Manifests

## ✏️ Summary of Changes
Use ACLs in seaweedfs to isolate access to artifacts by profile. When a new profile is created, credentials for this profile will be created by seaweedfs and put as secret into the new namespace. ACLs will be set to limited to access from this profile to his subdir in the bucket.

## 📦 Dependencies


## 🐛 Related Issues
Don't know if there is currently an open issue

## ✅ Contributor Checklist
  - [x] I have tested these changes with kustomize. See [Installation Prerequisites](https://github.com/kubeflow/manifests#prerequisites).
  - [x] All commits are [_signed-off_](https://github.com/kubeflow/community/tree/master/dco-signoff-hook#signing-off-commits) to satisfy the DCO check.

---     
 
> You can join the CNCF Slack and access our meetings at the [Kubeflow Community](https://www.kubeflow.org/docs/about/community/) website. Our channel on the CNCF Slack is here [**#kubeflow-platform**](https://app.slack.com/client/T08PSQ7BQ/C073W572LA2).
  



Diff:
diff --git a/.github/workflows/pipeline_swfs_test.yaml b/.github/workflows/pipeline_swfs_test.yaml
index 3cc96c51d8..481b7dcdf5 100644
--- a/.github/workflows/pipeline_swfs_test.yaml
+++ b/.github/workflows/pipeline_swfs_test.yaml
@@ -1,23 +1,24 @@
 name: Test pipelines with seaweedfs
 on:
   workflow_dispatch:
-#  pull_request:
-#    paths:
-#    - tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh
-#    - .github/workflows/pipeline_swfs_test.yaml
-#    - apps/pipeline/upstream/**
-#    - tests/gh-actions/install_istio.sh
-#    - tests/gh-actions/install_cert_manager.sh
-#    - tests/gh-actions/install_oauth2-proxy.sh
-#    - common/cert-manager/**
-#    - common/oauth2-proxy/**
-#    - common/istio*/**
-#    - experimental/seaweedfs/**
+  pull_request:
+    paths:
+    # - tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh
+    - .github/workflows/pipeline_swfs_test.yaml
+    - apps/pipeline/upstream/**
+    # - tests/gh-actions/install_istio.sh
+    # - tests/gh-actions/install_cert_manager.sh
+    # - tests/gh-actions/install_oauth2-proxy.sh
+    # - common/cert-manager/**
+    # - common/oauth2-proxy/**
+    # - common/istio*/**
+    - experimental/seaweedfs/**
 
 jobs:
   build:
     timeout-minutes: 15
-    runs-on: ubuntu-latest
+    runs-on:
+      labels: ubuntu-latest-16-cores
     steps:
     - name: Checkout
       uses: actions/checkout@v4
@@ -41,7 +42,7 @@ jobs:
       run: kustomize build common/kubeflow-namespace/base | kubectl apply -f -
 
     - name: Install KF Pipelines
-      run: ./tests/gh-actions/install_pipelines.sh
+      run: ./tests/gh-actions/install_pipelines_swfs.sh
 
     - name: Install KF Multi Tenancy
       run: ./tests/gh-actions/install_multi_tenancy.sh
@@ -61,13 +62,6 @@ jobs:
         fi
         kubectl get secret mlpipeline-minio-artifact -n "$KF_PROFILE" -o json | jq -r '.data | keys[] as $k | "\($k): \(. | .[$k] | @base64d)"' | tr '\n' ' '
 
-    - name: Install seaweedfs
-      run: |
-        kustomize build experimental/seaweedfs/istio | kubectl apply -f -
-        kubectl -n kubeflow wait --for=condition=available --timeout=120s deploy/seaweedfs
-        kubectl wait --for=condition=Ready pods --all --all-namespaces --timeout=60s --field-selector=status.phase!=Succeeded
-        kubectl -n kubeflow exec deploy/seaweedfs -c seaweedfs -- sh -c "echo \"s3.configure -user minio -access_key minio -secret_key minio123 -actions Read,Write,List -apply\" | /usr/bin/weed shell"
-
     - name: Port forward
       run: ./tests/gh-actions/port_forward_gateway.sh
 
diff --git a/experimental/seaweedfs/base/kustomization.yaml b/experimental/seaweedfs/base/kustomization.yaml
index 166c238dae..3546b1fe4d 100644
--- a/experimental/seaweedfs/base/kustomization.yaml
+++ b/experimental/seaweedfs/base/kustomization.yaml
@@ -3,7 +3,27 @@ kind: Kustomization
 namespace: kubeflow
 
 resources:
-- seaweedfs-deployment.yaml
-- seaweedfs-pvc.yaml
-- seaweedfs-service.yaml
-- seadweedfs-networkpolicy.yaml
+- seaweedfs/
+- ../../../apps/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user
+configMapGenerator:
+- name: kubeflow-pipelines-profile-controller-code
+  behavior: replace
+  files:
+  - pipeline-profile-controller/sync.py
+patches:
+- path: minio-service-patch.yaml
+- path: pipeline-profile-controller/deployment.yaml
+- patch: |-
+    apiVersion: apps/v1
+    kind: Deployment
+    metadata:
+      name: ml-pipeline-ui
+    spec:
+      template:
+        spec:
+          containers:
+          - name: ml-pipeline-ui
+            env:
+            - name: ARTIFACTS_SERVICE_PROXY_ENABLED
+              value: 'false'
+    $patch: merge
diff --git a/experimental/seaweedfs/base/seaweedfs-service.yaml b/experimental/seaweedfs/base/minio-service-patch.yaml
similarity index 100%
rename from experimental/seaweedfs/base/seaweedfs-service.yaml
rename to experimental/seaweedfs/base/minio-service-patch.yaml
diff --git a/experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml b/experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml
new file mode 100644
index 0000000000..114a3076dc
--- /dev/null
+++ b/experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml
@@ -0,0 +1,42 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kubeflow-pipelines-profile-controller
+spec:
+  template:
+    spec:
+      containers:
+      - name: profile-controller
+        securityContext:
+          allowPrivilegeEscalation: false
+          seccompProfile:
+            type: RuntimeDefault
+          runAsNonRoot: true
+          runAsUser: 1000
+          runAsGroup: 0
+          capabilities:
+            drop:
+            - ALL
+        # We just need an image with the python botocore library installed
+        image: docker.io/alpine/k8s:1.32.3
+        command: ["python", "/hooks/sync.py"]
+        env:
+        - name: KFP_VERSION
+          valueFrom:
+            configMapKeyRef:
+              name: pipeline-install-config
+              key: appVersion
+        - name: AWS_ENDPOINT_URL
+          value: http://seaweedfs:8111
+        - name: AWS_REGION
+          value: us-east-1
+        - name: AWS_ACCESS_KEY_ID
+          valueFrom:
+            secretKeyRef:
+              name: mlpipeline-minio-artifact
+              key: accesskey
+        - name: AWS_SECRET_ACCESS_KEY
+          valueFrom:
+            secretKeyRef:
+              name: mlpipeline-minio-artifact
+              key: secretkey
diff --git a/experimental/seaweedfs/base/pipeline-profile-controller/sync.py b/experimental/seaweedfs/base/pipeline-profile-controller/sync.py
new file mode 100644
index 0000000000..bf8c7618fa
--- /dev/null
+++ b/experimental/seaweedfs/base/pipeline-profile-controller/sync.py
@@ -0,0 +1,243 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+S3_BUCKET_NAME = 'mlpipeline'
+
+session = botocore.session.get_session()
+# To interact with seaweedfs user management. Region does not matter.
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://{S3_BUCKET_NAME}/private-artifacts/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": S3_BUCKET_NAME,
+                                "keyFormat": f"private-artifacts/{namespace}/{{{{workflow.name}}}}/{{{{workflow.creationTimestamp.Y}}}}/{{{{workflow.creationTimestamp.m}}}}/{{{{workflow.creationTimestamp.d}}}}/{{{{pod.name}}}}",
+                                "insecure": True,
+                                "accessKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "accesskey",
+                                },
+                                "secretKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "secretkey",
+                                }
+                            }
+                        })
+                    }
+                },
+            ]
+            print('Received request:\n', json.dumps(parent, sort_keys=True))
+            print('Desired resources except secrets:\n', json.dumps(desired_resources, sort_keys=True))
+
+            # Moved after the print argument because this is sensitive data.
+
+            # Check if secret is already there when the controller made the request. If yes, then
+            # use it. Else create a new credentials on seaweedfs for the namespace.
+            if s3_secret := attachments["Secret.v1"].get(f"{namespace}/mlpipeline-minio-artifact"):
+                desired_resources.append(s3_secret)
+                print('Using existing secret')
+            else:
+                print('Creating new access key.')
+                s3_access_key = iam.create_access_key(UserName=namespace)
+                # Use the AWS IAM API of seaweedfs to manage access policies to bucket.
+                # This policy ensures that a user can only access artifacts from his own profile.
+                iam.put_user_policy(
+                    UserName=namespace,
+                    PolicyName=f"KubeflowProject{namespace}",
+                    PolicyDocument=json.dumps(
+                        {
+                            "Version": "2012-10-17",
+                            "Statement": [{
+                                "Effect": "Allow",
+                                "Action": [
+                                    "s3:Put*",
+                                    "s3:Get*",
+                                    "s3:List*"
+                                ],
+                                "Resource": [
+                                    f"arn:aws:s3:::{S3_BUCKET_NAME}/artifacts/*",
+                                    f"arn:aws:s3:::{S3_BUCKET_NAME}/private-artifacts/{namespace}/*",
+                                    f"arn:aws:s3:::{S3_BUCKET_NAME}/private/{namespace}/*",
+                                    f"arn:aws:s3:::{S3_BUCKET_NAME}/shared/*",
+                                ]
+                            }]
+                        })
+                )
+                desired_resources.insert(
+                    0,
+                    {
+                        "apiVersion": "v1",
+                        "kind": "Secret",
+                        "metadata": {
+                            "name": "mlpipeline-minio-artifact",
+                            "namespace": namespace,
+                        },
+                        "data": {
+                            "accesskey": base64.b64encode(s3_access_key["AccessKey"]["AccessKeyId"].encode('utf-8')).decode("utf-8"),
+                            "secretkey": base64.b64encode(s3_access_key["AccessKey"]["SecretAccessKey"].encode('utf-8')).decode("utf-8"),
+                    },
+                })
+
+            return {"status": desired_status, "attachments": desired_resources}
+
+        def do_POST(self):
+            # Serve the sync() function as a JSON webhook.
+            observed = json.loads(
+                self.rfile.read(int(self.headers.get("content-length"))))
+            desired = self.sync(observed["object"], observed["attachments"])
+
+            self.send_response(200)
+            self.send_header("Content-type", "application/json")
+            self.end_headers()
+            self.wfile.write(bytes(json.dumps(desired), 'utf-8'))
+
+    return HTTPServer((url, int(controller_port)), Controller)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/experimental/seaweedfs/base/seaweedfs/kustomization.yaml b/experimental/seaweedfs/base/seaweedfs/kustomization.yaml
new file mode 100644
index 0000000000..774997572d
--- /dev/null
+++ b/experimental/seaweedfs/base/seaweedfs/kustomization.yaml
@@ -0,0 +1,11 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: kubeflow
+
+resources:
+- seaweedfs-deployment.yaml
+- seaweedfs-pvc.yaml
+- seaweedfs-networkpolicy.yaml
+- seaweedfs-create-admin-user-job.yaml
+- seaweedfs-service.yaml
+- seaweedfs-service-account.yaml
diff --git a/experimental/seaweedfs/base/seaweedfs/seaweedfs-create-admin-user-job.yaml b/experimental/seaweedfs/base/seaweedfs/seaweedfs-create-admin-user-job.yaml
new file mode 100644
index 0000000000..491ad460d5
--- /dev/null
+++ b/experimental/seaweedfs/base/seaweedfs/seaweedfs-create-admin-user-job.yaml
@@ -0,0 +1,67 @@
+kind: Job
+apiVersion: batch/v1
+metadata:
+  name: init-seaweedfs
+spec:
+  template:
+    metadata:
+      name: init-seaweedfs
+    spec:
+      restartPolicy: OnFailure
+      containers:
+      - name: init-seaweedfs
+        image: 'chrislusf/seaweedfs:3.85'
+        env:
+        - name: WEED_CLUSTER_DEFAULT
+          value: "sw"
+        - name: WEED_CLUSTER_SW_MASTER
+          value: "seaweedfs.kubeflow:9333"
+        envFrom:
+        - secretRef:
+            name: mlpipeline-minio-artifact
+        command:
+        - "/bin/sh"
+        - "-ec"
+        - |
+          wait_for_service() {
+            local url=$1
+            local max_attempts=60  # 5 minutes total (5s * 60)
+            local attempt=1
+          
+            echo "Waiting for service at $url..."
+            while [ $attempt -le $max_attempts ]; do
+              if wget -q --spider "$url" >/dev/null 2>&1; then
+                echo "Service at $url is up!"
+                return 0
+              fi
+              echo "Attempt $attempt: Service not ready yet, retrying in 5s..."
+              sleep 5
+              attempt=$((attempt + 1))
+            done
+            echo "Service at $url failed to become ready within 5 minutes"
+            exit 1
+          }
+          wait_for_service "http://minio-service.kubeflow:9000/status"
+          exec /bin/echo "s3.bucket.create --name mlpipeline" | /usr/bin/weed shell
+          exec /bin/echo \
+          "s3.configure -user kubeflow-admin \
+                        -access_key $accesskey \
+                        -secret_key $secretkey \
+                        -actions Admin \
+                        -apply" |\
+          /usr/bin/weed shell
+        securityContext:  # Using restricted profile
+          allowPrivilegeEscalation: false
+          privileged: false
+          runAsNonRoot: true
+          # image defaults to root user
+          runAsUser: 1001
+          runAsGroup: 1001
+          seccompProfile:
+            type: RuntimeDefault
+          capabilities:
+            drop:
+            - ALL
+            add:
+            - NET_BIND_SERVICE
+      serviceAccountName: seaweedfs
diff --git a/experimental/seaweedfs/base/seaweedfs-deployment.yaml b/experimental/seaweedfs/base/seaweedfs/seaweedfs-deployment.yaml
similarity index 84%
rename from experimental/seaweedfs/base/seaweedfs-deployment.yaml
rename to experimental/seaweedfs/base/seaweedfs/seaweedfs-deployment.yaml
index 002c4eef9d..a09d9b947c 100644
--- a/experimental/seaweedfs/base/seaweedfs-deployment.yaml
+++ b/experimental/seaweedfs/base/seaweedfs/seaweedfs-deployment.yaml
@@ -17,16 +17,22 @@ spec:
     metadata:
       labels:
         app: seaweedfs
+        application-crd-id: kubeflow-pipelines
     spec:
       containers:
       - name: seaweedfs
-        image: 'chrislusf/seaweedfs:3.69'
+        image: 'chrislusf/seaweedfs:3.85'
         args:
         - 'server'
         - '-dir=/data'
         - '-s3'
+        - '-iam'
         ports:
         - containerPort: 8333
+        - containerPort: 8111
+        - containerPort: 9333
+        - containerPort: 19333
+        - containerPort: 8888
         readinessProbe:
           httpGet:
             path: /status
@@ -63,3 +69,4 @@ spec:
       - name: data
         persistentVolumeClaim:
           claimName: seaweedfs-pvc
+      serviceAccountName: seaweedfs
diff --git a/experimental/seaweedfs/base/seadweedfs-networkpolicy.yaml b/experimental/seaweedfs/base/seaweedfs/seaweedfs-networkpolicy.yaml
similarity index 67%
rename from experimental/seaweedfs/base/seadweedfs-networkpolicy.yaml
rename to experimental/seaweedfs/base/seaweedfs/seaweedfs-networkpolicy.yaml
index 6d2cffbdc9..ab1a6cc665 100644
--- a/experimental/seaweedfs/base/seadweedfs-networkpolicy.yaml
+++ b/experimental/seaweedfs/base/seaweedfs/seaweedfs-networkpolicy.yaml
@@ -1,5 +1,18 @@
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
+metadata:
+  name: default-allow-same-namespace
+  namespace: kubeflow
+spec:
+  podSelector: {}
+  ingress:
+    - from:
+        - podSelector: {}
+  policyTypes:
+    - Ingress
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
 metadata:
   name: seaweedfs
 spec:
@@ -11,13 +24,15 @@ spec:
           operator: In
           values:
           - kubeflow-profile
+    ports:
+    - port: 8333
+  - from:
     - namespaceSelector:
         matchExpressions:
         - key: kubernetes.io/metadata.name
           operator: In
           values:
           - istio-system
-    - podSelector: {}
   podSelector:
     matchExpressions:
     - key: app
diff --git a/experimental/seaweedfs/base/seaweedfs-pvc.yaml b/experimental/seaweedfs/base/seaweedfs/seaweedfs-pvc.yaml
similarity index 100%
rename from experimental/seaweedfs/base/seaweedfs-pvc.yaml
rename to experimental/seaweedfs/base/seaweedfs/seaweedfs-pvc.yaml
diff --git a/experimental/seaweedfs/base/seaweedfs/seaweedfs-service-account.yaml b/experimental/seaweedfs/base/seaweedfs/seaweedfs-service-account.yaml
new file mode 100644
index 0000000000..9e0b2176f2
--- /dev/null
+++ b/experimental/seaweedfs/base/seaweedfs/seaweedfs-service-account.yaml
@@ -0,0 +1,4 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: seaweedfs
diff --git a/experimental/seaweedfs/base/seaweedfs/seaweedfs-service.yaml b/experimental/seaweedfs/base/seaweedfs/seaweedfs-service.yaml
new file mode 100644
index 0000000000..0134c34f39
--- /dev/null
+++ b/experimental/seaweedfs/base/seaweedfs/seaweedfs-service.yaml
@@ -0,0 +1,33 @@
+# Separate service for new ports of seaweedfs. If we add them to the existing minio-service there will be a problem
+# with the mlpipeline api server because it relies on MINIO_SERVICE_SERVICE_PORT pointing to the S3 port.
+# But with multiple ports on a service that is not really reliable. So we use the existing minio-service for
+# backwards-compatibility, but everything new, seaweedfs related is here.
+apiVersion: v1
+kind: Service
+metadata:
+  name: seaweedfs
+  namespace: kubeflow
+spec:
+  ports:
+  - name: http-iam
+    port: 8111
+    protocol: TCP
+    targetPort: 8111
+  - name: http-master
+    port: 9333
+    protocol: TCP
+    targetPort: 9333
+  - name: grpc-master
+    port: 19333
+    protocol: TCP
+    targetPort: 19333
+  - name: grpc-filer
+    port: 18888
+    protocol: TCP
+    targetPort: 18888
+  - name: http-filer
+    port: 8888
+    protocol: TCP
+    targetPort: 8888
+  selector:
+    app: seaweedfs
diff --git a/experimental/seaweedfs/istio/istio-authorization-policy.yaml b/experimental/seaweedfs/istio/istio-authorization-policy.yaml
index 409b2c1b4c..4c6ac2c163 100644
--- a/experimental/seaweedfs/istio/istio-authorization-policy.yaml
+++ b/experimental/seaweedfs/istio/istio-authorization-policy.yaml
@@ -16,7 +16,9 @@ spec:
     - source:
         principals:
         - cluster.local/ns/kubeflow/sa/ml-pipeline-ui
-  # Allow traffic from User Pipeline Pods, which don't have a sidecar.
+  # Allow traffic to s3 endpoint from User Pipeline Pods, which don't have a sidecar.
+  # Also needed for traffic from seaweedfs init pod. Seaweedfs gives the client an ip to connect to. This can not be
+  # handled well by istio (AuthPolicy). Instead, access to the sensitive ports will be limited by the NetworkPolicy.
   - {}
 ---
 apiVersion: "networking.istio.io/v1alpha3"
@@ -24,7 +26,7 @@ kind: DestinationRule
 metadata:
   name: ml-pipeline-seaweedfs
 spec:
-  host: minio-service.kubeflow.svc.cluster.local
+  host: seaweedfs.kubeflow.svc.cluster.local
   trafficPolicy:
     tls:
       mode: ISTIO_MUTUAL
diff --git a/tests/gh-actions/install_pipelines_swfs.sh b/tests/gh-actions/install_pipelines_swfs.sh
new file mode 100755
index 0000000000..e01abb6757
--- /dev/null
+++ b/tests/gh-actions/install_pipelines_swfs.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+set -euo pipefail
+echo "Installing Pipelines ..."
+kubectl apply -f apps/pipeline/upstream/third-party/metacontroller/base/crd.yaml
+echo "Waiting for crd/compositecontrollers.metacontroller.k8s.io to be available ..."
+kubectl wait --for condition=established --timeout=30s crd/compositecontrollers.metacontroller.k8s.io
+kustomize build experimental/seaweedfs/istio | kubectl apply -f -
+sleep 60
+kubectl wait --for=condition=Ready pods --all --all-namespaces --timeout=600s \
+  --field-selector=status.phase!=Succeeded
+
+kubectl wait --for=condition=Available deployment/ml-pipeline -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/ml-pipeline-ui -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/ml-pipeline-persistenceagent -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/ml-pipeline-scheduledworkflow -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/ml-pipeline-viewer-crd -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/cache-server -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/metadata-writer -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/seaweedfs -n kubeflow --timeout=10s
+kubectl wait --for=condition=Available deployment/mysql -n kubeflow --timeout=10s
+kubectl get deployment -n kubeflow -l app=ml-pipeline


Reviews:
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T15:58:09Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T16:03:00Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:07:07Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:14:17Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:20:21Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:21:54Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:32:47Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:35:35Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:37:05Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:38:55Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:40:54Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:41:21Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:44:49Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:49:59Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:54:40Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:55:24Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T17:56:58Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T18:13:16Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-16T18:20:35Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T07:37:24Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T07:53:33Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T07:59:17Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T08:11:01Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T13:01:25Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T13:07:27Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T13:13:35Z
user: login: juliusvonkohout
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T13:14:20Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T13:17:30Z
user: login: milinddethe15
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-25T17:27:50Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T18:07:25Z
user: login: milinddethe15
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-25T18:19:54Z
user: login: milinddethe15
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-25T18:26:45Z
user: login: pschoen-itsc
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-26T03:09:09Z

Review comments:
diff_hunk: @@ -0,0 +1,8 @@
+kind: Secret
+apiVersion: v1
+metadata:
+  name: seaweedfs-admin
+stringData:
+  accesskey: "kubeflow-admin"
+  secretkey: "ichah6aiko6Ubae4xahthahm9oeRaed2"
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-admin-secret.yaml
user: login: pschoen-itsc
body: This is something that should be generated for every new kubeflow installation
created_at: 2025-03-14T15:58:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,65 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kubeflow-pipelines-profile-controller
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: kubeflow-pipelines-profile-controller
+      app.kubernetes.io/component: ml-pipeline
+      app.kubernetes.io/name: kubeflow-pipelines
+      application-crd-id: kubeflow-pipelines
+  template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"
+      labels:
+        app: kubeflow-pipelines-profile-controller
+        app.kubernetes.io/component: ml-pipeline
+        app.kubernetes.io/name: kubeflow-pipelines
+        application-crd-id: kubeflow-pipelines
+    spec:
+      containers:
+      - name: profile-controller
+        securityContext:
+          allowPrivilegeEscalation: false
+          seccompProfile:
+            type: RuntimeDefault
+          runAsNonRoot: true
+          runAsUser: 1000
+          runAsGroup: 0
+          capabilities:
+            drop:
+            - ALL
+        image: alpine/k8s:1.29.13
path: experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml
user: login: pschoen-itsc
body: Change of image because we need the botocore libraries for interaction with the IAM service
created_at: 2025-03-14T16:03:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,65 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kubeflow-pipelines-profile-controller
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: kubeflow-pipelines-profile-controller
+      app.kubernetes.io/component: ml-pipeline
+      app.kubernetes.io/name: kubeflow-pipelines
+      application-crd-id: kubeflow-pipelines
+  template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"
+      labels:
+        app: kubeflow-pipelines-profile-controller
+        app.kubernetes.io/component: ml-pipeline
+        app.kubernetes.io/name: kubeflow-pipelines
+        application-crd-id: kubeflow-pipelines
+    spec:
+      containers:
+      - name: profile-controller
+        securityContext:
+          allowPrivilegeEscalation: false
+          seccompProfile:
+            type: RuntimeDefault
+          runAsNonRoot: true
+          runAsUser: 1000
+          runAsGroup: 0
+          capabilities:
+            drop:
+            - ALL
+        image: alpine/k8s:1.29.13
path: experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml
user: login: juliusvonkohout
body: Can you make it a small patch of the existing pipelines profile controller instead of a full resource ? We should try to stay as close as possible to the current setup. Can we also reuse the minio secrets and environment variables instead of creating new ones  as in https://github.com/kubeflow/pipelines/pull/7725 ?
created_at: 2025-03-16T17:07:07Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-seaweedfs-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": "kubeflow",
+                                "keyFormat": f"{namespace}/artifacts/{{{{workflow.name}}}}/{{{{workflow.creationTimestamp.Y}}}}/{{{{workflow.creationTimestamp.m}}}}/{{{{workflow.creationTimestamp.d}}}}/{{{{pod.name}}}}",
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: {namespace}/artifacts/ should be private-artifacts/{namespace}/


see also https://github.com/kubeflow/pipelines/pull/7725 for the structure. where aws:username is just the namespace name
```
"Resource": [
    "arn:aws:s3:::%s/artifacts/*"  % shared_bucket_name, # old shared artifacts for backwards compatibility
    "arn:aws:s3:::%s/private-artifacts/${aws:username}/*"  % shared_bucket_name, # private artifacts usually with a lifecycle cleanup policy
    "arn:aws:s3:::%s/private/${aws:username}/*"  % shared_bucket_name, # private storage
    "arn:aws:s3:::%s/shared/*"  % shared_bucket_name # shared storage for collaboration
]
```


created_at: 2025-03-16T17:14:17Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: what about jsut "default-namespaced" ?
created_at: 2025-03-16T17:20:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: {namespace}/v2/artifacts should be private-artifacts/{namespace}/v2/artifacts or so to allow for better pruning of old artifacts.
created_at: 2025-03-16T17:21:54Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-seaweedfs-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": "kubeflow",
+                                "keyFormat": f"{namespace}/artifacts/{{{{workflow.name}}}}/{{{{workflow.creationTimestamp.Y}}}}/{{{{workflow.creationTimestamp.m}}}}/{{{{workflow.creationTimestamp.d}}}}/{{{{pod.name}}}}",
+                                "insecure": True,
+                                "accessKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "accesskey",
+                                },
+                                "secretKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "secretkey",
+                                }
+                            }
+                        })
+                    }
+                },
+                # Visualization server related manifests below
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-visualizationserver"
+                        },
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            },
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-visualizationserver"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "image": f"{visualization_server_image}:{visualization_server_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "name":
+                                        "ml-pipeline-visualizationserver",
+                                    "ports": [{
+                                        "containerPort": 8888
+                                    }],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "50m",
+                                            "memory": "200Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "500m",
+                                            "memory": "1Gi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor",
+                            },
+                        },
+                    },
+                },
+                {
+                    "apiVersion": "networking.istio.io/v1alpha3",
+                    "kind": "DestinationRule",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "host": "ml-pipeline-visualizationserver",
+                        "trafficPolicy": {
+                            "tls": {
+                                "mode": "ISTIO_MUTUAL"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "security.istio.io/v1beta1",
+                    "kind": "AuthorizationPolicy",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            }
+                        },
+                        "rules": [{
+                            "from": [{
+                                "source": {
+                                    "principals": ["cluster.local/ns/kubeflow/sa/ml-pipeline"]
+                                }
+                            }]
+                        }]
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name": "http",
+                            "port": 8888,
+                            "protocol": "TCP",
+                            "targetPort": 8888,
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-visualizationserver",
+                        },
+                    },
+                },
+                # Artifact fetcher related resources below.
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        },
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-ui-artifact"
+                            }
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-ui-artifact"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "name":
+                                        "ml-pipeline-ui-artifact",
+                                    "image": f"{frontend_image}:{frontend_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "ports": [{
+                                        "containerPort": 3000
+                                    }],
+                                    "env": [
+                                        {
+                                            "name": "MINIO_ACCESS_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "accesskey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        },
+                                        {
+                                            "name": "MINIO_SECRET_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "secretkey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        }
+                                    ],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "10m",
+                                            "memory": "70Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "100m",
+                                            "memory": "500Mi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name":
+                                "http",  # name is required to let istio understand request protocol
+                            "port": 80,
+                            "protocol": "TCP",
+                            "targetPort": 3000
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    }
+                },
+            ]
+            print('Received request:\n', json.dumps(parent, sort_keys=True))
+            print('Desired resources except secrets:\n', json.dumps(desired_resources, sort_keys=True))
+
+            # Moved after the print argument because this is sensitive data.
+
+            # This could lead to the creation of multiple access keys for the same profile
+            # which is not an imminent problem, but yet not that elegant. But it is simple
+            # and ensures that there is always a secret with valid keys.
+            # That is likely why metacontroller docs recommend not having any side effects in the
+            # hook function.
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: Can you elaborate a bit on that ?
created_at: 2025-03-16T17:32:47Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-seaweedfs-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": "kubeflow",
+                                "keyFormat": f"{namespace}/artifacts/{{{{workflow.name}}}}/{{{{workflow.creationTimestamp.Y}}}}/{{{{workflow.creationTimestamp.m}}}}/{{{{workflow.creationTimestamp.d}}}}/{{{{pod.name}}}}",
+                                "insecure": True,
+                                "accessKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "accesskey",
+                                },
+                                "secretKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "secretkey",
+                                }
+                            }
+                        })
+                    }
+                },
+                # Visualization server related manifests below
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-visualizationserver"
+                        },
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            },
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-visualizationserver"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: please add a securitycontext as in https://github.com/kubeflow/manifests/pull/3050 for the deprecated visualization-server and artifact-proxy or even better aim for https://github.com/kubeflow/manifests/pull/3051/files#r1997669517
created_at: 2025-03-16T17:35:34Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,8 @@
+kind: Secret
+apiVersion: v1
+metadata:
+  name: seaweedfs-admin
+stringData:
+  accesskey: "kubeflow-admin"
+  secretkey: "ichah6aiko6Ubae4xahthahm9oeRaed2"
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-admin-secret.yaml
user: login: juliusvonkohout
body: can you use the existing minio admin secret in the namespace kubeflow ? I think this one is also used by ml-pipeline-ui and other deployments. We need it to later get rid of the artifact proxy and save tons of memory on large clusters.
created_at: 2025-03-16T17:37:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,52 @@
+kind: Job
+apiVersion: batch/v1
+metadata:
+  name: init-seaweedfs
+spec:
+  template:
+    metadata:
+      name: init-seaweedfs
+    spec:
+      restartPolicy: Never
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-create-admin-user-job.yaml
user: login: juliusvonkohout
body: I think it should restart until it is successful
created_at: 2025-03-16T17:38:55Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,8 @@
+kind: Secret
+apiVersion: v1
+metadata:
+  name: seaweedfs-admin
+stringData:
+  accesskey: "kubeflow-admin"
+  secretkey: "ichah6aiko6Ubae4xahthahm9oeRaed2"
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-admin-secret.yaml
user: login: juliusvonkohout
body: We can also do it directly in this PR here. The visualization-server can just be removed and the artifact proxy can be removed from sync.py if you add

```
patches:
- target:
    kind: Deployment
    name: ml-pipeline-ui
    namespace: kubeflow
  patch: |-
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: ml-pipeline-ui
      namespace: kubeflow
    spec:
      template:
        spec:
          containers:
          - name: ml-pipeline-ui
            env:
            - name: ARTIFACTS_SERVICE_PROXY_ENABLED
              value: 'false'
```
created_at: 2025-03-16T17:40:54Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,8 @@
+kind: Secret
+apiVersion: v1
+metadata:
+  name: seaweedfs-admin
+stringData:
+  accesskey: "kubeflow-admin"
+  secretkey: "ichah6aiko6Ubae4xahthahm9oeRaed2"
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-admin-secret.yaml
user: login: juliusvonkohout
body: Then we can also skip the securitycontexts for those visualization-server and artifact-proxy
created_at: 2025-03-16T17:41:21Z
author_association: MEMBER
diff_hunk: @@ -11,18 +11,20 @@ spec:
           operator: In
           values:
           - kubeflow-profile
+    ports:
+    - port: 8333
+  - from:
     - namespaceSelector:
         matchExpressions:
         - key: kubernetes.io/metadata.name
           operator: In
           values:
           - istio-system
-    - podSelector: {}
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-networkpolicy.yaml
user: login: juliusvonkohout
body: ml-pipeline-ui and maybe others need it as well without the artifact proxy
created_at: 2025-03-16T17:44:48Z
author_association: MEMBER
path: experimental/seaweedfs/istio/istio-authorization-policy.yaml
user: login: juliusvonkohout
body: Lets make sure that it is in line with

https://github.com/kubeflow/pipelines/blob/8d0ae5381e8366905c90009c56fd0e4807e94f0f/manifests/kustomize/third-party/minio/options/istio/istio-authorization-policy.yaml#L1-L30

created_at: 2025-03-16T17:49:58Z
author_association: MEMBER
diff_hunk: @@ -12,11 +12,23 @@ spec:
     - source:
         principals:
         - cluster.local/ns/kubeflow/sa/ml-pipeline
-  - from:
-    - source:
-        principals:
         - cluster.local/ns/kubeflow/sa/ml-pipeline-ui
-  # Allow traffic from User Pipeline Pods, which don't have a sidecar.
+    to:
+    - operation:
+        ports: ["8333"]
+  # Allow traffic to s3 endpoint from User Pipeline Pods, which don't have a sidecar.
+  - {}
+---
+apiVersion: security.istio.io/v1beta1
+kind: AuthorizationPolicy
+metadata:
+  name: pipeline-profile-controller
+spec:
+  action: ALLOW
+  selector:
+    matchLabels:
+      app: kubeflow-pipelines-profile-controller
+  rules:
   - {}
path: experimental/seaweedfs/istio/istio-authorization-policy.yaml
user: login: juliusvonkohout
body: What is this policy for ? Who should have access to pipelines-profile-controller? I think only metacontoller should call it. Please also compare with https://github.com/kubeflow/pipelines/blob/master/manifests/kustomize/base/installs/multi-user/istio-authorization-config.yaml and https://github.com/kubeflow/pipelines/blob/8d0ae5381e8366905c90009c56fd0e4807e94f0f/manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/deployment.yaml#L10
created_at: 2025-03-16T17:54:40Z
author_association: MEMBER
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: Please also highlight the changes to the original sync.py somwhow.
created_at: 2025-03-16T17:55:24Z
author_association: MEMBER
diff_hunk: @@ -3,7 +3,5 @@ kind: Kustomization
 namespace: kubeflow
 
 resources:
-- seaweedfs-deployment.yaml
-- seaweedfs-pvc.yaml
-- seaweedfs-service.yaml
-- seadweedfs-networkpolicy.yaml
+- seaweedfs/
+- pipeline-profile-controller/
path: experimental/seaweedfs/base/kustomization.yaml
user: login: juliusvonkohout
body: - pipeline-profile-controller/ should be a patch
created_at: 2025-03-16T17:56:58Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,65 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kubeflow-pipelines-profile-controller
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: kubeflow-pipelines-profile-controller
+      app.kubernetes.io/component: ml-pipeline
+      app.kubernetes.io/name: kubeflow-pipelines
+      application-crd-id: kubeflow-pipelines
+  template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"
+      labels:
+        app: kubeflow-pipelines-profile-controller
+        app.kubernetes.io/component: ml-pipeline
+        app.kubernetes.io/name: kubeflow-pipelines
+        application-crd-id: kubeflow-pipelines
+    spec:
+      containers:
+      - name: profile-controller
+        securityContext:
+          allowPrivilegeEscalation: false
+          seccompProfile:
+            type: RuntimeDefault
+          runAsNonRoot: true
+          runAsUser: 1000
+          runAsGroup: 0
+          capabilities:
+            drop:
+            - ALL
+        image: alpine/k8s:1.29.13
path: experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml
user: login: juliusvonkohout
body: You can also target the latest 1.32.x version instead of 1.29.13. please also do not use docker.io but Pleae also use https://explore.ggcr.dev/?repo=registry.k8s.io registry.k8s.io
created_at: 2025-03-16T18:13:16Z
author_association: MEMBER
path: experimental/seaweedfs/istio/istio-authorization-policy.yaml
user: login: juliusvonkohout
body: We might need

```
  - from:
    - source:
        principals:
        - cluster.local/ns/kubeflow/sa/ml-pipeline-ui
```

as well if we remove the artifact proxy.
created_at: 2025-03-16T18:20:35Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,8 @@
+kind: Secret
+apiVersion: v1
+metadata:
+  name: seaweedfs-admin
+stringData:
+  accesskey: "kubeflow-admin"
+  secretkey: "ichah6aiko6Ubae4xahthahm9oeRaed2"
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-admin-secret.yaml
user: login: pschoen-itsc
body: That should be no problem.
created_at: 2025-03-18T07:37:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-seaweedfs-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": "kubeflow",
+                                "keyFormat": f"{namespace}/artifacts/{{{{workflow.name}}}}/{{{{workflow.creationTimestamp.Y}}}}/{{{{workflow.creationTimestamp.m}}}}/{{{{workflow.creationTimestamp.d}}}}/{{{{pod.name}}}}",
+                                "insecure": True,
+                                "accessKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "accesskey",
+                                },
+                                "secretKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "secretkey",
+                                }
+                            }
+                        })
+                    }
+                },
+                # Visualization server related manifests below
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-visualizationserver"
+                        },
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            },
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-visualizationserver"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "image": f"{visualization_server_image}:{visualization_server_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "name":
+                                        "ml-pipeline-visualizationserver",
+                                    "ports": [{
+                                        "containerPort": 8888
+                                    }],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "50m",
+                                            "memory": "200Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "500m",
+                                            "memory": "1Gi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor",
+                            },
+                        },
+                    },
+                },
+                {
+                    "apiVersion": "networking.istio.io/v1alpha3",
+                    "kind": "DestinationRule",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "host": "ml-pipeline-visualizationserver",
+                        "trafficPolicy": {
+                            "tls": {
+                                "mode": "ISTIO_MUTUAL"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "security.istio.io/v1beta1",
+                    "kind": "AuthorizationPolicy",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            }
+                        },
+                        "rules": [{
+                            "from": [{
+                                "source": {
+                                    "principals": ["cluster.local/ns/kubeflow/sa/ml-pipeline"]
+                                }
+                            }]
+                        }]
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name": "http",
+                            "port": 8888,
+                            "protocol": "TCP",
+                            "targetPort": 8888,
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-visualizationserver",
+                        },
+                    },
+                },
+                # Artifact fetcher related resources below.
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        },
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-ui-artifact"
+                            }
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-ui-artifact"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "name":
+                                        "ml-pipeline-ui-artifact",
+                                    "image": f"{frontend_image}:{frontend_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "ports": [{
+                                        "containerPort": 3000
+                                    }],
+                                    "env": [
+                                        {
+                                            "name": "MINIO_ACCESS_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "accesskey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        },
+                                        {
+                                            "name": "MINIO_SECRET_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "secretkey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        }
+                                    ],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "10m",
+                                            "memory": "70Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "100m",
+                                            "memory": "500Mi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name":
+                                "http",  # name is required to let istio understand request protocol
+                            "port": 80,
+                            "protocol": "TCP",
+                            "targetPort": 3000
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    }
+                },
+            ]
+            print('Received request:\n', json.dumps(parent, sort_keys=True))
+            print('Desired resources except secrets:\n', json.dumps(desired_resources, sort_keys=True))
+
+            # Moved after the print argument because this is sensitive data.
+
+            # This could lead to the creation of multiple access keys for the same profile
+            # which is not an imminent problem, but yet not that elegant. But it is simple
+            # and ensures that there is always a secret with valid keys.
+            # That is likely why metacontroller docs recommend not having any side effects in the
+            # hook function.
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: pschoen-itsc
body: During development I observed that the sync endpoint was called multiple times when creating a namespace, which resulted in creating multiple keypairs in seaweedfs. That wasn't a problem by itself, just a bunch of access keys which will never be used. During my last test (after the rebase) I didn't observe this behauviour anymore. I will keep this in mind and will remove this comment if this was resolved  just by itself.
created_at: 2025-03-18T07:53:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,52 @@
+kind: Job
+apiVersion: batch/v1
+metadata:
+  name: init-seaweedfs
+spec:
+  template:
+    metadata:
+      name: init-seaweedfs
+    spec:
+      restartPolicy: Never
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-create-admin-user-job.yaml
user: login: pschoen-itsc
body: That was mostly just copied from their helm chart https://github.com/seaweedfs/seaweedfs/blob/master/k8s/charts/seaweedfs/templates/post-install-bucket-hook.yaml
But I agree with you. Until succeesful makes more sense.
created_at: 2025-03-18T07:59:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,65 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kubeflow-pipelines-profile-controller
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: kubeflow-pipelines-profile-controller
+      app.kubernetes.io/component: ml-pipeline
+      app.kubernetes.io/name: kubeflow-pipelines
+      application-crd-id: kubeflow-pipelines
+  template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"
+      labels:
+        app: kubeflow-pipelines-profile-controller
+        app.kubernetes.io/component: ml-pipeline
+        app.kubernetes.io/name: kubeflow-pipelines
+        application-crd-id: kubeflow-pipelines
+    spec:
+      containers:
+      - name: profile-controller
+        securityContext:
+          allowPrivilegeEscalation: false
+          seccompProfile:
+            type: RuntimeDefault
+          runAsNonRoot: true
+          runAsUser: 1000
+          runAsGroup: 0
+          capabilities:
+            drop:
+            - ALL
+        image: alpine/k8s:1.29.13
path: experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml
user: login: pschoen-itsc
body: @juliusvonkohout Do you know if any of theese images has botocore / boto3 included? The **only** reason I'm using that image is because it has that installed. 
An alternative would be to build an own image, but I would need some directions for that process in the kubeflow project.
created_at: 2025-03-18T08:11:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-seaweedfs-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": "kubeflow",
+                                "keyFormat": f"{namespace}/artifacts/{{{{workflow.name}}}}/{{{{workflow.creationTimestamp.Y}}}}/{{{{workflow.creationTimestamp.m}}}}/{{{{workflow.creationTimestamp.d}}}}/{{{{pod.name}}}}",
+                                "insecure": True,
+                                "accessKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "accesskey",
+                                },
+                                "secretKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "secretkey",
+                                }
+                            }
+                        })
+                    }
+                },
+                # Visualization server related manifests below
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-visualizationserver"
+                        },
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            },
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-visualizationserver"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "image": f"{visualization_server_image}:{visualization_server_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "name":
+                                        "ml-pipeline-visualizationserver",
+                                    "ports": [{
+                                        "containerPort": 8888
+                                    }],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "50m",
+                                            "memory": "200Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "500m",
+                                            "memory": "1Gi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor",
+                            },
+                        },
+                    },
+                },
+                {
+                    "apiVersion": "networking.istio.io/v1alpha3",
+                    "kind": "DestinationRule",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "host": "ml-pipeline-visualizationserver",
+                        "trafficPolicy": {
+                            "tls": {
+                                "mode": "ISTIO_MUTUAL"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "security.istio.io/v1beta1",
+                    "kind": "AuthorizationPolicy",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            }
+                        },
+                        "rules": [{
+                            "from": [{
+                                "source": {
+                                    "principals": ["cluster.local/ns/kubeflow/sa/ml-pipeline"]
+                                }
+                            }]
+                        }]
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name": "http",
+                            "port": 8888,
+                            "protocol": "TCP",
+                            "targetPort": 8888,
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-visualizationserver",
+                        },
+                    },
+                },
+                # Artifact fetcher related resources below.
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        },
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-ui-artifact"
+                            }
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-ui-artifact"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "name":
+                                        "ml-pipeline-ui-artifact",
+                                    "image": f"{frontend_image}:{frontend_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "ports": [{
+                                        "containerPort": 3000
+                                    }],
+                                    "env": [
+                                        {
+                                            "name": "MINIO_ACCESS_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "accesskey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        },
+                                        {
+                                            "name": "MINIO_SECRET_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "secretkey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        }
+                                    ],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "10m",
+                                            "memory": "70Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "100m",
+                                            "memory": "500Mi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name":
+                                "http",  # name is required to let istio understand request protocol
+                            "port": 80,
+                            "protocol": "TCP",
+                            "targetPort": 3000
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    }
+                },
+            ]
+            print('Received request:\n', json.dumps(parent, sort_keys=True))
+            print('Desired resources except secrets:\n', json.dumps(desired_resources, sort_keys=True))
+
+            # Moved after the print argument because this is sensitive data.
+
+            # This could lead to the creation of multiple access keys for the same profile
+            # which is not an imminent problem, but yet not that elegant. But it is simple
+            # and ensures that there is always a secret with valid keys.
+            # That is likely why metacontroller docs recommend not having any side effects in the
+            # hook function.
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: Cant you just check for existing secrets and reuse them as I do in https://github.com/kubeflow/pipelines/pull/7725 in sync.py def get_password(self, namespace) ?
created_at: 2025-03-18T13:01:24Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,65 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kubeflow-pipelines-profile-controller
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: kubeflow-pipelines-profile-controller
+      app.kubernetes.io/component: ml-pipeline
+      app.kubernetes.io/name: kubeflow-pipelines
+      application-crd-id: kubeflow-pipelines
+  template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"
+      labels:
+        app: kubeflow-pipelines-profile-controller
+        app.kubernetes.io/component: ml-pipeline
+        app.kubernetes.io/name: kubeflow-pipelines
+        application-crd-id: kubeflow-pipelines
+    spec:
+      containers:
+      - name: profile-controller
+        securityContext:
+          allowPrivilegeEscalation: false
+          seccompProfile:
+            type: RuntimeDefault
+          runAsNonRoot: true
+          runAsUser: 1000
+          runAsGroup: 0
+          capabilities:
+            drop:
+            - ALL
+        image: alpine/k8s:1.29.13
path: experimental/seaweedfs/base/pipeline-profile-controller/deployment.yaml
user: login: juliusvonkohout
body: Changing the image is fine, as long as it has python and boto3 and is available in a non-ratelimited official registry. If docker.io/alpine/k8s:1.32.3 is the best you can find so far, that is also fine for now, because I do not know which image contains boto3 by default. This image is also only pulled once per cluster, so its less dramatic.
created_at: 2025-03-18T13:07:26Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-seaweedfs-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": "kubeflow",
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: Can we stick with the mlpipeline bucket for backwards compatibility ?
created_at: 2025-03-18T13:13:34Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: juliusvonkohout
body: By the way lets stick to the bucket mlpipeline for backwards compatibility for now.
created_at: 2025-03-18T13:14:20Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,445 @@
+# Copyright 2020-2021 The Kubeflow Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import json
+import os
+import base64
+
+# From awscli installed in alpine/k8s image
+import botocore.session
+
+session = botocore.session.get_session()
+iam = session.create_client('iam', region_name='foobar')
+
+
+def main():
+    settings = get_settings_from_env()
+    server = server_factory(**settings)
+    server.serve_forever()
+
+
+def get_settings_from_env(controller_port=None,
+                          visualization_server_image=None, frontend_image=None,
+                          visualization_server_tag=None, frontend_tag=None, disable_istio_sidecar=None):
+    """
+    Returns a dict of settings from environment variables relevant to the controller
+
+    Environment settings can be overridden by passing them here as arguments.
+
+    Settings are pulled from the all-caps version of the setting name.  The
+    following defaults are used if those environment variables are not set
+    to enable backwards compatibility with previous versions of this script:
+        visualization_server_image: ghcr.io/kubeflow/kfp-visualization-server
+        visualization_server_tag: value of KFP_VERSION environment variable
+        frontend_image: ghcr.io/kubeflow/kfp-frontend
+        frontend_tag: value of KFP_VERSION environment variable
+        disable_istio_sidecar: Required (no default)
+        minio_access_key: Required (no default)
+        minio_secret_key: Required (no default)
+    """
+    settings = dict()
+    settings["controller_port"] = \
+        controller_port or \
+        os.environ.get("CONTROLLER_PORT", "8080")
+
+    settings["visualization_server_image"] = \
+        visualization_server_image or \
+        os.environ.get("VISUALIZATION_SERVER_IMAGE", "ghcr.io/kubeflow/kfp-visualization-server")
+
+    settings["frontend_image"] = \
+        frontend_image or \
+        os.environ.get("FRONTEND_IMAGE", "ghcr.io/kubeflow/kfp-frontend")
+
+    # Look for specific tags for each image first, falling back to
+    # previously used KFP_VERSION environment variable for backwards
+    # compatibility
+    settings["visualization_server_tag"] = \
+        visualization_server_tag or \
+        os.environ.get("VISUALIZATION_SERVER_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["frontend_tag"] = \
+        frontend_tag or \
+        os.environ.get("FRONTEND_TAG") or \
+        os.environ["KFP_VERSION"]
+
+    settings["disable_istio_sidecar"] = \
+        disable_istio_sidecar if disable_istio_sidecar is not None \
+            else os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
+
+    return settings
+
+
+def server_factory(visualization_server_image,
+                   visualization_server_tag, frontend_image, frontend_tag,
+                   disable_istio_sidecar, url="", controller_port=8080):
+    """
+    Returns an HTTPServer populated with Handler with customized settings
+    """
+    class Controller(BaseHTTPRequestHandler):
+        def sync(self, parent, attachments):
+            # parent is a namespace
+            namespace = parent.get("metadata", {}).get("name")
+
+            pipeline_enabled = parent.get("metadata", {}).get(
+                "labels", {}).get("pipelines.kubeflow.org/enabled")
+
+            if pipeline_enabled != "true":
+                return {"status": {}, "attachments": []}
+
+            # Compute status based on observed state.
+            desired_status = {
+                "kubeflow-pipelines-ready":
+                    len(attachments["Secret.v1"]) == 1 and
+                    len(attachments["ConfigMap.v1"]) == 3 and
+                    len(attachments["Deployment.apps/v1"]) == 2 and
+                    len(attachments["Service.v1"]) == 2 and
+                    len(attachments["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and
+                    len(attachments["AuthorizationPolicy.security.istio.io/v1beta1"]) == 1 and
+                    "True" or "False"
+            }
+
+            # Generate the desired attachment object(s).
+            desired_resources = [
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "kfp-launcher",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "defaultPipelineRoot": f"minio://kubeflow/{namespace}/v2/artifacts",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "metadata-grpc-configmap",
+                        "namespace": namespace,
+                    },
+                    "data": {
+                        "METADATA_GRPC_SERVICE_HOST":
+                            "metadata-grpc-service.kubeflow",
+                        "METADATA_GRPC_SERVICE_PORT": "8080",
+                    },
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "ConfigMap",
+                    "metadata": {
+                        "name": "artifact-repositories",
+                        "namespace": namespace,
+                        "annotations": {
+                            "workflows.argoproj.io/default-artifact-repository": "default-seaweedfs-namespaced"
+                        }
+                    },
+                    "data": {
+                        "default-seaweedfs-namespaced": json.dumps({
+                            "archiveLogs": True,
+                            "s3": {
+                                "endpoint": "minio-service.kubeflow:9000",
+                                "bucket": "kubeflow",
+                                "keyFormat": f"{namespace}/artifacts/{{{{workflow.name}}}}/{{{{workflow.creationTimestamp.Y}}}}/{{{{workflow.creationTimestamp.m}}}}/{{{{workflow.creationTimestamp.d}}}}/{{{{pod.name}}}}",
+                                "insecure": True,
+                                "accessKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "accesskey",
+                                },
+                                "secretKeySecret": {
+                                    "name": "mlpipeline-minio-artifact",
+                                    "key": "secretkey",
+                                }
+                            }
+                        })
+                    }
+                },
+                # Visualization server related manifests below
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-visualizationserver"
+                        },
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            },
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-visualizationserver"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "image": f"{visualization_server_image}:{visualization_server_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "name":
+                                        "ml-pipeline-visualizationserver",
+                                    "ports": [{
+                                        "containerPort": 8888
+                                    }],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "50m",
+                                            "memory": "200Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "500m",
+                                            "memory": "1Gi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor",
+                            },
+                        },
+                    },
+                },
+                {
+                    "apiVersion": "networking.istio.io/v1alpha3",
+                    "kind": "DestinationRule",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "host": "ml-pipeline-visualizationserver",
+                        "trafficPolicy": {
+                            "tls": {
+                                "mode": "ISTIO_MUTUAL"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "security.istio.io/v1beta1",
+                    "kind": "AuthorizationPolicy",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-visualizationserver"
+                            }
+                        },
+                        "rules": [{
+                            "from": [{
+                                "source": {
+                                    "principals": ["cluster.local/ns/kubeflow/sa/ml-pipeline"]
+                                }
+                            }]
+                        }]
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-visualizationserver",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name": "http",
+                            "port": 8888,
+                            "protocol": "TCP",
+                            "targetPort": 8888,
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-visualizationserver",
+                        },
+                    },
+                },
+                # Artifact fetcher related resources below.
+                {
+                    "apiVersion": "apps/v1",
+                    "kind": "Deployment",
+                    "metadata": {
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        },
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                    },
+                    "spec": {
+                        "selector": {
+                            "matchLabels": {
+                                "app": "ml-pipeline-ui-artifact"
+                            }
+                        },
+                        "template": {
+                            "metadata": {
+                                "labels": {
+                                    "app": "ml-pipeline-ui-artifact"
+                                },
+                                "annotations": disable_istio_sidecar and {
+                                    "sidecar.istio.io/inject": "false"
+                                } or {},
+                            },
+                            "spec": {
+                                "containers": [{
+                                    "name":
+                                        "ml-pipeline-ui-artifact",
+                                    "image": f"{frontend_image}:{frontend_tag}",
+                                    "imagePullPolicy":
+                                        "IfNotPresent",
+                                    "ports": [{
+                                        "containerPort": 3000
+                                    }],
+                                    "env": [
+                                        {
+                                            "name": "MINIO_ACCESS_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "accesskey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        },
+                                        {
+                                            "name": "MINIO_SECRET_KEY",
+                                            "valueFrom": {
+                                                "secretKeyRef": {
+                                                    "key": "secretkey",
+                                                    "name": "mlpipeline-minio-artifact"
+                                                }
+                                            }
+                                        }
+                                    ],
+                                    "resources": {
+                                        "requests": {
+                                            "cpu": "10m",
+                                            "memory": "70Mi"
+                                        },
+                                        "limits": {
+                                            "cpu": "100m",
+                                            "memory": "500Mi"
+                                        },
+                                    }
+                                }],
+                                "serviceAccountName":
+                                    "default-editor"
+                            }
+                        }
+                    }
+                },
+                {
+                    "apiVersion": "v1",
+                    "kind": "Service",
+                    "metadata": {
+                        "name": "ml-pipeline-ui-artifact",
+                        "namespace": namespace,
+                        "labels": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    },
+                    "spec": {
+                        "ports": [{
+                            "name":
+                                "http",  # name is required to let istio understand request protocol
+                            "port": 80,
+                            "protocol": "TCP",
+                            "targetPort": 3000
+                        }],
+                        "selector": {
+                            "app": "ml-pipeline-ui-artifact"
+                        }
+                    }
+                },
+            ]
+            print('Received request:\n', json.dumps(parent, sort_keys=True))
+            print('Desired resources except secrets:\n', json.dumps(desired_resources, sort_keys=True))
+
+            # Moved after the print argument because this is sensitive data.
+
+            # This could lead to the creation of multiple access keys for the same profile
+            # which is not an imminent problem, but yet not that elegant. But it is simple
+            # and ensures that there is always a secret with valid keys.
+            # That is likely why metacontroller docs recommend not having any side effects in the
+            # hook function.
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: pschoen-itsc
body: I think I implemented that at some point, but it didn't help much because the metacontroller made multiple requests at the same time. But I will look into that again. As I said, in the latest version this doesn't seem to be a problem anymore. Then I can also implement what you have already done in 7725
created_at: 2025-03-18T13:17:30Z
author_association: CONTRIBUTOR
path: .github/workflows/pipeline_swfs_test.yaml
user: login: milinddethe15
body: For this action, use `ubuntu-latest-16-cores` runner:
```yaml
    runs-on:
      labels: ubuntu-latest-16-cores
```
created_at: 2025-04-25T17:27:50Z
author_association: MEMBER
path: .github/workflows/pipeline_swfs_test.yaml
user: login: pschoen-itsc
body: Thanks
created_at: 2025-04-25T18:07:24Z
author_association: CONTRIBUTOR
path: experimental/seaweedfs/base/pipeline-profile-controller/sync.py
user: login: milinddethe15
body: workaround: https://www.diffchecker.com/h34UmQrY/
created_at: 2025-04-25T18:19:54Z
author_association: MEMBER
diff_hunk: @@ -17,16 +17,22 @@ spec:
     metadata:
       labels:
         app: seaweedfs
+        application-crd-id: kubeflow-pipelines
     spec:
       containers:
       - name: seaweedfs
-        image: 'chrislusf/seaweedfs:3.69'
+        image: 'chrislusf/seaweedfs:3.85'
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-deployment.yaml
user: login: milinddethe15
body: Can we use latest release v3.86? 
https://github.com/seaweedfs/seaweedfs/releases/
created_at: 2025-04-25T18:26:45Z
author_association: MEMBER
diff_hunk: @@ -17,16 +17,22 @@ spec:
     metadata:
       labels:
         app: seaweedfs
+        application-crd-id: kubeflow-pipelines
     spec:
       containers:
       - name: seaweedfs
-        image: 'chrislusf/seaweedfs:3.69'
+        image: 'chrislusf/seaweedfs:3.85'
path: experimental/seaweedfs/base/seaweedfs/seaweedfs-deployment.yaml
user: login: pschoen-itsc
body: I don't think so https://hub.docker.com/r/chrislusf/seaweedfs/tags
The Docker image is still missing. But this version does not seem the bring significant changes for our usecase, so 3.85 should be fine for now.
created_at: 2025-04-26T03:09:08Z
author_association: CONTRIBUTOR

PR comments:
user: login: pschoen-itsc
created_at: 2025-03-14T16:06:45Z
body: @juliusvonkohout I need to do a bit more testing (for example UI components), but the integration between seaweedfs and kf-pipelines works that way.
author_association: CONTRIBUTOR
user: login: juliusvonkohout
created_at: 2025-03-14T18:31:43Z
body: /retest
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-03-16T18:14:14Z
body: > @juliusvonkohout I need to do a bit more testing (for example UI components), but the integration between seaweedfs and kf-pipelines works that way.

Thank you very much for the PR. I have added a lot of comments and clarifications.

Now the question is why does the pipeline not stop running.
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-12T11:32:20Z
body: Do you mind squashing and signing your commits  (DCO) and rebasing to master?
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-12T11:35:47Z
body: I think you can copy from our new pipeline install script. See also https://github.com/kubeflow/manifests/pull/3093/files and https://github.com/kubeflow/manifests/blob/master/.github/workflows/pipeline_test.yaml as reference.
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-25T16:47:53Z
body: Thank you, i just re-enabled the tests
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-25T16:49:43Z
body: Ah i broke the DCO, but you can just squash my test enabling commit and take ownership of it if needed. Otherwise i will fix it later myself
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-25T17:05:46Z
body: I think https://github.com/pschoen-itsc/kf-manifests/blob/integrate-profiles-with-seaweedfs/tests/gh-actions/pipeline_test.py or https://github.com/kubeflow/manifests/actions/runs/14669563838/job/41172716531?pr=3051 needs some debugging. 
author_association: MEMBER
user: login: pschoen-itsc
created_at: 2025-04-25T18:07:12Z
body: The End-to-End Integration test succeded before, now it fails without changing anything...
@juliusvonkohout can you just retrigger it?
author_association: CONTRIBUTOR
user: login: milinddethe15
created_at: 2025-04-25T18:09:55Z
body: /retest
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-27T10:20:38Z
body: Amazing, thank you very much. It it such an improvement of the status quo.

In a follow up PR we should
1.  check whether ubuntu-latest-16-cores is really needed instead of the smaller nodes
2. test it also in https://github.com/kubeflow/manifests/blob/master/.github/workflows/full_kubeflow_integration_test.yaml
3. Add security checks to fest for unauthorized access
4. Adjust the Argo Workflow-Controller configuration map to achieve the same for V1 pipelines (easy, did so before)
5. See how we can upstream that to KFP
@akagami-harsh this could be interesting for you.

Thank you everyone!
I am pushing this for 4 years or so and even had google and redhat employees involved, back then even Amazon. It is fundamental for CVEs, maintainabiliy (minio is now stuck for 5 years or so) and hard multi-tenancy as basic requirement for a sane platform. We also had approaches there for several years with minio. it started all in 2020 here https://github.com/kubeflow/pipelines/issues/4649 and went via https://github.com/kubeflow/pipelines/pull/7725 (2022) and https://github.com/kubeflow/manifests/pull/2826 (October 2024) to https://github.com/kubeflow/manifests/pull/3051 (2025). Without that experimental and extended tests it would have been very hard to pull of and coordinate. I want to especially highlight @pschoen-itsc who spent his effort here for the public health sector in Germany where many insurances need hard multi-tenancy to process data.


/lgtm
/approve
author_association: MEMBER
user: login: google-oss-prow[bot]
created_at: 2025-04-27T10:20:53Z
body: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubeflow/manifests/pull/3051#issuecomment-2833371838" title="Approved">juliusvonkohout</a>*

The full list of commands accepted by this bot can be found [here](https://oss.gprow.dev/command-help?repo=kubeflow%2Fmanifests).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubeflow/manifests/blob/master/OWNERS)~~ [juliusvonkohout]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
author_association: NONE
user: login: juliusvonkohout
created_at: 2025-04-28T15:04:55Z
body: Tracked in https://github.com/kubeflow/manifests/issues/3119
author_association: MEMBER