Repository: kubeflow/pipelines

Issue:
[v2compat] configurable default pipeline root
Quoted from https://github.com/kubeflow/pipelines/issues/4649#issuecomment-844035914
Blocks https://github.com/kubeflow/pipelines/issues/5680

> FYI, KFP v2 compatible mode has been released, see documentation: https://www.kubeflow.org/docs/components/pipelines/sdk/v2/.
> 
> It doesn't support artifact repository configuration, this is one of the things we want to support too. So I'm posting early thoughts on this related issue.
> 
> Let me first try to summarize requirements for configuring artifact repositories for both KFP v2 compatible and v2.
> 
> ## Object store specific credentials requirements
> For GCS, AWS S3, we suggest setting up credentials, so that they represent identity of the pipeline step, so that not only artifact upload/download, calls to other Cloud services should also use the same credentials. For this reason, we don't recommend setting credentials in artifact repository config. The suggestion is to configure the identity transparently if possible using GCP workload identity or AWS IRSA. If credentials are really necessary, they can be configured using pipeline DSL via [kfp.gcp.use_gcp_secret](https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.extensions.html#kfp.gcp.use_gcp_secret) or [kfp.aws.use_aws_secret](https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.extensions.html#kfp.aws.use_aws_secret) etc. These principles should apply to other Cloud Providers that has credentials that can be used with all its services.
> 
> For on-prem object store like MinIO, the credentials do not represent an identity, they are only used to access a specified object store instance. Therefore, it's reasonable to include them in artifact repository config.
> 
> ## In summary
> * For GCS, only pipeline root needs to be configurable.
> * For AWS S3, besides pipeline root, we also need region, endpoint etc to be configurable.
> * For MinIO or similar on-prem object stores, besides pipeline root, we also need endpoint, credentials to be configurable.
> 
> We cannot implement a spec for every possible object stores, so likely we should use the same spec as what [Go CDK](https://gocloud.dev/) supports or depend on cloud provider contributions.
> 
> Go CDK supports provider specific query params to configure some things other than object key, so we might consider adopting these query params, so that pipeline root can be more expressive, so we might not need other configurations.
> e.g. for S3, it's possible to configure region via a query param: https://gocloud.dev/howto/blob/#s3
> 
> ```
> s3://my-bucket?region=us-west-1
> ```
> 
> ## How we configure pipeline root, other configs and credentials?
> Ideally, pipeline root can include all other configs, so that we can uniquely identify an artifact.
> Ideally, credentials should be configurable transparently.
> 
> When both ideal requirements are met, we only need to support namespace level default pipeline root. All other configurations can be done by specifying different pipeline roots.
> 
> However, now MinIO violates the requirement that credentials can be configured transparently. Therefore, we need a mechanism to either
> 
> * configure which credentials should be used with which pipeline root (probably, write rules like which pipeline_root prefix/query param should use which credentials)
> * or configure credentials with pipeline root together as artifact repository (but then we should specify artifact repos, not pipeline roots)
> * or ask users to configure credentials separately from pipeline_root
> 
> We probably need more thoughts on the exact config format, this seems like a complex problem.




Discussion:
/assign @capri-xiyue 

Resolving patch:
diff --git a/manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/sync.py b/manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/sync.py
--- a/manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/sync.py
+++ b/manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/sync.py
@@ -18,6 +18,8 @@
 import base64
 
 kfp_version = os.environ["KFP_VERSION"]
+# KFP_DEFAULT_PIPELINE_ROOT is optional
+kfp_default_pipeline_root = os.environ.get("KFP_DEFAULT_PIPELINE_ROOT")
 disable_istio_sidecar = os.environ.get("DISABLE_ISTIO_SIDECAR") == "true"
 mlpipeline_minio_access_key = base64.b64encode(
     bytes(os.environ.get("MINIO_ACCESS_KEY"), 'utf-8')).decode('utf-8')
@@ -27,17 +29,35 @@
 
 class Controller(BaseHTTPRequestHandler):
     def sync(self, parent, children):
+        # parent is a namespace
+        namespace = parent.get("metadata", {}).get("name")
         pipeline_enabled = parent.get("metadata", {}).get(
             "labels", {}).get("pipelines.kubeflow.org/enabled")
 
         if pipeline_enabled != "true":
             return {"status": {}, "children": []}
 
+        desired_configmap_count = 1
+        desired_resources = []
+        if kfp_default_pipeline_root:
+            desired_configmap_count = 2
+            desired_resources += [{
+                "apiVersion": "v1",
+                "kind": "ConfigMap",
+                "metadata": {
+                    "name": "kfp-launcher",
+                    "namespace": namespace,
+                },
+                "data": {
+                    "defaultPipelineRoot": kfp_default_pipeline_root,
+                },
+            }]
+
         # Compute status based on observed state.
         desired_status = {
             "kubeflow-pipelines-ready": \
                 len(children["Secret.v1"]) == 1 and \
-                len(children["ConfigMap.v1"]) == 1 and \
+                len(children["ConfigMap.v1"]) == desired_configmap_count and \
                 len(children["Deployment.apps/v1"]) == 2 and \
                 len(children["Service.v1"]) == 2 and \
                 len(children["DestinationRule.networking.istio.io/v1alpha3"]) == 1 and \
@@ -46,9 +66,7 @@ def sync(self, parent, children):
         }
 
         # Generate the desired child object(s).
-        # parent is a namespace
-        namespace = parent.get("metadata", {}).get("name")
-        desired_resources = [
+        desired_resources += [
             {
                 "apiVersion": "v1",
                 "kind": "ConfigMap",


Test patch:
diff --git a/manifests/kustomize/hack/test.sh b/manifests/kustomize/hack/test.sh
--- a/manifests/kustomize/hack/test.sh
+++ b/manifests/kustomize/hack/test.sh
@@ -48,7 +48,7 @@ kustomization_yamls_v3=(
 )
 for path in "${kustomization_yamls_v3[@]}"
 do
-  kustomize build --load_restrictor none "${MANIFESTS_DIR}/${path}" >/dev/null
+  kustomize build "${MANIFESTS_DIR}/${path}" >/dev/null
 done
 
 # verify these manifests work with kpt