Repository: kubeflow/pipelines
Pull request #11875: docs(KEP): Propose a new pipeline run workspace feature
Author: login: mprahl
Labels: name: lgtm
name: approved
name: size/XL
Created: 2025-04-29T15:12:39Z | Merged: 2025-05-16T17:41:23Z

**Description of your changes:**

Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
to clean up. This also doesn't work if the pipeline is run locally.

Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
duration of a pipeline run would greatly enhance the user experience.

**Checklist:**
- [x] You have [signed off your commits](https://www.kubeflow.org/docs/about/contributing/#sign-off-your-commits)
- [x] The title for your pull request (PR) should follow our title convention. [Learn more about the pull request title convention used in this repository](https://github.com/kubeflow/pipelines/blob/master/CONTRIBUTING.md#pull-request-title-convention). 


Diff:
diff --git a/hack/update-toc.sh b/hack/update-toc.sh
old mode 100644
new mode 100755
index d0e1872c856..6a24ebd1ed9
--- a/hack/update-toc.sh
+++ b/hack/update-toc.sh
@@ -21,12 +21,12 @@ set -o pipefail
 # cd to the root path
 ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
 cd "${ROOT}"
+PATH="${PATH}:${ROOT}/bin:$(go env GOPATH)/bin"
 
 echo "Updating tables of contents if necessary..."
 # Update tables of contents if necessary.
-find keps -name '*.md' \
-    | grep -Fxvf hack/.notableofcontents \
-    | xargs "${ROOT}/bin/mdtoc" --inplace --max-depth=5  || (
+find proposals -mindepth 2 -name '*.md' \
+    | xargs mdtoc --inplace --max-depth=5  || (
       echo "Failed generating TOC. If this failed silently and you are on mac, try 'brew install grep'"
       exit 1
     )
diff --git a/proposals/11551-kubernetes-native-api/README.md b/proposals/11551-kubernetes-native-api/README.md
index fc9e852018f..c2bae70cd3e 100644
--- a/proposals/11551-kubernetes-native-api/README.md
+++ b/proposals/11551-kubernetes-native-api/README.md
@@ -1,7 +1,6 @@
 # KEP-11551: Introducing a Kubernetes Native API for Pipelines and Pipeline Versions
 
 <!-- toc -->
-
 - [Summary](#summary)
 - [Motivation](#motivation)
   - [Goals](#goals)
@@ -16,6 +15,10 @@
     - [Migration](#migration)
 - [Design Details](#design-details)
   - [Custom Resource Definitions](#custom-resource-definitions)
+  - [Go Types](#go-types)
+    - [Pipelines](#pipelines)
+    - [Pipeline Versions](#pipeline-versions)
+    - [Anything Type](#anything-type)
   - [Kubeflow Pipelines API Server](#kubeflow-pipelines-api-server)
     - [Webhooks](#webhooks)
       - [Validating Webhook](#validating-webhook)
@@ -30,7 +33,8 @@
 - [Alternatives](#alternatives)
   - [Sync from Kubernetes to the Database](#sync-from-kubernetes-to-the-database)
   - [A Kubernetes Aggregation Layer API Server](#a-kubernetes-aggregation-layer-api-server)
-  <!-- /toc -->
+  - [Use Argo Workflows Directly](#use-argo-workflows-directly)
+<!-- /toc -->
 
 ## Summary
 
diff --git a/proposals/11875-pipeline-workspace/README.md b/proposals/11875-pipeline-workspace/README.md
new file mode 100644
index 00000000000..1c355aabc92
--- /dev/null
+++ b/proposals/11875-pipeline-workspace/README.md
@@ -0,0 +1,612 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+    - [1. Pipeline Configuration](#1-pipeline-configuration)
+    - [2. Repository Cloning Component](#2-repository-cloning-component)
+    - [3. Data Processing Component](#3-data-processing-component)
+    - [4. Model Import, Training, and Model Output](#4-model-import-training-and-model-output)
+    - [Key Benefits of this User Experience](#key-benefits-of-this-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+    - [Story 2: Custom PVC Configuration for Kubernetes-Savvy Users](#story-2-custom-pvc-configuration-for-kubernetes-savvy-users)
+    - [Story 3: Default PVC Configuration for Cluster Administrators](#story-3-default-pvc-configuration-for-cluster-administrators)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+      - [Importer Driver](#importer-driver)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+This proposal aims to simplify PVC usage in pipelines by enabling Kubernetes cluster administrators to set environment
+specific defaults. While this improvement benefits all users, it particularly targets those with minimal Kubernetes
+experience by making PVCs more accessible and easier to use.
+
+### Goals
+
+1. Define a workspace abstraction for pipeline runs defined as a pipeline configuration at the SDK level.
+1. Start with adding a PVC based implementation using Argo Workflows
+   [`spec.volumeClaimTemplates`](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) feature.
+1. Only require the user to provide a workspace size for most use cases. But expose setting other PVC options for
+   advanced use cases.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing pipeline components/tasks do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+1. Adding support for other storage backends beyond PVC.
+1. The workspace is meant to be ephemeral and is not meant as a storage of KFP artifacts. Output artifacts still must be
+   uploaded to object storage.
+1. It is not a goal to abstract pod affinity and/or node affinity rules to enhance scheduling for workspaces that
+   leverage `ReadWriteOnce` PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.set_path(model.path) # or trained_model.custom_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- For KFP remote, this is backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then
+  deleted after a successful run.
+- For KFP local (i.e. running the pipeline locally), this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+As a stretch goal, it'd be nice to make garbage collection of the PVC configurable. It'd default to `OnRunSuccess`. Here
+are the suggested options:
+
+- `OnRunSuccess` - when the pipeline run completes successfuly, the PVC is deleted. This maps to the Argo Workflows
+  `OnWorkflowSuccess` option.
+- `OnRunCompletion` - when the pipeline run completes (success or failure), the PVC is always deleted. This maps to the
+  Argo Workflow `OnWorkflowCompletion` option.
+- `None` - the PVC is never deleted. This would require a change in Argo Workflows to support this.
+
+Below is an example pipeline configuration with the `workspaceDeletion` option:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          workspaceDeletion="OnRunCompletion",
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.set_path(model.path) # or trained_model.custom_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by calling `trained_model.set_path(model.path)`. This prevents the need for copying the
+  output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e. `emptyDir`
+  volume). By setting `set_path`, the `train_model.path` now returns the custom path.
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+#### Story 2: Custom PVC Configuration for Kubernetes-Savvy Users
+
+As a data scientist with Kubernetes experience, I want to customize the PersistentVolumeClaim (PVC) properties when
+using workspaces, so that I can specify storage class, size, access modes, and other PVC parameters to match my specific
+requirements, just like I can with the existing `CreatePVC` component.
+
+#### Story 3: Default PVC Configuration for Cluster Administrators
+
+As a Kubeflow Pipelines administrator, I want to define default PersistentVolumeClaim (PVC) configurations at the
+cluster level, so that users can easily create workspaces with storage settings that match our cluster's capabilities
+and policies, without users needing to specify these details in every pipeline.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
+
+1. **Storage Exhaustion**: Unlike S3, which is often a cloud service with virtually unlimited storage, PVCs usually have
+   a finite amount of storage and too many pipeline runs could use up all the storage, causing pipelines to not run in
+   parallel. This is mostly mitigated by the automatic clean up PVCs after a pipeline run succeeds and with the TTL
+   feature to automatically clean up stale Argo Workflow resources.
+1. **ParallelFor**: When using `dsl.ParallelFor` and the PVC has a `ReadWriteOnce` access mode, it may end up running
+   serially if the pods get scheduled to different nodes. A workaround is to use pod affinity or node affinity rules but
+   abstracting that is out of the scope of this KEP.
+
+## Design Details
+
+### SDK and API
+
+#### Workspace Configuration
+
+The workspace will be configured at the pipeline level using `PipelineConfig`:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+            size="250GB",
+        ),
+    ),
+)
+def pipeline():
+    # Pipeline definition
+```
+
+An example of overriding the PVC configuration defaults set on the API server:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+This will lead to protocol buffer changes similar to the following:
+
+```diff
+diff --git a/api/v2alpha1/pipeline_spec.proto b/api/v2alpha1/pipeline_spec.proto
+index d986a048b..866696419 100644
+--- a/api/v2alpha1/pipeline_spec.proto
++++ b/api/v2alpha1/pipeline_spec.proto
+@@ -1104,6 +1104,19 @@ message PlatformDeploymentConfig {
+   map<string, google.protobuf.Struct> executors = 1;
+ }
+
++message WorkspaceConfig {
++  // Size of the workspace
++  string size = 1;
++
++  // Kubernetes specific configuration for the workspace
++  KubernetesWorkspaceConfig kubernetes = 2;
++}
++
++message KubernetesWorkspaceConfig {
++  // Patch of a PersistentVolumeClaim (PVC) spec to override defaults set on the API server for the workspace PVC
++  google.protobuf.Struct pvc_spec_patch = 1;
++}
++
+ // Spec for pipeline-level config options. See PipelineConfig DSL class.
+ message PipelineConfig {
+   // Name of the semaphore key to control pipeline concurrency
+@@ -1115,4 +1128,7 @@ message PipelineConfig {
+   // Time to live configuration after the pipeline run is completed for
+   // ephemeral resources created by the pipeline run.
+   int32 resource_ttl = 3;
++
++  // Configuration for the workspace
++  optional WorkspaceConfig workspace = 4;
+ }
+```
+
+#### Upload Artifacts From The Workspace
+
+Adding support for the `model.set_path()` function requires a new property named `custom_path` in the protocol buffer
+type to indicate the path to upload to S3 rather than needing to copy.
+
+```proto
+message RuntimeArtifact {
+  // The local path used to upload the artifact.
+  optional string custom_path = 7;
+}
+```
+
+It also requires setting this field to `None` on the Python `Artifact` class in the constructor. For example:
+
+```python
+  class Artifact:
+    def __init__(self,
+                 name: Optional[str] = None,
+                 uri: Optional[str] = None,
+                 metadata: Optional[Dict] = None) -> None:
+        self.custom_path: str = None
+```
+
+The Python executor also needs to pass this field back to the KFP Launcher in the executor output.
+
+#### Workspace Path Access
+
+Components will access the workspace path using the `dsl.WORKSPACE_PATH_PLACEHOLDER`:
+
+```python
+@dsl.component()
+def my_component(workspacePath: str):
+    # Use workspacePath to access the workspace directory
+    file_path = os.path.join(workspacePath, "my_file.txt")
+
+# In the pipeline definition:
+my_component(workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER)
+```
+
+The `dsl.WORKSPACE_PATH_PLACEHOLDER` will be defined as such:
+
+```python
+WORKSPACE_PATH_PLACEHOLDER = '{{$.workspace_path}}'
+```
+
+It is up to the KFP Driver to resolve this value, which will likely always be `/kfp-workspace`, but having the
+placeholder indicates to the KFP Driver that the component requires the workspace to be mounted.
+
+#### Downloading Artifacts
+
+Artifacts can be downloaded directly to the workspace:
+
+```python
+import_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+```
+
+This will require a new field of `download_to_workspace` to the `ImporterSpec` protocol buffer type:
+
+```proto
+  message ImporterSpec {
+    // Whether or not to download the artifact to the workspace.
+    optional bool download_to_workspace = 7;
+  }
+```
+
+The same change is needed in the corresponding `ImporterSpec` Python class and `importer` Python function.
+
+#### Accessing an Artifact in the Workspace
+
+The `Artifact` Python class' `_get_path` should check for a boolean metadata field `_kfp_workspace` on the artifact
+which is set by the KFP Driver (see the [Container Driver](#container-driver) section for context). If it's `True`, then
+`model.path` should return the path with the `/kfp-workspace/.artifacts/` prefix.
+
+### API Server
+
+#### PVC Default Configuration
+
+The KFP API server configuration (`config.json`) should accept the following new optional fields:
+
+```json
+{
+  "Workspace": {
+    "VolumeClaimTemplateSpec": {
+      "accessModes": ["ReadWriteMany"],
+      "storageClassName": "my-storage"
+    }
+  }
+}
+```
+
+These would be used as the default values when creating the PVC for the workspace.
+
+#### Argo Workflow Compiler
+
+Argo Workflows already supports PVC management with the
+[`spec.volumeClaimTemplates`](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) field. When
+a pipeline specifies a workspace, it'll be added to the Argo Workflow object such as the following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+spec:
+  volumeClaimTemplates:
+    - metadata:
+        name: kfp-workspace-46f1d52e-c72b-42fc-88ae-789edf7c33fd # The suffix is the run ID
+      spec:
+        accessModes:
+          - ReadWriteMany
+        storageClassName: my-storage
+        resources:
+          requests:
+            storage: 250Gi
+```
+
+By default, the volume claims are garbage collected if the workflow succeeds or the `Workflow` object is deleted.
+
+The base of the `volumeClaimTemplates` entry comes from the API server configuration but any overrides come from the
+user's pipeline config. Ideally the API server configuration defaults only requires the user to provide the `size` which
+gets set on `spec.resources.requests.storage`, but if the user supplies `storageClassName` and `accessModes` in the
+`pvcSpecPatch` option, they could use a workspace without the administrator configuring the defaults.
+
+Note that it could be possible that `pvcSpecPatch` sets the size, but that should be ignored by having `size` take
+priority.
+
+Additionally, if the artifact URI is an input parameter or has a static value starting with `oci://`, the new KFP
+Importer driver should be added to the Importer DAG. See the [Importer Driver](#importer-driver) section for more
+details.
+
+### Driver and Launcher Changes
+
+#### Importer
+
+##### Importer Driver
+
+To support downloading OCI artifacts in the Modelcar format to a workspace, we'll need to have a simple driver for the
+KFP Importer when the artifact URI is an input parameter or has a static value starting with `oci://`. It's only goal is
+to provide a spec patch with the logic from `addModelcarsToPodSpec` to set the proper sidecar containers.
+
+#### Importing to the Workspace
+
+The Importer now supports downloading artifacts to a workspace. To differentiate between artifacts to be downloaded by
+the KFP Launcher and those already in the workspace, the importer's execution type in MLMD should be
+`system.ImporterWorkspaceExecution` instead of `system.ImporterExecution`.
+
+Additionally, just before registering the artifact in MLMD, the artifacts should be downloaded to the workspace. The
+download path should be the same as the local paths returned from the `LocalPathForURI` function except that it will
+have the prefix of `/kfp-workspace/.artifacts/` in the path.
+
+#### Container Driver
+
+When resolving input artifacts, if the artifact is from a task with an execution type of
+`system.ImporterWorkspaceExecution`, `resolveUpstreamArtifacts` should set the metadata field `_kfp_workspace` to
+`True`. This is not persisted to MLMD but will be present in the executor input used by the KFP Launcher and the Python
+executor.
+
+When an input artifact is from a workspace, an input parameter is `dsl.WORKSPACE_PATH_PLACEHOLDER`, or an input
+parameter is a component output result that is a path to the workspace (e.g. starts with `/kfp-workspace`), the driver
+should set the volume mount of the KFP workspace in the `Pod` spec patch.
+
+Lastly, the Driver should disallow user mounted volumes in or under `/kfp-workspace` as this could lead to confusing
+behavior.
+
+#### Launcher
+
+The KFP Launcher will skip downloading any artifacts to the local paths (`emptyDir` volume mounts) if the artifact has
+the `_kfp_workspace` metadata field set to `True`, which is set the by the Container Driver.
+
+### Test Plan
+
+[x] I/we understand the owners of the involved components may require updates to existing tests to make this code solid
+enough prior to committing the changes necessary to implement this enhancement.
+
+#### Unit Tests
+
+The unit tests will be used to cover logic when possible but most coverage will come from integration tests.
+
+#### Integration tests
+
+- Add an E2E sample test with the example pipeline in this KEP or similar.
+- Add an additional E2E test that tests overriding the default PVC configurations.
+- Add an additional E2E test that tests downloading artifacts to the workspace.
+
+### Graduation Criteria
+
+N/A
+
+## Implementation History
+
+- Initial proposal: 2025-04-29
+
+## Drawbacks
+
+1. **Resource Overhead**: Workspaces consume additional storage resources when pipelines fail and there is no TTL set on
+   cleaning up the Argo Workflow objects. This is necessary to allow pipeline run retries though.
+1. **Potentially More Serialization**: Depending on the PVC's access modes (e.g. `ReadWriteOnce`), all pipeline steps
+   that leverage the workspace could be serial if the pods aren't scheduled to the same node.
+1. **Requires Administrator Configuration**: For the best user experience, the KFP administrator needs to configure the
+   default values for the PVC spec.
+
+## Alternatives
+
+1. **Improve CreatePVC**: We could consider improving the existing `CreatePVC` component to allow automatic deletion at
+   the end of a pipeline run and add owner references of the Argo Workflow object to the created PVC to clean up the
+   PVCs when the Argo Workflow objects are deleted. The API server default PVC spec configuration proposal from this KEP
+   could also be considered for improving `CreatePVC`.


Reviews:
user: login: HumairAK
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-12T18:05:20Z
user: login: hbelmiro
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T13:01:43Z
user: login: mprahl
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T13:25:25Z
user: login: mprahl
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T17:59:59Z
user: login: mprahl
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T18:16:24Z
user: login: mprahl
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T18:27:37Z
user: login: mprahl
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T20:04:14Z
user: login: HumairAK
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-15T12:48:37Z
user: login: hbelmiro
body: /lgtm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T15:38:45Z

Review comments:
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: suggestion: 

```suggestion
1. Define a workspace abstraction for pipeline runs defined as a pipeline configuration at the sdk level.
```
Argo PVC support at the backend level can be it's own point I think.
created_at: 2025-05-12T15:37:39Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: Optional: 

This has me thinking, there are 3 personas I can identify that we are targeting: 

a. Standard Pipeline Author, no k8s or limited k8s knowledge
b. Advanced Pipeline Author, with k8s knowledge 
c. KFP Administrator 

Point (2) applies to a/b, and point (3) aplies to (c). If you agree with this break down, maybe we can outline this in the motivation above (and continue to harden these personas in future keps?). 

created_at: 2025-05-12T15:42:17Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: ```suggestion
1. Only require the user to provide a workspace size for most use cases. But expose setting other PVC options for advanced use cases.
```
created_at: 2025-05-12T15:44:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: What is an "existing component" in this context? Can you also elaborate on this overall point, not sure I'm following. 
created_at: 2025-05-12T15:47:26Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: Optionally right? or did we decide to not go with that? 
created_at: 2025-05-12T15:47:57Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: Possibly another point to highlight either here or in motivation is, we are also not interested in data passing and having it metadata tracked, as such they are especially distinct from artifacts which persist beyond the scope of a pipeline, whereas this is not necessarily the case here. 
created_at: 2025-05-12T15:50:03Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: ```suggestion
- For KFP Remote, this is backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a successful run. 
- For KFP Local, i.e. running the pipeline locally, this would be a temporary directory instead.
```

nit here to highlight KFP local as a separate use case 
created_at: 2025-05-12T16:00:11Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: as a user I think I'm going to be confused about what the difference is between `trained_model.path` vs `trained_model.artifact_path`? Can we explain this in an intuitive way? 

Would any of these be better?
- trained_model.cached_path
- trained_model.fast_path
- trained_model.link_path

What happens when you assign this path but you weren't using a workspace? if it's only to be used in a workspace context then maybe `trained_model.workspace_path` ? 🤷🏾 

created_at: 2025-05-12T16:16:00Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: as per earlier comment there are at least 2 more stories: 

1. as a data scientist with k8s knowledge, I want the ability to configure underlying pvc options directly from my pipeline when I need to for advanced use cases
2. as a kfp administrator, I want to be able to configure at kfp deployment /management layer what default pvc storage options are available to use for workspaces 


created_at: 2025-05-12T16:20:50Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: The parallel case is another consideration, users will hit issues when using workspace with parallelFor for example, as it would require a pvc that has RWX and an underlying storage backend that supports this. AFAIK pods are not automatically scheduled on the same node in this case (would be good to confirm), unless we make use of affinities to force this somehow
created_at: 2025-05-12T16:26:46Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
+
+1. **Storage Exhaustion**: Unlike S3, which is often a cloud service with virtually unlimited storage, PVCs usually have
+   a finite amount of storage and too many pipeline runs could use up all the storage, causing pipelines to not run in
+   parallel. This is mostly mitigated by the automatic clean up PVCs after a pipeline run succeeds and with the TTL
+   feature to automatically clean up stale Argo Workflow resources.
+
+## Design Details
+
+### SDK and API
+
+#### Workspace Configuration
+
+The workspace will be configured at the pipeline level using `PipelineConfig`:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+            size="250GB",
+        ),
+    ),
+)
+def pipeline():
+    # Pipeline definition
+```
+
+An example of overriding the PVC configuration defaults set on the API server:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+This will lead to protocol buffer changes similar to the following:
+
+```diff
+diff --git a/api/v2alpha1/pipeline_spec.proto b/api/v2alpha1/pipeline_spec.proto
+index d986a048b..866696419 100644
+--- a/api/v2alpha1/pipeline_spec.proto
++++ b/api/v2alpha1/pipeline_spec.proto
+@@ -1104,6 +1104,19 @@ message PlatformDeploymentConfig {
+   map<string, google.protobuf.Struct> executors = 1;
+ }
+
++message WorkspaceConfig {
++  // Size of the workspace
++  string size = 1;
++
++  // Kubernetes specific configuration for the workspace
++  KubernetesWorkspaceConfig kubernetes = 2;
++}
++
++message KubernetesWorkspaceConfig {
++  // Patch of a PersistentVolumeClaim (PVC) spec to override defaults set on the API server for the workspace PVC
++  google.protobuf.Struct pvc_spec_patch = 1;
++}
++
+ // Spec for pipeline-level config options. See PipelineConfig DSL class.
+ message PipelineConfig {
+   // Name of the semaphore key to control pipeline concurrency
+@@ -1115,4 +1128,7 @@ message PipelineConfig {
+   // Time to live configuration after the pipeline run is completed for
+   // ephemeral resources created by the pipeline run.
+   int32 resource_ttl = 3;
++
++  // Configuration for the workspace
++  optional WorkspaceConfig workspace = 4;
+ }
+```
+
+#### Upload Artifacts From The Workspace
+
+Adding support for the `model.artifact_path` property to indicate the path to upload to S3 rather than needing to copy
+it to the local path from `model.path` requires a new field of `artifact_path` to the `RuntimeArtifact` protocol buffer
+type:
+
+```proto
+message RuntimeArtifact {
+  // The local path used to upload the artifact.
+  optional string artifact_path = 7;
+}
+```
+
+It also requires setting this field to `None` on the Python `Artifact` class in the constructor. For example:
+
+```python
+  class Artifact:
+    def __init__(self,
+                 name: Optional[str] = None,
+                 uri: Optional[str] = None,
+                 metadata: Optional[Dict] = None) -> None:
+        self.artifact_path: str = None
+```
+
+The Python executor also needs to pass this field back to the KFP Launcher in the executor output.
+
+#### Workspace Path Access
+
+Components will access the workspace path using the `dsl.WORKSPACE_PATH_PLACEHOLDER`:
+
+```python
+@dsl.component()
+def my_component(workspacePath: str):
+    # Use workspacePath to access the workspace directory
+    file_path = os.path.join(workspacePath, "my_file.txt")
+
+# In the pipeline definition:
+my_component(workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER)
+```
+
+The `dsl.WORKSPACE_PATH_PLACEHOLDER` will be defined as such:
+
+```python
+WORKSPACE_PATH_PLACEHOLDER = '{{$.workspace_path}}'
+```
+
+It is up to the KFP Driver to resolve this value, which will likely always be `/kfp-workspace`, but having the
+placeholder indicates to the KFP Driver that the component requires the workspace to be mounted.
+
+#### Downloading Artifacts
+
+Artifacts can be downloaded directly to the workspace:
+
+```python
+import_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+```
+
+This will require a new field of `download_to_workspace` to the `ImporterSpec` protocol buffer type:
+
+```proto
+  message ImporterSpec {
+    // Whether or not to download the artifact to the workspace.
+    optional bool download_to_workspace = 7;
+  }
+```
+
+The same change is needed in the corresponding `ImporterSpec` Python class and `importer` Python function.
+
+#### Accessing an Artifact in the Workspace
+
+The `Artifact` Python class' `_get_path` should check for a boolean metadata field `_kfp_workspace` on the artifact
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: `_kfp_workspace` is new here, can you provide a brief explanation of what this is? is this a new field in `TaskOutputArtifactSpec` ? 
created_at: 2025-05-12T16:44:09Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
+
+1. **Storage Exhaustion**: Unlike S3, which is often a cloud service with virtually unlimited storage, PVCs usually have
+   a finite amount of storage and too many pipeline runs could use up all the storage, causing pipelines to not run in
+   parallel. This is mostly mitigated by the automatic clean up PVCs after a pipeline run succeeds and with the TTL
+   feature to automatically clean up stale Argo Workflow resources.
+
+## Design Details
+
+### SDK and API
+
+#### Workspace Configuration
+
+The workspace will be configured at the pipeline level using `PipelineConfig`:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+            size="250GB",
+        ),
+    ),
+)
+def pipeline():
+    # Pipeline definition
+```
+
+An example of overriding the PVC configuration defaults set on the API server:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+This will lead to protocol buffer changes similar to the following:
+
+```diff
+diff --git a/api/v2alpha1/pipeline_spec.proto b/api/v2alpha1/pipeline_spec.proto
+index d986a048b..866696419 100644
+--- a/api/v2alpha1/pipeline_spec.proto
++++ b/api/v2alpha1/pipeline_spec.proto
+@@ -1104,6 +1104,19 @@ message PlatformDeploymentConfig {
+   map<string, google.protobuf.Struct> executors = 1;
+ }
+
++message WorkspaceConfig {
++  // Size of the workspace
++  string size = 1;
++
++  // Kubernetes specific configuration for the workspace
++  KubernetesWorkspaceConfig kubernetes = 2;
++}
++
++message KubernetesWorkspaceConfig {
++  // Patch of a PersistentVolumeClaim (PVC) spec to override defaults set on the API server for the workspace PVC
++  google.protobuf.Struct pvc_spec_patch = 1;
++}
++
+ // Spec for pipeline-level config options. See PipelineConfig DSL class.
+ message PipelineConfig {
+   // Name of the semaphore key to control pipeline concurrency
+@@ -1115,4 +1128,7 @@ message PipelineConfig {
+   // Time to live configuration after the pipeline run is completed for
+   // ephemeral resources created by the pipeline run.
+   int32 resource_ttl = 3;
++
++  // Configuration for the workspace
++  optional WorkspaceConfig workspace = 4;
+ }
+```
+
+#### Upload Artifacts From The Workspace
+
+Adding support for the `model.artifact_path` property to indicate the path to upload to S3 rather than needing to copy
+it to the local path from `model.path` requires a new field of `artifact_path` to the `RuntimeArtifact` protocol buffer
+type:
+
+```proto
+message RuntimeArtifact {
+  // The local path used to upload the artifact.
+  optional string artifact_path = 7;
+}
+```
+
+It also requires setting this field to `None` on the Python `Artifact` class in the constructor. For example:
+
+```python
+  class Artifact:
+    def __init__(self,
+                 name: Optional[str] = None,
+                 uri: Optional[str] = None,
+                 metadata: Optional[Dict] = None) -> None:
+        self.artifact_path: str = None
+```
+
+The Python executor also needs to pass this field back to the KFP Launcher in the executor output.
+
+#### Workspace Path Access
+
+Components will access the workspace path using the `dsl.WORKSPACE_PATH_PLACEHOLDER`:
+
+```python
+@dsl.component()
+def my_component(workspacePath: str):
+    # Use workspacePath to access the workspace directory
+    file_path = os.path.join(workspacePath, "my_file.txt")
+
+# In the pipeline definition:
+my_component(workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER)
+```
+
+The `dsl.WORKSPACE_PATH_PLACEHOLDER` will be defined as such:
+
+```python
+WORKSPACE_PATH_PLACEHOLDER = '{{$.workspace_path}}'
+```
+
+It is up to the KFP Driver to resolve this value, which will likely always be `/kfp-workspace`, but having the
+placeholder indicates to the KFP Driver that the component requires the workspace to be mounted.
+
+#### Downloading Artifacts
+
+Artifacts can be downloaded directly to the workspace:
+
+```python
+import_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+```
+
+This will require a new field of `download_to_workspace` to the `ImporterSpec` protocol buffer type:
+
+```proto
+  message ImporterSpec {
+    // Whether or not to download the artifact to the workspace.
+    optional bool download_to_workspace = 7;
+  }
+```
+
+The same change is needed in the corresponding `ImporterSpec` Python class and `importer` Python function.
+
+#### Accessing an Artifact in the Workspace
+
+The `Artifact` Python class' `_get_path` should check for a boolean metadata field `_kfp_workspace` on the artifact
+which is set by the KFP Driver. If it's `True`, then `model.path` should return the path with the
+`/kfp-workspace/.artifacts/` prefix.
+
+### API Server
+
+#### PVC Default Configuration
+
+The KFP API server configuration (`config.json`) should accept the following new optional fields:
+
+```json
+{
+  "Workspace": {
+    "VolumeClaimTemplateSpec": {
+      "accessModes": ["ReadWriteMany"],
+      "storageClassName": "my-storage"
+    }
+  }
+}
+```
+
+These would be used as the default values when creating the PVC for the workspace.
+
+#### Argo Workflow Compiler
+
+Argo Workflows already supports PVC management with the
+[`spec.volumeClaimTemplates`](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) field. When
+a pipeline specifies a workspace, it'll be added to the Argo Workflow object such as the following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+spec:
+  volumeClaimTemplates:
+    - metadata:
+        name: kfp-workspace-46f1d52e-c72b-42fc-88ae-789edf7c33fd # The suffix is the run ID
+      spec:
+        accessModes:
+          - ReadWriteMany
+        storageClassName: my-storage
+        resources:
+          requests:
+            storage: 250Gi
+```
+
+By default, the volume claims are garbage collected if the workflow succeeds or the `Workflow` object is deleted.
+
+The base of the `volumeClaimTemplates` entry comes from the API server configuration but any overrides come from the
+user's pipeline config. Ideally the API server configuration defaults only requires the user to provide the `size` which
+gets set on `spec.resources.requests.storage`, but if the user supplies `storageClassName` and `accessModes` in the
+`pvcSpecPatch` option, they could use a workspace without the administrator configuring the defaults.
+
+Note that it could be possible that `pvcSpecPatch` sets the size, but that should be ignored by `pvcSpecPatch` being set
+first followed by the value set in `size`.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: sorry it's a bit unclear, what takes higher precedence, `pvcSpecPatch` or `size` ? 
created_at: 2025-05-12T16:52:05Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
+
+1. **Storage Exhaustion**: Unlike S3, which is often a cloud service with virtually unlimited storage, PVCs usually have
+   a finite amount of storage and too many pipeline runs could use up all the storage, causing pipelines to not run in
+   parallel. This is mostly mitigated by the automatic clean up PVCs after a pipeline run succeeds and with the TTL
+   feature to automatically clean up stale Argo Workflow resources.
+
+## Design Details
+
+### SDK and API
+
+#### Workspace Configuration
+
+The workspace will be configured at the pipeline level using `PipelineConfig`:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+            size="250GB",
+        ),
+    ),
+)
+def pipeline():
+    # Pipeline definition
+```
+
+An example of overriding the PVC configuration defaults set on the API server:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+This will lead to protocol buffer changes similar to the following:
+
+```diff
+diff --git a/api/v2alpha1/pipeline_spec.proto b/api/v2alpha1/pipeline_spec.proto
+index d986a048b..866696419 100644
+--- a/api/v2alpha1/pipeline_spec.proto
++++ b/api/v2alpha1/pipeline_spec.proto
+@@ -1104,6 +1104,19 @@ message PlatformDeploymentConfig {
+   map<string, google.protobuf.Struct> executors = 1;
+ }
+
++message WorkspaceConfig {
++  // Size of the workspace
++  string size = 1;
++
++  // Kubernetes specific configuration for the workspace
++  KubernetesWorkspaceConfig kubernetes = 2;
++}
++
++message KubernetesWorkspaceConfig {
++  // Patch of a PersistentVolumeClaim (PVC) spec to override defaults set on the API server for the workspace PVC
++  google.protobuf.Struct pvc_spec_patch = 1;
++}
++
+ // Spec for pipeline-level config options. See PipelineConfig DSL class.
+ message PipelineConfig {
+   // Name of the semaphore key to control pipeline concurrency
+@@ -1115,4 +1128,7 @@ message PipelineConfig {
+   // Time to live configuration after the pipeline run is completed for
+   // ephemeral resources created by the pipeline run.
+   int32 resource_ttl = 3;
++
++  // Configuration for the workspace
++  optional WorkspaceConfig workspace = 4;
+ }
+```
+
+#### Upload Artifacts From The Workspace
+
+Adding support for the `model.artifact_path` property to indicate the path to upload to S3 rather than needing to copy
+it to the local path from `model.path` requires a new field of `artifact_path` to the `RuntimeArtifact` protocol buffer
+type:
+
+```proto
+message RuntimeArtifact {
+  // The local path used to upload the artifact.
+  optional string artifact_path = 7;
+}
+```
+
+It also requires setting this field to `None` on the Python `Artifact` class in the constructor. For example:
+
+```python
+  class Artifact:
+    def __init__(self,
+                 name: Optional[str] = None,
+                 uri: Optional[str] = None,
+                 metadata: Optional[Dict] = None) -> None:
+        self.artifact_path: str = None
+```
+
+The Python executor also needs to pass this field back to the KFP Launcher in the executor output.
+
+#### Workspace Path Access
+
+Components will access the workspace path using the `dsl.WORKSPACE_PATH_PLACEHOLDER`:
+
+```python
+@dsl.component()
+def my_component(workspacePath: str):
+    # Use workspacePath to access the workspace directory
+    file_path = os.path.join(workspacePath, "my_file.txt")
+
+# In the pipeline definition:
+my_component(workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER)
+```
+
+The `dsl.WORKSPACE_PATH_PLACEHOLDER` will be defined as such:
+
+```python
+WORKSPACE_PATH_PLACEHOLDER = '{{$.workspace_path}}'
+```
+
+It is up to the KFP Driver to resolve this value, which will likely always be `/kfp-workspace`, but having the
+placeholder indicates to the KFP Driver that the component requires the workspace to be mounted.
+
+#### Downloading Artifacts
+
+Artifacts can be downloaded directly to the workspace:
+
+```python
+import_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+```
+
+This will require a new field of `download_to_workspace` to the `ImporterSpec` protocol buffer type:
+
+```proto
+  message ImporterSpec {
+    // Whether or not to download the artifact to the workspace.
+    optional bool download_to_workspace = 7;
+  }
+```
+
+The same change is needed in the corresponding `ImporterSpec` Python class and `importer` Python function.
+
+#### Accessing an Artifact in the Workspace
+
+The `Artifact` Python class' `_get_path` should check for a boolean metadata field `_kfp_workspace` on the artifact
+which is set by the KFP Driver. If it's `True`, then `model.path` should return the path with the
+`/kfp-workspace/.artifacts/` prefix.
+
+### API Server
+
+#### PVC Default Configuration
+
+The KFP API server configuration (`config.json`) should accept the following new optional fields:
+
+```json
+{
+  "Workspace": {
+    "VolumeClaimTemplateSpec": {
+      "accessModes": ["ReadWriteMany"],
+      "storageClassName": "my-storage"
+    }
+  }
+}
+```
+
+These would be used as the default values when creating the PVC for the workspace.
+
+#### Argo Workflow Compiler
+
+Argo Workflows already supports PVC management with the
+[`spec.volumeClaimTemplates`](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) field. When
+a pipeline specifies a workspace, it'll be added to the Argo Workflow object such as the following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+spec:
+  volumeClaimTemplates:
+    - metadata:
+        name: kfp-workspace-46f1d52e-c72b-42fc-88ae-789edf7c33fd # The suffix is the run ID
+      spec:
+        accessModes:
+          - ReadWriteMany
+        storageClassName: my-storage
+        resources:
+          requests:
+            storage: 250Gi
+```
+
+By default, the volume claims are garbage collected if the workflow succeeds or the `Workflow` object is deleted.
+
+The base of the `volumeClaimTemplates` entry comes from the API server configuration but any overrides come from the
+user's pipeline config. Ideally the API server configuration defaults only requires the user to provide the `size` which
+gets set on `spec.resources.requests.storage`, but if the user supplies `storageClassName` and `accessModes` in the
+`pvcSpecPatch` option, they could use a workspace without the administrator configuring the defaults.
+
+Note that it could be possible that `pvcSpecPatch` sets the size, but that should be ignored by `pvcSpecPatch` being set
+first followed by the value set in `size`.
+
+Additionally, if the artifact URI is an input parameter or has a static value starting with `oci://`, the new KFP
+Importer driver should be added to the Importer DAG.
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: Caught me off guard since we don't start talking about the extra need for a driver for importer until later, maybe: 

```suggestion
Additionally, if the artifact URI is an input parameter or has a static value starting with `oci://`, the new KFP
Importer driver should be added to the Importer DAG (see "Importer Driver" section below for details).
```
created_at: 2025-05-12T16:59:09Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
path: proposals/11875-pipeline-workspace/README.md
user: login: hbelmiro
body: I'm worried about users misconfigurating workspace sizes, like setting too big sizes just to avoid errors. Wouldn't be necessary to allow admins to define quotas or something?
created_at: 2025-05-13T12:56:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
+
+1. **Storage Exhaustion**: Unlike S3, which is often a cloud service with virtually unlimited storage, PVCs usually have
+   a finite amount of storage and too many pipeline runs could use up all the storage, causing pipelines to not run in
+   parallel. This is mostly mitigated by the automatic clean up PVCs after a pipeline run succeeds and with the TTL
+   feature to automatically clean up stale Argo Workflow resources.
+
+## Design Details
+
+### SDK and API
+
+#### Workspace Configuration
+
+The workspace will be configured at the pipeline level using `PipelineConfig`:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+            size="250GB",
+        ),
+    ),
+)
+def pipeline():
+    # Pipeline definition
+```
+
+An example of overriding the PVC configuration defaults set on the API server:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+This will lead to protocol buffer changes similar to the following:
+
+```diff
+diff --git a/api/v2alpha1/pipeline_spec.proto b/api/v2alpha1/pipeline_spec.proto
+index d986a048b..866696419 100644
+--- a/api/v2alpha1/pipeline_spec.proto
++++ b/api/v2alpha1/pipeline_spec.proto
+@@ -1104,6 +1104,19 @@ message PlatformDeploymentConfig {
+   map<string, google.protobuf.Struct> executors = 1;
+ }
+
++message WorkspaceConfig {
++  // Size of the workspace
++  string size = 1;
++
++  // Kubernetes specific configuration for the workspace
++  KubernetesWorkspaceConfig kubernetes = 2;
++}
++
++message KubernetesWorkspaceConfig {
++  // Patch of a PersistentVolumeClaim (PVC) spec to override defaults set on the API server for the workspace PVC
++  google.protobuf.Struct pvc_spec_patch = 1;
++}
++
+ // Spec for pipeline-level config options. See PipelineConfig DSL class.
+ message PipelineConfig {
+   // Name of the semaphore key to control pipeline concurrency
+@@ -1115,4 +1128,7 @@ message PipelineConfig {
+   // Time to live configuration after the pipeline run is completed for
+   // ephemeral resources created by the pipeline run.
+   int32 resource_ttl = 3;
++
++  // Configuration for the workspace
++  optional WorkspaceConfig workspace = 4;
+ }
+```
+
+#### Upload Artifacts From The Workspace
+
+Adding support for the `model.artifact_path` property to indicate the path to upload to S3 rather than needing to copy
+it to the local path from `model.path` requires a new field of `artifact_path` to the `RuntimeArtifact` protocol buffer
+type:
+
+```proto
+message RuntimeArtifact {
+  // The local path used to upload the artifact.
+  optional string artifact_path = 7;
+}
+```
+
+It also requires setting this field to `None` on the Python `Artifact` class in the constructor. For example:
+
+```python
+  class Artifact:
+    def __init__(self,
+                 name: Optional[str] = None,
+                 uri: Optional[str] = None,
+                 metadata: Optional[Dict] = None) -> None:
+        self.artifact_path: str = None
+```
+
+The Python executor also needs to pass this field back to the KFP Launcher in the executor output.
+
+#### Workspace Path Access
+
+Components will access the workspace path using the `dsl.WORKSPACE_PATH_PLACEHOLDER`:
+
+```python
+@dsl.component()
+def my_component(workspacePath: str):
+    # Use workspacePath to access the workspace directory
+    file_path = os.path.join(workspacePath, "my_file.txt")
+
+# In the pipeline definition:
+my_component(workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER)
+```
+
+The `dsl.WORKSPACE_PATH_PLACEHOLDER` will be defined as such:
+
+```python
+WORKSPACE_PATH_PLACEHOLDER = '{{$.workspace_path}}'
+```
+
+It is up to the KFP Driver to resolve this value, which will likely always be `/kfp-workspace`, but having the
+placeholder indicates to the KFP Driver that the component requires the workspace to be mounted.
+
+#### Downloading Artifacts
+
+Artifacts can be downloaded directly to the workspace:
+
+```python
+import_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+```
+
+This will require a new field of `download_to_workspace` to the `ImporterSpec` protocol buffer type:
+
+```proto
+  message ImporterSpec {
+    // Whether or not to download the artifact to the workspace.
+    optional bool download_to_workspace = 7;
+  }
+```
+
+The same change is needed in the corresponding `ImporterSpec` Python class and `importer` Python function.
+
+#### Accessing an Artifact in the Workspace
+
+The `Artifact` Python class' `_get_path` should check for a boolean metadata field `_kfp_workspace` on the artifact
+which is set by the KFP Driver. If it's `True`, then `model.path` should return the path with the
+`/kfp-workspace/.artifacts/` prefix.
+
+### API Server
+
+#### PVC Default Configuration
+
+The KFP API server configuration (`config.json`) should accept the following new optional fields:
+
+```json
+{
+  "Workspace": {
+    "VolumeClaimTemplateSpec": {
+      "accessModes": ["ReadWriteMany"],
+      "storageClassName": "my-storage"
+    }
+  }
+}
+```
+
+These would be used as the default values when creating the PVC for the workspace.
+
+#### Argo Workflow Compiler
+
+Argo Workflows already supports PVC management with the
+[`spec.volumeClaimTemplates`](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) field. When
+a pipeline specifies a workspace, it'll be added to the Argo Workflow object such as the following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+spec:
+  volumeClaimTemplates:
+    - metadata:
+        name: kfp-workspace-46f1d52e-c72b-42fc-88ae-789edf7c33fd # The suffix is the run ID
+      spec:
+        accessModes:
+          - ReadWriteMany
+        storageClassName: my-storage
+        resources:
+          requests:
+            storage: 250Gi
+```
+
+By default, the volume claims are garbage collected if the workflow succeeds or the `Workflow` object is deleted.
+
+The base of the `volumeClaimTemplates` entry comes from the API server configuration but any overrides come from the
+user's pipeline config. Ideally the API server configuration defaults only requires the user to provide the `size` which
+gets set on `spec.resources.requests.storage`, but if the user supplies `storageClassName` and `accessModes` in the
+`pvcSpecPatch` option, they could use a workspace without the administrator configuring the defaults.
+
+Note that it could be possible that `pvcSpecPatch` sets the size, but that should be ignored by `pvcSpecPatch` being set
+first followed by the value set in `size`.
+
+Additionally, if the artifact URI is an input parameter or has a static value starting with `oci://`, the new KFP
+Importer driver should be added to the Importer DAG.
+
+### Driver and Launcher Changes
+
+#### Importer
+
+##### Importer Driver
+
+To support downloading OCI artifacts in the Modelcar format to a workspace, we'll need to have a simple driver for the
+KFP Importer when the artifact URI is an input parameter or has a static value starting with `oci://`. It's only goal is
+to provide a spec patch with the logic from `addModelcarsToPodSpec` to set the proper sidecar containers.
+
+#### Importing to the Workspace
+
+The importer now supports downloading artifacts to a workspace. To differentiate between artifacts to be downloaded by
+the KFP Launcher and those already in the workspace, the importer's execution type in MLMD should be
+`system.ImporterWorkspaceExecution` instead of `system.ImporterExecution`.
+
+Additionally, just before registering the artifact in MLMD, the artifacts should be downloaded to the workspace. The
+download path should be the same as the local paths returned from the `LocalPathForURI` function except that it will
+have the prefix of `/kfp-workspace/.artifacts/` in the path.
+
+#### Container Driver
+
+When resolving input artifacts, if the artifact is from a task with an execution type of
+`system.ImporterWorkspaceExecution`, `resolveUpstreamArtifacts` should set the metadata field `_kfp_workspace` to
+`True`. This is not persisted to MLMD but will be present in the executor input used by the KFP Launcher and the Python
+executor.
+
+When an input artifact is from a workspace, an input parameter is `dsl.WORKSPACE_PATH_PLACEHOLDER`, or an input
+parameter is a component output result that is a path to the workspace (e.g. starts with `/kfp-workspace`), the driver
+should set the volume mount of the KFP workspace in the `Pod` spec patch.
+
+Lastly, the Driver should disallow user mounted volumes in or under `/kfp-workspace` as this could lead to confusing
+behavior.
+
+#### Launcher
+
+The KFP Launcher will skip downloading any artifacts to the local paths (`emptyDir` volume mounts) if the artifact has
+the `_kfp_workspace` metadata field set to `True`, which is set the by the Container Driver.
+
+### Test Plan
+
+[x] I/we understand the owners of the involved components may require updates to existing tests to make this code solid
+enough prior to committing the changes necessary to implement this enhancement.
+
+#### Unit Tests
+
+The unit tests will be used to cover logic when possible but most coverage will come from integration tests.
+
+#### Integration tests
+
+- Add an E2E sample test with the example pipeline in this KEP or similar.
+- Add an additional E2E test that tests overriding the default PVC configurations.
path: proposals/11875-pipeline-workspace/README.md
user: login: hbelmiro
body: ```suggestion
- Add an E2E sample test with the example pipeline in this KEP or similar.
- Add an additional E2E test that tests overriding the default PVC configurations.
- Add an additional E2E test that tests downloading artifacts to the workspace.
```
created_at: 2025-05-13T13:00:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
path: proposals/11875-pipeline-workspace/README.md
user: login: mprahl
body: Yeah, that is confusing. I was referring to pipeline components/tasks.
created_at: 2025-05-13T13:25:25Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
path: proposals/11875-pipeline-workspace/README.md
user: login: mprahl
body: As discussed in Slack, let's go with a new method of `trained_model.set_path(model.path)` and this sets the `custom_path` property and causes `trained_model.path` to return `custom_path` if set.
created_at: 2025-05-13T17:59:59Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
+
+1. **Storage Exhaustion**: Unlike S3, which is often a cloud service with virtually unlimited storage, PVCs usually have
+   a finite amount of storage and too many pipeline runs could use up all the storage, causing pipelines to not run in
+   parallel. This is mostly mitigated by the automatic clean up PVCs after a pipeline run succeeds and with the TTL
+   feature to automatically clean up stale Argo Workflow resources.
+
+## Design Details
+
+### SDK and API
+
+#### Workspace Configuration
+
+The workspace will be configured at the pipeline level using `PipelineConfig`:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+            size="250GB",
+        ),
+    ),
+)
+def pipeline():
+    # Pipeline definition
+```
+
+An example of overriding the PVC configuration defaults set on the API server:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+This will lead to protocol buffer changes similar to the following:
+
+```diff
+diff --git a/api/v2alpha1/pipeline_spec.proto b/api/v2alpha1/pipeline_spec.proto
+index d986a048b..866696419 100644
+--- a/api/v2alpha1/pipeline_spec.proto
++++ b/api/v2alpha1/pipeline_spec.proto
+@@ -1104,6 +1104,19 @@ message PlatformDeploymentConfig {
+   map<string, google.protobuf.Struct> executors = 1;
+ }
+
++message WorkspaceConfig {
++  // Size of the workspace
++  string size = 1;
++
++  // Kubernetes specific configuration for the workspace
++  KubernetesWorkspaceConfig kubernetes = 2;
++}
++
++message KubernetesWorkspaceConfig {
++  // Patch of a PersistentVolumeClaim (PVC) spec to override defaults set on the API server for the workspace PVC
++  google.protobuf.Struct pvc_spec_patch = 1;
++}
++
+ // Spec for pipeline-level config options. See PipelineConfig DSL class.
+ message PipelineConfig {
+   // Name of the semaphore key to control pipeline concurrency
+@@ -1115,4 +1128,7 @@ message PipelineConfig {
+   // Time to live configuration after the pipeline run is completed for
+   // ephemeral resources created by the pipeline run.
+   int32 resource_ttl = 3;
++
++  // Configuration for the workspace
++  optional WorkspaceConfig workspace = 4;
+ }
+```
+
+#### Upload Artifacts From The Workspace
+
+Adding support for the `model.artifact_path` property to indicate the path to upload to S3 rather than needing to copy
+it to the local path from `model.path` requires a new field of `artifact_path` to the `RuntimeArtifact` protocol buffer
+type:
+
+```proto
+message RuntimeArtifact {
+  // The local path used to upload the artifact.
+  optional string artifact_path = 7;
+}
+```
+
+It also requires setting this field to `None` on the Python `Artifact` class in the constructor. For example:
+
+```python
+  class Artifact:
+    def __init__(self,
+                 name: Optional[str] = None,
+                 uri: Optional[str] = None,
+                 metadata: Optional[Dict] = None) -> None:
+        self.artifact_path: str = None
+```
+
+The Python executor also needs to pass this field back to the KFP Launcher in the executor output.
+
+#### Workspace Path Access
+
+Components will access the workspace path using the `dsl.WORKSPACE_PATH_PLACEHOLDER`:
+
+```python
+@dsl.component()
+def my_component(workspacePath: str):
+    # Use workspacePath to access the workspace directory
+    file_path = os.path.join(workspacePath, "my_file.txt")
+
+# In the pipeline definition:
+my_component(workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER)
+```
+
+The `dsl.WORKSPACE_PATH_PLACEHOLDER` will be defined as such:
+
+```python
+WORKSPACE_PATH_PLACEHOLDER = '{{$.workspace_path}}'
+```
+
+It is up to the KFP Driver to resolve this value, which will likely always be `/kfp-workspace`, but having the
+placeholder indicates to the KFP Driver that the component requires the workspace to be mounted.
+
+#### Downloading Artifacts
+
+Artifacts can be downloaded directly to the workspace:
+
+```python
+import_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+```
+
+This will require a new field of `download_to_workspace` to the `ImporterSpec` protocol buffer type:
+
+```proto
+  message ImporterSpec {
+    // Whether or not to download the artifact to the workspace.
+    optional bool download_to_workspace = 7;
+  }
+```
+
+The same change is needed in the corresponding `ImporterSpec` Python class and `importer` Python function.
+
+#### Accessing an Artifact in the Workspace
+
+The `Artifact` Python class' `_get_path` should check for a boolean metadata field `_kfp_workspace` on the artifact
path: proposals/11875-pipeline-workspace/README.md
user: login: mprahl
body: Sorry, it's mentioned in detail below in the Container Driver section but forgot this section came first.
created_at: 2025-05-13T18:16:24Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
+1. Allow the KFP administrator to set default configuration options for the PVCs created by the workspace.
+1. Enable efficient data sharing between pipeline components.
+1. Support artifact downloading directly to the workspace with `dsl.importer`.
+1. Existing components do not need to be aware that input artifacts are in a workspace.
+1. Automate the clean up of workspaces when a pipeline run succeeds or when it is cleaned up.
+1. Only mount the workspace when the component uses it. This is to prevent all pipeline tasks being tied to a single
+   Kubernetes node in `ReadWriteOnce` access mode.
+1. The API should not be limited to just PVC as other storage backends could be added in the future.
+
+### Non-Goals
+
+1. Multiple workspaces per pipeline run are not an initial goal. For advanced use cases such as this, the user should
+   leverage existing mechanisms to create the desired PVCs.
+
+## Proposal
+
+### SDK User Experience
+
+Below is a full example pipeline that highlights the desired SDK user experience. Below the code snippet will be a
+breakdown explaining each step.
+
+```python
+from kfp import dsl
+from kfp import compiler
+
+
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+def pipeline(repo: str, model_uri: str):
+    clone_repo_task = clone_repo(
+        workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+    )
+    process_data_task = process_data(repo_path=clone_repo_task.output)
+    import_base_model_task = dsl.importer(
+        artifact_class=dsl.Model, artifact_uri=model_uri, download_to_workspace=True,
+    )
+    train(model=import_base_model_task.output).after(process_data_task)
+```
+
+#### 1. Pipeline Configuration
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(size="250GB"),
+    ),
+)
+```
+
+- The pipeline is configured with a 250GB workspace using `dsl.WorkspaceConfig`.
+- This workspace is a shared storage space that persists for the duration of the pipeline run.
+- It's backed by a Kubernetes PersistentVolumeClaim (PVC) that's automatically created and then deleted after a
+  successful run. For running the pipeline locally, this would be a temporary directory instead.
+- The configuration of the PVC defaults to the workspace configuration on the KFP API server, but can be overridden as
+  needed.
+
+An example of overriding the PVC configuration:
+
+```python
+@dsl.pipeline(
+    name="my-pipeline",
+    pipeline_config=dsl.PipelineConfig(
+        workspace=dsl.WorkspaceConfig(
+          size="250GB",
+          kubernetes=dsl.KubernetesWorkspaceConfig(
+            pvcSpecPatch={
+              "storageClassName": "super-fast-storage",
+              "accessModes": ["ReadWriteMany"],
+            }
+          ),
+        ),
+    ),
+)
+```
+
+#### 2. Repository Cloning Component
+
+```python
+@dsl.component()
+def clone_repo(workspacePath: str, repo: str) -> str:
+    import os
+    import subprocess
+
+    clone_path = os.path.join(workspacePath, "repo")
+    subprocess.call(["git", "clone", repo, clone_path])
+
+    return clone_path
+
+# In the pipeline definition:
+clone_repo_task = clone_repo(
+    workspacePath=dsl.WORKSPACE_PATH_PLACEHOLDER, repo=repo,
+)
+```
+
+- This component takes two parameters:
+  - `workspacePath`: A parameter that when provided `dsl.WORKSPACE_PATH_PLACEHOLDER`, gets the path to the shared
+    workspace (i.e. `/kfp-workspace`). This indicates the workspace should be mounted to this component.
+  - `repo`: The Git repository URL to clone.
+- It clones the repository into a subdirectory of the workspace.
+- Returns the path to the cloned repository for use by subsequent components.
+
+#### 3. Data Processing Component
+
+```python
+@dsl.component()
+def process_data(repo_path: str):
+    print("Processing the data at " + repo_path)
+
+# In the pipeline definition:
+process_data_task = process_data(repo_path=clone_repo_task.output)
+```
+
+- Takes the path to the cloned repository as input.
+- Since the path is provided by the output of `clone_repo` and that path is in a workspace, the workspace is
+  automatically mounted for this component.
+
+#### 4. Model Import, Training, and Model Output
+
+```python
+@dsl.component()
+def train(model: dsl.Model, trained_model: dsl.Output[dsl.Model]):
+    with open(model.path, "r") as model_file:
+        print("Training the model")
+
+    # Upload the model to S3 and register it in MLMD
+    trained_model.artifact_path = model.path
+
+# In the pipeline definition:
+import_base_model_task = dsl.importer(
+    artifact_class=dsl.Model,
+    artifact_uri=model_uri,
+    download_to_workspace=True,
+)
+train(model=import_base_model_task.output).after(process_data_task)
+```
+
+- Uses `dsl.importer` with `download_to_workspace=True` to download a model directly to the workspace.
+- The `train` component receives the model from the workspace rather than downloading it again.
+- The `train` component has the `trained_model` output artifact and sets a path on the workspace of where the KFP
+  Launcher should upload to S3 by setting `trained_model.artifact_path = model.path`. This prevents the need for copying
+  the output artifact from the workspace to `trained_model.path`, which uses the Kubernetes node's storage (i.e.
+  `emptyDir` volume).
+
+#### Key Benefits of this User Experience
+
+1. **Efficiency**: Artifacts are downloaded only once and shared between components.
+1. **Simplicity**: Users don't need to manage PVCs or understand Kubernetes storage.
+1. **Automatic Cleanup**: The workspace's PVC is automatically deleted after the pipeline completes.
+1. **Flexibility**: Components only have access to the workspace when needed.
+
+### User Stories
+
+#### Story 1: Data Scientist Working with Large Datasets
+
+As a data scientist, I want to process large datasets across multiple pipeline steps without worrying about data
+transfer overhead and Kubernetes configuration, so that I can focus on my analysis rather than infrastructure concerns.
+
+### Notes/Constraints/Caveats
+
+N/A
+
+### Risks and Mitigations
path: proposals/11875-pipeline-workspace/README.md
user: login: mprahl
body: You're right. It doesn't seem like Kubernetes schedules pods based on this and would require affinities. I think since abstracting this is challenging, it should be a separate KEP.
created_at: 2025-05-13T18:27:37Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,555 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
+  - [Notes/Constraints/Caveats](#notesconstraintscaveats)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [SDK and API](#sdk-and-api)
+    - [Workspace Configuration](#workspace-configuration)
+    - [Upload Artifacts From The Workspace](#upload-artifacts-from-the-workspace)
+    - [Workspace Path Access](#workspace-path-access)
+    - [Downloading Artifacts](#downloading-artifacts)
+    - [Accessing an Artifact in the Workspace](#accessing-an-artifact-in-the-workspace)
+  - [API Server](#api-server)
+    - [PVC Default Configuration](#pvc-default-configuration)
+    - [Argo Workflow Compiler](#argo-workflow-compiler)
+  - [Driver and Launcher Changes](#driver-and-launcher-changes)
+    - [Importer](#importer)
+    - [Importing to the Workspace](#importing-to-the-workspace)
+    - [Container Driver](#container-driver)
+    - [Launcher](#launcher)
+  - [Test Plan](#test-plan)
+    - [Unit Tests](#unit-tests)
+    - [Integration tests](#integration-tests)
+  - [Graduation Criteria](#graduation-criteria)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. Doing so with artifact
+inputs and outputs introduces overhead and requires additional S3 storage. The primary workaround is to create a
+persistent volume claim (PVC) for the pipeline run but this requires Kubernetes knowledge, can be complicated, and hard
+to clean up. This also doesn't work if the pipeline is run locally.
+
+Abstracting this away with a new feature of KFP workspaces that provides shared storage between components for the
+duration of a pipeline run would greatly enhance the user experience.
+
+## Motivation
+
+Many Kubeflow Pipelines (KFP) pipelines require passing large amounts of data between components. The current mechanism
+of artifact inputs and outputs can cause unnecessary overhead as each component that requires an artifact must download
+it from S3 and any intermediate artifact that needs to be passed to other components must upload the artifact to S3.
+This increases pipeline run execution times and requires additional storage in S3, while also cluttering MLMD and KFP
+with unnecessary artifacts. The primary workaround is to create a persistent volume claim (PVC) for the pipeline run,
+mount the PVC on components that require it, and delete the PVC at the end of the pipeline run. This requires Kubernetes
+knowledge as well as cluster specific knowledge of which storage classes are available on the cluster. If the pipeline
+run fails, PVCs might not get cleaned up, causing unnecessary storage usage. This also doesn't work if the pipeline is
+run locally.
+
+This can be abstracted away from the user with the addition of a pipeline run workspace concept similar to
+[Argo Workflows volumes](https://argo-workflows.readthedocs.io/en/latest/walk-through/volumes/#volumes) and
+[Tekton workspaces](https://tekton.dev/docs/pipelines/workspaces/#workspaces-in-pipelines-and-pipelineruns) but with
+more default values that are environment specific (e.g. `storageClassName`, `accessModes`, and etc.). This would greatly
+enhance the user experience.
+
+### Goals
+
+1. Define a workspace (backed by a PVC) for pipeline runs defined as a pipeline configuration.
+1. Only require the user to provide a workspace size but allow setting other PVC options.
path: proposals/11875-pipeline-workspace/README.md
user: login: mprahl
body: I had that thought too but there isn't a restriction on that now for `CreatePVC` so I guess the administrator has to configure this at the Kubernetes level for now. Any further restrictions should be a broader effort.
created_at: 2025-05-13T20:04:14Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,605 @@
+# KEP-11875: Pipeline Run Workspace in Kubeflow Pipelines
+
+<!-- toc -->
+
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [SDK User Experience](#sdk-user-experience)
+  - [User Stories](#user-stories)
+    - [Story 1: Data Scientist Working with Large Datasets](#story-1-data-scientist-working-with-large-datasets)
path: proposals/11875-pipeline-workspace/README.md
user: login: HumairAK
body: you'll need to update the list here to add the other 2 stories 
created_at: 2025-05-15T12:48:37Z
author_association: COLLABORATOR

PR comments:
user: login: andreyvelich
created_at: 2025-04-29T15:34:39Z
body: cc @akshaychitneni @bigsur0 @shravan-achar
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-29T17:40:32Z
body: This is automatically available via KfP V1 data_passing_method. Maybe you can take this as reference for the V2 implementation 
author_association: MEMBER
user: login: juliusvonkohout
created_at: 2025-04-29T17:42:29Z
body: I think there are also a few related issues to link
author_association: MEMBER
user: login: mprahl
created_at: 2025-04-29T20:49:48Z
body: > This is automatically available via KfP V1 data_passing_method. Maybe you can take this as reference for the V2 implementation

Thanks for that! I wasn't aware of that v1 feature and see it here:
https://github.com/kubeflow/pipelines/commit/54a596abd837586e294b03c31d10ac61d9273f1a#diff-19db57446e9b12ad2beb5df35a5094eb

I like the idea but couldn't find much documentation on it. It seems like it was an experimental feature still as described in this GitHub issue: https://github.com/kubeflow/pipelines/issues/4574#issuecomment-771835329.

From what I could gather, it seemed to be "all or nothing" in that all artifacts are in the PVC when this is enabled. I do like the simplicity of this, however, my proposal is more flexible but it's debatable which is preferable.

One thing this KEP is missing and this v1 experimental feature supports is an output artifact not being uploaded to S3. I'll think on that specific aspect.
author_association: COLLABORATOR
user: login: juliusvonkohout
created_at: 2025-04-30T09:16:01Z
body: > > This is automatically available via KfP V1 data_passing_method. Maybe you can take this as reference for the V2 implementation
> 
> Thanks for that! I wasn't aware of that v1 feature and see it here: [54a596a#diff-19db57446e9b12ad2beb5df35a5094eb](https://github.com/kubeflow/pipelines/commit/54a596abd837586e294b03c31d10ac61d9273f1a#diff-19db57446e9b12ad2beb5df35a5094eb)
> 
> I like the idea but couldn't find much documentation on it. It seems like it was an experimental feature still as described in this GitHub issue: [#4574 (comment)](https://github.com/kubeflow/pipelines/issues/4574#issuecomment-771835329).
> 
> From what I could gather, it seemed to be "all or nothing" in that all artifacts are in the PVC when this is enabled. I do like the simplicity of this, however, my proposal is more flexible but it's debatable which is preferable.
> 
> One thing this KEP is missing and this v1 experimental feature supports is an output artifact not being uploaded to S3. I'll think on that specific aspect.

I know a lot using it in production in v1. I also fixed it a bit in a later commit to still store metrics, main logs and visualizations on S3, since otherwise the UI breaks. So by default everything should stay on the pvc except for things needed by the UI. and maybe this should be user configurable as well. lets make it flexible but with a simple default
author_association: MEMBER
user: login: dandawg
created_at: 2025-05-07T18:00:27Z
body: Love this proposal! I also remember working with PVCs in V1, including KFP support for creating a PVC, or using an existing PVC.

@mprahl adding an existing PVC workspace would be a nice addition, as well as keeping a workspace after a run, since ML is very iterative. Consider two things:
1) The ability to run a pipeline more than once on a workspace. This could support use-cases like breaking up training epocs, for example, or just doing iterative experimentation.
2) PVCs can also be attached to Kubeflow Notebooks, so it would be possible to go back and forth between Notebooks and Pipelines. I actually implemented this for customers on V1 a while back--it would be really cool to see this with these workspaces. This would score points for the Kubeflow ecosystem!!

A couple of challenges I see:
1) how do we deal with the local env runs?
2) We may need ReadWriteMany to support attaching the PVC to parallel running components, but not all clusters have ReadWriteMany PVC support.
3) What happens with metadata/artifacts when I go to retrieve them later? For example through the SDK (i.e. client.get_artifact(..))? Do I still have a record that the run occurred? What about the ability to retrieve metrics, parameters, artifacts for it? If the PVC is gone, do I get an intelligible error message?
author_association: CONTRIBUTOR
user: login: mprahl
created_at: 2025-05-07T19:58:52Z
body: Thanks for the review! I'll reply inline.

> Love this proposal! I also remember working with PVCs in V1, including KFP support for creating a PVC, or using an existing PVC.
> 
> @mprahl adding an existing PVC workspace would be a nice addition, as well as keeping a workspace after a run, since ML is very iterative. Consider two things:
> 
> 1. The ability to run a pipeline more than once on a workspace. This could support use-cases like breaking up training epocs, for example, or just doing iterative experimentation.

Interesting idea. The thought here was that any artifact that lives beyond the pipeline run should be an output artifact that is uploaded to S3 for traceability and potential reuse across pipelines. Is the overhead cost of storage and data transfers too high for that approach?

> 2. PVCs can also be attached to Kubeflow Notebooks, so it would be possible to go back and forth between Notebooks and Pipelines. I actually implemented this for customers on V1 a while back--it would be really cool to see this with these workspaces. This would score points for the Kubeflow ecosystem!!

I think the potential solution is the same for 1. Essentially, allow an option to keep the PVC around after the pipeline run and to allow the user to name the workspace so that it's identifiable in other pipeline runs and notebooks. Does that capture it?

> 
> A couple of challenges I see:
> 
> 1. how do we deal with the local env runs?

Do you mean KFP local? In that case, KFP local would just use a temporary directory on disk. I can clarify that point in the doc.

> 2. We may need ReadWriteMany to support attaching the PVC to parallel running components, but not all clusters have ReadWriteMany PVC support.

Yes, that is the downside of this approach and the design attempts to limit the potential impact by only mounting the PVC when it's needed by the component. I can't think of an alternative other than allowing multiple workspaces.

> 3. What happens with metadata/artifacts when I go to retrieve them later? For example through the SDK (i.e. client.get_artifact(..))? Do I still have a record that the run occurred? What about the ability to retrieve metrics, parameters, artifacts for it? If the PVC is gone, do I get an intelligible error message?

The content in the workspace/PVC are not output artifacts. Any output artifacts that come from the workspace by specifying the `artifact_path` property are uploaded from the workspace to S3. I think in the v1 approach, this was an issue but this proposal specifically doesn't allow output artifacts in a PVC for the issue you mention.
author_association: COLLABORATOR
user: login: HumairAK
created_at: 2025-05-15T12:48:51Z
body: just the one comment above, otherwise /lgtm
great write up! 🎉 
author_association: COLLABORATOR
user: login: mprahl
created_at: 2025-05-15T13:19:08Z
body: @HumairAK thanks! I updated the table of contents but also fixed the script to update the table of contents in the second commit.
author_association: COLLABORATOR
user: login: VaniHaripriya
created_at: 2025-05-15T19:55:11Z
body: /lgtm

Great write up @mprahl !
author_association: CONTRIBUTOR
user: login: mprahl
created_at: 2025-05-15T21:11:07Z
body: /approve
author_association: COLLABORATOR
user: login: mprahl
created_at: 2025-05-15T21:11:48Z
body: @HumairAK can you approve it please?
author_association: COLLABORATOR
user: login: HumairAK
created_at: 2025-05-16T17:40:17Z
body: /approve
author_association: COLLABORATOR
user: login: google-oss-prow[bot]
created_at: 2025-05-16T17:40:24Z
body: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubeflow/pipelines/pull/11875#issuecomment-2887325595" title="Approved">HumairAK</a>*, *<a href="https://github.com/kubeflow/pipelines/pull/11875#issuecomment-2885062015" title="Approved">mprahl</a>*

The full list of commands accepted by this bot can be found [here](https://oss.gprow.dev/command-help?repo=kubeflow%2Fpipelines).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubeflow/pipelines/blob/master/OWNERS)~~ [HumairAK]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
author_association: NONE