Repository: zenml-io/zenml
Pull request #3292: Pyproject.toml support for DockerSettings
Author: login: schustmi
Labels: name: enhancement
description: New feature or request
name: internal
description: To filter out internal PRs and issues
Created: 2025-01-17T15:59:01Z | Merged: 2025-05-13T13:14:01Z

## Describe changes

This PR adds the following functionality:
- support for `pyproject.toml` files to be specified for installing requirements in Docker images.
ZenML will extract the requirements from the pyproject.toml using one of the default or a custom user-provided command.
- install packages from a `requirements.txt` or `pyproject.toml` file in the source root if no requirements or pyproject file was specified in the `DockerSettings`.
- an attribute to disable the above automatic installation.
- a way to specify a command that installs a local python package.

## Pre-requisites
Please ensure you have done the following:
- [ ] I have read the **CONTRIBUTING.md** document.
- [ ] I have added tests to cover my changes.
- [ ] I have based my new branch on `develop` and the open PR is targeting `develop`. If your branch wasn't based on develop read [Contribution guide on rebasing branch to develop](https://github.com/zenml-io/zenml/blob/main/CONTRIBUTING.md#-pull-requests-rebase-your-branch-on-develop).
- [ ] **IMPORTANT**: I made sure that my changes are reflected properly in the following resources:
  - [ ] [ZenML Docs](https://docs.zenml.io)
  - [ ] Dashboard: Needs to be communicated to the frontend team.
  - [ ] Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
  - [ ] [Projects](https://github.com/zenml-io/zenml-projects): Depending on the version dependencies, different projects might get affected.

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Other (add details above)




Diff:
diff --git a/docs/book/how-to/containerization/containerization.md b/docs/book/how-to/containerization/containerization.md
index 45e5334a7ba..3a7799c1d53 100644
--- a/docs/book/how-to/containerization/containerization.md
+++ b/docs/book/how-to/containerization/containerization.md
@@ -245,65 +245,102 @@ ZenML offers several ways to specify dependencies for your Docker containers:
 
 ### Python Dependencies
 
+By default, ZenML automatically installs all packages required by your active ZenML stack. 
+
+{% hint style="warning" %}
+In future versions, if none of the `replicate_local_python_environment`, `pyproject_path` or `requirements` attributes on `DockerSettings` are specified, ZenML will try to automatically find a `requirements.txt` and `pyproject.toml` files inside your current source root and install packages from the first one it finds. You can disable this behavior by setting `disable_automatic_requirements_detection=True`. If
+you already want this automatic detection in current versions of ZenML, set `disable_automatic_requirements_detection=False`.
+{% endhint %}
+
 1.  **Replicate Local Environment**:
+    ```python
+    docker_settings = DockerSettings(replicate_local_python_environment=True)
+
+
+    @pipeline(settings={"docker": docker_settings})
+    def my_pipeline(...):
+        ...
+    ```
+
+    This will run `pip freeze` to get a list of the installed packages in your local Python environment and will install them in the Docker image. This ensures that the same
+    exact dependencies will be installed.
+    {% hint style="warning" %}
+    This does not work when you have a local project installed. To install local projects, check out the `Install Local Projects` section below.
+    {% endhint %}
+2.  **Specify a `pyproject.toml` file**:
 
     ```python
-    # Use pip freeze (outputs a requirements file with exact package versions)
-    from zenml.config import DockerSettings, PythonEnvironmentExportMethod
-    docker_settings = DockerSettings(
-        replicate_local_python_environment=PythonEnvironmentExportMethod.PIP_FREEZE
-    )
-    # Or as a string
-    docker_settings = DockerSettings(replicate_local_python_environment="pip_freeze")
-
-    # Or use poetry (requires Poetry to be installed)
-    docker_settings = DockerSettings(
-        replicate_local_python_environment=PythonEnvironmentExportMethod.POETRY_EXPORT
-    )
-    # Or as a string
-    docker_settings = DockerSettings(replicate_local_python_environment="poetry_export")
-
-    # Use custom command (provide a list of command arguments)
-    docker_settings = DockerSettings(replicate_local_python_environment=[
-        "poetry", "export", "--extras=train", "--format=requirements.txt"
-    ])
+    docker_settings = DockerSettings(pyproject_path="/path/to/pyproject.toml")
+
+    @pipeline(settings={"docker": docker_settings})
+    def my_pipeline(...):
+        ...
     ```
 
-    This feature allows you to easily replicate your local Python environment in the Docker container, ensuring that your pipeline runs with the same dependencies.
-2.  **Specify Requirements Directly**:
+    By default, ZenML will try to export the dependencies specified in the `pyproject.toml` by trying to run `uv export` and `poetry export`.
+    If both of these commands do not work for your `pyproject.toml` file or you want to customize the command (for example to install certain
+    extras), you can specify a custom command using the `pyproject_export_command` attribute. This command must output a list of requirements following the format of the [requirements file](https://pip.pypa.io/en/stable/reference/requirements-file-format/). The command can contain a `{directory}` placeholder which will be replaced with the directory in which the `pyproject.toml` file is stored.
+
+    ```python
+    from zenml.config import DockerSettings
+
+    docker_settings = DockerSettings(pyproject_export_command=[
+        "uv",
+        "export",
+        "--extra=train",
+        "--format=requirements-txt"
+        "--directory={directory}
+    ])
+
+
+    @pipeline(settings={"docker": docker_settings})
+    def my_pipeline(...):
+        ...
+    ```
+3.  **Specify Requirements Directly**:
 
     ```python
     docker_settings = DockerSettings(requirements=["torch==1.12.0", "torchvision"])
     ```
-3.  **Use Requirements File**:
+4.  **Use Requirements File**:
 
     ```python
     docker_settings = DockerSettings(requirements="/path/to/requirements.txt")
     ```
-4.  **Specify ZenML Integrations**:
+5.  **Specify ZenML Integrations**:
 
     ```python
     from zenml.integrations.constants import PYTORCH, EVIDENTLY
 
     docker_settings = DockerSettings(required_integrations=[PYTORCH, EVIDENTLY])
     ```
-5.  **Control Stack Requirements**:\
+6.  **Control Stack Requirements**:
     By default, ZenML installs the requirements needed by your active stack. You can disable this behavior if needed:
 
     ```python
     docker_settings = DockerSettings(install_stack_requirements=False)
     ```
 
-{% hint style="info" %}
-You can combine these methods but do make sure that your list of requirements does not overlap with ones specified explicitly in the Docker settings to avoid version conflicts.
-{% endhint %}
+7.  **Install Local Projects**:
+    If your code requires the installation of some local code files as a python package, you can specify a command
+    that installs it as follows:
+    ```python
+    docker_settings = DockerSettings(local_project_install_command="pip install . --no-deps")
+    ```
+
+    {% hint style="warning" %}
+    Installing a local python package only works if your code files are included in the Docker image, so make sure you have
+    `allow_including_files_in_images=True` in your Docker settings. If you want to instead use the [code download functionality](#source-code-management)
+    to avoid building new Docker images for each pipeline run, you can follow [this example](https://github.com/zenml-io/zenml-patterns/tree/main/docker-local-pkg). 
+    {% endhint %}
 
 Depending on the options specified in your Docker settings, ZenML installs the requirements in the following order (each step optional):
 
 1. The packages installed in your local Python environment
 2. The packages required by the stack (unless disabled by setting `install_stack_requirements=False`)
 3. The packages specified via the `required_integrations`
-4. The packages specified via the `requirements` attribute
+4. The packages defined in the pyproject.toml file specified by the `pyproject_path` attribute
+5. The packages specified via the `requirements` attribute
 
 ### System Packages
 
diff --git a/src/zenml/config/build_configuration.py b/src/zenml/config/build_configuration.py
index 3a441f4537a..3fa4e185094 100644
--- a/src/zenml/config/build_configuration.py
+++ b/src/zenml/config/build_configuration.py
@@ -135,6 +135,9 @@ def should_include_files(
         Returns:
             Whether files should be included in the image.
         """
+        if self.settings.local_project_install_command:
+            return True
+
         if self.should_download_files(code_repository=code_repository):
             return False
 
@@ -153,6 +156,9 @@ def should_download_files(
         Returns:
             Whether files should be downloaded in the image.
         """
+        if self.settings.local_project_install_command:
+            return False
+
         if self.should_download_files_from_code_repository(
             code_repository=code_repository
         ):
@@ -176,6 +182,9 @@ def should_download_files_from_code_repository(
         Returns:
             Whether files should be downloaded from the code repository.
         """
+        if self.settings.local_project_install_command:
+            return False
+
         if (
             code_repository
             and self.settings.allow_download_from_code_repository
diff --git a/src/zenml/config/docker_settings.py b/src/zenml/config/docker_settings.py
index f4f32461f89..7fee0e90c8e 100644
--- a/src/zenml/config/docker_settings.py
+++ b/src/zenml/config/docker_settings.py
@@ -91,12 +91,21 @@ class DockerSettings(BaseSettings):
     --------------------------------
     Depending on the configuration of this object, requirements will be
     installed in the following order (each step optional):
-    - The packages installed in your local python environment
+    - The packages installed in your local python environment (extracted using
+      `pip freeze`)
     - The packages required by the stack unless this is disabled by setting
-      `install_stack_requirements=False`.
+      `install_stack_requirements=False`
     - The packages specified via the `required_integrations`
+    - The packages defined inside a pyproject.toml file given by the
+      `pyproject_path` attribute.
     - The packages specified via the `requirements` attribute
 
+    If neither `replicate_local_python_environment`, `pyproject_path` or
+    `requirements` are specified, ZenML will try to automatically find a
+    requirements.txt or pyproject.toml file in your current source root
+    and installs packages from the first one it finds. You can disable this
+    behavior by setting `disable_automatic_requirements_detection=True`.
+
     Attributes:
         parent_image: Full name of the Docker image that should be
             used as the parent for the image that will be built. Defaults to
@@ -137,10 +146,29 @@ class DockerSettings(BaseSettings):
             packages.
         python_package_installer_args: Arguments to pass to the python package
             installer.
-        replicate_local_python_environment: If not `None`, ZenML will use the
-            specified method to generate a requirements file that replicates
-            the packages installed in the currently running python environment.
-            This requirements file will then be installed in the Docker image.
+        disable_automatic_requirements_detection: If set to True, ZenML will
+            not automatically detect requirements.txt files or pyproject.toml
+            files in your source root.
+        replicate_local_python_environment: If set to True, ZenML will run
+            `pip freeze` to gather the requirements of the local Python
+            environment and then install them in the Docker image.
+        pyproject_path: Path to a pyproject.toml file. If given, the
+            dependencies will be exported to a requirements.txt
+            formatted file using the `pyproject_export_command` and then
+            installed inside the Docker image.
+        pyproject_export_command: Command to export the dependencies inside a
+            pyproject.toml file to a requirements.txt formatted file. If not
+            given and ZenML needs to export the requirements anyway, `uv export`
+            and `poetry export` will be tried to see if one of them works. This
+            command can contain a `{directory}` placeholder which will be
+            replaced with the directory in which the pyproject.toml file is
+            stored.
+            **Note**: This command will be run before any code files are copied
+            into the image. It is therefore not possible to install a local
+            project using this command. This command should exclude any local
+            projects, and you can specify a `local_project_install_command`
+            instead which will be run after the code files are copied into the
+            image.
         requirements: Path to a requirements file or a list of required pip
             packages. During the image build, these requirements will be
             installed using pip. If you need to use a different tool to
@@ -149,13 +177,16 @@ class DockerSettings(BaseSettings):
         required_integrations: List of ZenML integrations that should be
             installed. All requirements for the specified integrations will
             be installed inside the Docker image.
-        required_hub_plugins: DEPRECATED/UNUSED.
         install_stack_requirements: If `True`, ZenML will automatically detect
             if components of your active stack are part of a ZenML integration
             and install the corresponding requirements and apt packages.
             If you set this to `False` or use custom components in your stack,
             you need to make sure these get installed by specifying them in
             the `requirements` and `apt_packages` attributes.
+        local_project_install_command: Command to install a local project in
+            the Docker image. This is run after the code files are copied into
+            the image, and it is therefore only possible when code is included
+            in the image, not downloaded at runtime.
         apt_packages: APT packages to install inside the Docker image.
         environment: Dictionary of environment variables to set inside the
             Docker image.
@@ -170,14 +201,6 @@ class DockerSettings(BaseSettings):
             from a code repository if possible.
         allow_download_from_artifact_store: If `True`, code can be downloaded
             from the artifact store.
-        build_options: DEPRECATED, use parent_image_build_config.build_options
-            instead.
-        dockerignore: DEPRECATED, use build_config.dockerignore instead.
-        copy_files: DEPRECATED/UNUSED.
-        copy_global_config: DEPRECATED/UNUSED.
-        source_files: DEPRECATED. Use allow_including_files_in_images,
-            allow_download_from_code_repository and
-            allow_download_from_artifact_store instead.
     """
 
     parent_image: Optional[str] = None
@@ -191,14 +214,18 @@ class DockerSettings(BaseSettings):
         PythonPackageInstaller.PIP
     )
     python_package_installer_args: Dict[str, Any] = {}
+    disable_automatic_requirements_detection: bool = True
     replicate_local_python_environment: Optional[
-        Union[List[str], PythonEnvironmentExportMethod]
+        Union[List[str], PythonEnvironmentExportMethod, bool]
     ] = Field(default=None, union_mode="left_to_right")
+    pyproject_path: Optional[str] = None
+    pyproject_export_command: Optional[List[str]] = None
     requirements: Union[None, str, List[str]] = Field(
         default=None, union_mode="left_to_right"
     )
     required_integrations: List[str] = []
     install_stack_requirements: bool = True
+    local_project_install_command: Optional[str] = None
     apt_packages: List[str] = []
     environment: Dict[str, Any] = {}
     user: Optional[str] = None
@@ -221,6 +248,8 @@ class DockerSettings(BaseSettings):
         "copy_global_config",
         "source_files",
         "required_hub_plugins",
+        "build_options",
+        "dockerignore",
     )
 
     @model_validator(mode="before")
@@ -305,6 +334,58 @@ def _validate_skip_build(self) -> "DockerSettings":
 
         return self
 
+    @model_validator(mode="after")
+    def _validate_code_files_included_if_installing_local_project(
+        self,
+    ) -> "DockerSettings":
+        """Ensures that files are included when installing a local package.
+
+        Raises:
+            ValueError: If files are not included in the Docker image
+                when trying to install a local package.
+
+        Returns:
+            The validated settings values.
+        """
+        if (
+            self.local_project_install_command
+            and not self.allow_including_files_in_images
+        ):
+            raise ValueError(
+                "Files must be included in the Docker image when trying to "
+                "install a local python package. You can do so by setting "
+                "the `allow_including_files_in_images` attribute of your "
+                "DockerSettings to `True`."
+            )
+
+        return self
+
+    @model_validator(mode="after")
+    def _deprecate_replicate_local_environment_commands(
+        self,
+    ) -> "DockerSettings":
+        """Deprecates some values for `replicate_local_python_environment`.
+
+        Returns:
+            The validated settings values.
+        """
+        if isinstance(
+            self.replicate_local_python_environment,
+            (str, list, PythonEnvironmentExportMethod),
+        ):
+            logger.warning(
+                "Specifying a command (`%s`) for "
+                "`DockerSettings.replicate_local_python_environment` is "
+                "deprecated. If you want to replicate your exact local "
+                "environment using `pip freeze`, set "
+                "`DockerSettings.replicate_local_python_environment=True`. "
+                "If you want to export requirements from a pyproject.toml "
+                "file, use `DockerSettings.pyproject_path` and "
+                "`DockerSettings.pyproject_export_command` instead."
+            )
+
+        return self
+
     model_config = ConfigDict(
         # public attributes are immutable
         frozen=True,
diff --git a/src/zenml/pipelines/build_utils.py b/src/zenml/pipelines/build_utils.py
index f23873ee50e..1d11dae0daa 100644
--- a/src/zenml/pipelines/build_utils.py
+++ b/src/zenml/pipelines/build_utils.py
@@ -80,6 +80,11 @@ def requires_included_code(
     for step in deployment.step_configurations.values():
         docker_settings = step.config.docker_settings
 
+        if docker_settings.local_project_install_command:
+            # When installing a local package, we need to include the code
+            # files in the container image.
+            return True
+
         if docker_settings.allow_download_from_artifact_store:
             return False
 
@@ -136,6 +141,9 @@ def code_download_possible(
         Whether code download is possible for the deployment.
     """
     for step in deployment.step_configurations.values():
+        if step.config.docker_settings.local_project_install_command:
+            return False
+
         if step.config.docker_settings.allow_download_from_artifact_store:
             continue
 
diff --git a/src/zenml/utils/pipeline_docker_image_builder.py b/src/zenml/utils/pipeline_docker_image_builder.py
index b82a260fe42..9cc8a276f75 100644
--- a/src/zenml/utils/pipeline_docker_image_builder.py
+++ b/src/zenml/utils/pipeline_docker_image_builder.py
@@ -68,6 +68,11 @@
 }
 UV_DEFAULT_ARGS = {"no-cache-dir": None}
 
+DEFAULT_PYPROJECT_EXPORT_COMMANDS = [
+    "uv export --format=requirements-txt --directory={directory} --no-hashes --no-emit-project",
+    "poetry export --format=requirements.txt --directory={directory} --without-hashes",
+]
+
 
 class PipelineDockerImageBuilder:
     """Builds Docker images to run a ZenML pipeline."""
@@ -156,10 +161,11 @@ def build_docker_image(
         requires_zenml_build = any(
             [
                 docker_settings.requirements,
+                docker_settings.pyproject_path,
                 docker_settings.required_integrations,
-                docker_settings.required_hub_plugins,
                 docker_settings.replicate_local_python_environment,
                 docker_settings.install_stack_requirements,
+                docker_settings.local_project_install_command,
                 docker_settings.apt_packages,
                 docker_settings.environment,
                 include_files,
@@ -434,8 +440,56 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
+
+        if (
+            implicit_requirements
+            and docker_settings.disable_automatic_requirements_detection
+        ):
+            # TODO: This is only temporary to log a warning notifying users
+            # that we will soon switch the default behavior to
+            # `disable_automatic_requirements_detection=False`. Remove and
+            # adjust the logic once we've made this change.
+            if log:
+                logger.warning(
+                    "Detected `requirements.txt` or `pyproject.toml` files in "
+                    "the source root. In future versions of ZenML, these will "
+                    "be automatically picked up and installed in Docker images "
+                    "by default. To disable this behavior and keep the "
+                    "current behavior, set "
+                    "`DockerSettings.disable_automatic_requirements_detection` "
+                    "to `True`. If you want to enable this behavior right away, "
+                    "you can do so by setting "
+                    "`DockerSettings.disable_automatic_requirements_detection` "
+                    "to `False`."
+                )
+            implicit_requirements = False
+            requirements = None
+            pyproject_path = None
+
         requirements_files: List[Tuple[str, str, List[str]]] = []
 
         # Generate requirements file for the local environment if configured
@@ -447,6 +501,10 @@ def gather_requirements_files(
                 command = (
                     docker_settings.replicate_local_python_environment.command
                 )
+            elif isinstance(
+                docker_settings.replicate_local_python_environment, bool
+            ):
+                command = PythonEnvironmentExportMethod.PIP_FREEZE.command
             else:
                 command = " ".join(
                     docker_settings.replicate_local_python_environment
@@ -520,9 +578,69 @@ def gather_requirements_files(
                     ", ".join(f"`{r}`" for r in integration_requirements_list),
                 )
 
+        if pyproject_path:
+            path = os.path.abspath(pyproject_path)
+
+            if not os.path.exists(path):
+                raise FileNotFoundError(
+                    f"Pyproject file {path} does not exist."
+                )
+
+            def _run_command(command: str) -> str:
+                command = command.format(directory=os.path.dirname(path))
+
+                result = subprocess.run(
+                    command,
+                    capture_output=True,
+                    check=True,
+                    shell=True,  # nosec
+                )
+                return result.stdout.decode()
+
+            if docker_settings.pyproject_export_command:
+                command = " ".join(docker_settings.pyproject_export_command)
+
+                try:
+                    pyproject_requirements = _run_command(command)
+                except subprocess.CalledProcessError as e:
+                    raise RuntimeError(
+                        "Failed to export pyproject dependencies with "
+                        f"command `{command}`: {e.stderr.decode()}"
+                    )
+            else:
+                command_errors = {}
+
+                for command in DEFAULT_PYPROJECT_EXPORT_COMMANDS:
+                    try:
+                        pyproject_requirements = _run_command(command)
+                    except subprocess.CalledProcessError as e:
+                        command_errors[command] = e.stderr.decode()
+                    else:
+                        break
+                else:
+                    raise RuntimeError(
+                        "Failed to export pyproject dependencies with the "
+                        f"following commands: {command_errors}. Please specify "
+                        "a working command to export your pyproject.toml "
+                        "dependencies to a requirements.txt formatted file "
+                        "using `DockerSettings.pyproject_export_command`."
+                    )
+
+            requirements_files.append(
+                (".zenml_pyproject_requirements", pyproject_requirements, [])
+            )
+            if log:
+                logger.info(
+                    "- %s python packages from file `%s`",
+                    "Implicitly including"
+                    if implicit_requirements
+                    else "Including",
+                    path,
+                )
+
         # Generate/Read requirements file for user-defined requirements
-        if isinstance(docker_settings.requirements, str):
-            path = os.path.abspath(docker_settings.requirements)
+        if isinstance(requirements, str):
+            path = os.path.abspath(requirements)
             try:
                 user_requirements = io_utils.read_file_contents_as_string(path)
             except FileNotFoundError as e:
@@ -531,15 +649,18 @@ def gather_requirements_files(
                 ) from e
             if log:
                 logger.info(
-                    "- Including user-defined requirements from file `%s`",
+                    "- %s user-defined requirements from file `%s`",
+                    "Implicitly including"
+                    if implicit_requirements
+                    else "Including",
                     path,
                 )
-        elif isinstance(docker_settings.requirements, List):
-            user_requirements = "\n".join(docker_settings.requirements)
+        elif isinstance(requirements, List):
+            user_requirements = "\n".join(requirements)
             if log:
                 logger.info(
                     "- Including user-defined requirements: %s",
-                    ", ".join(f"`{r}`" for r in docker_settings.requirements),
+                    ", ".join(f"`{r}`" for r in requirements),
                 )
         else:
             user_requirements = None
@@ -638,6 +759,11 @@ def _generate_zenml_pipeline_dockerfile(
         lines.append("COPY . .")
         lines.append("RUN chmod -R a+rw .")
 
+        if docker_settings.local_project_install_command:
+            lines.append(
+                f"RUN {docker_settings.local_project_install_command}"
+            )
+
         if docker_settings.user:
             # Change file ownership to specified user
             lines.append(f"RUN chown -R {docker_settings.user} .")
diff --git a/tests/unit/utils/test_pipeline_docker_image_builder.py b/tests/unit/utils/test_pipeline_docker_image_builder.py
index e4befd5952f..2dd126895b6 100644
--- a/tests/unit/utils/test_pipeline_docker_image_builder.py
+++ b/tests/unit/utils/test_pipeline_docker_image_builder.py
@@ -12,6 +12,7 @@
 #  or implied. See the License for the specific language governing
 #  permissions and limitations under the License.
 
+import subprocess
 from pathlib import Path
 
 import pytest
@@ -48,9 +49,59 @@ def test_check_user_is_set():
 def test_requirements_file_generation(mocker, local_stack, tmp_path: Path):
     """Tests that the requirements get included in the correct order and only when configured."""
     mocker.patch("subprocess.check_output", return_value=b"local_requirements")
+
+    mocker.patch(
+        "subprocess.run",
+        return_value=subprocess.CompletedProcess(
+            args="", returncode=0, stdout=b"pyproject_requirements"
+        ),
+    )
     mocker.patch.object(
         local_stack, "requirements", return_value={"stack_requirements"}
     )
+    from zenml.utils import source_utils
+
+    mocker.patch.object(
+        source_utils,
+        "get_source_root",
+        return_value=str(tmp_path),
+    )
+
+    # Nothing specified -> No requirements
+    settings = DockerSettings(
+        install_stack_requirements=False,
+    )
+    files = PipelineDockerImageBuilder.gather_requirements_files(
+        settings, stack=local_stack
+    )
+    assert len(files) == 0
+
+    # Nothing specified, no requirements.txt exists -> Use pyproject.toml
+    pyproject_file = tmp_path / "pyproject.toml"
+    pyproject_file.write_text("pyproject")
+
+    settings = DockerSettings(
+        install_stack_requirements=False,
+        disable_automatic_requirements_detection=False,
+    )
+    files = PipelineDockerImageBuilder.gather_requirements_files(
+        settings, stack=local_stack
+    )
+    assert len(files) == 1
+    assert files[0][1] == "pyproject_requirements"
+
+    # Nothing specified, requirements.txt exists -> Use requirements.txt
+    requirements_file = tmp_path / "requirements.txt"
+    requirements_file.write_text("user_requirements")
+
+    files = PipelineDockerImageBuilder.gather_requirements_files(
+        settings, stack=local_stack
+    )
+    assert len(files) == 1
+    assert files[0][1] == "user_requirements"
+
+    pyproject_file.unlink()
+    requirements_file.unlink()
 
     # just local requirements
     settings = DockerSettings(
@@ -94,16 +145,22 @@ def test_requirements_file_generation(mocker, local_stack, tmp_path: Path):
     # all values set
     requirements_file = tmp_path / "requirements.txt"
     requirements_file.write_text("user_requirements")
+
+    pyproject_file = tmp_path / "pyproject.toml"
+    pyproject_file.write_text("pyproject")
+
     settings = DockerSettings(
+        disable_automatic_requirements_detection=False,
         install_stack_requirements=True,
         requirements=str(requirements_file),
         required_integrations=[SKLEARN],
         replicate_local_python_environment="pip_freeze",
+        pyproject_path=str(pyproject_file),
     )
     files = PipelineDockerImageBuilder.gather_requirements_files(
         settings, stack=local_stack
     )
-    assert len(files) == 4
+    assert len(files) == 5
     # first up the local python requirements
     assert files[0][1] == "local_requirements"
     # then the stack requirements
@@ -113,8 +170,10 @@ def test_requirements_file_generation(mocker, local_stack, tmp_path: Path):
         sorted(SklearnIntegration.REQUIREMENTS)
     )
     assert files[2][1] == expected_integration_requirements
+    # then the pyproject requirements
+    assert files[3][1] == "pyproject_requirements"
     # last the user requirements
-    assert files[3][1] == "user_requirements"
+    assert files[4][1] == "user_requirements"
 
 
 def test_build_skipping():


Reviews:
user: login: bcdurak
body: Leaving some initial comments.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-01-30T13:51:24Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-01-31T12:11:53Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-01-31T12:13:05Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-01-31T12:19:45Z
user: login: bcdurak
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-13T15:30:11Z
user: login: AlexejPenner
body: 🦞 
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T15:15:17Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T15:17:33Z
user: login: stefannica
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T08:55:02Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:00:15Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:02:02Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:03:13Z
user: login: stefannica
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:32:48Z
user: login: htahir1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:35:23Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:36:08Z
user: login: htahir1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T09:55:19Z
user: login: schustmi
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T10:00:24Z
user: login: htahir1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T10:48:25Z
user: login: stefannica
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T11:34:55Z

Review comments:
diff_hunk: @@ -68,6 +68,12 @@
 }
 UV_DEFAULT_ARGS = {"no-cache-dir": None}
 
+# TODO: these don't actually install any extras. Should we include all extras? No extras? Exclude dev extras?
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: bcdurak
body: Just a reminder for the TODO.
created_at: 2025-01-30T12:46:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -91,12 +91,19 @@ class DockerSettings(BaseSettings):
     --------------------------------
     Depending on the configuration of this object, requirements will be
     installed in the following order (each step optional):
-    - The packages installed in your local python environment
+    - The packages installed in your local python environment (extracted using
+      `pip freeze`)
     - The packages required by the stack unless this is disabled by setting
-      `install_stack_requirements=False`.
+      `install_stack_requirements=False`
     - The packages specified via the `required_integrations`
+    - The packages defined inside a pyproject.toml file given by the
path: src/zenml/config/docker_settings.py
user: login: bcdurak
body: I think there is a misalignment with the docs here. In the docs, we say the priority is as follows:

1. Packages in the local env 
2. Packages for the stack
3. Required Integrations
4. Requirements (could be a list or a path to a requirements.txt)
5. Pyproject.toml next

In the docstring here, 4 and 5 are switched.
created_at: 2025-01-30T13:14:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -437,8 +443,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: bcdurak
body: Same issue with the order as above.
created_at: 2025-01-30T13:15:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -437,8 +443,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.replicate_local_python_environment,
+                docker_settings.required_integrations,
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: bcdurak
body: Does this mean that adding an extra integration (or a package) to be installed disables the discovery of the implicit requirements? So, if I was not using any of these settings before and now want to add `sklearn` to my docker settings, suddenly the `requirements.txt` will not be detected. I am not sure if this is the optimal behavior here.
created_at: 2025-01-30T13:24:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -192,8 +209,10 @@ class DockerSettings(BaseSettings):
     )
     python_package_installer_args: Dict[str, Any] = {}
     replicate_local_python_environment: Optional[
-        Union[List[str], PythonEnvironmentExportMethod]
+        Union[List[str], PythonEnvironmentExportMethod, bool]
     ] = Field(default=None, union_mode="left_to_right")
+    pyproject_path: Optional[str] = None
+    pyproject_export_command: Optional[List[str]] = None
path: src/zenml/config/docker_settings.py
user: login: bcdurak
body: As far as I can see, it is possible to set this as follows: `DockerSettings(pyproject_export_command=[......])` without an explicit `pyproject_path` as long as there is a `pyproject.toml` file to work with, in the current source root. But as soon as someone puts an integration, requirement or the replicate flag, this becomes unused.
created_at: 2025-01-30T13:48:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -192,8 +209,10 @@ class DockerSettings(BaseSettings):
     )
     python_package_installer_args: Dict[str, Any] = {}
     replicate_local_python_environment: Optional[
-        Union[List[str], PythonEnvironmentExportMethod]
+        Union[List[str], PythonEnvironmentExportMethod, bool]
     ] = Field(default=None, union_mode="left_to_right")
+    pyproject_path: Optional[str] = None
+    pyproject_export_command: Optional[List[str]] = None
path: src/zenml/config/docker_settings.py
user: login: bcdurak
body: Perhaps a validator can be helpful to detect this early.
created_at: 2025-01-30T13:48:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -91,12 +91,19 @@ class DockerSettings(BaseSettings):
     --------------------------------
     Depending on the configuration of this object, requirements will be
     installed in the following order (each step optional):
-    - The packages installed in your local python environment
+    - The packages installed in your local python environment (extracted using
+      `pip freeze`)
     - The packages required by the stack unless this is disabled by setting
-      `install_stack_requirements=False`.
+      `install_stack_requirements=False`
     - The packages specified via the `required_integrations`
+    - The packages defined inside a pyproject.toml file given by the
path: src/zenml/config/docker_settings.py
user: login: bcdurak
body: I see that the docstring here is correct in the code and the docs need to be updated.
created_at: 2025-01-30T13:49:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -437,8 +443,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.replicate_local_python_environment,
+                docker_settings.required_integrations,
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: schustmi
body: Correct. I'm not super convinced by it myself to be honest, but what would be a better alternative in your opinion?
created_at: 2025-01-31T12:11:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -192,8 +209,10 @@ class DockerSettings(BaseSettings):
     )
     python_package_installer_args: Dict[str, Any] = {}
     replicate_local_python_environment: Optional[
-        Union[List[str], PythonEnvironmentExportMethod]
+        Union[List[str], PythonEnvironmentExportMethod, bool]
     ] = Field(default=None, union_mode="left_to_right")
+    pyproject_path: Optional[str] = None
+    pyproject_export_command: Optional[List[str]] = None
path: src/zenml/config/docker_settings.py
user: login: schustmi
body: What would you validate here? I don't see the problem of this being unused if there is no `pyproject.toml` to be honest
created_at: 2025-01-31T12:13:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -91,12 +91,19 @@ class DockerSettings(BaseSettings):
     --------------------------------
     Depending on the configuration of this object, requirements will be
     installed in the following order (each step optional):
-    - The packages installed in your local python environment
+    - The packages installed in your local python environment (extracted using
+      `pip freeze`)
     - The packages required by the stack unless this is disabled by setting
-      `install_stack_requirements=False`.
+      `install_stack_requirements=False`
     - The packages specified via the `required_integrations`
+    - The packages defined inside a pyproject.toml file given by the
path: src/zenml/config/docker_settings.py
user: login: schustmi
body: You're right, I fixed the error in the docs.

In general, there are two scenarios:
- You don't explicitly specify any requirements using docker settings: In this case we first look for a `requirements.txt` (-> highest priority), and if it doesn't exist we look for a `pyproject.toml`
- You do explicitly specify some requirements (using e.g. `DockerSettings.required_integrations` or `DockerSettings.requirements`): In this case the order in the docstring is the one that is being used. Requirements being last means it is the highest priority (it will override values from all previous steps), which is in line with the first case.

Do you think these make sense?

created_at: 2025-01-31T12:19:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -437,8 +443,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.replicate_local_python_environment,
+                docker_settings.required_integrations,
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: bcdurak
body: Yeah, I guess I do not have a super clear alternative either. 

The reason why I was a bit hesitant here was this. We have a default behavior that decides whether we look for these files or not. By adding an additional integration, (that I intuitively thought would not change this behavior), I change the way that it looks for these files. I guess the `replicate_local_python_environment` flag at least allows people to control the behavior. Perhaps we can write a sentence about this in the docs?
created_at: 2025-02-13T15:21:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -68,6 +68,12 @@
 }
 UV_DEFAULT_ARGS = {"no-cache-dir": None}
 
+# TODO: these don't actually install any extras. Should we include all extras? No extras? Exclude dev extras?
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: bcdurak
body: Is this still relevant for this PR?
created_at: 2025-02-13T15:21:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -192,8 +209,10 @@ class DockerSettings(BaseSettings):
     )
     python_package_installer_args: Dict[str, Any] = {}
     replicate_local_python_environment: Optional[
-        Union[List[str], PythonEnvironmentExportMethod]
+        Union[List[str], PythonEnvironmentExportMethod, bool]
     ] = Field(default=None, union_mode="left_to_right")
+    pyproject_path: Optional[str] = None
+    pyproject_export_command: Optional[List[str]] = None
path: src/zenml/config/docker_settings.py
user: login: bcdurak
body: I would validate the following two scenarios:
- If `pyproject_export_command` is set, there needs to be a `pyproject.toml` in the directory.
- If `required_integrations` are set, `pyproject_export_command ` is unused. 

However, both are not so critical, I just thought early detection might help us improve the UX a bit.
created_at: 2025-02-13T15:28:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -91,12 +91,19 @@ class DockerSettings(BaseSettings):
     --------------------------------
     Depending on the configuration of this object, requirements will be
     installed in the following order (each step optional):
-    - The packages installed in your local python environment
+    - The packages installed in your local python environment (extracted using
+      `pip freeze`)
     - The packages required by the stack unless this is disabled by setting
-      `install_stack_requirements=False`.
+      `install_stack_requirements=False`
     - The packages specified via the `required_integrations`
+    - The packages defined inside a pyproject.toml file given by the
path: src/zenml/config/docker_settings.py
user: login: bcdurak
body: I left a comment about the confusion that I had with the interaction between the implicit and explicit definition of requirements but outside of that, the explanation looks good here.
created_at: 2025-02-13T15:29:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -245,49 +245,67 @@ ZenML offers several ways to specify dependencies for your Docker containers:
 
 ### Python Dependencies
 
+By default, ZenML automatically installs all packages required by your active ZenML stack. Additionally, if none of the `replicate_local_python_environment`, `pyproject_path` or `requirements` attributes on `DockerSettings` are specified, ZenML will try to automatically find a `requirements.txt` and `pyproject.toml` files inside your current [source root](./which-files-are-built-into-the-image.md) and installs packages from the first one it finds. You can disable this behavior by setting `disable_automatic_requirements_detection=True`.
+
 1.  **Replicate Local Environment**:
+    ```python
+    docker_settings = DockerSettings(replicate_local_python_environment=True)
+
+
+    @pipeline(settings={"docker": docker_settings})
+    def my_pipeline(...):
+        ...
+    ```
+
+    This will run `pip freeze` to get a list of your local packages, and allows you to easily replicate your local Python environment in the Docker container, ensuring that your pipeline runs with the same dependencies.
path: docs/book/how-to/containerization/containerization.md
user: login: AlexejPenner
body: If this does not pick up locally installed libraries we should say so here
created_at: 2025-05-08T15:06:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -305,6 +334,53 @@ def _validate_skip_build(self) -> "DockerSettings":
 
         return self
 
+    @model_validator(mode="after")
+    def _validate_code_files_included_if_installing_local_project(
+        self,
+    ) -> "DockerSettings":
+        """Ensures that files are included when installing a local package.
+
+        Raises:
+            ValueError: If files are not included in the Docker image
+                when trying to install a local package.
+
+        Returns:
+            The validated settings values.
+        """
+        if (
+            self.local_project_install_command
+            and not self.allow_including_files_in_images
+        ):
+            raise ValueError(
+                "Files must be included in the Docker image when trying to "
+                "install a local python package."
path: src/zenml/config/docker_settings.py
user: login: AlexejPenner
body: ```suggestion
                "Files must be included in the Docker image when trying to "
                "install a local python package. You can do so by using the `allow_including_files_in_images` attribute of your docker settings: `DockerSettings(..., allow_including_files_in_images=True)`"
```

Lets save the user an additional odyssey to find out how to do this.
created_at: 2025-05-08T15:14:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -305,6 +334,53 @@ def _validate_skip_build(self) -> "DockerSettings":
 
         return self
 
+    @model_validator(mode="after")
+    def _validate_code_files_included_if_installing_local_project(
+        self,
+    ) -> "DockerSettings":
+        """Ensures that files are included when installing a local package.
+
+        Raises:
+            ValueError: If files are not included in the Docker image
+                when trying to install a local package.
+
+        Returns:
+            The validated settings values.
+        """
+        if (
+            self.local_project_install_command
+            and not self.allow_including_files_in_images
+        ):
+            raise ValueError(
+                "Files must be included in the Docker image when trying to "
+                "install a local python package."
path: src/zenml/config/docker_settings.py
user: login: schustmi
body: ~~Isn't this literally in 100 code snippets on the same docs page?~~ Ah this is in code, i'll update it.
created_at: 2025-05-08T15:17:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -149,13 +177,16 @@ class DockerSettings(BaseSettings):
         required_integrations: List of ZenML integrations that should be
             installed. All requirements for the specified integrations will
             be installed inside the Docker image.
-        required_hub_plugins: DEPRECATED/UNUSED.
         install_stack_requirements: If `True`, ZenML will automatically detect
             if components of your active stack are part of a ZenML integration
             and install the corresponding requirements and apt packages.
             If you set this to `False` or use custom components in your stack,
             you need to make sure these get installed by specifying them in
             the `requirements` and `apt_packages` attributes.
+        local_project_install_command: Command to install a local project in
+            the Docker image. This is run after the code files are copied into
+            the image, and it is therefore only possible when code is included
+            in the image, not downloaded at runtime.
path: src/zenml/config/docker_settings.py
user: login: stefannica
body: I would argue that the package could theoretically be installed at runtime,  using the container entrypoint, before running any Python code, but this is a bigger change and not at all necessary here.
created_at: 2025-05-13T07:48:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: stefannica
body: Shouldn't you make `disable_automatic_requirements_detection` False by default? Otherwise the upgrade might be breaking for some users: existing installations might pick up `requirements.txt` or `pyproject.toml` files after they upgrade, even though these files were silently ignored before.
created_at: 2025-05-13T07:53:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -305,6 +334,55 @@ def _validate_skip_build(self) -> "DockerSettings":
 
         return self
 
+    @model_validator(mode="after")
+    def _validate_code_files_included_if_installing_local_project(
+        self,
+    ) -> "DockerSettings":
+        """Ensures that files are included when installing a local package.
+
+        Raises:
+            ValueError: If files are not included in the Docker image
+                when trying to install a local package.
+
+        Returns:
+            The validated settings values.
+        """
+        if (
+            self.local_project_install_command
+            and not self.allow_including_files_in_images
+        ):
+            raise ValueError(
+                "Files must be included in the Docker image when trying to "
+                "install a local python package. You can do so by setting "
+                "the `allow_including_files_in_images` attribute of your "
+                "DockerSettings to `True`."
+            )
+
+        return self
+
+    @model_validator(mode="after")
+    def _deprecate_replicate_local_environment_commands(
+        self,
+    ) -> "DockerSettings":
+        """Deprecates some values for `replicate_local_python_environment`.
+
+        Returns:
+            The validated settings values.
+        """
+        if isinstance(self.replicate_local_python_environment, (str, list)):
path: src/zenml/config/docker_settings.py
user: login: stefannica
body: ```suggestion
        if isinstance(self.replicate_local_python_environment, (str, list, PythonEnvironmentExportMethod)):
```

It might also be the enum type, right ?
created_at: 2025-05-13T07:59:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: schustmi
body: You mean `True`, to disable the automatic behavior? I guess we want this to be the default going forward, so at some point this will have to be a breaking change? I'm not sure whether this should be now or some time in the future, @htahir1 maybe has an opinion?
created_at: 2025-05-13T09:00:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -149,13 +177,16 @@ class DockerSettings(BaseSettings):
         required_integrations: List of ZenML integrations that should be
             installed. All requirements for the specified integrations will
             be installed inside the Docker image.
-        required_hub_plugins: DEPRECATED/UNUSED.
         install_stack_requirements: If `True`, ZenML will automatically detect
             if components of your active stack are part of a ZenML integration
             and install the corresponding requirements and apt packages.
             If you set this to `False` or use custom components in your stack,
             you need to make sure these get installed by specifying them in
             the `requirements` and `apt_packages` attributes.
+        local_project_install_command: Command to install a local project in
+            the Docker image. This is run after the code files are copied into
+            the image, and it is therefore only possible when code is included
+            in the image, not downloaded at runtime.
path: src/zenml/config/docker_settings.py
user: login: schustmi
body: Yep agreed, I think we even have a ticket for this to enable it in a more generic way (and only for certain orchestrators), but this is out of scope for now and therefore the above is true at least for the current implementation
created_at: 2025-05-13T09:02:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -305,6 +334,55 @@ def _validate_skip_build(self) -> "DockerSettings":
 
         return self
 
+    @model_validator(mode="after")
+    def _validate_code_files_included_if_installing_local_project(
+        self,
+    ) -> "DockerSettings":
+        """Ensures that files are included when installing a local package.
+
+        Raises:
+            ValueError: If files are not included in the Docker image
+                when trying to install a local package.
+
+        Returns:
+            The validated settings values.
+        """
+        if (
+            self.local_project_install_command
+            and not self.allow_including_files_in_images
+        ):
+            raise ValueError(
+                "Files must be included in the Docker image when trying to "
+                "install a local python package. You can do so by setting "
+                "the `allow_including_files_in_images` attribute of your "
+                "DockerSettings to `True`."
+            )
+
+        return self
+
+    @model_validator(mode="after")
+    def _deprecate_replicate_local_environment_commands(
+        self,
+    ) -> "DockerSettings":
+        """Deprecates some values for `replicate_local_python_environment`.
+
+        Returns:
+            The validated settings values.
+        """
+        if isinstance(self.replicate_local_python_environment, (str, list)):
path: src/zenml/config/docker_settings.py
user: login: schustmi
body: Yep true, applied it.
created_at: 2025-05-13T09:03:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: stefannica
body: I meant `True`, of course.
created_at: 2025-05-13T09:32:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: htahir1
body: I guess it isnt much of a breaking change but we can add it to the breaking change section in the release notes.... I would make it default tbh
created_at: 2025-05-13T09:35:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: schustmi
body: We can also disable this by default and if it's set to `True`, log a warning that this will change in the future? And then we change the default in a few releases?
created_at: 2025-05-13T09:36:07Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: htahir1
body: @schustmi i wouldnt worry so much about it - whats the worst that could happen tbh
created_at: 2025-05-13T09:55:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: schustmi
body: @htahir1 It potentially breaks CI (or any automated) pipelines
created_at: 2025-05-13T10:00:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -434,8 +440,32 @@ def gather_requirements_files(
             - Packages installed in the local Python environment
             - Requirements defined by stack integrations
             - Requirements defined by user integrations
+            - Requirements exported from a pyproject.toml
             - User-defined requirements
         """
+        implicit_requirements = False
+        pyproject_path = docker_settings.pyproject_path
+        requirements = docker_settings.requirements
+
+        if not any(
+            [
+                docker_settings.disable_automatic_requirements_detection,
+                docker_settings.replicate_local_python_environment,
+                requirements,
+                pyproject_path,
+            ]
+        ):
+            root = source_utils.get_source_root()
+            requirements_path = os.path.join(root, "requirements.txt")
+            pyproject_file_path = os.path.join(root, "pyproject.toml")
+
+            if os.path.exists(requirements_path):
+                implicit_requirements = True
+                requirements = requirements_path
+            elif os.path.exists(pyproject_file_path):
+                implicit_requirements = True
+                pyproject_path = pyproject_file_path
path: src/zenml/utils/pipeline_docker_image_builder.py
user: login: htahir1
body: @schustmi hmm but if we set this to false nobody will ever find out about it... but ur right... I'm unsure... maybe lets flip it for now but then we need docs to explain how to flip it back 
created_at: 2025-05-13T10:48:24Z
author_association: CONTRIBUTOR

PR comments:
user: login: coderabbitai[bot]
created_at: 2025-01-17T15:59:08Z
body: <!-- This is an auto-generated comment: summarize by coderabbit.ai -->
<!-- This is an auto-generated comment: skip review by coderabbit.ai -->

> [!IMPORTANT]
> ## Review skipped
> 
> Auto reviews are disabled on this repository.
> 
> Please check the settings in the CodeRabbit UI or the `.coderabbit.yaml` file in this repository. To trigger a single review, invoke the `@coderabbitai review` command.
> 
> You can disable this status message by setting the `reviews.review_status` to `false` in the CodeRabbit configuration file.

<!-- end of auto-generated comment: skip review by coderabbit.ai -->
<!-- tips_start -->

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

<details>
<summary>❤️ Share</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai)
- [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

</details>

<details>
<summary>🪧 Tips</summary>

### Chat

There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=zenml-io/zenml&utm_content=3292):

- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  - `I pushed a fix in commit <commit_id>, please review it.`
  - `Explain this complex logic.`
  - `Open a follow-up GitHub issue for this discussion.`
- Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples:
  - `@coderabbitai explain this code block.`
  -	`@coderabbitai modularize this function.`
- PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.`
  - `@coderabbitai read src/utils.ts and explain its main purpose.`
  - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.`
  - `@coderabbitai help me debug CodeRabbit configuration file.`

### Support

Need help? Create a ticket on our [support page](https://www.coderabbit.ai/contact-us/support) for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

### CodeRabbit Commands (Invoked using PR comments)

- `@coderabbitai pause` to pause the reviews on a PR.
- `@coderabbitai resume` to resume the paused reviews.
- `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
- `@coderabbitai full review` to do a full review from scratch and review all the files again.
- `@coderabbitai summary` to regenerate the summary of the PR.
- `@coderabbitai generate docstrings` to [generate docstrings](https://docs.coderabbit.ai/finishing-touches/docstrings) for this PR.
- `@coderabbitai generate sequence diagram` to generate a sequence diagram of the changes in this PR.
- `@coderabbitai resolve` resolve all the CodeRabbit review comments.
- `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository.
- `@coderabbitai help` to get help.

### Other keywords and placeholders

- Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed.
- Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description.
- Add `@coderabbitai` anywhere in the PR title to generate the title automatically.

### Documentation and Community

- Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit.
- Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback.
- Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

</details>

<!-- tips_end -->
author_association: CONTRIBUTOR
user: login: schustmi
created_at: 2025-01-20T13:43:03Z
body: @htahir1 Any ideas for the general docs structure of this? Or do we just leave it as-is for now?
author_association: CONTRIBUTOR
user: login: htahir1
created_at: 2025-01-20T14:19:18Z
body: So I think the docs need to change in these places too:

- In production guide on [this page](https://docs.zenml.io/user-guide/production-guide/configure-pipeline)

> Now that we have our pipeline up and running in the cloud, you might be wondering how ZenML figured out what sort of dependencies to install in the Docker image that we just ran on the VM. The answer lies in the [runner script we executed (i.e. run.py)](https://github.com/zenml-io/zenml/blob/main/examples/quickstart/run.py#L215), in particular, these lines:

- Probably i'd include the default behavior of trying to find requirements.txt and pyproject.toml implicitly here in the top page: https://docs.zenml.io/how-to/customize-docker-builds

- Probably makes sense to add the implicit behavior here as we mention a lot about allow_download_from_code_repository and allow_download_from_artifact_store? https://docs.zenml.io/how-to/customize-docker-builds/which-files-are-built-into-the-image

- Maybe we need to add it somewhere herE? https://docs.zenml.io/how-to/pipeline-development/configure-python-environments/handling-dependencies#use-a-tool-like-pip-compile-for-reproducibility Not sure exactly

- Finally, I would include it here as a note that pyproject.toml behavior is not recommended but if you include it this is what happens: https://docs.zenml.io/how-to/project-setup-and-management/setting-up-a-project-repository/set-up-repository

author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-01-31T12:22:14Z
body: ✅ No broken links found!
author_association: CONTRIBUTOR
user: login: avishniakov
created_at: 2025-03-28T07:47:52Z
body: @michael-zenml @htahir1 May I add my 5 cents here? Does adding `pyproject.toml` support mean that it will also rely on lock file or it will lock each time? In our setting it will be crucial to have it bounded to locks, since normally locks are also committed for consistency and reproducibility.
Aside from that - great addition to have in ZenML!

P.S. Just noticed that it is a long runner already - I would love to see it priotorised, as it will ease the use of Dockers for our case, without the need to rely on external build process.
author_association: CONTRIBUTOR
user: login: avishniakov
created_at: 2025-03-28T08:32:33Z
body: A small companion PR to that Poetry activity: https://github.com/zenml-io/zenml/pull/3470
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-06T12:20:13Z
body: ## Documentation Link Check Results
✅ **Absolute links check passed**
✅ **Relative links check passed**
<sub>Last checked: 2025-05-13 12:15:04 UTC</sub>
author_association: CONTRIBUTOR
user: login: schustmi
created_at: 2025-05-07T11:35:57Z
body: @avishniakov Users will have full control to specify a command that will be run to extract those requirements from the `pyproject.toml`. This will depend on what package manager you're using, for example in the case of `uv`, you could specify `uv export --frozen` which exports the requirements from the current lock file. Does that clarify, or do you think we need additional changes in this PR to make it work for your case?
author_association: CONTRIBUTOR
user: login: avishniakov
created_at: 2025-05-07T11:46:15Z
body: > @avishniakov Users will have full control to specify a command that will be run to extract those requirements from the `pyproject.toml`. This will depend on what package manager you're using, for example in the case of `uv`, you could specify `uv export --frozen` which exports the requirements from the current lock file. Does that clarify, or do you think we need additional changes in this PR to make it work for your case?

Yeah, if we can do something like `poetry export --without-hashes --format=requirements.txt` to specify - just good.
author_association: CONTRIBUTOR
user: login: schustmi
created_at: 2025-05-07T12:01:25Z
body: @avishniakov Yep, we will use exactly this command as one of the default values even. I wasn't sure however if `poetry` will reuse existing lockfiles or if it will regenerate it.
author_association: CONTRIBUTOR