Repository: snakemake/snakemake

Issue:
multiext support for named output
**Is your feature request related to a problem? Please describe.**

Right now it is not possible to cache a rule's output when there are several named output files. With several output files we can use `multiext`, but it does not allow for named output.

**Describe the solution you'd like**

I was thinking that it would be nice to have a way to have multiple named output files with `multiext`, for example:

```
output:
    multiext("path/to/output_file.", out1 = "ext1",  out2 = "ext2")
```

or:
```
output:
    out1, out2 = multiext("path/to/output_file.", "ext1", "ext2")
```

Discussion:
@notestaff: I'd like to second the request for this feature.@johanneskoester: Yes, this should be possible. I like the syntax of the first suggestion more. This would have to be implemented in snakemake.rules.Rule._set_inoutput_item and snakemake.io.multiext.

Solution patch:
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
index a9c5371f..bf305ec4 100644
--- a/.github/workflows/docs.yml
+++ b/.github/workflows/docs.yml
@@ -18,20 +18,17 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      - uses: actions/setup-python@v4
+      - name: Install Pixi
+        uses: prefix-dev/setup-pixi@v0.8.3
         with:
-          python-version: '3.x'
-
-      - name: Setup doc dependencies
-        run: |
-          python -m pip install --upgrade pip
-          python -m pip install -r docs/requirements.txt
-          python -m pip install .
-
-      - name: Build docs
+          environments: "docs"
+          pixi-version: v0.42.1
+          cache: false # no pixi.lock
+          locked: false
+      
+      - name: Build Docs
         run: |
-          cd docs
-          make html SPHINXOPTS="-W --keep-going -n"
+          pixi run --environment docs build-docs
 
       - name: Upload documentation
         uses: actions/upload-artifact@v4
@@ -45,20 +42,17 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      - uses: actions/setup-python@v4
+      - name: Install Pixi
+        uses: prefix-dev/setup-pixi@v0.8.3
         with:
-          python-version: '3.x'
-
-      - name: Setup doc dependencies
-        run: |
-          python -m pip install --upgrade pip
-          python -m pip install -r apidocs/requirements.txt
-          python -m pip install .
+          environments: "docs"
+          pixi-version: v0.42.1
+          cache: false # no pixi.lock
+          locked: false
 
-      - name: Build docs
+      - name: Build API Docs
         run: |
-          cd apidocs
-          make html SPHINXOPTS="--keep-going -n"
+          pixi run --environment docs build-apidocs
 
       - name: Upload documentation
         uses: actions/upload-artifact@v4
diff --git a/docs/project_info/contributing.rst b/docs/project_info/contributing.rst
index 031b04b4..bcbb8c8a 100644
--- a/docs/project_info/contributing.rst
+++ b/docs/project_info/contributing.rst
@@ -85,6 +85,43 @@ If you want to create more pull requests, first run :code:`git checkout main` an
 
 Feel free to ask questions about this if you want to contribute to Snakemake :)
 
+.. _pixi-getting_started:
+
+Development with ``pixi``
+=========================
+
+`pixi <https://pixi.sh/>`_ is a tool that is designed to help you manage 
+your development environment.It acts as a drop-in replacement for
+`conda <https://docs.conda.io/en/latest/>`_, offering:
+
+- **Easy installation & Updating**: `install pixi <https://pixi.sh/latest/#installation>`_ 
+  through many methods and for different shells.
+  Updating ``pixi`` is as simple as ``pixi self-update``
+
+- **Ease of Use**: A streamlined CLI (similar to Yarn or Cargo) for quick
+  environment creation and management. Try commands like ``pixi init``,
+  ``pixi add <package>``, or ``pixi run`` to see how intuitive it is.
+
+- **Multiple Environments**: Define and switch between multiple sets of
+  dependencies under one project.
+  Pixi uses a ``feature`` system to compose an ``environment``.
+  Think of ``features`` as a way to group dependencies and settings together.
+  and an environment is a collection of features.
+  This allows easy management of different environments for multiple use.
+  See the ``pyproject.toml`` file for an example of how the ``test`` feature
+  is used to define the ``dev``, ``py311`` and ``py312`` environments.
+
+- **Cross-Platform Solving**: Target Linux, macOS, and Windows from a single
+  config. Pixi resolves the correct packages for each platform and captures
+  them in a lockfile for reproducible setups—no Docker needed.
+
+- **Speed & Conda Compatibility**: Written in Rust, Pixi downloads and solves
+  packages in parallel for faster operations. It uses the Conda ecosystem
+  and channels, so you get the same packages with improved performance. In
+  many cases, Pixi can outperform both Conda and Mamba.
+
+To learn more, visit the `Pixi docs <https://pixi.sh>`__ or check out helpful
+guides on `prefix.dev <https://prefix.dev/>`__. 
 
 Testing Guidelines
 ==================
@@ -103,57 +140,74 @@ Setup to run the test suite locally
 
 Unit tests and regression tests are written to be run by `pytest <https://docs.pytest.org/en/stable/>`_.
 
-Assuming you are on a Linux system, and have a working Conda installation, the standard way to run the tests is explained in the following.
-With sufficient experience, it is however possible to run the tests in different setups, given the expected dependencies are installed.
 
-1. Ensure your Conda version is at least 24.7.1 (Snakemake's minimum required Conda version).
-   Check the output of ``conda --version`` and update conda if necessary.
-2. Activate strict channel priorities (this is always a good idea when using Conda, see `here <https://conda-forge.org/docs/user/tipsandtricks/#using-multiple-channels>`__ for the rationale), by running ``conda config --set channel_priority strict``.
-3. After checking out the branch you want to test, run these commands:
+.. _pixi-test-guide:
+
+Testing Guide using ``pixi``
+=============================
+
+**Prerequisites**: Make sure you have ``pixi`` installed: See :ref:`pixi-getting_started`.
+
+**Activate your environment**:
+--------------------------------
+
+There are a few environments you can use to run the tests.
+
+The ``dev`` environment is most useful for overall development.
+This environment will also install the ``docs`` and ``style`` features
+which will allow you to also build documentation and run ``black``.
+
+.. code-block:: console
+
+    $ pixi shell -e dev
+
+The ``py311`` and ``py312`` environments are what are used in the 
+CI tests which isolate the Python version and the `test` dependencies.
+Use this if you want to test your code against the same environment
+as any failing CI tests.
+
+.. code-block:: console
 
-   .. code-block:: console
+    $ pixi shell -e py311
 
-       $ conda env create -f test-environment.yml -n snakemake-testing
-       $ conda activate snakemake-testing
-       $ pip install -e .
+**Run a comprehensive, simple, or single test**:
+The test suite defines two types of tests via ``pixi tasks`` that you can run:
 
-   You may want to set a specific Python version by editing the constraint in ``test-environment.yml`` before doing this.
-   Use of the ``-e``/``--editable`` option to ``pip`` will make your development version of Snakemake the one called when running Snakemake and all the unit tests. You only need to run the ``pip`` command once, not after each time you make code changes.
+**test-all**: This task runs the comprehensive test suite, which includes 
+*most* of the tests in the ``tests/`` directory.
 
-4. From the base Snakemake folder you may now run any specific test:
+.. code-block:: console
 
-   .. code-block:: console
+    $ pixi run test-all
 
-      $ pytest tests/tests.py::test_log_input
+**test-simple**: This task runs the main tests located in ``tests/tests.py``.
 
-   You can also use the ``-k`` flag to select tests by substring match, rather than by the full name, and the ``--co`` option to preview which tests will be run. Try, for example:
+.. code-block:: console
 
-   .. code-block:: console
+    $ pixi run test-simple
 
-      $ pytest --co tests/tests.py -k test_modules_all
+**Single test**: You can also run a single test by using ``pytest`` 
+directly with the test file and the test name.
 
-Running the full test suite
----------------------------
+.. code-block:: console
 
-If you simply run ``pytest`` in the top level directory it will scan for and attempt to run every test in the directory, but you will almost certainly get errors as not all tests are working and current.
+    $ pixi run pytest tests/tests.py::test_log_input
 
-The core test suite is run as a `GitHub action <https://docs.github.com/en/actions/about-github-actions/understanding-github-actions>`_ by the code under ``.github/workflows/main.yml``, so you should look in this file for the list of tests actually expected to pass in a regular test environment.
+.. tip::
+    This test suite is quite long, and can be run in parts similar to the 
+    CI/CD tests which run it in 1/10 parts.
 
-At the time of writing this text, the suite is:
+    To do so, you can use the ``--splits`` and ``--group`` flags to run
+    a subset of the tests. For example, to run the first group of tests
+    in a 10 part split:
 
-.. code-block::
+    .. code-block:: console
 
-   tests/tests.py
-   tests/tests_using_conda.py
-   tests/test_expand.py
-   tests/test_io.py
-   tests/test_schema.py
-   tests/test_linting.py
-   tests/test_executor_test_suite.py
-   tests/test_api.py
-   tests/test_internals.py
+        $ pixi run test-simple \
+            --splits 10 \
+            --group 1 \
+            --splitting-algorithm=least_duration
 
-Other tests in the directory may or may not work.
 
 Warnings and oddities
 ---------------------
@@ -168,9 +222,6 @@ This is likely due to system security profiles that conda, being a non-root appl
 The Debian/Ubuntu ``singularity-container`` DEB package, which must be installed by the system administrator, does work.
 The equivalent RPM package should also work on RedHat-type systems.
 
-Tests in ``tests/test_api.py`` require a working ``git``.
-This is not included in ``test-environment.yml`` as it's assumed you must have GIT installed to be working on the source code, but installing git into the conda environment should work if need be.
-
 Depending on how the Snakemake code was downloaded and installed in the test environment, Snakemake may not be able to determine its own version and may think that it is version 0.
 The existing unit tests should all cope with this, and in general you should avoid writing tests that rely on explicit version checks.
 
@@ -217,46 +268,39 @@ For the documentation, please adhere to the following guidelines:
 Documentation Setup
 -------------------
 
-For building the documentation, you have to install the Sphinx.
-If you have already installed Conda, all you need to do is to create a
-Snakemake development environment via
+To get started, make sure you have ``pixi`` installed: 
+See :ref:`pixi-getting_started`.
+We use ``pixi`` to manage the docs environment and tasks to streamline
+the developer experience.
 
 .. code-block:: console
 
-    $ git clone git@github.com:snakemake/snakemake.git
-    $ cd snakemake
-    $ conda env create -f doc-environment.yml -n snakemake_docs
-    $ conda activate snakemake_docs
-
-You will also need to install your development version of Snakemake for the docs to be built correctly
-
-.. code-block:: console
-
-    $ pip install -e .
-
-Then, the docs can be built with
-
-.. code-block:: console
+    $ ➜ pixi task list --environment docs
+    Tasks that can run on this machine:
+    -----------------------------------
+    build-apidocs, build-docs, docs
 
-    $ cd docs
-    $ make html
-    $ make clean && make html  # force rebuild
+    - build-apidocs   Build the API documentation in the apidocs/ directory
+    - build-docs      Build the documentation in the docs/ directory
+    - docs            Serve the documentation on http://localhost:8000 with live reload
 
-Alternatively, you can use virtualenv.
-The following assumes you have a working Python 3 setup.
+**Test if the docs build**:
+To only build the documentation, you can use the ``build-docs`` task.
 
 .. code-block:: console
 
-    $ git clone git@github.org:snakemake/snakemake.git
-    $ cd snakemake/docs
-    $ virtualenv -p python3 .venv
-    $ source .venv/bin/activate
-    $ pip install --upgrade -r requirements.txt
+    $ pixi run build-docs
 
-Afterwards, the docs can be built with
+**Live server with auto-reload**:
+To serve the documentation on a local server with live reload, 
+use the ``docs`` task.
 
 .. code-block:: console
 
-    $ source .venv/bin/activate
-    $ make html  # rebuild for changed files only
-    $ make clean && make html  # force rebuild
\ No newline at end of file
+    $ pixi run docs
+    [sphinx-autobuild] Starting initial build
+    [sphinx-autobuild] > python -m sphinx build docs/ docs/_build/html
+    ...
+    The HTML pages are in docs/_build/html.
+    [sphinx-autobuild] Serving on http://0.0.0.0:8000
+    [sphinx-autobuild] Waiting to detect changes...
\ No newline at end of file
diff --git a/docs/snakefiles/rules.rst b/docs/snakefiles/rules.rst
index 4afbe347..7398051f 100644
--- a/docs/snakefiles/rules.rst
+++ b/docs/snakefiles/rules.rst
@@ -308,6 +308,27 @@ The multiext function
 
 The effect is the same as if you would write ``expand("some/plot{ext}", ext=[".pdf", ".svg", ".png"])``, however, using a simpler syntax.
 Moreover, defining output with ``multiext`` is the only way to use :ref:`between workflow caching <caching>` for rules with multiple output files.
+It's also possible to get named input/output files in the following way:
+
+.. code-block:: python
+
+    rule plot:
+        input:
+            ...
+        output:
+            multiext("some/plot", out1=".pdf", out2=".svg")
+            "some_other_output"
+            named_output="another_output"
+        shell:
+            """
+            somecommand > {output.out1}
+            othercommand > {output.out2}
+            anothercommand > {output[2]}
+            finalcommand > {output.named_output}
+            """
+
+Do note that all the multiext extensions should be named, or all of them should be unnamed (not both).
+Additionally, if additional input/output statements are given, multiext should be treated as positional arguments (before other named input/output files).
 
 .. _snakefiles-semantic-helpers:
 
diff --git a/pyproject.toml b/pyproject.toml
index 4d38cbf3..a17e21c2 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -225,6 +225,10 @@ recommonmark = ">=0.7.1,<0.8"
 commonmark = ">=0.9.1,<0.10"
 myst-parser = ">=3.0.1,<5"
 sphinxcontrib-napoleon = ">=0.7,<0.8"
+sphinx-autobuild = ">=2024.10.3,<2025"
+sphinx-tabs = ">=3.4.1,<4"
+sphinx-autodoc-typehints = ">=2.0.1,<3"
+
 
 [tool.pixi.feature.docs.tasks.build-docs]
 description = "Build the documentation in the docs/ directory"
@@ -232,9 +236,13 @@ cwd = "docs"
 cmd = 'make html SPHINXOPTS="-W --keep-going -n"'
 
 [tool.pixi.feature.docs.tasks.docs]
-description = "Serve the documentation on http://localhost:8000"
-cmd = "python -m http.server 8000 --directory docs/_build/html"
-depends-on = ["build-docs"]
+description = "Serve the documentation on http://localhost:8000 with live reload"
+cmd = "sphinx-autobuild docs/ docs/_build/html --host 0.0.0.0 --port 8000 --watch docs"
+
+[tool.pixi.feature.docs.tasks.build-apidocs]
+description = "Build the API documentation in the apidocs/ directory"
+cwd = "apidocs"
+cmd = 'make html SPHINXOPTS="--keep-going -n"'
 
 # Style
 [tool.pixi.feature.style.dependencies]
diff --git a/src/snakemake/cli.py b/src/snakemake/cli.py
index 8a285f53..ae71e709 100644
--- a/src/snakemake/cli.py
+++ b/src/snakemake/cli.py
@@ -2093,7 +2093,7 @@ def args_to_api(args, parser):
                     elif args.archive:
                         dag_api.archive(args.archive)
                     elif args.delete_all_output:
-                        dag_api.delete_output()
+                        dag_api.delete_output(dryrun=args.dryrun)
                     elif args.delete_temp_output:
                         dag_api.delete_output(only_temp=True, dryrun=args.dryrun)
                     else:
diff --git a/src/snakemake/io.py b/src/snakemake/io.py
index 358f8bde..d81b2760 100755
--- a/src/snakemake/io.py
+++ b/src/snakemake/io.py
@@ -26,6 +26,7 @@ from typing import (
     Any,
     Callable,
     Dict,
+    Iterable,
     List,
     Optional,
     Set,
@@ -389,7 +390,8 @@ class _IOFile(str, AnnotatedStringInterface):
 
     @property
     def multiext_prefix(self):
-        return get_flag_value(self._file, "multiext")
+        multiext_value = get_flag_value(self._file, "multiext")
+        return multiext_value.prefix if multiext_value is not None else None
 
     @property
     def should_keep_local(self):
@@ -1436,13 +1438,52 @@ def expand(*args, **wildcard_values):
         return do_expand(wildcard_values)
 
 
-def multiext(prefix, *extensions):
+@dataclass
+class MultiextValue:
+    prefix: str
+    name: Optional[str] = None
+
+
+def multiext(prefix, *extensions, **named_extensions):
     """Expand a given prefix with multiple extensions (e.g. .txt, .csv, _peaks.bed, ...)."""
-    if any((r"/" in ext or r"\\" in ext) for ext in extensions):
+    if any(
+        (r"/" in ext or r"\\" in ext)
+        for ext in chain(extensions, named_extensions.values())
+    ):
         raise WorkflowError(
             r"Extensions for multiext may not contain path delimiters (/,\)."
         )
-    return [flag(prefix + ext, "multiext", flag_value=prefix) for ext in extensions]
+    # Ensure either all extensions are named or all are positional
+    if not (
+        (extensions and not named_extensions) or (not extensions and named_extensions)
+    ):
+        raise WorkflowError(
+            "multiext should be given with all named extensions or all not-named extensions, not a mix."
+        )
+    if extensions:
+        return [
+            flag(prefix + ext, "multiext", flag_value=MultiextValue(prefix=prefix))
+            for ext in extensions
+        ]
+    else:
+        return [
+            flag(
+                prefix + ext,
+                "multiext",
+                flag_value=MultiextValue(name=name, prefix=prefix),
+            )
+            for name, ext in named_extensions.items()
+        ]
+
+
+def is_multiext_items(
+    items: Union[str, _IOFile, Iterable[str], Iterable[_IOFile]],
+) -> bool:
+    return (
+        isinstance(items, collections.abc.Iterable)
+        and not isinstance(items, str)
+        and all(is_flagged(subitem, "multiext") for subitem in items)
+    )
 
 
 def limit(pattern: Union[str, AnnotatedString], **wildcards):
diff --git a/src/snakemake/jobs.py b/src/snakemake/jobs.py
index eb96179f..d5832b27 100644
--- a/src/snakemake/jobs.py
+++ b/src/snakemake/jobs.py
@@ -56,6 +56,7 @@ from snakemake.exceptions import (
 
 from snakemake.logging import logger
 from snakemake.common import (
+    get_function_params,
     is_local_file,
     get_uuid,
     IO_PROP_LIMIT,
@@ -482,6 +483,14 @@ class Job(AbstractJob, SingleJobExecutorInterface, JobReportInterface):
         self._resources = None
         self._attempt = attempt
 
+    def _get_resources_to_skip(self):
+        """Return a set of resource names that are callable and depend on input files."""
+        return {
+            name
+            for name, val in self.rule.resources.items()
+            if is_callable(val) and "input" in get_function_params(val)
+        }
+
     @property
     def resources(self):
         if self._resources is None:
@@ -492,12 +501,8 @@ class Job(AbstractJob, SingleJobExecutorInterface, JobReportInterface):
                 skip_evaluation = {"tmpdir"}
             if not self._params_and_resources_resetted:
                 # initial evaluation, input files of job are probably not yet present.
-                # Therefore skip all functions
-                skip_evaluation.update(
-                    name
-                    for name, val in self.rule.resources.items()
-                    if is_callable(val)
-                )
+                # Therefore skip all functions that depend on input files.
+                skip_evaluation.update(self._get_resources_to_skip())
             self._resources = self.rule.expand_resources(
                 self.wildcards_dict,
                 self.input,
diff --git a/src/snakemake/path_modifier.py b/src/snakemake/path_modifier.py
index 40d79b62..d543d393 100644
--- a/src/snakemake/path_modifier.py
+++ b/src/snakemake/path_modifier.py
@@ -7,7 +7,14 @@ import os
 
 from snakemake.common.prefix_lookup import PrefixLookup
 from snakemake.exceptions import WorkflowError
-from snakemake.io import is_callable, is_flagged, AnnotatedString, flag, get_flag_value
+from snakemake.io import (
+    is_callable,
+    is_flagged,
+    AnnotatedString,
+    flag,
+    get_flag_value,
+    MultiextValue,
+)
 from snakemake.logging import logger
 
 PATH_MODIFIER_FLAG = "path_modified"
@@ -52,8 +59,9 @@ class PathModifier:
                 modified_path = AnnotatedString(modified_path)
             modified_path.flags.update(path.flags)
             if is_flagged(modified_path, "multiext"):
-                modified_path.flags["multiext"] = self.apply_default_storage(
-                    self.replace_prefix(modified_path.flags["multiext"], property)
+                multiext_value = modified_path.flags["multiext"]
+                multiext_value.prefix = self.apply_default_storage(
+                    self.replace_prefix(multiext_value.prefix, property)
                 )
         # Flag the path as modified and return.
         modified_path = flag(modified_path, PATH_MODIFIER_FLAG)
diff --git a/src/snakemake/rules.py b/src/snakemake/rules.py
index 2b74c2ba..949b2ab9 100644
--- a/src/snakemake/rules.py
+++ b/src/snakemake/rules.py
@@ -27,6 +27,7 @@ from snakemake.io import (
     AnnotatedString,
     contains_wildcard,
     contains_wildcard_constraints,
+    is_multiext_items,
     update_wildcard_constraints,
     flag,
     get_flag_value,
@@ -276,9 +277,20 @@ class Rule(RuleInterface):
         consider_ancient = self.workflow.workflow_settings.consider_ancient.get(
             self.name, frozenset()
         )
-
         for i, item in enumerate(input):
-            self._set_inoutput_item(item, mark_ancient=i in consider_ancient)
+            if is_multiext_items(item):
+                for ifile in item:
+                    self._set_inoutput_item(
+                        ifile,
+                        name=get_flag_value(ifile, "multiext").name,
+                        mark_ancient=i in consider_ancient,
+                    )
+            else:
+                self._set_inoutput_item(
+                    item,
+                    mark_ancient=i in consider_ancient,
+                )
+
         for name, item in kwinput.items():
             self._set_inoutput_item(
                 item, name=name, mark_ancient=name in consider_ancient
@@ -334,7 +346,17 @@ class Rule(RuleInterface):
         output -- the list of output files
         """
         for item in output:
-            self._set_inoutput_item(item, output=True)
+            # Named multiext have their name set under the flag (MultiextValue), if the first one is named, all of them are named.
+            # Any of the output files in item can be multiext, so we do need to check all of them.
+            if is_multiext_items(item):
+                for ofile in item:
+                    self._set_inoutput_item(
+                        ofile,
+                        name=get_flag_value(ofile, "multiext").name,
+                        output=True,
+                    )
+            else:
+                self._set_inoutput_item(item, output=True)
         for name, item in kwoutput.items():
             self._set_inoutput_item(item, output=True, name=name)
 


Test patch:
diff --git a/tests/tests.py b/tests/tests.py
index d8409fb0..2d36055a 100644
--- a/tests/tests.py
+++ b/tests/tests.py
@@ -44,6 +44,11 @@ def test_delete_all_output():
     run(dpath("test_delete_all_output"))
 
 
+@skip_on_windows
+def test_github_issue_3265_respect_dryrun_delete_all():
+    run(dpath("test_github_issue_3265_respect_dryrun_delete_all"))
+
+
 def test_github_issue_14():
     """Add cleanup_scripts argument to allow the user to keep scripts"""
     # Return temporary directory for inspection - we should keep scripts here
@@ -1351,6 +1356,10 @@ def test_multiext():
     run(dpath("test_multiext"))
 
 
+def test_multiext_named():
+    run(dpath("test_multiext_named"))
+
+
 def test_core_dependent_threads():
     run(dpath("test_core_dependent_threads"))
 
@@ -2057,3 +2066,13 @@ def test_failed_intermediate():
 @skip_on_windows  # OS agnostic
 def test_issue3338():
     run(dpath("test_issue3338"), targets=["all"])
+
+
+def test_github_issue_3374():
+    run(dpath("test_github_issue3271"), check_results=False)
+    run(
+        dpath("test_github_issue3271"),
+        snakefile="Snakefile_should_fail",
+        shouldfail=True,
+        check_results=False,
+    )