Repository: pdm-project/pdm

Issue:
 Environment variable substitution in `cmd` scripts
## Is your feature/enhancement proposal related to a problem? Please describe.

When I tried migrating a project's tasks from [Poe the Poet](https://poethepoet.natn.io), an independent task runner for Poetry, I ran into a feature that is important to me missing.

Poe can substitute environment variables in `cmd` tasks, which are like PDM `cmd` scripts. Here is an example of how I use it:

```toml
[tool.poe.env]
"PYTHON_SOURCES" = "src tests"

[tool.poe.tasks.format]
cmd = "ruff format ${PYTHON_SOURCES}"
help = "Run the Ruff formatter"

[tool.poe.tasks.lint]
cmd = "ruff check ${PYTHON_SOURCES}"
help = "Run Ruff"
```

I find this useful because it allows you to avoid repeating yourself without invoking an external shell. Why `shell` scripts aren't a sufficient alternative: the shell depends on the platform. On Windows this is still `cmd.exe`, which has an incompatible syntax for variable substitution.

## Describe the solution you'd like

I would like to be able to substitute environment variables as `{env:PYTHON_SOURCES}` or something similar consistent with existing PDM syntax (`{args}` and `{pdm}`) in `cmd` scripts.


Discussion:
Currently `"ruff check ${PYTHON_SOURCES}"` also works in PDM `cmd`, doesn't it work for you?
﻿No. Here is a demo of what doesn't work for me.

```none
> cat pyproject.toml
[project]
name = "pdm-test"
version = "0.1.0"
description = "Default template for PDM package"
authors = [
    {name = "D. Bohdan", email = "dbohdan@dbohdan.com"},
]
dependencies = []
requires-python = "==3.11.*"
readme = "README.md"
license = {text = "MIT"}


[tool.pdm]
package-type = "application"

[tool.pdm.scripts]
_.env = { PYTHON_SOURCES = "test.py" }

foo.shell = "python ${PYTHON_SOURCES}"
bar.cmd = "python ${PYTHON_SOURCES}"

> cat test.py
print("Hello")

> pdm run foo
Hello

> pdm run bar
/home/dbohdan/.local/share/pdm/venvs/pdm-test-q1q4PxnD-3.11/bin/python: can't open file '/tmp/pdm-test/${PYTHON_SOURCES}': [Errno 2] No such file or directory

> pdm --version
PDM, version 2.11.2
```
> No. Here is a demo of what doesn't work for me.

Okay, the problem is `_.env` isn't loaded into the current process, but only takes effect in the subprocess
The scripts also succeed/fail the same  way with

```toml
[tool.pdm.scripts]
foo.shell = "python ${PYTHON_SOURCES}"
foo.env = { PYTHON_SOURCES = "test.py" }

bar.cmd = "python ${PYTHON_SOURCES}"
bar.env = { PYTHON_SOURCES = "test.py" }
```


if you set env before executing you will find it runs successful 



Patch:
diff --git a/news/2488.bugfix.md b/news/2488.bugfix.md
deleted file mode 100644
index 695c78ccaf..0000000000
--- a/news/2488.bugfix.md
+++ /dev/null
@@ -1,1 +0,0 @@
-Fix a KeyError raised when resolving a URL dependency without package name given.
diff --git a/news/2495.bugfix.md b/news/2495.bugfix.md
deleted file mode 100644
index a7693b03c7..0000000000
--- a/news/2495.bugfix.md
+++ /dev/null
@@ -1,1 +0,0 @@
-`pdm update --update-eager` can hit InconsistentCandidate error when dependency is included both through default dependencies and extra.
diff --git a/news/2502.bugfix.md b/news/2502.bugfix.md
deleted file mode 100644
index e80458b772..0000000000
--- a/news/2502.bugfix.md
+++ /dev/null
@@ -1,1 +0,0 @@
-`pdm install` should not warn when overwriting its own symlinks on `install`/`update`.
diff --git a/news/2507.bugfix.md b/news/2507.bugfix.md
deleted file mode 100644
index 3c79cc4c2b..0000000000
--- a/news/2507.bugfix.md
+++ /dev/null
@@ -1,1 +0,0 @@
-Fix a bug that candidates without local version are rejected when the local version is pinned.
diff --git a/news/2510.doc.md b/news/2510.doc.md
deleted file mode 100644
index a1f2132eef..0000000000
--- a/news/2510.doc.md
+++ /dev/null
@@ -1,1 +0,0 @@
-Add maturin as a compatible build backend in the docs.
diff --git a/news/2542.bugfix.md b/news/2542.bugfix.md
new file mode 100644
index 0000000000..a8410ee94c
--- /dev/null
+++ b/news/2542.bugfix.md
@@ -0,0 +1,1 @@
+Fix the environment variable substitution for `cmd` scripts.
diff --git a/src/pdm/cli/commands/run.py b/src/pdm/cli/commands/run.py
index 722375956e..01587002af 100644
--- a/src/pdm/cli/commands/run.py
+++ b/src/pdm/cli/commands/run.py
@@ -22,7 +22,7 @@
 from pdm.exceptions import PdmUsageError
 from pdm.project import Project
 from pdm.signals import pdm_signals
-from pdm.utils import is_path_relative_to
+from pdm.utils import expand_env_vars, is_path_relative_to
 
 if TYPE_CHECKING:
     from typing import Any, Callable, Iterator, TypedDict
@@ -44,8 +44,8 @@ def exec_opts(*options: TaskOptions | None) -> dict[str, Any]:
     )
 
 
-RE_ARGS_PLACEHOLDER = re.compile(r"{args(?::(?P<default>[^}]*))?}")
-RE_PDM_PLACEHOLDER = re.compile(r"{pdm}")
+RE_ARGS_PLACEHOLDER = re.compile(r"\{args(?::(?P<default>[^}]*))?\}")
+RE_PDM_PLACEHOLDER = re.compile(r"\{pdm\}")
 
 
 def _interpolate_args(script: str, args: Sequence[str]) -> tuple[str, bool]:
@@ -63,11 +63,9 @@ def replace(m: re.Match[str]) -> str:
 def _interpolate_pdm(script: str) -> str:
     """Interpolate the `{pdm} placeholder in a string"""
     executable_path = Path(sys.executable)
+    pdm_executable = shlex.join([executable_path.as_posix(), "-m", "pdm"])
 
-    def replace(m: re.Match[str]) -> str:
-        return shlex.join([executable_path.as_posix(), "-m", "pdm"])
-
-    interpolated = RE_PDM_PLACEHOLDER.sub(replace, script)
+    interpolated = RE_PDM_PLACEHOLDER.sub(pdm_executable, script)
     return interpolated
 
 
@@ -141,7 +139,7 @@ def get_task(self, script_name: str) -> Task | None:
         return Task(kind, script_name, value, cast("TaskOptions", options))
 
     def expand_command(self, command: str) -> str:
-        expanded_command = os.path.expanduser(os.path.expandvars(command))
+        expanded_command = os.path.expanduser(command)
         if expanded_command.replace(os.sep, "/").startswith(("./", "../")):
             abspath = os.path.abspath(expanded_command)
             if not os.path.isfile(abspath):
@@ -191,16 +189,17 @@ def _run_process(
             process_env.update(env)
         if shell:
             assert isinstance(args, str)
-            expanded_args: str | Sequence[str] = os.path.expandvars(args)
+            # environment variables will be expanded by shell
+            process_cmd: str | Sequence[str] = args
         else:
             assert isinstance(args, Sequence)
-            command, *args = args
+            command, *args = (expand_env_vars(arg, env=process_env) for arg in args)
             if command.endswith(".py"):
                 args = [command, *args]
                 command = str(project.environment.interpreter.executable)
             expanded_command = self.expand_command(command)
             real_command = os.path.realpath(expanded_command)
-            expanded_args = [os.path.expandvars(arg) for arg in [expanded_command, *args]]
+            process_cmd = [expanded_command, *args]
             if (
                 project_env.is_local
                 and not site_packages
@@ -222,7 +221,7 @@ def forward_signal(signum: int, frame: FrameType | None) -> None:
 
         handle_term = signal.signal(signal.SIGTERM, forward_signal)
         handle_int = signal.signal(signal.SIGINT, forward_signal)
-        process = subprocess.Popen(expanded_args, cwd=cwd, env=process_env, shell=shell, bufsize=0)
+        process = subprocess.Popen(process_cmd, cwd=cwd, env=process_env, shell=shell, bufsize=0)
         process.wait()
         signal.signal(signal.SIGTERM, handle_term)
         signal.signal(signal.SIGINT, handle_int)
diff --git a/src/pdm/utils.py b/src/pdm/utils.py
index 4cbd901903..db9afdcd05 100644
--- a/src/pdm/utils.py
+++ b/src/pdm/utils.py
@@ -18,7 +18,7 @@
 import warnings
 from os import name as os_name
 from pathlib import Path
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, Mapping
 
 from packaging.version import Version, _cmpkey
 
@@ -221,17 +221,19 @@ def path_to_url(path: str) -> str:
     return url
 
 
-def expand_env_vars(credential: str, quote: bool = False) -> str:
+def expand_env_vars(credential: str, quote: bool = False, env: Mapping[str, str] | None = None) -> str:
     """A safe implementation of env var substitution.
     It only supports the following forms:
 
         ${ENV_VAR}
 
-    Neither $ENV_VAR and %ENV_VAR is not supported.
+    Neither $ENV_VAR and %ENV_VAR is supported.
     """
+    if env is None:
+        env = os.environ
 
     def replace_func(match: Match) -> str:
-        rv = os.getenv(match.group(1), match.group(0))
+        rv = env.get(match.group(1), match.group(0))
         return parse.quote(rv) if quote else rv
 
     return re.sub(r"\$\{(.+?)\}", replace_func, credential)


Test patch:
diff --git a/tests/cli/test_run.py b/tests/cli/test_run.py
index 2ac983fd7f..c2536822fb 100644
--- a/tests/cli/test_run.py
+++ b/tests/cli/test_run.py
@@ -325,11 +325,11 @@ def test_run_shell_script_with_pdm_placeholder(project, pdm):
 def test_run_expand_env_vars(project, pdm, capfd, monkeypatch):
     (project.root / "test_script.py").write_text("import os; print(os.getenv('FOO'))")
     project.pyproject.settings["scripts"] = {
-        "test_cmd": 'python -c "foo, bar = 0, 1;print($FOO)"',
-        "test_cmd_no_expand": "python -c 'print($FOO)'",
+        "test_cmd": 'python -c "foo, bar = 0, 1;print(${FOO})"',
+        "test_cmd_no_expand": "python -c 'print(${FOO})'",
         "test_script": "python test_script.py",
         "test_cmd_array": ["python", "test_script.py"],
-        "test_shell": {"shell": "echo $FOO"},
+        "test_shell": {"shell": "echo %FOO%" if os.name == "nt" else "echo $FOO"},
     }
     project.pyproject.write()
     capfd.readouterr()
@@ -351,6 +351,35 @@ def test_run_expand_env_vars(project, pdm, capfd, monkeypatch):
         assert capfd.readouterr()[0].strip() == "bar"
 
 
+def test_run_expand_env_vars_from_config(project, pdm, capfd):
+    (project.root / "test_script.py").write_text("import os; print(os.getenv('FOO'))")
+    project.pyproject.settings["scripts"] = {
+        "test_cmd": 'python -c "foo, bar = 0, 1;print(${FOO})"',
+        "test_cmd_no_expand": "python -c 'print(${FOO})'",
+        "test_script": "python test_script.py",
+        "test_cmd_array": ["python", "test_script.py"],
+        "test_shell": {"shell": "echo %FOO%" if os.name == "nt" else "echo $FOO"},
+        "_": {"env": {"FOO": "bar"}},
+    }
+    project.pyproject.write()
+    capfd.readouterr()
+    with cd(project.root):
+        pdm(["run", "test_cmd"], obj=project)
+        assert capfd.readouterr()[0].strip() == "1"
+
+        result = pdm(["run", "test_cmd_no_expand"], obj=project)
+        assert result.exit_code == 1
+
+        pdm(["run", "test_script"], obj=project)
+        assert capfd.readouterr()[0].strip() == "bar"
+
+        pdm(["run", "test_cmd_array"], obj=project)
+        assert capfd.readouterr()[0].strip() == "bar"
+
+        pdm(["run", "test_shell"], obj=project)
+        assert capfd.readouterr()[0].strip() == "bar"
+
+
 def test_run_script_with_env_defined(project, pdm, capfd):
     (project.root / "test_script.py").write_text("import os; print(os.getenv('FOO'))")
     project.pyproject.settings["scripts"] = {"test_script": {"cmd": "python test_script.py", "env": {"FOO": "bar"}}}