Repository: pdm-project/pdm
Pull request #1898 (2023-05-08 09:19:45)

feat: option to create non-crossplatform lockfile
Signed-off-by: Frost Ming <me@frostming.com>

## Pull Request Checklist

- [ ] A news fragment is added in `news/` describing what is new.
- [x] Test cases added for changed code.

## Describe what you have changed in this PR.


codecov-commenter: ## [Codecov](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) Report
Patch coverage: **`100.00`**% and project coverage change: **`+0.02`** :tada:
> Comparison is base [(`d52c79c`)](https://app.codecov.io/gh/pdm-project/pdm/commit/d52c79cb771fcdf5871b2bd2d170bca27ede5cc5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) 85.35% compared to head [(`55f90c7`)](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) 85.37%.

> :exclamation: Current head 55f90c7 differs from pull request most recent head 16c0b3d. Consider uploading reports for the commit 16c0b3d to get more accurate results

:mega: This organization is not using Codecov’s [GitHub App Integration](https://github.com/apps/codecov). We recommend you install it so Codecov can continue to function properly for your repositories. [Learn more](https://about.codecov.io/blog/codecov-is-updating-its-github-integration/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project)

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1898      +/-   ##
==========================================
+ Coverage   85.35%   85.37%   +0.02%     
==========================================
  Files          99       99              
  Lines        9172     9185      +13     
  Branches     1983     1989       +6     
==========================================
+ Hits         7829     7842      +13     
  Misses        909      909              
  Partials      434      434              
```

| Flag | Coverage Δ | |
|---|---|---|
| unittests | `85.18% <100.00%> (+0.02%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) | Coverage Δ | |
|---|---|---|
| [src/pdm/cli/actions.py](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9jbGkvYWN0aW9ucy5weQ==) | `80.60% <100.00%> (+0.21%)` | :arrow_up: |
| [src/pdm/cli/commands/lock.py](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9jbGkvY29tbWFuZHMvbG9jay5weQ==) | `100.00% <100.00%> (ø)` | |
| [src/pdm/project/core.py](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9wcm9qZWN0L2NvcmUucHk=) | `90.45% <100.00%> (+0.04%)` | :arrow_up: |
| [src/pdm/project/lockfile.py](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9wcm9qZWN0L2xvY2tmaWxlLnB5) | `87.23% <100.00%> (+1.51%)` | :arrow_up: |


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/pdm-project/pdm/pull/1898?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project).


Patch:
diff --git a/docs/docs/usage/dependency.md b/docs/docs/usage/dependency.md
index 949c838b..9b9c8868 100644
--- a/docs/docs/usage/dependency.md
+++ b/docs/docs/usage/dependency.md
@@ -316,6 +316,16 @@ pdm lock --prod -L pdm.prod.lock
 
 Check the `metadata.groups` field in the lockfile to see which groups are included.
 
+## Cross-platform lockfile
+
+By default, the generated lockfile is **cross-platform**, which means the current platform isn't taken into account when resolving the dependencies. The result lockfile will contain wheels and dependencies for all possible platforms and Python versions.
+However, sometimes this will result in a wrong lockfile when a release doesn't contain all wheels. To avoid this, you can tell PDM
+to create a lockfile that works for **this platform** only, trimming the wheels not relevant to the current platform. This can be done by passing the `--no-cross-platform` option to `pdm lock`:
+
+```bash
+pdm lock --no-cross-platform
+```
+
 ## Show what packages are installed
 
 Similar to `pip list`, you can list all packages installed in the packages directory:
diff --git a/news/1868.feature.md b/news/1868.feature.md
new file mode 100644
index 00000000..1e8a61af
--- /dev/null
+++ b/news/1868.feature.md
@@ -0,0 +1,1 @@
+Support [Trusted Publisher](https://docs.pypi.org/trusted-publishers/using-a-publisher/).
diff --git a/news/1898.feature.md b/news/1898.feature.md
new file mode 100644
index 00000000..5a9de4a8
--- /dev/null
+++ b/news/1898.feature.md
@@ -0,0 +1,1 @@
+Add an option `--no-cross-platform` to `pdm lock` to create a non-cross-platform lockfile.
diff --git a/src/pdm/cli/actions.py b/src/pdm/cli/actions.py
index bf311229..f04a51d9 100644
--- a/src/pdm/cli/actions.py
+++ b/src/pdm/cli/actions.py
@@ -57,6 +57,7 @@ def do_lock(
     dry_run: bool = False,
     refresh: bool = False,
     groups: list[str] | None = None,
+    cross_platform: bool | None = None,
     hooks: HookManager | None = None,
 ) -> dict[str, Candidate]:
     """Performs the locking process and update lockfile."""
@@ -80,11 +81,16 @@ def do_lock(
         project.write_lockfile(lockfile, groups=groups)
         return mapping
     # TODO: multiple dependency definitions for the same package.
-    provider = project.get_provider(strategy, tracked_names)
+    if cross_platform is None:
+        cross_platform = project.lockfile.cross_platform
+    provider = project.get_provider(strategy, tracked_names, ignore_compatibility=cross_platform)
     if not requirements:
         requirements = [
             r for g, deps in project.all_dependencies.items() if groups is None or g in groups for r in deps.values()
         ]
+    if not cross_platform:
+        this_env = project.environment.marker_environment
+        requirements = [req for req in requirements if not req.marker or req.marker.evaluate(this_env)]
     resolve_max_rounds = int(project.config["strategy.resolve_max_rounds"])
     ui = project.core.ui
     with ui.logging("lock"):
@@ -120,7 +126,7 @@ def do_lock(
         else:
             data = format_lockfile(project, mapping, dependencies)
             ui.echo(f"{termui.Emoji.LOCK} Lock successful")
-            project.write_lockfile(data, write=not dry_run, groups=groups)
+            project.write_lockfile(data, write=not dry_run, groups=groups, cross_platform=cross_platform)
             hooks.try_emit("post_lock", resolution=mapping, dry_run=dry_run)
 
     return mapping
diff --git a/src/pdm/cli/commands/lock.py b/src/pdm/cli/commands/lock.py
index 4e4d2109..25b9539e 100644
--- a/src/pdm/cli/commands/lock.py
+++ b/src/pdm/cli/commands/lock.py
@@ -32,6 +32,13 @@ class Command(BaseCommand):
             action="store_true",
             help="Check if the lock file is up to date and quit",
         )
+        parser.add_argument(
+            "--no-cross-platform",
+            action="store_false",
+            default=True,
+            dest="cross_platform",
+            help="Only lock packages for the current platform",
+        )
 
     def handle(self, project: Project, options: argparse.Namespace) -> None:
         if options.check:
@@ -55,5 +62,6 @@ class Command(BaseCommand):
             project,
             refresh=options.refresh,
             groups=selection.all(),
+            cross_platform=options.cross_platform,
             hooks=HookManager(project, options.skip),
         )
diff --git a/src/pdm/cli/commands/publish/repository.py b/src/pdm/cli/commands/publish/repository.py
index 7d5bf3cb..b15d8f28 100644
--- a/src/pdm/cli/commands/publish/repository.py
+++ b/src/pdm/cli/commands/publish/repository.py
@@ -1,13 +1,15 @@
 from __future__ import annotations
 
+import os
 import pathlib
 import weakref
 from typing import Any, Iterable
-from urllib.parse import urlparse
+from urllib.parse import urlparse, urlunparse
 
 import requests
 import requests_toolbelt
 import rich.progress
+from unearth.auth import get_keyring_provider
 
 from pdm import termui
 from pdm.cli.commands.publish.package import PackageFile
@@ -15,10 +17,7 @@ from pdm.exceptions import PdmUsageError
 from pdm.project import Project
 from pdm.project.config import DEFAULT_REPOSITORIES
 
-try:
-    import keyring
-except ImportError:
-    keyring = None
+keyring = get_keyring_provider()
 
 
 class Repository:
@@ -44,6 +43,11 @@ class Repository:
         netloc = urlparse(self.url).netloc
         if username and password:
             return username, password
+        if password:
+            return "__token__", password
+        token = self._get_pypi_token_via_oidc()
+        if token is not None:
+            return "__token__", token
         if not termui.is_interactive():
             raise PdmUsageError("Username and password are required")
         username, password, save = self._prompt_for_credentials(netloc, username)
@@ -51,19 +55,53 @@ class Repository:
             self._credentials_to_save = (netloc, username, password)
         return username, password
 
+    def _get_pypi_token_via_oidc(self) -> str | None:
+        ACTIONS_ID_TOKEN_REQUEST_TOKEN = os.getenv("ACTIONS_ID_TOKEN_REQUEST_TOKEN")
+        ACTIONS_ID_TOKEN_REQUEST_URL = os.getenv("ACTIONS_ID_TOKEN_REQUEST_URL")
+        if not ACTIONS_ID_TOKEN_REQUEST_TOKEN or not ACTIONS_ID_TOKEN_REQUEST_URL:
+            return None
+        self.ui.echo("Getting PyPI token via GitHub Actions OIDC...")
+        try:
+            parsed_url = urlparse(self.url)
+            audience_url = urlunparse(parsed_url._replace(path="/_/oidc/audience"))
+            resp = self.session.get(audience_url)
+            resp.raise_for_status()
+
+            resp = self.session.get(
+                ACTIONS_ID_TOKEN_REQUEST_URL,
+                params=resp.json(),
+                headers={"Authorization": f"bearer {ACTIONS_ID_TOKEN_REQUEST_TOKEN}"},
+            )
+            resp.raise_for_status()
+            oidc_token = resp.json()["value"]
+
+            mint_token_url = urlunparse(parsed_url._replace(path="/_/oidc/github/mint-token"))
+            resp = self.session.post(mint_token_url, json={"token": oidc_token})
+            resp.raise_for_status()
+            token = resp.json()["token"]
+        except requests.RequestException:
+            self.ui.echo("Failed to get PyPI token via GitHub Actions OIDC", err=True)
+            return None
+        else:
+            if os.getenv("GITHUB_ACTIONS"):
+                # tell GitHub Actions to mask the token in any console logs
+                print(f"::add-mask::{token}")
+            return token
+
     def _prompt_for_credentials(self, service: str, username: str | None) -> tuple[str, str, bool]:
         if keyring is not None:
-            cred = keyring.get_credential(service, username)
+            cred = keyring.get_auth_info(service, username)
             if cred is not None:
-                return cred.username, cred.password, False
+                return cred[0], cred[1], False
         if username is None:
             username = termui.ask("[primary]Username")
         password = termui.ask("[primary]Password", password=True)
         return username, password, True
 
     def _save_credentials(self, service: str, username: str, password: str) -> None:
+        assert keyring is not None
         self.ui.echo("Saving credentials to keyring")
-        keyring.set_password(service, username, password)
+        keyring.save_auth_info(service, username, password)
 
     @staticmethod
     def _convert_to_list_of_tuples(data: dict[str, Any]) -> list[tuple[str, Any]]:
diff --git a/src/pdm/cli/completions/pdm.bash b/src/pdm/cli/completions/pdm.bash
index ef16090f..557ea483 100644
--- a/src/pdm/cli/completions/pdm.bash
+++ b/src/pdm/cli/completions/pdm.bash
@@ -77,7 +77,7 @@ _pdm_a919b69078acdf0a_complete()
             ;;
 
             (lock)
-            opts="--check --dev --global --group --help --lockfile --no-default --no-isolation --production --project --refresh --skip --verbose"
+            opts="--check --dev --global --group --help --lockfile --no-cross-platform --no-default --no-isolation --production --project --refresh --skip --verbose"
             ;;
 
             (plugin)
diff --git a/src/pdm/cli/completions/pdm.fish b/src/pdm/cli/completions/pdm.fish
index 6b21ae31..77873e53 100644
--- a/src/pdm/cli/completions/pdm.fish
+++ b/src/pdm/cli/completions/pdm.fish
@@ -200,6 +200,7 @@ complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l global -d 'Use the g
 complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l group -d 'Select group of optional-dependencies separated by comma or dev-dependencies (with `-d`). Can be supplied multiple times, use ":all" to include all groups under the same species.'
 complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l help -d 'Show this help message and exit.'
 complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l lockfile -d 'Specify another lockfile path. Default: pdm.lock. [env var: PDM_LOCKFILE]'
+complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l no-cross-platform -d 'Only lock packages for the current platform'
 complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l no-default -d 'Don\'t include dependencies from the default group'
 complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l no-isolation -d 'Do not isolate the build in a clean environment'
 complete -c pdm -A -n '__fish_seen_subcommand_from lock' -l production -d 'Unselect dev dependencies'
diff --git a/src/pdm/cli/completions/pdm.ps1 b/src/pdm/cli/completions/pdm.ps1
index 7d57be56..9e2f1184 100644
--- a/src/pdm/cli/completions/pdm.ps1
+++ b/src/pdm/cli/completions/pdm.ps1
@@ -315,7 +315,7 @@ function TabExpansion($line, $lastWord) {
             "lock" {
                 $completer.AddOpts(
                     @(
-                        [Option]::new(@("--global", "-g", "--no-isolation", "--refresh", "-L", "--lockfile", "--check", "--dev", "--prod", "--production", "-d", "--no-default")),
+                        [Option]::new(@("--global", "-g", "--no-isolation", "--refresh", "-L", "--lockfile", "--check", "--dev", "--prod", "--production", "-d", "--no-default", "--no-cross-platform")),
                         $skipOption,
                         $sectionOption,
                         $projectOption
diff --git a/src/pdm/cli/completions/pdm.zsh b/src/pdm/cli/completions/pdm.zsh
index a6684e5d..872827e7 100644
--- a/src/pdm/cli/completions/pdm.zsh
+++ b/src/pdm/cli/completions/pdm.zsh
@@ -237,6 +237,7 @@ _pdm() {
         {-d,--dev}"[Select dev dependencies]"
         {--prod,--production}"[Unselect dev dependencies]"
         "--no-default[Don\'t include dependencies from the default group]"
+        "--no-cross-platform[Only lock packages for the current platform]"
       )
       ;;
     self)
diff --git a/src/pdm/project/core.py b/src/pdm/project/core.py
index 3e158238..d3ae3a8f 100644
--- a/src/pdm/project/core.py
+++ b/src/pdm/project/core.py
@@ -491,22 +491,32 @@ class Project:
 
         return SpinnerReporter(spinner, requirements)
 
-    def get_lock_metadata(self, groups: Iterable[str] | None = None) -> dict[str, Any]:
+    def get_lock_metadata(
+        self, groups: Iterable[str] | None = None, cross_platform: bool | None = None
+    ) -> dict[str, Any]:
         content_hash = tomlkit.string("sha256:" + self.pyproject.content_hash("sha256"))
         content_hash.trivia.trail = "\n\n"
         if groups is None:
             groups = self.iter_groups()
+        if cross_platform is None:
+            cross_platform = self.lockfile.cross_platform
         return {
             "lock_version": self.lockfile.spec_version,
+            "cross_platform": cross_platform,
             "groups": sorted(groups, key=lambda x: (x != "default", x)),
             "content_hash": content_hash,
         }
 
     def write_lockfile(
-        self, toml_data: dict, show_message: bool = True, write: bool = True, groups: Iterable[str] | None = None
+        self,
+        toml_data: dict,
+        show_message: bool = True,
+        write: bool = True,
+        groups: Iterable[str] | None = None,
+        cross_platform: bool | None = None,
     ) -> None:
         """Write the lock file to disk."""
-        toml_data["metadata"].update(self.get_lock_metadata(groups))
+        toml_data["metadata"].update(self.get_lock_metadata(groups, cross_platform=cross_platform))
         self.lockfile.set_data(toml_data)
 
         if write:
diff --git a/src/pdm/project/lockfile.py b/src/pdm/project/lockfile.py
index 5dc7aab3..9dcb0816 100644
--- a/src/pdm/project/lockfile.py
+++ b/src/pdm/project/lockfile.py
@@ -28,6 +28,12 @@ class Lockfile(TOMLBase):
     def groups(self) -> list[str] | None:
         return self._data.get("metadata", {}).get("groups")
 
+    @property
+    def cross_platform(self) -> bool:
+        if self.empty():
+            return True
+        return self._data.get("metadata", {}).get("cross_platform", True)
+
     def compare_groups(self, groups: Iterable[str]) -> list[str]:
         if not self.groups:
             return []


Test patch:
diff --git a/tests/cli/test_lock.py b/tests/cli/test_lock.py
index 7155fe42..489e9e4f 100644
--- a/tests/cli/test_lock.py
+++ b/tests/cli/test_lock.py
@@ -1,3 +1,4 @@
+import sys
 from unittest.mock import ANY
 
 import pytest
@@ -5,12 +6,13 @@ from unearth import Link
 
 from pdm.cli import actions
 from pdm.models.requirements import parse_requirement
+from pdm.models.specifiers import PySpecSet
 
 
 def test_lock_command(project, pdm, mocker):
     m = mocker.patch.object(actions, "do_lock")
     pdm(["lock"], obj=project)
-    m.assert_called_with(project, refresh=False, groups=["default"], hooks=ANY)
+    m.assert_called_with(project, refresh=False, groups=["default"], hooks=ANY, cross_platform=True)
 
 
 @pytest.mark.usefixtures("repository")
@@ -126,3 +128,24 @@ def test_lock_self_referencing_groups(project, pdm, to_dev):
     pdm(["lock", "-G", "dev"], obj=project, strict=True)
     assert project.lockfile.groups == ["default", "dev"]
     assert "requests" in project.locked_repository.all_candidates
+
+
+@pytest.mark.usefixtures("local_finder")
+def test_lock_multiple_platform_wheels(project, pdm):
+    project.environment.python_requires = PySpecSet(">=3.7")
+    project.add_dependencies({"pdm-hello": parse_requirement("pdm-hello")})
+    pdm(["lock"], obj=project, strict=True)
+    assert project.lockfile.cross_platform
+    file_hashes = project.lockfile["metadata"]["files"]["pdm-hello 0.1.0"]
+    assert len(file_hashes) == 2
+
+
+@pytest.mark.usefixtures("local_finder")
+def test_lock_current_platform_wheels(project, pdm):
+    project.environment.python_requires = PySpecSet(">=3.7")
+    project.add_dependencies({"pdm-hello": parse_requirement("pdm-hello")})
+    pdm(["lock", "--no-cross-platform"], obj=project, strict=True)
+    assert project.lockfile.cross_platform is False
+    file_hashes = project.lockfile["metadata"]["files"]["pdm-hello 0.1.0"]
+    wheels_num = 2 if sys.platform == "win32" and not project.python.is_32bit else 1
+    assert len(file_hashes) == wheels_num