Repository: pdm-project/pdm
Pull request #1481 (2022-10-31 20:21:30)

Do not add local package to lockfile
Replaces #1470 

## Pull Request Check List

- [x] A news fragment is added in `news/` describing what is new.
- [x] Test cases added for changed code.

## Describe what you have changed in this PR.

If you have a pyproject where you depend on yourself using the pattern described in https://github.com/pdm-project/pdm/issues/1441, with a config such as this:

```
[project]
name = "pkg"
version = "0.2.0"
dependencies = []

requires-python = ">=3.8"
license = {text = "MIT"}

[project.optional-dependencies]
foobar = ["pyyaml"]
ci = ["pkg[foobar]"]
```

The project will appear multiple times in the lockfile. Changing the version or name of the local package will desync from the lockfile; leading to `pdm sync` failing. This PR removes the local package from the lockfile, and instead adds a generator for it when running from a `LockedRepository`.

This works and all tests pass. I did however notice that when using this pattern the local package gets installed twice; but that happens on main as well:

```
$ pdm --version
PDM, version 2.2.0
$ pdm install -G ci
Synchronizing working set with lock file: 2 to add, 0 to update, 0 to remove

  ✔ Install pkg 0.2.0 successful
  ✔ Install pyyaml 6.0 successful
Installing the project as an editable package...
  ✔ Install pkg 0.2.0 successful

🎉 All complete!
```

Not sure if there's a good opportunity to fix that here as well.

frostming: > I did however notice that when using this pattern the local package gets installed twice; but that happens on main as well:

It can be fixed by dropping the self-package from the resolution result. Then no need to skip it in `format_lockfile` anymore.

Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 07514793..0706b6d4 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -2,12 +2,12 @@ ci:
   autoupdate_schedule: monthly
 repos:
   - repo: https://github.com/psf/black
-    rev: 22.8.0
+    rev: 22.10.0
     hooks:
       - id: black
 
   - repo: https://github.com/codespell-project/codespell
-    rev: v2.2.1
+    rev: v2.2.2
     hooks:
       - id: codespell  # See setup.cfg for args
 
@@ -24,7 +24,7 @@ repos:
       - id: isort
 
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v0.981
+    rev: v0.982
     hooks:
       - id: mypy
         args: [src]
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8f0578cc..827926c0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,15 @@
+Release v2.2.1 (2022-11-03)
+---------------------------
+
+### Features & Improvements
+
+- Make `sitecustomize.py` respect the `PDM_PROJECT_MAX_DEPTH` environment variable [#1471](https://github.com/pdm-project/pdm/issues/1471)
+
+### Bug Fixes
+
+- Fix the comparison of `python_version` in the environment marker. When the version contains only one digit, the result was incorrect. [#1484](https://github.com/pdm-project/pdm/issues/1484)
+
+
 Release v2.2.0 (2022-10-31)
 ---------------------------
 
diff --git a/news/1481.bugfix.md b/news/1481.bugfix.md
new file mode 100644
index 00000000..1204482b
--- /dev/null
+++ b/news/1481.bugfix.md
@@ -0,0 +1,1 @@
+The local package is now treated specially during installation and locking. This means it will no longer be included in the lockfile, and should never be installed twice even when using nested extras. This will ensure the lockdown stays relevant when the version changes.
diff --git a/src/pdm/builders/base.py b/src/pdm/builders/base.py
index 85d7ad6b..aaea88e8 100644
--- a/src/pdm/builders/base.py
+++ b/src/pdm/builders/base.py
@@ -206,9 +206,6 @@ class EnvBuilder:
             overlay=self.get_overlay_env(os.path.normcase(self.src_dir).rstrip("\\/")),
         )
 
-        if build_system["build-backend"].startswith("setuptools"):
-            self.ensure_setup_py()
-
     @property
     def _env_vars(self) -> dict[str, str]:
         paths = self._prefix.bin_dirs
@@ -307,16 +304,3 @@ class EnvBuilder:
         return the absolute path of the built result.
         """
         raise NotImplementedError("Should be implemented in subclass")
-
-    def ensure_setup_py(self) -> str:
-        from pdm.pep517.base import Builder
-        from pdm.project.metadata import MutableMetadata
-
-        builder = Builder(self.src_dir)
-        project_file = os.path.join(self.src_dir, "pyproject.toml")
-        if os.path.exists(project_file):
-            try:
-                builder._meta = MutableMetadata.from_file(project_file)
-            except ValueError:
-                builder._meta = None
-        return builder.ensure_setup_py().as_posix()
diff --git a/src/pdm/installers/core.py b/src/pdm/installers/core.py
index 159aa4c0..deb8eb5b 100644
--- a/src/pdm/installers/core.py
+++ b/src/pdm/installers/core.py
@@ -22,7 +22,10 @@ def install_requirements(
     resolver = project.core.resolver_class(provider, reporter)
     resolve_max_rounds = int(project.config["strategy.resolve_max_rounds"])
     resolved, _ = resolve(
-        resolver, reqs, environment.python_requires, max_rounds=resolve_max_rounds
+        resolver,
+        reqs,
+        environment.python_requires,
+        max_rounds=resolve_max_rounds,
     )
     manager = InstallManager(environment, use_install_cache=use_install_cache)
     working_set = environment.get_working_set()
diff --git a/src/pdm/models/markers.py b/src/pdm/models/markers.py
index 723f799e..d2ec9c64 100644
--- a/src/pdm/models/markers.py
+++ b/src/pdm/models/markers.py
@@ -136,6 +136,8 @@ def _build_pyspec_from_marker(markers: List[Any]) -> PySpecSet:
             if key == "python_version":
                 if op == ">":
                     int_versions = [int(ver) for ver in version.split(".")]
+                    if len(int_versions) < 2:
+                        int_versions.append(0)
                     int_versions[-1] += 1
                     version = ".".join(str(v) for v in int_versions)
                     op = ">="
diff --git a/src/pdm/models/repositories.py b/src/pdm/models/repositories.py
index f04d4b83..0ec80f80 100644
--- a/src/pdm/models/repositories.py
+++ b/src/pdm/models/repositories.py
@@ -436,8 +436,33 @@ class LockedRepository(BaseRepository):
     def _get_dependencies_from_lockfile(self, candidate: Candidate) -> CandidateInfo:
         return self.candidate_info[self._identify_candidate(candidate)]
 
+    def _get_dependency_from_local_package(self, candidate: Candidate) -> CandidateInfo:
+        """Adds the local package as a candidate only if the candidate
+        name is the same as the local package."""
+        if candidate.name != self.environment.project.name:
+            raise CandidateInfoNotFound(candidate) from None
+
+        reqs = self.environment.project.meta.dependencies
+        if candidate.req.extras is not None:
+            reqs = sum(
+                (
+                    self.environment.project.meta.optional_dependencies[g]
+                    for g in candidate.req.extras
+                ),
+                [],
+            )
+
+        return (
+            reqs,
+            str(self.environment.python_requires),
+            self.environment.project.meta.description,
+        )
+
     def dependency_generators(self) -> Iterable[Callable[[Candidate], CandidateInfo]]:
-        return (self._get_dependencies_from_lockfile,)
+        return (
+            self._get_dependency_from_local_package,
+            self._get_dependencies_from_lockfile,
+        )
 
     def find_candidates(
         self,
diff --git a/src/pdm/pep582/sitecustomize.py b/src/pdm/pep582/sitecustomize.py
index 2b083503..e7e36fdb 100644
--- a/src/pdm/pep582/sitecustomize.py
+++ b/src/pdm/pep582/sitecustomize.py
@@ -3,13 +3,13 @@ import site
 import sys
 
 
-def get_pypackages_path(maxdepth=5):
+def get_pypackages_path():
     def find_pypackage(path, version):
         if not os.path.exists(path):
             return None
 
         packages_name = f"__pypackages__/{version}/lib"
-        for _ in range(maxdepth):
+        for _ in range(int(os.getenv("PDM_PROJECT_MAX_DEPTH", "5"))):
             if os.path.exists(os.path.join(path, packages_name)):
                 return os.path.join(path, packages_name)
             if os.path.dirname(path) == path:
diff --git a/src/pdm/resolver/core.py b/src/pdm/resolver/core.py
index 4fe85333..d6a85741 100644
--- a/src/pdm/resolver/core.py
+++ b/src/pdm/resolver/core.py
@@ -3,8 +3,11 @@ from __future__ import annotations
 from typing import TYPE_CHECKING, Dict, cast
 
 from pdm.models.candidates import Candidate
+from pdm.models.repositories import BaseRepository
+from pdm.models.requirements import strip_extras
 from pdm.resolver.providers import BaseProvider
 from pdm.resolver.python import PythonRequirement
+from pdm.utils import normalize_name
 
 if TYPE_CHECKING:
     from resolvelib.resolvers import Resolver
@@ -27,13 +30,22 @@ def resolve(
     """
     requirements.append(PythonRequirement.from_pyspec_set(requires_python))
     provider = cast(BaseProvider, resolver.provider)
+    repository = cast(BaseRepository, provider.repository)
+
     result = resolver.resolve(requirements, max_rounds)
 
     mapping = cast(Dict[str, Candidate], result.mapping)
     mapping.pop("python", None)
+
+    local_name = (
+        normalize_name(repository.environment.project.meta.name)
+        if repository.environment.project.name
+        else None
+    )
     for key, candidate in list(result.mapping.items()):
         if key is None:
             continue
+
         # For source distribution whose name can only be determined after it is built,
         # the key in the resolution map should be updated.
         if key.startswith(":empty:"):
@@ -41,4 +53,7 @@ def resolve(
             mapping[new_key] = mapping.pop(key)
             key = new_key
 
+        if strip_extras(key)[0] == local_name:
+            del mapping[key]
+
     return mapping, provider.fetched_dependencies


Test patch:
diff --git a/tests/models/test_marker.py b/tests/models/test_marker.py
index 9285fd89..7250f12f 100644
--- a/tests/models/test_marker.py
+++ b/tests/models/test_marker.py
@@ -1,6 +1,6 @@
 import pytest
 
-from pdm.models.markers import split_marker_extras
+from pdm.models.markers import Marker, split_marker_extras
 
 
 @pytest.mark.parametrize(
@@ -31,3 +31,29 @@ from pdm.models.markers import split_marker_extras
 def test_split_marker_extras(original, extras, rest):
     result = split_marker_extras(original)
     assert result == (extras, rest)
+
+
+@pytest.mark.parametrize(
+    "original,marker,py_spec",
+    [
+        ("python_version > '3'", None, ">=3.1"),
+        ("python_version > '3.8'", None, ">=3.9"),
+        ("python_version != '3.8'", None, "!=3.8.*"),
+        ("python_version == '3.7'", None, ">=3.7,<3.8"),
+        ("python_version in '3.6 3.7'", None, ">=3.6,<3.8"),
+        ("python_full_version >= '3.6.0'", None, ">=3.6"),
+        ("python_full_version not in '3.8.3'", None, "!=3.8.3"),
+        # mixed marker and python version
+        ("python_version > '3.7' and os_name == 'nt'", 'os_name == "nt"', ">=3.8"),
+        (
+            "python_version > '3.7' or os_name == 'nt'",
+            'python_version > "3.7" or os_name == "nt"',
+            "",
+        ),
+    ],
+)
+def test_split_pyspec(original, marker, py_spec):
+    m = Marker(original)
+    a, b = m.split_pyspec()
+    assert marker == (str(a) if a is not None else None)
+    assert py_spec == str(b)