Repository: pdm-project/pdm

Issue:
Failure of Editable Installs from Relative Paths outside Project Root
I see that this has been brought up before in #464, but am not sure what the final conclusion is for pdm 2+. Currently PDM is unable to add editable packages outside of the project's directory (aka `${PROJECT_ROOT}`).

This can be reproduced given the following fictitious package structure:

```
project-a
├── pdm.lock
├── pyproject.toml
project-b
├── pdm.lock
├── pyproject.toml
```

And the `pyproject.toml` in project-a includes:

```toml
[tool.pdm.dev-dependencies]
dev = [
    "-e file:///${PROJECT_ROOT}/../project-b",
]
```

This causes a failure similar to below during `pdm install`, on pdm 2.1.3.

```
Traceback (most recent call last):
  File "/Users/maestro/.asdf/installs/python/3.7.13/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/core.py", line 254, in main
    return Core().main(args)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/core.py", line 187, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/core.py", line 182, in main
    f(options.project, options)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/cli/commands/install.py", line 69, in handle
    hooks=hooks,
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/cli/actions.py", line 203, in do_sync
    candidates = resolve_candidates_from_lockfile(project, requirements)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/cli/actions.py", line 150, in resolve_candidates_from_lockfile
    resolve_max_rounds,
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/resolver/core.py", line 30, in resolve
    result = resolver.resolve(requirements, max_rounds)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/resolver/providers.py", line 180, in get_dependencies
    deps, requires_python, _ = self.repository.get_dependencies(candidate)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/models/repositories.py", line 426, in get_dependencies
    reqs, python, summary = super().get_dependencies(candidate)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/models/repositories.py", line 70, in get_dependencies
    requirements, requires_python, summary = getter(candidate)
  File "/Users/maestro/.asdf/installs/python/3.7.13/lib/python3.7/site-packages/pdm/models/repositories.py", line 418, in _get_dependencies_from_lockfile
    return self.candidate_info[self._identify_candidate(candidate)]
KeyError: ('project-b', None, 'file:///${PROJECT_ROOT}/../project-b', True)
```

## Expected behavior

In this case I would expect pdm to properly resolve the relative path without failure.

I understand that you can get around this by using symlinks and possibly other clever file path magic, but it would be nice if relative editable installs worked with pdm by default, as many other project do seem to support this capability (pipenv and poetry AFAIK)

**What is the expected behavior here? Is there a suggested way to manage interproject dependencies with PDM that I may have missed?**

Discussion:
codecov-commenter: # [Codecov](https://codecov.io/gh/pdm-project/pdm/pull/1432?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) Report
Base: **84.94**% // Head: **84.97**% // Increases project coverage by **`+0.03%`** :tada:
> Coverage data is based on head [(`2e56273`)](https://codecov.io/gh/pdm-project/pdm/pull/1432?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) compared to base [(`901c5e2`)](https://codecov.io/gh/pdm-project/pdm/commit/901c5e2c11143c6be640afa3aaa944e90906f374?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1432      +/-   ##
==========================================
+ Coverage   84.94%   84.97%   +0.03%     
==========================================
  Files          88       88              
  Lines        7605     7607       +2     
  Branches     1777     1778       +1     
==========================================
+ Hits         6460     6464       +4     
+ Misses        762      761       -1     
+ Partials      383      382       -1     
```

| Flag | Coverage Δ | |
|---|---|---|
| unittests | `84.79% <100.00%> (+0.03%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/pdm-project/pdm/pull/1432?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) | Coverage Δ | |
|---|---|---|
| [src/pdm/models/requirements.py](https://codecov.io/gh/pdm-project/pdm/pull/1432/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9tb2RlbHMvcmVxdWlyZW1lbnRzLnB5) | `95.23% <100.00%> (+0.03%)` | :arrow_up: |
| [src/pdm/models/caches.py](https://codecov.io/gh/pdm-project/pdm/pull/1432/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9tb2RlbHMvY2FjaGVzLnB5) | `87.71% <0.00%> (-1.17%)` | :arrow_down: |
| [src/pdm/cli/actions.py](https://codecov.io/gh/pdm-project/pdm/pull/1432/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9jbGkvYWN0aW9ucy5weQ==) | `80.37% <0.00%> (+0.23%)` | :arrow_up: |
| [src/pdm/termui.py](https://codecov.io/gh/pdm-project/pdm/pull/1432/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS90ZXJtdWkucHk=) | `90.78% <0.00%> (+0.70%)` | :arrow_up: |
| [src/pdm/models/repositories.py](https://codecov.io/gh/pdm-project/pdm/pull/1432/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9tb2RlbHMvcmVwb3NpdG9yaWVzLnB5) | `78.77% <0.00%> (+0.81%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/pdm-project/pdm/pull/1432?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project).


Patch:
diff --git a/news/1220.bugfix.md b/news/1220.bugfix.md
new file mode 100644
index 00000000..677d71f2
--- /dev/null
+++ b/news/1220.bugfix.md
@@ -0,0 +1,1 @@
+Fix the issue that relative paths don't work well with `--project` argument.
diff --git a/news/1381.bugfix.md b/news/1381.bugfix.md
new file mode 100644
index 00000000..d64a0593
--- /dev/null
+++ b/news/1381.bugfix.md
@@ -0,0 +1,1 @@
+Fix the support for relative paths from outside the project.
diff --git a/src/pdm/cli/actions.py b/src/pdm/cli/actions.py
index 618b67f3..bcf47d68 100644
--- a/src/pdm/cli/actions.py
+++ b/src/pdm/cli/actions.py
@@ -48,7 +48,7 @@ from pdm.models.requirements import Requirement, parse_requirement, strip_extras
 from pdm.models.specifiers import get_specifier
 from pdm.project import Project
 from pdm.resolver import resolve
-from pdm.utils import normalize_name
+from pdm.utils import cd, normalize_name
 
 PEP582_PATH = os.path.join(os.path.dirname(os.path.dirname(__file__)), "pep582")
 
@@ -259,6 +259,8 @@ def do_add(
     for r in [parse_requirement(line, True) for line in editables] + [
         parse_requirement(line) for line in packages
     ]:
+        if r.is_file_or_url:
+            r.relocate(project.root)  # type: ignore
         key = r.identify()
         r.prerelease = prerelease
         tracked_names.add(key)
@@ -438,18 +440,19 @@ def do_remove(
         f"{'dev-' if dev else ''}dependencies: "
         + ", ".join(f"[bold green]{name}[/]" for name in packages)
     )
-    for name in packages:
-        req = parse_requirement(name)
-        matched_indexes = sorted(
-            (i for i, r in enumerate(deps) if req.matches(r)), reverse=True
-        )
-        if not matched_indexes:
-            raise ProjectError(
-                f"[bold green]{name}[/] does not exist in "
-                f"[bold cyan]{group}[/] dependencies."
+    with cd(project.root):
+        for name in packages:
+            req = parse_requirement(name)
+            matched_indexes = sorted(
+                (i for i, r in enumerate(deps) if req.matches(r)), reverse=True
             )
-        for i in matched_indexes:
-            del deps[i]
+            if not matched_indexes:
+                raise ProjectError(
+                    f"[bold green]{name}[/] does not exist in "
+                    f"[bold cyan]{group}[/] dependencies."
+                )
+            for i in matched_indexes:
+                del deps[i]
     cast(Array, deps).multiline(True)
 
     if not dry_run:
diff --git a/src/pdm/models/candidates.py b/src/pdm/models/candidates.py
index b5d43e44..48964167 100644
--- a/src/pdm/models/candidates.py
+++ b/src/pdm/models/candidates.py
@@ -28,6 +28,7 @@ from pdm.models.specifiers import PySpecSet
 from pdm.project.metadata import MutableMetadata, SetupDistribution
 from pdm.utils import (
     cached_property,
+    cd,
     convert_hashes,
     create_tracked_tempdir,
     expand_env_vars_in_auth,
@@ -236,14 +237,15 @@ class Candidate:
             if not self.req.editable:
                 result.update(revision=self.get_revision())
         elif not self.req.is_named:
-            if self.req.is_file_or_url and self.req.is_local_dir:
-                result.update(path=path_replace(root_path, ".", self.req.str_path))
-            else:
-                result.update(
-                    url=path_replace(
-                        root_path.lstrip("/"), "${PROJECT_ROOT}", self.req.url
+            with cd(project_root):
+                if self.req.is_file_or_url and self.req.is_local_dir:
+                    result.update(path=path_replace(root_path, ".", self.req.str_path))
+                else:
+                    result.update(
+                        url=path_replace(
+                            root_path.lstrip("/"), "${PROJECT_ROOT}", self.req.url
+                        )
                     )
-                )
         return {k: v for k, v in result.items() if v}
 
     def format(self) -> str:
@@ -329,7 +331,8 @@ class PreparedCandidate:
                 }
             )
         elif isinstance(req, FileRequirement):
-            if req.is_local_dir:
+            assert self.link is not None
+            if self.link.is_file and self.link.file_path.is_dir():
                 return _filter_none(
                     {
                         "url": url_without_fragments(req.url),
@@ -337,22 +340,14 @@ class PreparedCandidate:
                         "subdirectory": req.subdirectory,
                     }
                 )
-            url = expand_env_vars_in_auth(
-                req.url.replace(
-                    "${PROJECT_ROOT}",
-                    self.environment.project.root.as_posix().lstrip(  # type: ignore
-                        "/"
-                    ),
-                )
-            )
             with self.environment.get_finder() as finder:
                 hash_cache = self.environment.project.make_hash_cache()
                 return _filter_none(
                     {
-                        "url": url_without_fragments(req.url),
+                        "url": self.link.url_without_fragment,
                         "archive_info": {
                             "hash": hash_cache.get_hash(
-                                Link(url), finder.session
+                                self.link, finder.session
                             ).replace(":", "=")
                         },
                         "subdirectory": req.subdirectory,
diff --git a/src/pdm/models/repositories.py b/src/pdm/models/repositories.py
index aba03c4a..3bb9e9d0 100644
--- a/src/pdm/models/repositories.py
+++ b/src/pdm/models/repositories.py
@@ -17,7 +17,7 @@ from pdm.models.requirements import (
 )
 from pdm.models.search import SearchResultParser
 from pdm.models.specifiers import PySpecSet
-from pdm.utils import normalize_name, url_without_fragments
+from pdm.utils import cd, normalize_name, url_without_fragments
 
 if TYPE_CHECKING:
     from pdm._types import CandidateInfo, SearchResult, Source
@@ -397,26 +397,27 @@ class LockedRepository(BaseRepository):
         return {can.req.identify(): can for can in self.packages.values()}
 
     def _read_lockfile(self, lockfile: Mapping[str, Any]) -> None:
-        for package in lockfile.get("package", []):
-            version = package.get("version")
-            if version:
-                package["version"] = f"=={version}"
-            package_name = package.pop("name")
-            req_dict = {
-                k: v
-                for k, v in package.items()
-                if k not in ("dependencies", "requires_python", "summary")
-            }
-            req = Requirement.from_req_dict(package_name, req_dict)
-            can = make_candidate(req, name=package_name, version=version)
-            can_id = self._identify_candidate(can)
-            self.packages[can_id] = can
-            candidate_info: CandidateInfo = (
-                package.get("dependencies", []),
-                package.get("requires_python", ""),
-                package.get("summary", ""),
-            )
-            self.candidate_info[can_id] = candidate_info
+        with cd(self.environment.project.root):
+            for package in lockfile.get("package", []):
+                version = package.get("version")
+                if version:
+                    package["version"] = f"=={version}"
+                package_name = package.pop("name")
+                req_dict = {
+                    k: v
+                    for k, v in package.items()
+                    if k not in ("dependencies", "requires_python", "summary")
+                }
+                req = Requirement.from_req_dict(package_name, req_dict)
+                can = make_candidate(req, name=package_name, version=version)
+                can_id = self._identify_candidate(can)
+                self.packages[can_id] = can
+                candidate_info: CandidateInfo = (
+                    package.get("dependencies", []),
+                    package.get("requires_python", ""),
+                    package.get("summary", ""),
+                )
+                self.candidate_info[can_id] = candidate_info
 
         for key, hashes in lockfile.get("metadata", {}).get("files", {}).items():
             self.file_hashes[tuple(key.split(None, 1))] = {  # type: ignore
diff --git a/src/pdm/models/requirements.py b/src/pdm/models/requirements.py
index cdadefac..796a4071 100644
--- a/src/pdm/models/requirements.py
+++ b/src/pdm/models/requirements.py
@@ -254,7 +254,7 @@ class FileRequirement(Requirement):
         super().__post_init__()
         self._parse_url()
         if self.path and not self.path.exists():
-            raise RequirementError(f"The local path {self.path} does not exist.")
+            raise RequirementError(f"The local path '{self.path}' does not exist.")
         if self.is_local_dir:
             self._check_installable()
 
@@ -272,21 +272,22 @@ class FileRequirement(Requirement):
         if not self.path:
             return None
         result = self.path.as_posix()
-        if (
-            not self.path.is_absolute()
-            and not result.startswith("./")
-            and not result.startswith("../")
-        ):
+        if not self.path.is_absolute() and not result.startswith(("./", "../")):
             result = "./" + result
+        if result.startswith("./../"):
+            result = result[2:]
         return result
 
     def _parse_url(self) -> None:
         if not self.url:
             if self.path:
-                self.url = path_to_url(self.path.absolute().as_posix())
                 if not self.path.is_absolute():
-                    project_root = Path(".").absolute().as_posix().lstrip("/")
-                    self.url = self.url.replace(project_root, "${PROJECT_ROOT}")
+                    str_path = cast(str, self.str_path)
+                    if str_path.startswith("./"):
+                        str_path = str_path[2:]
+                    self.url = f"file:///${{PROJECT_ROOT}}/{str_path}"
+                else:
+                    self.url = path_to_url(self.path.absolute().as_posix())
         else:
             try:
                 self.path = Path(
@@ -301,6 +302,15 @@ class FileRequirement(Requirement):
                 pass
         self._parse_name_from_url()
 
+    def relocate(self, root: str | Path) -> None:
+        """Change the project root to the given path"""
+        if self.path is None or self.path.is_absolute():
+            return
+        # self.path is relative
+        self.path = Path(os.path.relpath(self.path, root))
+        self.url = ""
+        self._parse_url()
+
     @property
     def is_local(self) -> bool:
         return self.path and self.path.exists() or False
diff --git a/src/pdm/project/core.py b/src/pdm/project/core.py
index c29fe999..5446ab3c 100644
--- a/src/pdm/project/core.py
+++ b/src/pdm/project/core.py
@@ -103,7 +103,7 @@ class Project:
                     err=True,
                 )
 
-        self.root = Path(root_path or "").absolute()
+        self.root: Path = Path(root_path or "").absolute()
         self.is_global = is_global
         self.init_global_project()
         self._lockfile_file = self.root / "pdm.lock"


Test patch:
diff --git a/tests/models/test_requirements.py b/tests/models/test_requirements.py
index 20644ad7..3503fdb8 100644
--- a/tests/models/test_requirements.py
+++ b/tests/models/test_requirements.py
@@ -1,9 +1,10 @@
 import os
+import shutil
 
 import pytest
 
 from pdm.models.requirements import RequirementError, parse_requirement
-from pdm.utils import path_to_url
+from pdm.utils import cd, path_to_url
 from tests import FIXTURES
 
 FILE_PREFIX = "file:///" if os.name == "nt" else "file://"
@@ -86,3 +87,24 @@ def test_not_supported_editable_requirement(line):
         RequirementError, match="Editable requirement is only supported"
     ):
         parse_requirement(line, True)
+
+
+def test_convert_req_with_relative_path_from_outside(tmp_path):
+    shutil.copytree(FIXTURES / "projects/demo", tmp_path / "demo")
+    tmp_path.joinpath("project").mkdir()
+    with cd(tmp_path / "project"):
+        r = parse_requirement("../demo")
+        assert r.path.resolve() == tmp_path / "demo"
+        assert r.url == "file:///${PROJECT_ROOT}/../demo"
+
+
+def test_file_req_relocate(tmp_path):
+    shutil.copytree(FIXTURES / "projects/demo", tmp_path / "demo")
+    tmp_path.joinpath("project").mkdir()
+    with cd(tmp_path):
+        r = parse_requirement("./demo")
+        assert r.path.resolve() == tmp_path / "demo"
+        assert r.url == "file:///${PROJECT_ROOT}/demo"
+        r.relocate(tmp_path / "project")
+        assert r.str_path == "../demo"
+        assert r.url == "file:///${PROJECT_ROOT}/../demo"