Repository: pypa/pip

Issue:
Call for clarification: acceptable values for build-system.requires in pyproject.toml
**Environment**

* pip version: 19.0.3
* Python version: 3.7
* OS: GNU/Linux

<!-- Feel free to add more information about your environment here -->

**Description**
<!-- A clear and concise description of what the bug is. -->
It's not clear nor is clearly specified by PEP518. But I was having a need to have an in-tree build back-end.
So I've hacked it via installing with a relative path.
*And it works!*
Yet, @pradyunsg has pointed out that the way I used probably doesn't conform to PEP 508.
So I tried some other ways to point to the in-tree distribution. And those didn't work.

**How to Reproduce**
<!-- Describe the steps to reproduce this bug. -->
(this works)
```toml
[build-system]
requires = ["./build-aiohttp", ]
build-backend = "build_aiohttp.api"

# not yet supported, so it doesn't influence anything, it's for forward-compat:
backend-path = "./build-aiohttp"
```

But if instead of `"./build-aiohttp"` in `requires` I try any of `"file://build-aiohttp"`, `"file:///./build-aiohttp"`, `"build_aiohttp @ file://./build-aiohttp"`, `"build_aiohttp @ file:./build-aiohttp"` pip fails to recognize those as installables.

**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->

I don't know what to expect. The method which works seems to be undefined in PEPs so I probably shouldn't rely on it.

Pip may either decide to improve the filtering of `requires` option or document it being permissive...

P.S. Oh and, by the way, I was able to test my other PEP517 backend outside of the project tree via
```toml
[build-system]
requires = ["../fortunate_pkg"]
```
so this relative path feature proves to be quite useful for development/debugging purposes.


Discussion:
[In-tree backend support](https://www.python.org/dev/peps/pep-0517/#in-tree-build-backends) was recently added to PEP 517. It's not yet supported in pip, however.
Hi Paul, I do know that. My question is about whether it's legal to put a relative path in `requires`.
Ah, sorry, my misunderstanding. I don't believe so, PEP 518 states what can go in `requires`:

> This key must have a value of a list of strings representing PEP 508 dependencies

So they aren't paths at all, and if pip is accepting paths, that's wrong.
Here is where pip's validation logic is currently (it only checks that it's a list of strings): https://github.com/pypa/pip/blob/54b6a91405adc79cdb8a2954e9614d6860799ccb/src/pip/_internal/pyproject.py#L89-L95
I'm inclined to describe this as "user error" - which is not to say that pip couldn't give better messages, but it's the user's responsibility to supply valid input. I don't really think we need to do full PEP 508 validation here.

The odd sort-of-works result is probably because pip accepts more than just PEP 508 dependencies as "things to install", so this is arguably "behaviour undefined by the PEP that triggers an implementation defined result". The classic [Undefined Behaviour](https://en.wikipedia.org/wiki/Undefined_behavior) nasal demons case ;-)
Right, I'm just trying to decide whether to rely on it. Because, if it turns out broken in the next version, then it would affect already published sdists.
@webknjaz you can use `intreehooks:loader` as was done for `flit` [here](https://github.com/takluyver/flit/blob/a2e79f34d3a9edbd8bedb45c1376a3a3cb0344dd/pyproject.toml#L4). This should continue to work regardless of what pip does.
I'm inclined to say let's actually abort in this edge case.

I don't want people trying things they shouldn't and it sort-of working.
Agreed, especially since this use case is covered by `intreehooks` and (soon) `backend-path`.

Resolving patch:
diff --git a/src/pip/_internal/pyproject.py b/src/pip/_internal/pyproject.py
--- a/src/pip/_internal/pyproject.py
+++ b/src/pip/_internal/pyproject.py
@@ -5,6 +5,7 @@
 import sys
 
 from pip._vendor import pytoml, six
+from pip._vendor.packaging.requirements import InvalidRequirement, Requirement
 
 from pip._internal.exceptions import InstallationError
 from pip._internal.utils.typing import MYPY_CHECK_RUNNING
@@ -150,6 +151,21 @@ def load_pyproject_toml(
             reason="'build-system.requires' is not a list of strings.",
         ))
 
+    # Each requirement must be valid as per PEP 508
+    for requirement in requires:
+        try:
+            Requirement(requirement)
+        except InvalidRequirement:
+            raise InstallationError(
+                error_template.format(
+                    package=req_name,
+                    reason=(
+                        "'build-system.requires' contains an invalid "
+                        "requirement: {!r}".format(requirement)
+                    ),
+                )
+            )
+
     backend = build_system.get("build-backend")
     check = []  # type: List[str]
     if backend is None:


Test patch:
diff --git a/tests/unit/test_pep517.py b/tests/unit/test_pep517.py
--- a/tests/unit/test_pep517.py
+++ b/tests/unit/test_pep517.py
@@ -1,3 +1,5 @@
+from textwrap import dedent
+
 import pytest
 
 from pip._internal.exceptions import InstallationError
@@ -9,11 +11,11 @@
     ("pep517_setup_only", False),
     ("pep517_pyproject_only", True),
 ])
-def test_use_pep517(data, source, expected):
+def test_use_pep517(shared_data, source, expected):
     """
     Test that we choose correctly between PEP 517 and legacy code paths
     """
-    src = data.src.joinpath(source)
+    src = shared_data.src.joinpath(source)
     req = InstallRequirement(None, None, source_dir=src)
     req.load_pyproject_toml()
     assert req.use_pep517 is expected
@@ -23,11 +25,11 @@ def test_use_pep517(data, source, expected):
     ("pep517_setup_and_pyproject", "specifies a build backend"),
     ("pep517_pyproject_only", "does not have a setup.py"),
 ])
-def test_disabling_pep517_invalid(data, source, msg):
+def test_disabling_pep517_invalid(shared_data, source, msg):
     """
     Test that we fail if we try to disable PEP 517 when it's not acceptable
     """
-    src = data.src.joinpath(source)
+    src = shared_data.src.joinpath(source)
     req = InstallRequirement(None, None, source_dir=src)
 
     # Simulate --no-use-pep517
@@ -39,3 +41,23 @@ def test_disabling_pep517_invalid(data, source, msg):
     err_msg = e.value.args[0]
     assert "Disabling PEP 517 processing is invalid" in err_msg
     assert msg in err_msg
+
+
+@pytest.mark.parametrize(
+    ("spec",), [("./foo",), ("git+https://example.com/pkg@dev#egg=myproj",)]
+)
+def test_pep517_parsing_checks_requirements(tmpdir, spec):
+    tmpdir.joinpath("pyproject.toml").write_text(dedent(
+        """
+        [build-system]
+        requires = [{!r}]
+        build-backend = "foo"
+        """.format(spec)
+    ))
+    req = InstallRequirement(None, None, source_dir=tmpdir)
+
+    with pytest.raises(InstallationError) as e:
+        req.load_pyproject_toml()
+
+    err_msg = e.value.args[0]
+    assert "contains an invalid requirement" in err_msg