Repository: python-poetry/poetry

Issue:
1.1.0b2 causes EnvCommandError (Non-posix paths with file:// protocol on Windows)
<!--
  Hi there! Thank you for discovering and submitting an issue.

  Before you submit this; let's make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill these first.
-->

<!-- Checked checkbox should look like this: [x] -->
- [x] I am on the [latest](https://github.com/python-poetry/poetry/releases/latest) Poetry version.
- [x] I have searched the [issues](https://github.com/python-poetry/poetry/issues) of this repo and believe that this is not a duplicate.
- [x] If an exception occurs when executing a command, I executed it again in debug mode (`-vvv` option).

<!--
  Once those are done, if you're able to fill in the following list with your information,
  it'd be very helpful to whoever handles the issue.
-->

- **OS version and name**: Windows 10
- **Poetry version**: 1.1.0b2
- **Link of a [Gist](https://gist.github.com/) with the contents of your pyproject.toml file**: <!-- Gist Link Here -->

## Issue

When on Poetry 1.0.10 there are no issues. But after having updated to poetry 1.1.0b2, I constantly get an EnvCommandError caused by a FileNotFoundError (a temporary setup.py file is not found):

```bash
$ poetry --version
Poetry version 1.1.0b2

$ poetry lock
Updating dependencies
Resolving dependencies...

Writing lock file

$ poetry install -vvv                                         
Using virtualenv: C:\Users\fredrik\code\repos\mylittleci\venv
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 2 installs, 0 updates, 0 removals

  • Installing zipp (3.1.0)

  Stack trace:

  7  c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:183 in _execute_operation
      181│
      182│             try:
    → 183│                 result = self._do_execute_operation(operation)
      184│             except EnvCommandError as e:
      185│                 if e.e.returncode == -2:

  6  c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:257 in _do_execute_operation
      255│             return 0
      256│
    → 257│         result = getattr(self, "_execute_{}".format(method))(operation)
      258│
      259│         if result != 0:

  5  c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:394 in _execute_install
      392│
      393│     def _execute_install(self, operation):  # type: (Install) -> None
    → 394│         return self._install(operation)
      395│
      396│     def _execute_update(self, operation):  # type: (Update) -> None

  4  c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:432 in _install
      430│             args.insert(2, "-U")
      431│
    → 432│         return self.run_pip(*args)
      433│
      434│     def _update(self, operation):

  3  c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:283 in run_pip
      281│     def run_pip(self, *args, **kwargs):  # type: (...) -> int
      282│         try:
    → 283│             self._env.run_pip(*args, **kwargs)
      284│         except EnvCommandError as e:
      285│             output = decode(e.e.output)

  2  c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\utils\env.py:883 in run_pip
       881│         pip = self.get_pip_command()
       882│         cmd = pip + list(args)
    →  883│         return self._run(cmd, **kwargs)
       884│
       885│     def _run(self, cmd, **kwargs):

  1  c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\utils\env.py:1151 in _run
      1149│             self.unset_env("__PYVENV_LAUNCHER__")
      1150│
    → 1151│             return super(VirtualEnv, self)._run(cmd, **kwargs)
      1152│
      1153│     def execute(self, bin, *args, **kwargs):

  EnvCommandError

  Command C:\Users\fredrik\code\repos\mylittleci\venv\Scripts\pip.exe install --no-deps file://C:\Users\fredrik\AppData\Local\pypoetry\Cache\artifacts\3f\29\e0\445941d0045028dce97085f4263281587225a49fa7dd09f41e4c2752af\zipp-3.1.0-py3-none-any.whl errored with the following return code 1, and output:
  Processing c:\users\fredrik\appdata\local\pypoetry\cache\artifacts\3f\29\e0\445941d0045028dce97085f4263281587225a49fa7dd09f41e4c2752af\zipp-3.1.0-py3-none-any.whl
      ERROR: Command errored out with exit status 1:
       command: 'c:\users\fredrik\code\repos\mylittleci\venv\scripts\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\fredrik\\AppData\\Local\\Temp\\pip-req-build-gsp4euj0\\setup.py'"'"'; __file__='"'"'C:\\Users\\fredrik\\AppData\\Local\\Temp\\pip-req-build-gsp4euj0\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\fredrik\AppData\Local\Temp\pip-pip-egg-info-_md_1odm'
           cwd: C:\Users\fredrik\AppData\Local\Temp\pip-req-build-gsp4euj0\
      Complete output (5 lines):
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "C:\Users\fredrik\AppData\Local\Programs\Python\Python37\lib\tokenize.py", line 447, in open
          buffer = _builtin_open(filename, 'rb')
      FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\fredrik\\AppData\\Local\\Temp\\pip-req-build-gsp4euj0\\setup.py'
      ----------------------------------------
  ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.


  at c:\users\fredrik\.local\pipx\venvs\poetry\lib\site-packages\poetry\utils\env.py:915 in _run
       911│                 output = subprocess.check_output(
       912│                     cmd, stderr=subprocess.STDOUT, **kwargs
       913│                 )
       914│         except CalledProcessError as e:
    →  915│             raise EnvCommandError(e, input=input_)
       916│
       917│         return decode(output)
       918│
       919│     def execute(self, bin, *args, **kwargs):
```

The contents of my `pyproject.toml`:

```
[tool.poetry]
name = "mylittleci"
version = "0.0.1"
authors = ["Fredrik <***@***.***>"]
description = "mylittleci"


[tool.poetry.dependencies]
python = "^3.7"


[tool.poetry.dev-dependencies]
importlib_metadata = "^1.7.0"


[tool.poetry.scripts]
calculator = "mylittleci.cli.calculator:main"


[build-system]
# https://python-poetry.org/docs/faq/#is-tox-supported
requires = ["poetry-core>=1.0.0a5"]
build-backend = "poetry.core.masonry.api"
```



Discussion:
I am re-opening this, at I am hitting this constantly now, with poetry 1.1.0b2.

I have attempted the `poetry install` command in both Powershell and in git bash, both on the same Windows machine. In both cases, I get the FileNotFoundError (to the temporary setup.py) as seen in the code block in my post above.

If I downgrade back to poetry 1.0.10 I can see that the dummy setup.py file is being created in my repo root and the `poetry install` command completes without errors. With Poetry 1.1.0b2, it seems as if the temporary setup.py file is either not yet created or has already been deleted when an attempt to access it is being made.
@dispater I tracked down what the issues is here, and I could set up a PR to fix this. But I want to hear where you think this should get fixed, as I see two possible approaches.

I suppose this is a Windows-only problem. When inside the `poetry.utils.env.py:Env._run`, I can see what the `cmd` variable holds with a simple print statement.

In my case, `cmd` comes into the method as:

```
['C:\\Users\\fredrik\\code\\repos\\mylittleci\\venv\\Scripts\\pip.exe', 'install', '--no-deps', 'file://C:\\Users\\fredrik\\AppData\\Local\\pypoetry\\Cache\\artifacts\\47\\bb\\ec\\bf68abe7d5464fced474bdf20de54de7737439f79a80b8bfc247e33c40\\smmap-3.0.4-py2.py3-none-any.whl']
```

Since I am on Windows, the list is converted into a string:

```
C:\Users\fredrik\code\repos\mylittleci\venv\Scripts\pip.exe install --no-deps file://C:\Users\fredrik\AppData\Local\pypoetry\Cache\artifacts\47\bb\ec\bf68abe7d5464fced474bdf20de54de7737439f79a80b8bfc247e33c40\smmap-3.0.4-py2.py3-none-any.whl
```

If I, after the list-to-string conversion, add a simple `cmd = cmd.replace("\\", "/")`, the pip installation completes without errors, and this is on Windows.

It also works removing the protocol and leaving the backslashes in: `cmd = cmd.replace("file://", "")`

The deadly combination is to combine the protocol with backslashes.

Now, to my question before attempting a PR...

Is there a reason why this `file://` protocol is used?
I'm thinking the problem can either be solved under `self._is_windows` in the `poetry.utils.env.py::_run` method, or it can be solved e.g. in the `poetry.installation.chef.py::get_cached_archives_for_link` method. What do you think would be best?
I updated the comment above. Please re-read, if you've already had time to read it.
@fredrikaverpil thank you for the details you have provided. The issue here is the following piece of code.

https://github.com/python-poetry/poetry/blob/c77299ed373df50ec459907671f65f5da3de5070/poetry/installation/executor.py#L425

This is problematic because `str()` as this is not cross-platform safe. Looking into this, at the moment the type of `archive` is not necessarily guarenteed to be a `Path` instance either, this needs to be fixed and the above use of `str(archive)` needs to be replaced with `archive.as_posix()` instead.
@abn exactly. I was actually working on a fix and a regression test for this. Do you want me to make a branch with this in it?

```python
    def get_cached_archives_for_link(self, link):  # type: (Link) -> List[Link]
        cache_dir = self.get_cache_directory_for_link(link)

        archive_types = ["whl", "tar.gz", "tar.bz2", "bz2", "zip"]
        links = []
        for archive_type in archive_types:
            for archive in cache_dir.glob("*.{}".format(archive_type)):
                if self._env._is_windows:
                    archive = archive.as_posix()
                links.append(Link("file://{}".format(str(archive))))

        return links
```

@fredrikaverpil freel free to submit a PR for the code you linked above by replacing `str(archive)` with `archive.as_posix()`, there is no need to use the windows check, let `pathlib` worry about that :)

I'll clean up the other bits later anyway.

Resolving patch:
diff --git a/poetry/installation/chef.py b/poetry/installation/chef.py
--- a/poetry/installation/chef.py
+++ b/poetry/installation/chef.py
@@ -81,7 +81,7 @@ def get_cached_archives_for_link(self, link):  # type: (Link) -> List[Link]
         links = []
         for archive_type in archive_types:
             for archive in cache_dir.glob("*.{}".format(archive_type)):
-                links.append(Link("file://{}".format(str(archive))))
+                links.append(Link(archive.as_uri()))
 
         return links
 


Test patch:
diff --git a/tests/installation/test_chef.py b/tests/installation/test_chef.py
--- a/tests/installation/test_chef.py
+++ b/tests/installation/test_chef.py
@@ -44,17 +44,19 @@ def test_get_cached_archives_for_link(config, mocker):
         ),
     )
 
+    distributions = Path(__file__).parent.parent.joinpath("fixtures/distributions")
     mocker.patch.object(
-        chef,
-        "get_cache_directory_for_link",
-        return_value=Path(__file__).parent.parent.joinpath("fixtures/distributions"),
+        chef, "get_cache_directory_for_link", return_value=distributions,
     )
 
     archives = chef.get_cached_archives_for_link(
         Link("https://files.python-poetry.org/demo-0.1.0.tar.gz")
     )
 
-    assert 2 == len(archives)
+    assert archives
+    assert set(archives) == {
+        Link(path.as_uri()) for path in distributions.glob("demo-0.1.0*")
+    }
 
 
 def test_get_cache_directory_for_link(config):