Repository: python-poetry/poetry

Issue:
Empty line in RECORD for git dependency, fails on uninstall
- [ ] I am on the [latest](https://github.com/python-poetry/poetry/releases/latest) Poetry version.
- [x] I have searched the [issues](https://github.com/python-poetry/poetry/issues) of this repo and believe that this is not a duplicate.
- [x] If an exception occurs when executing a command, I executed it again in debug mode (`-vvv` option).


- **OS version and name**: Windows 10
- **Python version**: 3.9
- **Poetry version**: `1.2.0b2`
- **Link of a [Gist](https://gist.github.com/) with the contents of the pyproject.toml file after `poetry init`**: https://gist.github.com/greschd/a17ef8858e3959d18d3bc78817a77e6c

## Summary

When adding a dependency from a `git+` link, the `RECORD` file contains a trailing empty line. This causes an exception when trying to uninstall the dependency.

## Steps to Reproduce

In an empty directory, execute the following steps:
- `poetry init`
- When asked for dependencies, add a `git` url, e.g. `git+https://github.com/python-poetry/poetry.git`
- `poetry install`
- Remove the git dependency, e.g. `poetry remove poetry`

## Observed behavior

The `poetry remove` step fails. Output of `poetry remove -vvv poetry`:

<details><summary><b>Click to expand</b></summary>

```
poetry remove -vvv poetry
Loading configuration file C:\Users\dgresch\AppData\Roaming\pypoetry\config.toml
Using virtualenv: C:\Users\dgresch\AppData\Local\pypoetry\Cache\virtualenvs\test-m45U-ywY-py3.9
Updating dependencies
Resolving dependencies...
   1: fact: test is 0.1.0
   1: derived: test
   1: selecting test (0.1.0)
   1: Version solving took 0.000 seconds.
   1: Tried 1 solutions.

Writing lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 0 updates, 36 removals

  • Removing cachecontrol (0.12.11)
  • Removing cachy (0.3.0)
  • Removing certifi (2022.5.18.1)
  • Removing charset-normalizer (2.0.12)
  • Removing cleo (1.0.0a5)
  • Removing crashtest (0.3.1)
  • Removing distlib (0.3.4)
  • Removing dulwich (0.20.43)
  • Removing entrypoints (0.4)
  • Removing filelock (3.7.1)
  • Removing html5lib (1.1)
  • Removing idna (3.3)
  • Removing importlib-metadata (4.11.4)
  • Removing keyring (23.5.1)
  • Removing lockfile (0.12.2)
  • Removing msgpack (1.0.4)
  • Removing packaging (21.3)
  • Removing pexpect (4.8.0)
  • Removing pkginfo (1.8.2)
  • Removing platformdirs (2.5.2)
  • Removing poetry (1.2.0b3.dev0 2afe984)

  Stack trace:

  2  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\utils\env.py:1493 in _run
      1491│                 )
      1492│             else:
    → 1493│                 output = subprocess.check_output(
      1494│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1495│                 )

  1  ~\AppData\Local\Programs\Python\Python39\lib\subprocess.py:424 in check_output
       422│         kwargs['input'] = empty
       423│
    →  424│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       425│                **kwargs).stdout
       426│

  CalledProcessError

  Command 'C:\Users\dgresch\AppData\Local\pypoetry\Cache\virtualenvs\test-m45U-ywY-py3.9\Scripts\python.exe C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip uninstall poetry -y' returned non-zero exit status 2.

  at ~\AppData\Local\Programs\Python\Python39\lib\subprocess.py:528 in run
       524│             # We don't call process.wait() as .__exit__ does that for us.
       525│             raise
       526│         retcode = process.poll()
       527│         if check and retcode:
    →  528│             raise CalledProcessError(retcode, process.args,
       529│                                      output=stdout, stderr=stderr)
       530│     return CompletedProcess(process.args, retcode, stdout, stderr)
       531│
       532│

The following error occurred when trying to handle this error:


  Stack trace:

  7  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:246 in _execute_operation
      244│
      245│             try:
    → 246│                 result = self._do_execute_operation(operation)
      247│             except EnvCommandError as e:
      248│                 if e.e.returncode == -2:

  6  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:318 in _do_execute_operation
      316│             return 0
      317│
    → 318│         result: int = getattr(self, f"_execute_{method}")(operation)
      319│
      320│         if result != 0:

  5  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:456 in _execute_uninstall
      454│         self._write(operation, message)
      455│
    → 456│         return self._remove(operation)
      457│
      458│     def _install(self, operation: Install | Update) -> int:

  4  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:496 in _remove
      494│
      495│         try:
    → 496│             return self.run_pip("uninstall", package.name, "-y")
      497│         except CalledProcessError as e:
      498│             if "not installed" in str(e):

  3  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\installation\executor.py:341 in run_pip
      339│     def run_pip(self, *args: Any, **kwargs: Any) -> int:
      340│         try:
    → 341│             self._env.run_pip(*args, **kwargs)
      342│         except EnvCommandError as e:
      343│             output = decode(e.e.output)

  2  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\utils\env.py:1456 in run_pip
      1454│         pip = self.get_pip_command(embedded=True)
      1455│         cmd = pip + list(args)
    → 1456│         return self._run(cmd, **kwargs)
      1457│
      1458│     def run_python_script(self, content: str, **kwargs: Any) -> int | str:

  1  ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\utils\env.py:1746 in _run
      1744│     def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
      1745│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
    → 1746│         return super()._run(cmd, **kwargs)
      1747│
      1748│     def get_temp_environ(

  EnvCommandError

  Command C:\Users\dgresch\AppData\Local\pypoetry\Cache\virtualenvs\test-m45U-ywY-py3.9\Scripts\python.exe C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip uninstall poetry -y errored with the following return code 2, and output:
  Found existing installation: poetry 1.2.0b3.dev0
  ERROR: Exception:
  Traceback (most recent call last):
    File "C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip\_internal\cli\base_command.py", line 167, in exc_logging_wrapper
      status = run_func(*args)
    File "C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip\_internal\commands\uninstall.py", line 97, in run
      uninstall_pathset = req.uninstall(
    File "C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip\_internal\req\req_install.py", line 637, in uninstall
      uninstalled_pathset = UninstallPathSet.from_dist(dist)
    File "C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip\_internal\req\req_uninstall.py", line 522, in from_dist
      for path in uninstallation_paths(dist):
    File "C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip\_internal\req\req_uninstall.py", line 41, in unique
      for item in fn(*args, **kw):
    File "C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip\_internal\req\req_uninstall.py", line 81, in uninstallation_paths
      for entry in entries:
    File "C:\Users\dgresch\.local\pipx\venvs\poetry\lib\site-packages\virtualenv\seed\wheels\embed\pip-22.0.4-py3-none-any.whl\pip\_internal\metadata\base.py", line 405, in <genexpr>
      return (str(pathlib.Path(row[0])) for row in csv.reader(text.splitlines()))


  at ~\.local\pipx\venvs\poetry\lib\site-packages\poetry\utils\env.py:1497 in _run
      1493│                 output = subprocess.check_output(
      1494│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1495│                 )
      1496│         except CalledProcessError as e:
    → 1497│             raise EnvCommandError(e, input=input_)
      1498│
      1499│         return decode(output)
      1500│
      1501│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

  • Removing poetry-core (1.1.0b2)
  • Removing poetry-plugin-export (1.0.4)
  • Removing pylev (1.4.0)
  • Removing pywin32-ctypes (0.2.0)
  • Removing requests (2.27.1)
  • Removing requests-toolbelt (0.9.1)
  • Removing shellingham (1.4.0)
  • Removing six (1.16.0)
  • Removing tomlkit (0.11.0)
  • Removing urllib3 (1.26.9)
  • Removing virtualenv (20.14.1)
  • Removing webencodings (0.5.1)
  • Removing zipp (3.8.0)
```
</details>

## Expected behavior

The git dependency is removed.

## Additional information

File `<virtualenv_path>\Lib\site-packages\poetry-1.2.0b3.dev0.dist-info\RECORD`: https://gist.github.com/greschd/e07a5321515eec72a4ccff3e90c3906b

Note the empty lines at the end. I'm not sure which part of the system is responsible for writing this `RECORD` file, but installing directly with `pip install git+https://github.com/python-poetry/poetry.git` doesn't appear to have the same problem.

**Environments that _do_ reproduce the issue:**
- Windows 10, Python 3.9, official `poetry` installer
- Windows 10, Python 3.9, `pipx` installer

**Environments that _do not_ reproduce the issue:**
- Ubuntu 20.04 (WSL) with Python 3.8, official `poetry` installer
- Ubuntu 20.04 (WSL) with Python 3.9, official `poetry` installer
- Ubuntu 20.04 (WSL) with Python 3.8, `pipx` installer

At first glance, this seems to be a Windows-specific issue.

Pip freeze, run from the _failing_ Windows 10 pipx environment:

<details><summary><b>Click to expand:</b></summary>

```
CacheControl==0.12.11
cachy==0.3.0
certifi==2022.5.18.1
charset-normalizer==2.0.12
cleo==1.0.0a5
crashtest==0.3.1
distlib==0.3.4
dulwich==0.20.43
entrypoints==0.4
filelock==3.7.1
html5lib==1.1
idna==3.3
importlib-metadata==4.11.4
keyring==23.6.0
lockfile==0.12.2
msgpack==1.0.4
packaging==21.3
pexpect==4.8.0
pkginfo==1.8.2
platformdirs==2.5.2
poetry==1.2.0b2
poetry-core==1.1.0b2
poetry-plugin-export==1.0.4
ptyprocess==0.7.0
pylev==1.4.0
pyparsing==3.0.9
pywin32-ctypes==0.2.0
requests==2.27.1
requests-toolbelt==0.9.1
shellingham==1.4.0
six==1.16.0
tomlkit==0.11.0
urllib3==1.26.9
virtualenv==20.14.1
webencodings==0.5.1
zipp==3.8.0
```
</details>

Not shown in the `freeze`, but these also seem to be available:

```python
>>> import wheel; wheel.__version__
'0.37.1'
>>> import pip; pip.__version__
'22.1.2'
```

Repository with Github Actions that show the error: https://github.com/greschd/poetry_issue_5799_repro
See e.g. this run: https://github.com/greschd/poetry_issue_5799_repro/actions/runs/2464176553


Discussion:

Resolving patch:
diff --git a/src/poetry/installation/executor.py b/src/poetry/installation/executor.py
--- a/src/poetry/installation/executor.py
+++ b/src/poetry/installation/executor.py
@@ -761,7 +761,7 @@ def _save_url_reference(self, operation: Operation) -> None:
 
                 record = dist_path / "RECORD"
                 if record.exists():
-                    with record.open(mode="a", encoding="utf-8") as f:
+                    with record.open(mode="a", encoding="utf-8", newline="") as f:
                         writer = csv.writer(f)
                         path = url.relative_to(record.parent.parent)
                         writer.writerow([str(path), "", ""])
diff --git a/src/poetry/masonry/builders/editable.py b/src/poetry/masonry/builders/editable.py
--- a/src/poetry/masonry/builders/editable.py
+++ b/src/poetry/masonry/builders/editable.py
@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import csv
 import hashlib
 import json
 import os
@@ -255,14 +256,15 @@ def _add_dist_info(self, added_files: list[Path]) -> None:
         added_files.append(direct_url_json)
 
         record = dist_info.joinpath("RECORD")
-        with record.open("w", encoding="utf-8") as f:
+        with record.open("w", encoding="utf-8", newline="") as f:
+            csv_writer = csv.writer(f)
             for path in added_files:
                 hash = self._get_file_hash(path)
                 size = path.stat().st_size
-                f.write(f"{path!s},sha256={hash},{size}\n")
+                csv_writer.writerow((path, f"sha256={hash}", size))
 
             # RECORD itself is recorded with no hash or size
-            f.write(f"{record},,\n")
+            csv_writer.writerow((record, "", ""))
 
     def _get_file_hash(self, filepath: Path) -> str:
         hashsum = hashlib.sha256()


Test patch:
diff --git a/tests/installation/test_executor.py b/tests/installation/test_executor.py
--- a/tests/installation/test_executor.py
+++ b/tests/installation/test_executor.py
@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import csv
 import json
 import re
 import shutil
@@ -400,12 +401,14 @@ def verify_installed_distribution(
 
     if url_reference is not None:
         record_file = distribution._path.joinpath("RECORD")
+        with open(record_file, encoding="utf-8", newline="") as f:
+            reader = csv.reader(f)
+            rows = list(reader)
+        assert all(len(row) == 3 for row in rows)
+        record_entries = {row[0] for row in rows}
         direct_url_entry = direct_url_file.relative_to(record_file.parent.parent)
         assert direct_url_file.exists()
-        assert str(direct_url_entry) in {
-            row.split(",")[0]
-            for row in record_file.read_text(encoding="utf-8").splitlines()
-        }
+        assert str(direct_url_entry) in record_entries
         assert json.loads(direct_url_file.read_text(encoding="utf-8")) == url_reference
     else:
         assert not direct_url_file.exists()
diff --git a/tests/masonry/builders/test_editable_builder.py b/tests/masonry/builders/test_editable_builder.py
--- a/tests/masonry/builders/test_editable_builder.py
+++ b/tests/masonry/builders/test_editable_builder.py
@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import csv
 import json
 import os
 import shutil
@@ -158,17 +159,22 @@ def test_builder_installs_proper_files_for_standard_packages(
 """
     assert metadata == dist_info.joinpath("METADATA").read_text(encoding="utf-8")
 
-    records = dist_info.joinpath("RECORD").read_text()
+    with open(dist_info.joinpath("RECORD"), encoding="utf-8", newline="") as f:
+        reader = csv.reader(f)
+        records = list(reader)
+
+    assert all(len(row) == 3 for row in records)
+    record_entries = {row[0] for row in records}
     pth_file = "simple_project.pth"
     assert tmp_venv.site_packages.exists(pth_file)
-    assert str(tmp_venv.site_packages.find(pth_file)[0]) in records
-    assert str(tmp_venv._bin_dir.joinpath("foo")) in records
-    assert str(tmp_venv._bin_dir.joinpath("baz")) in records
-    assert str(dist_info.joinpath("METADATA")) in records
-    assert str(dist_info.joinpath("INSTALLER")) in records
-    assert str(dist_info.joinpath("entry_points.txt")) in records
-    assert str(dist_info.joinpath("RECORD")) in records
-    assert str(dist_info.joinpath("direct_url.json")) in records
+    assert str(tmp_venv.site_packages.find(pth_file)[0]) in record_entries
+    assert str(tmp_venv._bin_dir.joinpath("foo")) in record_entries
+    assert str(tmp_venv._bin_dir.joinpath("baz")) in record_entries
+    assert str(dist_info.joinpath("METADATA")) in record_entries
+    assert str(dist_info.joinpath("INSTALLER")) in record_entries
+    assert str(dist_info.joinpath("entry_points.txt")) in record_entries
+    assert str(dist_info.joinpath("RECORD")) in record_entries
+    assert str(dist_info.joinpath("direct_url.json")) in record_entries
 
     baz_script = f"""\
 #!{tmp_venv.python}