Repository: jazzband/pip-tools

Issue:
[6.5.0] Packages needlessly uninstalled during sync due to non-canonicalized names (dots vs dashes)
Packages with dots in them are not compiled to the same file in version 6.4.0 and 6.5.0, where dots are replaced with dashes.

E.g. `zope.event` becomes `zope-event`



#### Environment Versions

1. OS Type: Arch Linux
1. Python version: `Python 3.10.2`
1. pip version: `pip 20.3.4 from /usr/lib/python3.10/site-packages/pip (python 3.10)`
1. pip-tools version: `pip-compile, version 6.5.0`

#### Steps to replicate

<details>
  <summary>On version 6.4.0</summary>

 - `requirements.in`

```
pip-tools==6.4.0
gevent==21.12.0
```

 - `requirements.txt` (after `pip-compile`)

```
#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
#
#    pip-compile
#
click==8.0.3
    # via pip-tools
gevent==21.12.0
    # via -r requirements.in
greenlet==1.1.2
    # via gevent
pep517==0.12.0
    # via pip-tools
pip-tools==6.4.0
    # via -r requirements.in
tomli==2.0.0
    # via pep517
wheel==0.37.1
    # via pip-tools
zope.event==4.5.0
    # via gevent
zope.interface==5.4.0
    # via gevent

# The following packages are considered to be unsafe in a requirements file:
# pip
# setuptools
```

</details>

<details>
  <summary>On version 6.5.0</summary>


 - `requirements.in`

```
pip-tools==6.5.0
gevent==21.12.0
```

 - `requirements.txt` (after `pip-compile`)

```
#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
#
#    pip-compile
#
click==8.0.3
    # via pip-tools
gevent==21.12.0
    # via -r requirements.in
greenlet==1.1.2
    # via gevent
pep517==0.12.0
    # via pip-tools
pip-tools==6.5.0
    # via -r requirements.in
tomli==2.0.0
    # via pep517
wheel==0.37.1
    # via pip-tools
zope-event==4.5.0
    # via gevent
zope-interface==5.4.0
    # via gevent

# The following packages are considered to be unsafe in a requirements file:
# pip
# setuptools
```

</details>

#### Expected result

 - The `zope.event` package being listed as `zope.event` in generated `requirements.txt`
 - Running `pip-sync` twice in a raw will not do anything on second run (except printing `Everything up-to-date`


#### Actual result

 - The `zope.event` package has been renamed to `zope-event` in generated `requirements.txt`
 - Running `pip-sync` does the following even if nothing has changed (i.e. it does so in two consecutive runs)

```
Found existing installation: zope.event 4.5.0
Uninstalling zope.event-4.5.0:
  Successfully uninstalled zope.event-4.5.0
Found existing installation: zope.interface 5.4.0
Uninstalling zope.interface-5.4.0:
  Successfully uninstalled zope.interface-5.4.0
Collecting zope-event==4.5.0
  Using cached zope.event-4.5.0-py2.py3-none-any.whl (6.8 kB)
Collecting zope-interface==5.4.0
  Using cached zope.interface-5.4.0-cp310-cp310-linux_x86_64.whl
Requirement already satisfied: setuptools in ./env/lib/python3.10/site-packages (from zope-event==4.5.0->-r /tmp/tmpcg_uqvf4 (line 1)) (58.1.0)
Installing collected packages: zope-interface, zope-event
Successfully installed zope-event-4.5.0 zope-interface-5.4.0
WARNING: You are using pip version 21.2.4; however, version 22.0.3 is available.
You should consider upgrading via the '/tmp/foo/env/bin/python -m pip install --upgrade pip' command.
```


Discussion:
This is expected and explained at https://github.com/jazzband/pip-tools/pull/1567#issuecomment-1030069457
@webknjaz 

This is introduced there but can (I think) easily be resolved by using `canonicalize_name` in `key_from_req`. I have done so just now in the process of merging the new release back into #1329, and I think this particular valid issue is resolved in that PR.
Oh to be clear, I'm not referring to the issue as titled, but this part

> Running `pip-sync` does the following even if nothing has changed (i.e. it does so in two consecutive runs)

I may re-title this.
The fix specifically, in `key_from_req`:

```diff
-    key = key.replace("_", "-").lower()
-    return key
+    return str(canonicalize_name(key))
```
> This is expected and explained at https://github.com/jazzband/pip-tools/pull/1567#issuecomment-1030069457

Thank you for the pointer. I hope that having created this issue will point users that are searching for keywords like `dot` or `dash` will arrive here now! :smiley: 

---

I'm confirming that #1329 is fixing the `pip-sync` consecutive runs issue.

The `.` are still converted into `-` but you know better than I do about this one, so it's ok with me.

Many thanks!
Given how large #1329 is, I think it probably makes sense to pull [1eac955](https://github.com/jazzband/pip-tools/pull/1329/commits/1eac9553257be5d3f8b7d5e07eefb3c50c92fd8f) out into a separate PR (and add a test 😉) in order to get it out sooner than later.
I'll take it out and rebase if someone merges the tiny change alone, it should have been part of the recent merge.

If no one does by the time I get back to this, I'll submit a small PR myself. 

Resolving patch:
diff --git a/piptools/utils.py b/piptools/utils.py
--- a/piptools/utils.py
+++ b/piptools/utils.py
@@ -69,9 +69,7 @@ def key_from_req(
     else:
         # from packaging, such as install requirements from requirements.txt
         key = req.name
-    assert isinstance(key, str)
-    key = key.replace("_", "-").lower()
-    return key
+    return str(canonicalize_name(key))
 
 
 def comment(text: str) -> str:


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -22,6 +22,7 @@
     get_sys_path_for_python_executable,
     is_pinned_requirement,
     is_url_requirement,
+    key_from_ireq,
     lookup_table,
     lookup_table_from_tuples,
 )
@@ -266,6 +267,13 @@ def test_is_pinned_requirement_editable(from_editable):
     assert not is_pinned_requirement(ireq)
 
 
+def test_key_from_ireq_normalization(from_line):
+    keys = set()
+    for line in ("zope.event", "zope-event", "zope_event", "ZOPE.event"):
+        keys.add(key_from_ireq(from_line(line)))
+    assert len(keys) == 1
+
+
 @pytest.mark.parametrize(
     ("line", "expected"),
     (