Repository: jazzband/pip-tools

Issue:
Order of packages in requirements.txt -- pip-compile vs. pip freeze
The issue regards packages with names containing `-` character or a digit. This may cause an unexpected order of packages in the output file, different than with `pip freeze`. It seems that the output file is sorted by the whole lines, together with `==` and version name, instead of just by the package names.

#### Environment Versions

1. Windows 10
2. Python version: 3.7.6
3. pip version: 21.1.2
4. pip-tools version: 6.1.0

#### Steps to replicate

`pip-compile` with `requirements.in` file:

```
django
django-redis
django-sendfile
django-sendfile2
djangorestframework
```

#### Expected result

(without comments and additional libraries, for clarity)

```
django==3.2.4
django-redis==5.0.0
django-sendfile==0.3.11
django-sendfile2==0.6.0
djangorestframework==3.12.4
```

#### Actual result

```
django-redis==5.0.0
django-sendfile2==0.6.0
django-sendfile==0.3.11
django==3.2.4
djangorestframework==3.12.4
```



Discussion:
I am not sure we really need a fix for this, it would add extra complexity for a very minor benefit.
It's certainly not a big issue, but it causes some inconveniences:
* When migrating from `pip freeze`, the shuffle of the packages creates a bigger diff than necessary, and thus makes it harder to review the changes introduced by `pip-compile`.
* When reading the generated list of packages, it's harder to find the one we're looking for, since we know how lexicographic order should work.

It's a bit surprising to read that fixing this would add extra complexity. I've looked at the code and it seems like the issue is here:

https://github.com/jazzband/pip-tools/blob/f953ceb8840b5b653ef94d7b9bc172192413e0a8/piptools/writer.py#L91-L92

I've tried changing locally `str(ireq.req).lower()` to `ireq.req.name.lower()` and it works like expected.
Yeah, this would make sense to sync packages ordering with `pip freeze`, which is [a case-insensitive sorted order](https://pip.pypa.io/en/stable/cli/pip_freeze/).

> I've tried changing locally `str(ireq.req).lower()` to `ireq.req.name.lower()` and it works like expected.

Try also `key_from_ireq(ireq))`.

@adamsol feel free to open a PR 👍🏻 

Resolving patch:
diff --git a/piptools/writer.py b/piptools/writer.py
--- a/piptools/writer.py
+++ b/piptools/writer.py
@@ -89,7 +89,7 @@ def __init__(
         self.emit_find_links = emit_find_links
 
     def _sort_key(self, ireq: InstallRequirement) -> Tuple[bool, str]:
-        return (not ireq.editable, str(ireq.req).lower())
+        return (not ireq.editable, key_from_ireq(ireq))
 
     def write_header(self) -> Iterator[str]:
         if self.emit_header:


Test patch:
diff --git a/tests/test_cli_compile.py b/tests/test_cli_compile.py
--- a/tests/test_cli_compile.py
+++ b/tests/test_cli_compile.py
@@ -827,10 +827,10 @@ def test_generate_hashes_with_annotations(runner):
 @pytest.mark.network
 def test_generate_hashes_with_long_annotations(runner):
     with open("requirements.in", "w") as fp:
+        fp.write("Django==1.11.29\n")
         fp.write("django-debug-toolbar==1.11\n")
         fp.write("django-storages==1.9.1\n")
         fp.write("django-taggit==0.24.0\n")
-        fp.write("Django==1.11.29\n")
         fp.write("pytz==2020.4\n")
         fp.write("sqlparse==0.3.1\n")
 
@@ -844,6 +844,14 @@ def test_generate_hashes_with_long_annotations(runner):
         #
         #    pip-compile --generate-hashes
         #
+        django==1.11.29 \\
+            --hash=sha256:014e3392058d94f40569206a24523ce254d55ad2f9f46c6550b0fe2e4f94cf3f \\
+            --hash=sha256:4200aefb6678019a0acf0005cd14cfce3a5e6b9b90d06145fcdd2e474ad4329c
+            # via
+            #   -r requirements.in
+            #   django-debug-toolbar
+            #   django-storages
+            #   django-taggit
         django-debug-toolbar==1.11 \\
             --hash=sha256:89d75b60c65db363fb24688d977e5fbf0e73386c67acf562d278402a10fc3736 \\
             --hash=sha256:c2b0134119a624f4ac9398b44f8e28a01c7686ac350a12a74793f3dd57a9eea0
@@ -856,14 +864,6 @@ def test_generate_hashes_with_long_annotations(runner):
             --hash=sha256:710b4d15ec1996550cc68a0abbc41903ca7d832540e52b1336e6858737e410d8 \\
             --hash=sha256:bb8f27684814cd1414b2af75b857b5e26a40912631904038a7ecacd2bfafc3ac
             # via -r requirements.in
-        django==1.11.29 \\
-            --hash=sha256:014e3392058d94f40569206a24523ce254d55ad2f9f46c6550b0fe2e4f94cf3f \\
-            --hash=sha256:4200aefb6678019a0acf0005cd14cfce3a5e6b9b90d06145fcdd2e474ad4329c
-            # via
-            #   -r requirements.in
-            #   django-debug-toolbar
-            #   django-storages
-            #   django-taggit
         pytz==2020.4 \\
             --hash=sha256:3e6b7dd2d1e0a59084bcee14a17af60c5c562cdc16d828e8eba2e683d3a7e268 \\
             --hash=sha256:5c55e189b682d420be27c6995ba6edce0c0a77dd67bfbe2ae6607134d5851ffd
diff --git a/tests/test_writer.py b/tests/test_writer.py
--- a/tests/test_writer.py
+++ b/tests/test_writer.py
@@ -344,3 +344,24 @@ def test_write_find_links(writer, find_links, expected_lines):
     """
     writer.find_links = find_links
     assert list(writer.write_find_links()) == expected_lines
+
+
+def test_write_order(writer, from_line):
+    """
+    Order of packages should match that of `pip freeze`.
+    """
+    writer.emit_header = False
+
+    packages = [
+        from_line("package_a==0.1"),
+        from_line("Package-b==2.3.4"),
+        from_line("Package==5.6"),
+        from_line("package2==7.8.9"),
+    ]
+    expected_lines = [
+        "package==5.6",
+        "package_a==0.1",
+        "package-b==2.3.4",
+        "package2==7.8.9",
+    ]
+    assert list(writer._iter_lines(packages)) == expected_lines