Repository: jazzband/pip-tools

Issue:
Sync raises IncompatibleRequirements even when environment markers indicate the incompatible requirements to be irrelevant for the current platform
IMO this is simply more of #206, and I've already demonstrated the problem there. But it's been a month and no one has re-opened, so I'm opening this more specific issue to address the problem.

Here's an example `dev-requirements.txt` from the `plumbum` project:

```python
pytest
pytest-cov
pytest-mock
idna<2.8 ; python_version < '2.7'
pycparser<2.18 ; python_version < '2.7'
paramiko<2.4 ; python_version < '2.7'
paramiko ; python_version >= '2.7'
setuptools
wheel ; python_version >= '2.7'
psutil
```

```bash
pip-sync dev-requirements.txt
```

Identical output whether in a Python `3.7.4` or `2.7.16` env:

```
Incompatible requirements found: paramiko (from -r dev-requirements.txt (line 7)) and paramiko<2.4 (from -r dev-requirements.txt (line 6))
```

No packages end up installed aside from `pip-tools` and its deps.

#### Environment Versions

1. Arch Linux
1. Python version: `3.7.4`
1. pip version:  `19.2.3`
1. pip-tools version: `4.1.0`

#### Steps to replicate

```bash
echo "paramiko==2.4.0 ; python_version < '2.7'" > mark.txt
echo "paramiko==2.6.0 ; python_version >= '2.7'" >> mark.txt
pip-sync mark.txt
```

Note that this works:

```bash
pip install --no-deps -r mark.txt
```

#### Expected result

`pip-sync` should ignore non-matching requirements when environment markers are present.

#### Actual result

`pip-sync` checks for conflicts as if it wants to install requirements for all platforms.

#### Further notes

```bash
mv mark.txt mark.in
pip-compile --no-header mark.in
```

```python
asn1crypto==1.0.1         # via cryptography
bcrypt==3.1.7             # via paramiko
cffi==1.12.3              # via bcrypt, cryptography, pynacl
cryptography==2.7         # via paramiko
paramiko==2.6.0 ; python_version >= "2.7"
pycparser==2.19           # via cffi
pynacl==1.3.0             # via paramiko
six==1.12.0               # via bcrypt, cryptography, pynacl
```

Currently, compiling such an in-file will only include the compile-time platform's matching reqs. This hides the issue under discussion, and arguably means it's not a bug. But I believe it is generally desired for pip-sync to honor environment markers, as evidenced by the contents of #206 (closed, but not solved), #600 (merged), #459 (replaced), #460 (merged), #518 (open 2yrs), #563 (open 2yrs), #585 (open 2yrs), #896 (open), etc.

This is probably even more relevant for working with a single python version across different platforms.


Discussion:

Resolving patch:
diff --git a/piptools/sync.py b/piptools/sync.py
--- a/piptools/sync.py
+++ b/piptools/sync.py
@@ -82,18 +82,19 @@ def merge(requirements, ignore_conflicts):
         # Limitation: URL requirements are merged by precise string match, so
         # "file:///example.zip#egg=example", "file:///example.zip", and
         # "example==1.0" will not merge with each other
-        key = key_from_ireq(ireq)
-
-        if not ignore_conflicts:
-            existing_ireq = by_key.get(key)
-            if existing_ireq:
-                # NOTE: We check equality here since we can assume that the
-                # requirements are all pinned
-                if ireq.specifier != existing_ireq.specifier:
-                    raise IncompatibleRequirements(ireq, existing_ireq)
-
-        # TODO: Always pick the largest specifier in case of a conflict
-        by_key[key] = ireq
+        if ireq.match_markers():
+            key = key_from_ireq(ireq)
+
+            if not ignore_conflicts:
+                existing_ireq = by_key.get(key)
+                if existing_ireq:
+                    # NOTE: We check equality here since we can assume that the
+                    # requirements are all pinned
+                    if ireq.specifier != existing_ireq.specifier:
+                        raise IncompatibleRequirements(ireq, existing_ireq)
+
+            # TODO: Always pick the largest specifier in case of a conflict
+            by_key[key] = ireq
     return by_key.values()
 
 


Test patch:
diff --git a/tests/test_cli_sync.py b/tests/test_cli_sync.py
--- a/tests/test_cli_sync.py
+++ b/tests/test_cli_sync.py
@@ -93,21 +93,34 @@ def test_force_files_with_dot_in_extension(runner):
     assert out.exit_code == 0
 
 
-def test_merge_error(runner):
+@pytest.mark.parametrize(
+    ("req_lines", "should_raise"),
+    [
+        (["six>1.10.0", "six<1.10.0"], True),
+        (
+            ["six>1.10.0 ; python_version>='3.0'", "six<1.10.0 ; python_version<'3.0'"],
+            False,
+        ),
+    ],
+)
+def test_merge_error(req_lines, should_raise, runner):
     """
     Sync command should raise an error if there are merge errors.
+    It should not raise an error if otherwise incompatible requirements
+    are isolated by exclusive environment markers.
     """
     with open("requirements.txt", "w") as req_in:
-        req_in.write("six>1.10.0\n")
-
-        # Add incompatible package
-        req_in.write("six<1.10.0")
+        for line in req_lines:
+            req_in.write(line + "\n")
 
     with mock.patch("piptools.sync.check_call"):
         out = runner.invoke(cli, ["-n"])
 
-    assert out.exit_code == 2
-    assert "Incompatible requirements found" in out.stderr
+    if should_raise:
+        assert out.exit_code == 2
+        assert "Incompatible requirements found" in out.stderr
+    else:
+        assert out.exit_code == 0
 
 
 @pytest.mark.parametrize(