Repository: python-poetry/poetry

Issue:
1.1.3 regression. poetry exports wrong constraints
<!--
  Hi there! Thank you for discovering and submitting an issue.

  Before you submit this; let's make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill these first.
-->

<!-- Checked checkbox should look like this: [x] -->
- [x] I am on the [latest](https://github.com/python-poetry/poetry/releases/latest) Poetry version.
- [x] I have searched the [issues](https://github.com/python-poetry/poetry/issues) of this repo and believe that this is not a duplicate.
- [x] If an exception occurs when executing a command, I executed it again in debug mode (`-vvv` option).

<!--
  Once those are done, if you're able to fill in the following list with your information,
  it'd be very helpful to whoever handles the issue.
-->

- **OS version and name**: Fedora 33
- **Poetry version**: 1.1.3
- **Link of a [Gist](https://gist.github.com/) with the contents of your pyproject.toml file**: https://gist.github.com/apollo13/cda3ea5e3a135d67eeed4be099bcb7e9

## Issue

The linked pyproject.toml generates a broken requirements.txt (it wrongly applies `sys_platform=darwin`). But first off lets see the graph:
```
pyinstaller 4.0 PyInstaller bundles a Python application and all its dependencies into a single package.
├── altgraph *
├── macholib >=1.8
│   └── altgraph >=0.15 
├── pefile >=2017.8.1
│   └── future * 
├── pyinstaller-hooks-contrib >=2020.6
└── pywin32-ctypes >=0.2.0
```

as we can see the toplevel dependency `pyinstaller` directly requires `althgraph` and then `macholib` also requires `altgraph`.

Looking at `poetry.lock` we see:
```
[[package]]
name = "pyinstaller"
version = "4.0"
description = "PyInstaller bundles a Python application and all its dependencies into a single package."
category = "main"
optional = false
python-versions = "*"

[package.dependencies]
altgraph = "*"
macholib = {version = ">=1.8", markers = "sys_platform == \"darwin\""}
pefile = {version = ">=2017.8.1", markers = "sys_platform == \"win32\""}
pyinstaller-hooks-contrib = ">=2020.6"
pywin32-ctypes = {version = ">=0.2.0", markers = "sys_platform == \"win32\""}

[[package]]
name = "macholib"
version = "1.14"
description = "Mach-O header analysis and editing"
category = "main"
optional = false
python-versions = "*"

[package.dependencies]
altgraph = ">=0.15"
```

This tells us that `pyinstaller` depends on `macholib` on darwin systems. The export looks like this:
```
altgraph==0.17; sys_platform == "darwin" \
    --hash=sha256:c623e5f3408ca61d4016f23a681b9adb100802ca3e3da5e718915a9e4052cebe \
    --hash=sha256:1f05a47122542f97028caf78775a095fbe6a2699b5089de8477eb583167d69aa
future==0.18.2; python_version >= "2.6" and python_full_version < "3.0.0" and sys_platform == "win32" or python_full_version >= "3.3.0" and sys_platform == "win32" \
    --hash=sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d
macholib==1.14; sys_platform == "darwin" \
    --hash=sha256:c500f02867515e6c60a27875b408920d18332ddf96b4035ef03beddd782d4281 \
    --hash=sha256:0c436bc847e7b1d9bda0560351bf76d7caf930fb585a828d13608839ef42c432
pefile==2019.4.18; sys_platform == "win32" \
    --hash=sha256:a5d6e8305c6b210849b47a6174ddf9c452b2888340b8177874b862ba6c207645
pyinstaller-hooks-contrib==2020.9 \
    --hash=sha256:a5fd45a920012802e3f2089e1d3501ef2f49265dfea8fc46c3310f18e3326c91 \
    --hash=sha256:c382f3ac1a42b45cfecd581475c36db77da90e479b2f5bcb6d840d21fa545114
pyinstaller==4.0 \
    --hash=sha256:970beb07115761d5e4ec317c1351b712fd90ae7f23994db914c633281f99bab0
pywin32-ctypes==0.2.0; sys_platform == "win32" \
    --hash=sha256:24ffc3b341d457d48e8922352130cf2644024a4ff09762a2261fd34c36ee5942 \
    --hash=sha256:9dc2d991b3479cc2df15930958b674a48a227d5361d413827a4cfd0b5876fc98
```

Here `altgraph` is incorrectly labeled as `darwin`, propbably due to the fact that `macholib` is only required on `darwin` and as such the constraints trickles down. This makes sense but should not happen if there is another dependency on the same package.



Discussion:
I have the same problem with the `six` package on Windows:

    [tool.poetry]
    name = "test"
    version = "0.4.0"
    description = ""
    authors = [""]
    
    [tool.poetry.dependencies]
    python = ">=3.6"
    junit-xml = "*"
    poetry = "^1.1.4"
    
    [build-system]
    requires = ["poetry_core>=1.0.0"]
    build-backend = "poetry.core.masonry.api"

In this scenario, `junit-xml` requires `six` on all platforms.

But `poetry <- keyring <- secretstorage <- cryptography <- six` markers are merged to the `six` DependencyPackage instance [here](https://github.com/python-poetry/poetry/blob/369ced14be5218a6237d8eb0a579f6592fd922ee/poetry/packages/locker.py#L269) which transforms `six` into a linux-only package, because `keyring` needs `secretstorage` on linux only.
I feel like I start recognizing a pattern here... There is at least 1 (maybe 2) other issue like that, where something similar seems to happen. Two unrelated dependencies on the same library are incorrectly merged.
I don't understand why markers should be propagated up the dependency tree. It looks like https://github.com/python-poetry/poetry/blob/de0b32c245c72568cf546090600d4626917cd3a4/poetry/packages/locker.py#L514 suggests that a marker from a dependency is a constraint for a package. But if the dependency is optional then should it even matter?
@jonapich My case got fixed by the following diff:
```
diff --git a/poetry/packages/locker.py b/poetry/packages/locker.py
index 48d68e95..f06d36d6 100644
--- a/poetry/packages/locker.py
+++ b/poetry/packages/locker.py
@@ -269,9 +269,9 @@ class Locker(object):
             if key not in nested_dependencies:
                 nested_dependencies[key] = requirement
             else:
-                nested_dependencies[key].marker = nested_dependencies[
-                    key
-                ].marker.intersect(requirement.marker)
+                nested_dependencies[key].marker = nested_dependencies[key].marker.union(
+                    requirement.marker
+                )
 
         return cls.__walk_dependency_level(
             dependencies=next_level_dependencies,
```

Could you try if this also fixes your case?

Resolving patch:
diff --git a/poetry/packages/locker.py b/poetry/packages/locker.py
--- a/poetry/packages/locker.py
+++ b/poetry/packages/locker.py
@@ -269,9 +269,9 @@ def __walk_dependency_level(
             if key not in nested_dependencies:
                 nested_dependencies[key] = requirement
             else:
-                nested_dependencies[key].marker = nested_dependencies[
-                    key
-                ].marker.intersect(requirement.marker)
+                nested_dependencies[key].marker = nested_dependencies[key].marker.union(
+                    requirement.marker
+                )
 
         return cls.__walk_dependency_level(
             dependencies=next_level_dependencies,


Test patch:
diff --git a/tests/utils/test_exporter.py b/tests/utils/test_exporter.py
--- a/tests/utils/test_exporter.py
+++ b/tests/utils/test_exporter.py
@@ -175,6 +175,200 @@ def test_exporter_can_export_requirements_txt_with_standard_packages_and_markers
     assert expected == content
 
 
+def test_exporter_can_export_requirements_txt_poetry(tmp_dir, poetry):
+    """Regression test for #3254"""
+
+    poetry.locker.mock_lock_data(
+        {
+            "package": [
+                {
+                    "name": "poetry",
+                    "version": "1.1.4",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                    "dependencies": {"keyring": "*"},
+                },
+                {
+                    "name": "junit-xml",
+                    "version": "1.9",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                    "dependencies": {"six": "*"},
+                },
+                {
+                    "name": "keyring",
+                    "version": "21.8.0",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                    "dependencies": {
+                        "SecretStorage": {
+                            "version": "*",
+                            "markers": "sys_platform == 'linux'",
+                        }
+                    },
+                },
+                {
+                    "name": "secretstorage",
+                    "version": "3.3.0",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                    "dependencies": {"cryptography": "*"},
+                },
+                {
+                    "name": "cryptography",
+                    "version": "3.2",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                    "dependencies": {"six": "*"},
+                },
+                {
+                    "name": "six",
+                    "version": "1.15.0",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                },
+            ],
+            "metadata": {
+                "python-versions": "*",
+                "content-hash": "123456789",
+                "hashes": {
+                    "poetry": [],
+                    "keyring": [],
+                    "secretstorage": [],
+                    "cryptography": [],
+                    "six": [],
+                    "junit-xml": [],
+                },
+            },
+        }
+    )
+    set_package_requires(
+        poetry, skip={"keyring", "secretstorage", "cryptography", "six"}
+    )
+
+    exporter = Exporter(poetry)
+
+    exporter.export("requirements.txt", Path(tmp_dir), "requirements.txt")
+
+    with (Path(tmp_dir) / "requirements.txt").open(encoding="utf-8") as f:
+        content = f.read()
+
+    # The dependency graph:
+    # junit-xml 1.9 Creates JUnit XML test result documents that can be read by tools such as Jenkins
+    # └── six *
+    # poetry 1.1.4 Python dependency management and packaging made easy.
+    # ├── keyring >=21.2.0,<22.0.0
+    # │   ├── importlib-metadata >=1
+    # │   │   └── zipp >=0.5
+    # │   ├── jeepney >=0.4.2
+    # │   ├── pywin32-ctypes <0.1.0 || >0.1.0,<0.1.1 || >0.1.1
+    # │   └── secretstorage >=3.2 -- On linux only
+    # │       ├── cryptography >=2.0
+    # │       │   └── six >=1.4.1
+    # │       └── jeepney >=0.6 (circular dependency aborted here)
+    expected = {
+        "poetry": dependency_from_pep_508("poetry==1.1.4"),
+        "junit-xml": dependency_from_pep_508("junit-xml==1.9"),
+        "keyring": dependency_from_pep_508("keyring==21.8.0"),
+        "secretstorage": dependency_from_pep_508(
+            "secretstorage==3.3.0; sys_platform=='linux'"
+        ),
+        "cryptography": dependency_from_pep_508(
+            "cryptography==3.2; sys_platform=='linux'"
+        ),
+        "six": dependency_from_pep_508("six==1.15.0"),
+    }
+
+    for line in content.strip().split("\n"):
+        dependency = dependency_from_pep_508(line)
+        assert dependency.name in expected
+        expected_dependency = expected.pop(dependency.name)
+        assert dependency == expected_dependency
+        print(dependency.marker)
+        print(expected_dependency.marker)
+        assert dependency.marker == expected_dependency.marker
+
+
+def test_exporter_can_export_requirements_txt_pyinstaller(tmp_dir, poetry):
+    """Regression test for #3254"""
+
+    poetry.locker.mock_lock_data(
+        {
+            "package": [
+                {
+                    "name": "pyinstaller",
+                    "version": "4.0",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                    "dependencies": {
+                        "altgraph": "*",
+                        "macholib": {
+                            "version": "*",
+                            "markers": "sys_platform == 'darwin'",
+                        },
+                    },
+                },
+                {
+                    "name": "altgraph",
+                    "version": "0.17",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                },
+                {
+                    "name": "macholib",
+                    "version": "1.8",
+                    "category": "main",
+                    "optional": False,
+                    "python-versions": "*",
+                    "dependencies": {"altgraph": ">=0.15"},
+                },
+            ],
+            "metadata": {
+                "python-versions": "*",
+                "content-hash": "123456789",
+                "hashes": {"pyinstaller": [], "altgraph": [], "macholib": []},
+            },
+        }
+    )
+    set_package_requires(poetry, skip={"altgraph", "macholib"})
+
+    exporter = Exporter(poetry)
+
+    exporter.export("requirements.txt", Path(tmp_dir), "requirements.txt")
+
+    with (Path(tmp_dir) / "requirements.txt").open(encoding="utf-8") as f:
+        content = f.read()
+
+    # Rationale for the results:
+    #  * PyInstaller has an explicit dependency on altgraph, so it must always be installed.
+    #  * PyInstaller requires macholib on Darwin, which in turn requires altgraph.
+    # The dependency graph:
+    # pyinstaller 4.0 PyInstaller bundles a Python application and all its dependencies into a single package.
+    # ├── altgraph *
+    # ├── macholib >=1.8 -- only on Darwin
+    # │   └── altgraph >=0.15
+    expected = {
+        "pyinstaller": dependency_from_pep_508("pyinstaller==4.0"),
+        "altgraph": dependency_from_pep_508("altgraph==0.17"),
+        "macholib": dependency_from_pep_508("macholib==1.8; sys_platform == 'darwin'"),
+    }
+
+    for line in content.strip().split("\n"):
+        dependency = dependency_from_pep_508(line)
+        assert dependency.name in expected
+        expected_dependency = expected.pop(dependency.name)
+        assert dependency == expected_dependency
+        assert dependency.marker == expected_dependency.marker
+
+
 def test_exporter_can_export_requirements_txt_with_nested_packages_and_markers(
     tmp_dir, poetry
 ):
@@ -241,7 +435,7 @@ def test_exporter_can_export_requirements_txt_with_nested_packages_and_markers(
             "c==7.8.9; sys_platform == 'win32' and python_version < '3.7'"
         ),
         "d": dependency_from_pep_508(
-            "d==0.0.1; python_version < '3.7' and platform_system == 'Windows' and sys_platform == 'win32'"
+            "d==0.0.1; platform_system == 'Windows' and python_version < '3.7' or sys_platform == 'win32' and python_version < '3.7'"
         ),
     }