Repository: pantsbuild/pants

Issue:
Pants fails to generate coverage for Python modules importing PySide2
When running tests with Pants and generating coverage

```
$ ./pants test :: --print-stacktrace --no-process-execution-local-cleanup
```

an exception is raised:

```
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Generate Pytest html coverage report.' failed with exit code 1.
stdout:
No source for code: '/private/var/folders/k0/3nssm8v15r5b08k1k1wyhs2r0000gp/T/process-executionNiGXI5/(builtin)'.
Aborting report output, consider using -i.

stderr:
```

The tests pass, but the coverage report generation fails.

Please see a minimal monorepo to reproduce Pants issue with `coverage.py` here: https://github.com/alexey-tereshenkov-oxb/monorepo-coverage-pants




Discussion:
The minimal repro actually repros directly with coverage as such:
```
$ cp /tmp/process-executionPHckm6/.coveragerc .
$ PYTHONPATH="./minimalcov:$PYTHONPATH" coverage run --rcfile .coveragerc -m pytest minimalcov/minimalcov/tests; coverage html
==================================================================================================================================== test session starts =====================================================================================================================================
platform linux -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/jsirois/dev/alexey-tereshenkov-oxb/monorepo-coverage-pants
collected 1 item                                                                                                                                                                                                                                                                             

minimalcov/minimalcov/tests/test_foo.py .                                                                                                                                                                                                                                              [100%]

===================================================================================================================================== 1 passed in 0.07s ======================================================================================================================================
No source for code: '/home/jsirois/dev/alexey-tereshenkov-oxb/monorepo-coverage-pants/(builtin)'.
Aborting report output, consider using -i.
```

So the issue is involved in the way we setup coverage although I'm not sure what that issue is yet:
```
$ cat .coveragerc 
[run]
relative_files = True
omit = 
	pytest.pex/*

```
You get a "(builtin)" entry in the coverage data you don't get otherwise:
```
$ sqlite3 .coverage
SQLite version 3.36.0 2021-06-18 18:36:39
Enter ".help" for usage hints.
sqlite> select * from file;
1|/home/jsirois/.local/lib/python3.9/site-packages/pytest/__init__.py
2|/home/jsirois/.local/lib/python3.9/site-packages/pytest/collect.py
3|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/__init__.py
4|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_version.py
5|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/deprecated.py
6|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/warning_types.py
7|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/compat.py
8|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/outcomes.py
9|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/assertion/__init__.py
10|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/assertion/rewrite.py
11|/home/jsirois/.local/lib/python3.9/site-packages/py/__init__.py
12|/home/jsirois/.local/lib/python3.9/site-packages/py/_error.py
13|/home/jsirois/.local/lib/python3.9/site-packages/py/_vendored_packages/__init__.py
14|/home/jsirois/.local/lib/python3.9/site-packages/py/_vendored_packages/apipkg/__init__.py
15|/home/jsirois/.local/lib/python3.9/site-packages/py/_vendored_packages/apipkg/version.py
16|/home/jsirois/.local/lib/python3.9/site-packages/py/_version.py
17|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_io/__init__.py
18|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_io/terminalwriter.py
19|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_io/wcwidth.py
20|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_io/saferepr.py
21|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/assertion/util.py
22|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_code/__init__.py
23|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_code/code.py
24|/home/jsirois/.local/lib/python3.9/site-packages/pluggy/__init__.py
25|/home/jsirois/.local/lib/python3.9/site-packages/pluggy/_version.py
26|/home/jsirois/.local/lib/python3.9/site-packages/pluggy/manager.py
27|/home/jsirois/.local/lib/python3.9/site-packages/pluggy/_tracing.py
28|/home/jsirois/.local/lib/python3.9/site-packages/pluggy/callers.py
29|/home/jsirois/.local/lib/python3.9/site-packages/pluggy/hooks.py
30|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_code/source.py
31|/home/jsirois/.local/lib/python3.9/site-packages/py/_path/__init__.py
32|/home/jsirois/.local/lib/python3.9/site-packages/py/_path/local.py
33|/home/jsirois/.local/lib/python3.9/site-packages/py/_path/common.py
34|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/config/__init__.py
35|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/hookspec.py
36|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/config/exceptions.py
37|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/config/findpaths.py
38|/home/jsirois/.local/lib/python3.9/site-packages/iniconfig/__init__.py
39|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/pathlib.py
40|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/store.py
41|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/main.py
42|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/nodes.py
43|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/mark/__init__.py
44|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/mark/expression.py
45|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/mark/structures.py
46|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/config/argparsing.py
47|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/fixtures.py
48|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/reports.py
49|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/runner.py
50|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/timing.py
51|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/assertion/truncate.py
52|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/cacheprovider.py
53|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/python.py
54|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/capture.py
55|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/debugging.py
56|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/freeze_support.py
57|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/logging.py
58|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/terminal.py
59|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/monkeypatch.py
60|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/pytester.py
61|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/tmpdir.py
62|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/python_api.py
63|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/recwarn.py
64|/home/jsirois/.local/lib/python3.9/site-packages/pytest/__main__.py
65|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/helpconfig.py
66|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/unittest.py
67|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/skipping.py
68|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/pastebin.py
69|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/nose.py
70|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/junitxml.py
71|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/doctest.py
72|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/setuponly.py
73|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/setupplan.py
74|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/stepwise.py
75|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/warnings.py
76|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/unraisableexception.py
77|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/threadexception.py
78|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/faulthandler.py
79|/home/jsirois/.local/lib/python3.9/site-packages/_pytest/_argcomplete.py
80|/home/jsirois/.local/lib/python3.9/site-packages/py/_builtin.py
81|/home/jsirois/.local/lib/python3.9/site-packages/py/_code/__init__.py
82|/home/jsirois/.local/lib/python3.9/site-packages/py/_code/code.py
83|minimalcov/minimalcov/__init__.py
84|minimalcov/minimalcov/tests/__init__.py
85|minimalcov/minimalcov/tests/test_foo.py
86|minimalcov/minimalcov/src/__init__.py
87|minimalcov/minimalcov/src/foo.py
88|/home/jsirois/.local/lib/python3.9/site-packages/PySide2/__init__.py
89|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/__init__.py
90|(builtin)
91|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/__init__.py
92|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/__init__.py
93|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/loader.py
94|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/__feature__.py
95|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/mapping.py
96|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/lib/__init__.py
97|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/lib/tool.py
98|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/errorhandler.py
99|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/layout.py
100|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/parser.py
101|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/importhandler.py
102|/home/jsirois/.local/lib/python3.9/site-packages/PySide2/support/__init__.py
103|/home/jsirois/.local/lib/python3.9/site-packages/PySide2/support/deprecated.py
104|/home/jsirois/.local/lib/python3.9/site-packages/shiboken2/files.dir/shibokensupport/signature/lib/enum_sig.py
sqlite> 
```
Ok. Error persists adding `--source` back in:
```
$ PYTHONPATH="./minimalcov:$PYTHONPATH" coverage run --rcfile .coveragerc --source . -m pytest minimalcov/minimalcov/tests; coverage html
==================================================================================================================================== test session starts =====================================================================================================================================
platform linux -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/jsirois/dev/alexey-tereshenkov-oxb/monorepo-coverage-pants
collected 1 item                                                                                                                                                                                                                                                                             

minimalcov/minimalcov/tests/test_foo.py .                                                                                                                                                                                                                                              [100%]

===================================================================================================================================== 1 passed in 0.07s ======================================================================================================================================
No source for code: '/home/jsirois/dev/alexey-tereshenkov-oxb/monorepo-coverage-pants/(builtin)'.
Aborting report output, consider using -i.
```

But then goes away with the smallest possible restriction:
```
$ PYTHONPATH="./minimalcov:$PYTHONPATH" coverage run --rcfile .coveragerc --source minimalcov/ -m pytest minimalcov/minimalcov/tests; coverage html
==================================================================================================================================== test session starts =====================================================================================================================================
platform linux -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/jsirois/dev/alexey-tereshenkov-oxb/monorepo-coverage-pants
collected 1 item                                                                                                                                                                                                                                                                             

minimalcov/minimalcov/tests/test_foo.py .                                                                                                                                                                                                                                              [100%]

===================================================================================================================================== 1 passed in 0.07s ======================================================================================================================================
```
And now coverage data is more sane:
```
$ sqlite3 .coverage
SQLite version 3.36.0 2021-06-18 18:36:39
Enter ".help" for usage hints.
sqlite> select * from file;
1|minimalcov/minimalcov/__init__.py
2|minimalcov/minimalcov/tests/__init__.py
3|minimalcov/minimalcov/tests/test_foo.py
4|minimalcov/minimalcov/src/__init__.py
5|minimalcov/minimalcov/src/foo.py
sqlite> 

```

Of course this is all running pytest via coverage. Pants runs coverage via pytest (pytest-cov) - so there may need to be some juggling of our sandbox structure under test to get this working. I'll poke.

Resolving patch:
diff --git a/src/python/pants/backend/python/goals/coverage_py.py b/src/python/pants/backend/python/goals/coverage_py.py
--- a/src/python/pants/backend/python/goals/coverage_py.py
+++ b/src/python/pants/backend/python/goals/coverage_py.py
@@ -122,11 +122,12 @@ def register_options(cls, register):
             member_type=str,
             default=None,
             help=(
-                "A list of Python modules to use in the coverage report, e.g. "
-                "`['helloworld_test', 'helloworld.util.dirutil'].\n\nThe modules are recursive: "
-                "any submodules will be included.\n\nIf you leave this off, the coverage report "
-                "will include every file in the transitive closure of the address/file arguments; "
-                "for example, `test ::` will include every Python file in your project, whereas "
+                "A list of Python modules or filesystem paths to use in the coverage report, e.g. "
+                "`['helloworld_test', 'helloworld/util/dirutil'].\n\nBoth modules and directory "
+                "paths are recursive: any submodules or child paths, respectively, will be "
+                "included.\n\nIf you leave this off, the coverage report will include every file "
+                "in the transitive closure of the address/file arguments; for example, `test ::` "
+                "will include every Python file in your project, whereas "
                 "`test project/app_test.py` will include `app_test.py` and any of its transitive "
                 "dependencies."
             ),
diff --git a/src/python/pants/backend/python/goals/pytest_runner.py b/src/python/pants/backend/python/goals/pytest_runner.py
--- a/src/python/pants/backend/python/goals/pytest_runner.py
+++ b/src/python/pants/backend/python/goals/pytest_runner.py
@@ -1,7 +1,6 @@
 # Copyright 2018 Pants project contributors (see CONTRIBUTORS.md).
 # Licensed under the Apache License, Version 2.0 (see LICENSE).
 
-import itertools
 import logging
 from abc import ABC, abstractmethod
 from dataclasses import dataclass
@@ -278,11 +277,21 @@ async def setup_pytest_for_target(
     coverage_args = []
     if test_subsystem.use_coverage and not request.is_debug:
         output_files.append(".coverage")
-        cov_paths = coverage_subsystem.filter if coverage_subsystem.filter else (".",)
+
+        if coverage_subsystem.filter:
+            cov_args = [f"--cov={morf}" for morf in coverage_subsystem.filter]
+        else:
+            # N.B.: Passing `--cov=` or `--cov=.` to communicate "record coverage for all sources"
+            # fails in certain contexts as detailed in:
+            #   https://github.com/pantsbuild/pants/issues/12390
+            # Instead we focus coverage on just the directories containing python source files
+            # materialized to the Process chroot.
+            cov_args = [f"--cov={source_root}" for source_root in prepared_sources.source_roots]
+
         coverage_args = [
             "--cov-report=",  # Turn off output.
             f"--cov-config={coverage_config.path}",
-            *itertools.chain.from_iterable(["--cov", cov_path] for cov_path in cov_paths),
+            *cov_args,
         ]
 
     extra_env = {


Test patch:
diff --git a/src/python/pants/backend/python/goals/coverage_py_integration_test.py b/src/python/pants/backend/python/goals/coverage_py_integration_test.py
--- a/src/python/pants/backend/python/goals/coverage_py_integration_test.py
+++ b/src/python/pants/backend/python/goals/coverage_py_integration_test.py
@@ -140,13 +140,14 @@ def test_coverage() -> None:
             {tmpdir}/src/python/project/__init__.py                        0      0   100%
             {tmpdir}/src/python/project/lib.py                             6      0   100%
             {tmpdir}/src/python/project/lib_test.py                        3      0   100%
+            {tmpdir}/src/python/project/random.py                          2      2     0%
             {tmpdir}/tests/python/project_test/__init__.py                 0      0   100%
             {tmpdir}/tests/python/project_test/no_src/__init__.py          0      0   100%
             {tmpdir}/tests/python/project_test/no_src/test_no_src.py       2      0   100%
             {tmpdir}/tests/python/project_test/test_arithmetic.py          3      0   100%
             {tmpdir}/tests/python/project_test/test_multiply.py            3      0   100%
             ---------------------------------------------------------------------------------
-            TOTAL                                                            17      0   100%
+            TOTAL                                                            19      2    89%
             """
         )
         in result.stderr
@@ -239,3 +240,58 @@ def test_coverage_html_xml_json() -> None:
     assert "Wrote json coverage report to `dist/coverage/python`" in result.stderr
     json_coverage = coverage_path / "coverage.json"
     assert json_coverage.exists() is True
+
+
+def test_default_coverage_issues_12390() -> None:
+    # N.B.: This ~replicates the repo used to reproduce this issue at
+    # https://github.com/alexey-tereshenkov-oxb/monorepo-coverage-pants.
+    files = {
+        "requirements.txt": "PySide2==5.15.2",
+        "BUILD": dedent(
+            """\
+            python_requirements(
+                module_mapping={{
+                    "PySide2": ["PySide2"],
+                }},
+            )
+            """
+        ),
+        "minimalcov/minimalcov/src/foo.py": 'print("In the foo module!")',
+        "minimalcov/minimalcov/src/BUILD": "python_library()",
+        "minimalcov/minimalcov/tests/test_foo.py": dedent(
+            """\
+            import minimalcov.src.foo
+
+            from PySide2.QtWidgets import QApplication
+
+            def test_1():
+                assert True
+            """
+        ),
+        "minimalcov/minimalcov/tests/BUILD": "python_tests()",
+    }
+    with setup_tmpdir(files) as tmpdir:
+        command = [
+            "--backend-packages=pants.backend.python",
+            "test",
+            "--use-coverage",
+            "::",
+            f"--source-root-patterns=['/{tmpdir}/minimalcov']",
+            "--coverage-py-report=raw",
+        ]
+        result = run_pants(command)
+        result.assert_success()
+
+    assert (
+        dedent(
+            f"""\
+            Name                                                  Stmts   Miss  Cover
+            -------------------------------------------------------------------------
+            {tmpdir}/minimalcov/minimalcov/src/foo.py              1      0   100%
+            {tmpdir}/minimalcov/minimalcov/tests/test_foo.py       4      0   100%
+            -------------------------------------------------------------------------
+            TOTAL                                                     5      0   100%
+            """
+        )
+        in result.stderr
+    ), result.stderr