Repository: cocotb/cocotb

Issue:
Cocotb overrides user coverage configuration
I have code coverage enabled in cocotb with the COVERAGE flag. I wanted to modify what it reports, so I created [a configuration file for coverage.py](https://coverage.readthedocs.io/en/6.4/config.html) and defined COVERAGE_RCFILE environment variable pointing to it. It worked except files I wanted to omit from the report were still appearing there. It turns out cocotb passes its own omit argument to coverage in the regression module, overriding the user configuration.

I think cocotb should leave all coverage configuration to the config file if COVERAGE_RCFILE is defined.
Cocotb overrides user coverage configuration
I have code coverage enabled in cocotb with the COVERAGE flag. I wanted to modify what it reports, so I created [a configuration file for coverage.py](https://coverage.readthedocs.io/en/6.4/config.html) and defined COVERAGE_RCFILE environment variable pointing to it. It worked except files I wanted to omit from the report were still appearing there. It turns out cocotb passes its own omit argument to coverage in the regression module, overriding the user configuration.

I think cocotb should leave all coverage configuration to the config file if COVERAGE_RCFILE is defined.


Discussion:
`COCOTB_LIBRARY_COVERAGE` collects coverage on the cocotb library, and `COVERAGE` collects user coverage omitting the library coverage. If you really want the cocotb library coverage included in your coverage, simply enable the library coverage and merge the coverage reports.
What I am trying to do is to omit other Python packages in addition to cocotb. In the .coveragerc file I have the local/lib folder omitted but the setting is not going through because of cocotb passing its own omit argument. As explained in [coverage.py docs](https://coverage.readthedocs.io/en/6.4.1/api_coverage.html): "Many of these arguments duplicate and override values that can be provided in a configuration file. Parameters that are missing here will use values from the config file."
I can't think of a way to get around that.
So how do you propose we best address this? Wouldn't specifying `COVERAGE_RCFILE` affect the configurations passed to both call to `coverage.coverage`? Should we go down to a single `coverage.coverage` call and have `COCOTB_LIBRARY_COVERAGE` and `COVERAGE` simply turn on arguments to a default call, but if `COVERAGE_RCFILE` is specified pass no arguments?
I think I see what you mean. So there are actually two `coverage.coverage` calls, one in regression module and one in \_\_init__ for the library coverage. And if, for example, you had Python site-packages folder omitted in a configuration file, then `COCOTB_LIBRARY_COVERAGE` doesn't work. Or passing the config file to both could cause some conflict between the measurements. To prevent that, you could pass `config_file=False` to the library `coverage.coverage`. For the regression one, I was thinking that if `COVERAGE_RCFILE` is specified then `coverage.coverage` should be passed no arguments. Else call like it is called right now.
I think that should work fine. We are open to PRs and 1.7 is closing in soon.
`COCOTB_LIBRARY_COVERAGE` collects coverage on the cocotb library, and `COVERAGE` collects user coverage omitting the library coverage. If you really want the cocotb library coverage included in your coverage, simply enable the library coverage and merge the coverage reports.
What I am trying to do is to omit other Python packages in addition to cocotb. In the .coveragerc file I have the local/lib folder omitted but the setting is not going through because of cocotb passing its own omit argument. As explained in [coverage.py docs](https://coverage.readthedocs.io/en/6.4.1/api_coverage.html): "Many of these arguments duplicate and override values that can be provided in a configuration file. Parameters that are missing here will use values from the config file."
I can't think of a way to get around that.
So how do you propose we best address this? Wouldn't specifying `COVERAGE_RCFILE` affect the configurations passed to both call to `coverage.coverage`? Should we go down to a single `coverage.coverage` call and have `COCOTB_LIBRARY_COVERAGE` and `COVERAGE` simply turn on arguments to a default call, but if `COVERAGE_RCFILE` is specified pass no arguments?
I think I see what you mean. So there are actually two `coverage.coverage` calls, one in regression module and one in \_\_init__ for the library coverage. And if, for example, you had Python site-packages folder omitted in a configuration file, then `COCOTB_LIBRARY_COVERAGE` doesn't work. Or passing the config file to both could cause some conflict between the measurements. To prevent that, you could pass `config_file=False` to the library `coverage.coverage`. For the regression one, I was thinking that if `COVERAGE_RCFILE` is specified then `coverage.coverage` should be passed no arguments. Else call like it is called right now.
I think that should work fine. We are open to PRs and 1.7 is closing in soon.

Resolving patch:
diff --git a/cocotb/__init__.py b/cocotb/__init__.py
--- a/cocotb/__init__.py
+++ b/cocotb/__init__.py
@@ -224,6 +224,7 @@ def _initialise_testbench(argv_):  # pragma: no cover
             global _library_coverage
             _library_coverage = coverage.coverage(
                 data_file=".coverage.cocotb",
+                config_file=False,
                 branch=True,
                 include=["{}/*".format(os.path.dirname(__file__))],
             )
diff --git a/cocotb/regression.py b/cocotb/regression.py
--- a/cocotb/regression.py
+++ b/cocotb/regression.py
@@ -121,9 +121,16 @@ def __init__(self, dut: SimHandle, tests: Iterable[Test]):
 
         if coverage is not None:
             self.log.info("Enabling coverage collection of Python code")
-            # Exclude cocotb itself from coverage collection.
-            cocotb_package_dir = os.path.dirname(__file__)
-            self._cov = coverage.coverage(branch=True, omit=[f"{cocotb_package_dir}/*"])
+            config_filepath = os.getenv("COVERAGE_RCFILE")
+            if config_filepath is None:
+                # Exclude cocotb itself from coverage collection.
+                cocotb_package_dir = os.path.dirname(__file__)
+                self._cov = coverage.coverage(
+                    branch=True, omit=[f"{cocotb_package_dir}/*"]
+                )
+            else:
+                # Allow the config file to handle all configuration
+                self._cov = coverage.coverage()
             self._cov.start()
 
         # Test Discovery


Test patch: