Repository: lbl-anp/becquerel (language: python)

Issue:
Run tests in parallel with `pytest-xdist`
@jvavrek can we change the tests to use xdist and run in parallel?

_Originally posted by @jccurtis in https://github.com/lbl-anp/becquerel/issues/333#issuecomment-1087928469_

I was unable to get `pytest-xdist` to work properly—it seems to cause race conditions when used in conjunction with `pytest-cov`. Perhaps we can revisit this in the future. See the original post for further discussion on the original attempt.
Run tests in parallel with `pytest-xdist`
@jvavrek can we change the tests to use xdist and run in parallel?

_Originally posted by @jccurtis in https://github.com/lbl-anp/becquerel/issues/333#issuecomment-1087928469_

I was unable to get `pytest-xdist` to work properly—it seems to cause race conditions when used in conjunction with `pytest-cov`. Perhaps we can revisit this in the future. See the original post for further discussion on the original attempt.


Discussion / hints:
I found [this post on SO](https://stackoverflow.com/questions/4637036/is-there-a-way-to-control-how-pytest-xdist-runs-tests-in-parallel) that suggests adding `--dist=loadfile` to the call to `pytest` to get all the tests in a given file to run on the same worker.  I tried this on a Docker container running the Python 3.9 image and went from getting multiple failed tests to none.  I'm only running on a small number of cores so maybe it still fails with more cores.  Seems like this should help though if not solve the issue.

Happy to test on other versions of Python or share my Dockerfile if it would help.
@RobJY that is a great find. Thank you for that! @jvavrek want me to make a separate PR with just feature?
> @RobJY that is a great find. Thank you for that! @jvavrek want me to make a separate PR with just feature?

@jccurtis if you have time for it. If not, I can try it this afternoon.
I found [this post on SO](https://stackoverflow.com/questions/4637036/is-there-a-way-to-control-how-pytest-xdist-runs-tests-in-parallel) that suggests adding `--dist=loadfile` to the call to `pytest` to get all the tests in a given file to run on the same worker.  I tried this on a Docker container running the Python 3.9 image and went from getting multiple failed tests to none.  I'm only running on a small number of cores so maybe it still fails with more cores.  Seems like this should help though if not solve the issue.

Happy to test on other versions of Python or share my Dockerfile if it would help.
@RobJY that is a great find. Thank you for that! @jvavrek want me to make a separate PR with just feature?
> @RobJY that is a great find. Thank you for that! @jvavrek want me to make a separate PR with just feature?

@jccurtis if you have time for it. If not, I can try it this afternoon.

Fix patch:
diff --git a/requirements-dev.txt b/requirements-dev.txt
index 1d4834f8..fa475d2c 100644
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -7,3 +7,4 @@ pytest<7.0.0
 pytest-black
 pytest-cov
 pytest-rerunfailures
+pytest-xdist


Test patch:
diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml
index e8b2fbaa..e6dcc305 100644
--- a/.github/workflows/tests.yaml
+++ b/.github/workflows/tests.yaml
@@ -42,5 +42,10 @@ jobs:
         run: python -c "import becquerel;"
       - name: Install dev dependencies
         run: pip install -r requirements-dev.txt
+      # Run tests on 4 nodes (processes) in parallel while guaranteeing that the
+      # tests from each file in tests/ run in sequence on the same node. The
+      # `loadfile` is required for the code coverage to complete in parallel
+      # without failing the pipeline.
+      # See: https://github.com/lbl-anp/becquerel/issues/336
       - name: Test with pytest
-        run: pytest
+        run: pytest -n 4 --dist=loadfile