Repository: modin-project/modin
Pull request: FIX-#3031: Added -n 2 to long running pytest tasks and paralellized windows tasks
Signed-off-by: Gregory Shimansky <gregory.shimansky@intel.com>

<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/CONTRIBUTING.html
if you have questions about contributing.
-->

## What do these changes do?

<!-- Please give a short brief about these changes. -->

- [x] commit message follows format outlined [here](https://modin.readthedocs.io/en/latest/contributing.html)
- [x] passes `flake8 modin`
- [x] passes `black --check modin`
- [x] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [x] Resolves #3031 <!-- issue must be created for each patch -->
- [x] tests added and passing
- [x] module layout described at `docs/developer/architecture.rst` is up-to-date <!-- if you have added, renamed or removed files or directories please update the documentation accordingly -->


Problem statement:
Improve CI performance with xdist pytests
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. What kind of performance improvements would you like to see with this new API?

All tests currently executed on GitHub Actions work in single process. This is not optimal since even free GA workers contain 2 CPU cores, so we could speed up CI tasks at least 2x times. More speedup may be possible if we utilize our own GA workers with more CPU cores.

Hints from issue discussion:
Windows on Ray tests fail because of Ray bug https://github.com/ray-project/ray/issues/13213

Commit message: FIX-#3031: Removed xdist for test_concat

Signed-off-by: Gregory Shimansky <gregory.shimansky@intel.com>

Patch to review:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 72e8ce86199..9a8901b0c8a 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -45,7 +45,7 @@ jobs:
         with:
           python-version: "3.7.x"
           architecture: "x64"
-      - run: pip install pytest pytest-cov pydocstyle numpydoc
+      - run: pip install pytest pytest-cov pytest-xdist pydocstyle numpydoc
       - run: pytest scripts/test
       - run: pip install -e .[all]
       - run: |
@@ -219,7 +219,7 @@ jobs:
       - shell: bash -l {0}
         run: python -m pytest modin/test/test_envvar_npartitions.py
       - shell: bash -l {0}
-        run: python -m pytest modin/test/test_partition_api.py
+        run: python -m pytest -n 2 modin/test/test_partition_api.py
 
   test-defaults:
     needs: [lint-commit, lint-flake8, lint-black, test-api, test-headers]
@@ -252,39 +252,39 @@ jobs:
       - shell: bash -l {0}
         run: pytest modin/experimental/xgboost/test/test_default.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/test/backends/base/test_internals.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/test/backends/base/test_internals.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_binary.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_binary.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_default.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_default.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_indexing.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_indexing.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_iter.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_iter.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_join_sort.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_join_sort.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_map_metadata.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_map_metadata.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_reduction.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_reduction.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_udf.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_udf.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_window.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_window.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_pickle.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_pickle.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_series.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_rolling.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_rolling.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_concat.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_concat.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_groupby.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_groupby.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_reshape.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_reshape.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_general.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_general.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
         run: bash <(curl -s https://codecov.io/bash)
 
@@ -406,39 +406,39 @@ jobs:
       - shell: bash -l {0}
         run: MODIN_BENCHMARK_MODE=True pytest modin/pandas/test/internals/test_benchmark_mode.py
       - shell: bash -l {0}
-        run: pytest modin/experimental/xgboost/test/test_default.py
+        run: pytest -n 2 modin/experimental/xgboost/test/test_default.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_binary.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_binary.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_default.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_default.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_indexing.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_indexing.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_iter.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_iter.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_join_sort.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_join_sort.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_map_metadata.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_map_metadata.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_reduction.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_reduction.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_udf.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_udf.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_window.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_window.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_pickle.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_pickle.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py
+        run: python -m pytest -n 2 modin/pandas/test/test_series.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_rolling.py
+        run: python -m pytest -n 2 modin/pandas/test/test_rolling.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_concat.py
+        run: python -m pytest modin/pandas/test/test_concat.py # Ray version fails with -n 2
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_groupby.py
+        run: python -m pytest -n 2 modin/pandas/test/test_groupby.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_reshape.py
+        run: python -m pytest -n 2 modin/pandas/test/test_reshape.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_general.py
+        run: python -m pytest -n 2 modin/pandas/test/test_general.py
       - shell: bash -l {0}
         run: python -m pytest modin/pandas/test/test_io.py
       - shell: bash -l {0}
@@ -473,9 +473,9 @@ jobs:
           conda info
           conda list
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/dataframe/test_map_metadata.py
+        run: python -m pytest -n 2 modin/pandas/test/dataframe/test_map_metadata.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py
+        run: python -m pytest -n 2 modin/pandas/test/test_series.py
       - shell: bash -l {0}
         run: python -m pytest modin/pandas/test/test_io.py
       - shell: bash -l {0}
@@ -513,11 +513,30 @@ jobs:
   test-windows:
     needs: [lint-commit, lint-flake8, lint-black, test-api, test-headers]
     runs-on: windows-latest
+    defaults:
+      run:
+        shell: bash -l {0}
     strategy:
       matrix:
         python-version: ["3.7", "3.8"]
         engine: ["ray", "dask"]
-        part: ["DataFrame", 3]
+        test-task:
+          - modin/pandas/test/dataframe/test_binary.py
+          - modin/pandas/test/dataframe/test_default.py
+          - modin/pandas/test/dataframe/test_indexing.py
+          - modin/pandas/test/dataframe/test_iter.py
+          - modin/pandas/test/dataframe/test_join_sort.py
+          - modin/pandas/test/dataframe/test_map_metadata.py
+          - modin/pandas/test/dataframe/test_reduction.py
+          - modin/pandas/test/dataframe/test_udf.py
+          - modin/pandas/test/dataframe/test_window.py
+          - modin/pandas/test/dataframe/test_pickle.py
+          - modin/pandas/test/test_series.py
+          - modin/pandas/test/test_rolling.py
+          - modin/pandas/test/test_concat.py
+          - modin/pandas/test/test_groupby.py
+          - modin/pandas/test/test_reshape.py
+          - modin/pandas/test/test_general.py
     env:
       MODIN_ENGINE: ${{matrix.engine}}
       MODIN_MEMORY: 1000000000
@@ -535,66 +554,16 @@ jobs:
           use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
           auto-update-conda: true # this enable `use-only-tar-bz2` feature on Windows
       - name: Conda environment
-        shell: bash -l {0}
         run: |
           conda info
           conda list
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_binary.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_default.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_indexing.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_iter.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_join_sort.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_map_metadata.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_reduction.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_udf.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_window.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_pickle.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_rolling.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_concat.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_groupby.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_reshape.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_general.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        timeout-minutes: 20
+      - run: python -m pytest ${{matrix.test-task}}
+        if: matrix.test-task != 'modin/pandas/test/test_io.py'
+      - timeout-minutes: 30
         run: python -m pytest modin/pandas/test/test_io.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: choco install codecov
-      - shell: bash -l {0}
-        run: codecov -f ./coverage.xml
+        if: matrix.test-task == 'modin/pandas/test/test_io.py'
+      - run: choco install codecov
+      - run: codecov -f ./coverage.xml
 
   test-pyarrow:
     needs: [lint-commit, lint-flake8, lint-black, test-api, test-headers]
diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml
index 8ec84b38e9a..ed6c1baf4fc 100644
--- a/.github/workflows/push.yml
+++ b/.github/workflows/push.yml
@@ -62,39 +62,39 @@ jobs:
       - name: Install HDF5
         run: sudo apt update && sudo apt install -y libhdf5-dev
       - shell: bash -l {0}
-        run: pytest modin/experimental/xgboost/test/test_default.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/experimental/xgboost/test/test_default.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_binary.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_binary.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_default.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_default.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_indexing.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_indexing.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_iter.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_iter.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_join_sort.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_join_sort.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_map_metadata.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_map_metadata.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_reduction.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_reduction.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_udf.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_udf.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_window.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_window.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_pickle.py --backend=${{ matrix.backend }}
+        run: pytest -n 2 modin/pandas/test/dataframe/test_pickle.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_series.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_rolling.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_rolling.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_concat.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_concat.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_groupby.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_groupby.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_reshape.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_reshape.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_general.py --backend=${{ matrix.backend }}
+        run: python -m pytest -n 2 modin/pandas/test/test_general.py --backend=${{ matrix.backend }}
       - shell: bash -l {0}
         run: bash <(curl -s https://codecov.io/bash)
 
@@ -159,39 +159,39 @@ jobs:
       - name: Install HDF5
         run: sudo apt update && sudo apt install -y libhdf5-dev
       - shell: bash -l {0}
-        run: pytest modin/experimental/xgboost/test/test_default.py
+        run: pytest -n 2 modin/experimental/xgboost/test/test_default.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_binary.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_binary.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_default.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_default.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_indexing.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_indexing.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_iter.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_iter.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_join_sort.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_join_sort.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_map_metadata.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_map_metadata.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_reduction.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_reduction.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_udf.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_udf.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_window.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_window.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_pickle.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_pickle.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py
+        run: python -m pytest -n 2 modin/pandas/test/test_series.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_rolling.py
+        run: python -m pytest -n 2 modin/pandas/test/test_rolling.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_concat.py
+        run: python -m pytest modin/pandas/test/test_concat.py # Ray version fails with -n 2
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_groupby.py
+        run: python -m pytest -n 2 modin/pandas/test/test_groupby.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_reshape.py
+        run: python -m pytest -n 2 modin/pandas/test/test_reshape.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_general.py
+        run: python -m pytest -n 2 modin/pandas/test/test_general.py
       - shell: bash -l {0}
         run: python -m pytest modin/pandas/test/test_io.py
       - shell: bash -l {0}
@@ -201,11 +201,30 @@ jobs:
 
   test-windows:
     runs-on: windows-latest
+    defaults:
+      run:
+        shell: bash -l {0}
     strategy:
       matrix:
         python-version: ["3.7", "3.8"]
         engine: ["ray", "dask"]
-        part: ["DataFrame", 3]
+        test-task:
+          - modin/pandas/test/dataframe/test_binary.py
+          - modin/pandas/test/dataframe/test_default.py
+          - modin/pandas/test/dataframe/test_indexing.py
+          - modin/pandas/test/dataframe/test_iter.py
+          - modin/pandas/test/dataframe/test_join_sort.py
+          - modin/pandas/test/dataframe/test_map_metadata.py
+          - modin/pandas/test/dataframe/test_reduction.py
+          - modin/pandas/test/dataframe/test_udf.py
+          - modin/pandas/test/dataframe/test_window.py
+          - modin/pandas/test/dataframe/test_pickle.py
+          - modin/pandas/test/test_series.py
+          - modin/pandas/test/test_rolling.py
+          - modin/pandas/test/test_concat.py
+          - modin/pandas/test/test_groupby.py
+          - modin/pandas/test/test_reshape.py
+          - modin/pandas/test/test_general.py
     env:
       MODIN_ENGINE: ${{matrix.engine}}
       MODIN_MEMORY: 1000000000
@@ -223,66 +242,16 @@ jobs:
           use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
           auto-update-conda: true # this enable `use-only-tar-bz2` feature on Windows
       - name: Conda environment
-        shell: bash -l {0}
         run: |
           conda info
           conda list
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_binary.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_default.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_indexing.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_iter.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_join_sort.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_map_metadata.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_reduction.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_udf.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_window.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_pickle.py
-        if: matrix.part == 'DataFrame'
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_rolling.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_concat.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_groupby.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_reshape.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_general.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        timeout-minutes: 20
+      - run: python -m pytest ${{matrix.test-task}}
+        if: matrix.test-task != 'modin/pandas/test/test_io.py'
+      - timeout-minutes: 30
         run: python -m pytest modin/pandas/test/test_io.py
-        if: matrix.part == 3
-      - shell: bash -l {0}
-        run: choco install codecov
-      - shell: bash -l {0}
-        run: codecov -f ./coverage.xml
+        if: matrix.test-task == 'modin/pandas/test/test_io.py'
+      - run: choco install codecov
+      - run: codecov -f ./coverage.xml
 
   test-pyarrow:
     runs-on: ubuntu-latest


Code review comments:
diff_hunk: @@ -45,7 +45,7 @@ jobs:
         with:
           python-version: "3.7.x"
           architecture: "x64"
-      - run: pip install pytest pytest-cov pydocstyle numpydoc
+      - run: pip install pytest pytest-cov pytest-xdist pydocstyle numpydoc
path: .github/workflows/ci.yml
text: @user1:
I don't think we need `xdist` here because this is about checking docs, they're fast enough without that :)

diff_hunk: @@ -406,39 +406,39 @@ jobs:
       - shell: bash -l {0}
         run: MODIN_BENCHMARK_MODE=True pytest modin/pandas/test/internals/test_benchmark_mode.py
       - shell: bash -l {0}
-        run: pytest modin/experimental/xgboost/test/test_default.py
+        run: pytest -n 2 modin/experimental/xgboost/test/test_default.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_binary.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_binary.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_default.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_default.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_indexing.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_indexing.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_iter.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_iter.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_join_sort.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_join_sort.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_map_metadata.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_map_metadata.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_reduction.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_reduction.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_udf.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_udf.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_window.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_window.py
       - shell: bash -l {0}
-        run: pytest modin/pandas/test/dataframe/test_pickle.py
+        run: pytest -n 2 modin/pandas/test/dataframe/test_pickle.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_series.py
+        run: python -m pytest -n 2 modin/pandas/test/test_series.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_rolling.py
+        run: python -m pytest -n 2 modin/pandas/test/test_rolling.py
       - shell: bash -l {0}
-        run: python -m pytest modin/pandas/test/test_concat.py
+        run: python -m pytest modin/pandas/test/test_concat.py # Ray version fails with -n 2
path: .github/workflows/ci.yml
text: @user1:
can we use a `yml if` here somehow to use `-n 2` on non-Ray backends?