Repository: modin-project/modin
Pull request: FIX-#6587: Use different env files for unidist engine for windows and linux
<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/development/contributing.html
if you have questions about contributing.
-->

## What do these changes do?

<!-- Please give a short brief about these changes. -->

- [x] first commit message and PR title follow format outlined [here](https://modin.readthedocs.io/en/latest/development/contributing.html#commit-message-formatting)
  > **_NOTE:_**  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.
- [x] passes `flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] passes `black --check modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [x] Resolves #6587  <!-- issue must be created for each patch -->
- [x] tests passing
- [ ] module layout described at `docs/development/architecture.rst` is up-to-date <!-- if you have added, renamed or removed files or directories please update the documentation accordingly -->


Problem statement:
Use different env files for unidist engine for windows and linux
unidist depends on mpi4py only and conda solver may pick different MPI implementations in depend on a concrete development envionment. We should be explicit and specify the exact MPI implementation we test on.

Hints from issue discussion:

Commit message: fix

Signed-off-by: Igoshev, Iaroslav <iaroslav.igoshev@intel.com>

Patch to review:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 00fa76d5c05..b3e28ae4ad5 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -349,7 +349,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: ./.github/actions/mamba-env
         with:
-          environment-file: requirements/env_unidist.yml
+          environment-file: requirements/env_unidist_linux.yml
           activate-environment: modin_on_unidist
           python-version: ${{matrix.python-version}}
       - name: Install HDF5
@@ -556,7 +556,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: ./.github/actions/mamba-env
         with:
-          environment-file: ${{ matrix.execution.name == 'unidist' && 'requirements/env_unidist.yml' || 'environment-dev.yml' }}
+          environment-file: ${{ matrix.os = 'ubuntu' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_linux.yml' || matrix.os = 'windows' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_win.yml' || 'environment-dev.yml' }}
           activate-environment: ${{ matrix.execution.name == 'unidist' && 'modin_on_unidist' || 'modin' }}
           python-version: ${{matrix.python-version}}
       - name: Install HDF5
diff --git a/requirements/env_unidist.yml b/requirements/env_unidist_linux.yml
similarity index 99%
rename from requirements/env_unidist.yml
rename to requirements/env_unidist_linux.yml
index 4f8995d4861..7589560c2b9 100644
--- a/requirements/env_unidist.yml
+++ b/requirements/env_unidist_linux.yml
@@ -8,6 +8,7 @@ dependencies:
   - pandas>=2.1,<2.2
   - numpy>=1.22.4
   - unidist-mpi>=0.2.1
+  - mpich
   - fsspec>=2022.05.0
   - packaging>=21.0
   - psutil>=5.8.0
diff --git a/requirements/env_unidist_win.yml b/requirements/env_unidist_win.yml
new file mode 100644
index 00000000000..f3b3459dab6
--- /dev/null
+++ b/requirements/env_unidist_win.yml
@@ -0,0 +1,61 @@
+name: modin_on_unidist
+channels:
+  - conda-forge
+dependencies:
+  - pip
+
+  # required dependencies
+  - pandas>=2.1,<2.2
+  - numpy>=1.22.4
+  - unidist-mpi>=0.2.1
+  - msmpi
+  - fsspec>=2022.05.0
+  - packaging>=21.0
+  - psutil>=5.8.0
+
+  # optional dependencies
+  - pyarrow>=7.0.0
+  - xarray>=2022.03.0
+  - jinja2>=3.1.2
+  - scipy>=1.8.1
+  - s3fs>=2022.05.0
+  - lxml>=4.8.0
+  - openpyxl>=3.0.10
+  - xlrd>=2.0.1
+  - matplotlib>=3.6.1
+  - sqlalchemy>=1.4.0,<1.4.46
+  - pandas-gbq>=0.15.0
+  - pytables>=3.7.0
+  # pymssql==2.2.8 broken: https://github.com/modin-project/modin/issues/6429
+  - pymssql>=2.1.5,!=2.2.8
+  - psycopg2>=2.9.3
+  - fastparquet>=0.8.1
+  - tqdm>=4.60.0
+  # pandas isn't compatible with numexpr=2.8.5: https://github.com/modin-project/modin/issues/6469
+  - numexpr<2.8.5
+
+  # dependencies for making release
+  - pygithub>=v1.58.0
+  - pygit2>=1.9.2
+
+  # test dependencies
+  - coverage>=7.1.0
+  - moto>=4.1.0
+  - pytest>=7.3.2
+  - pytest-cov>=4.0.0
+  - pytest-xdist>=3.2.0
+
+  # code linters
+  - black>=23.1.0
+  - flake8>=6.0.0
+  - flake8-no-implicit-concat>=0.3.4
+  - flake8-print>=5.0.0
+  - mypy>=1.0.0
+  - pandas-stubs>=2.0.0
+
+  - pip:
+      # Fixes breaking ipywidgets changes, but didn't release yet.
+      - git+https://github.com/modin-project/modin-spreadsheet.git@49ffd89f683f54c311867d602c55443fb11bf2a5
+      - connectorx>=0.2.6a4
+      # The `numpydoc` version should match the version installed in the `lint-pydocstyle` job of the CI.
+      - numpydoc==1.1.0


Code review comments:
diff_hunk: @@ -556,7 +556,7 @@ jobs:
       - uses: actions/checkout@v3
       - uses: ./.github/actions/mamba-env
         with:
-          environment-file: ${{ matrix.execution.name == 'unidist' && 'requirements/env_unidist.yml' || 'environment-dev.yml' }}
+          environment-file: ${{ matrix.os = 'ubuntu' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_linux.yml' || matrix.os = 'windows' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_win.yml' || 'environment-dev.yml' }}
path: .github/workflows/ci.yml
text: @author:
```suggestion
          environment-file: ${{ matrix.os == 'ubuntu' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_linux.yml' || matrix.os == 'windows' && matrix.execution.name == 'unidist' && 'requirements/env_unidist_win.yml' || 'environment-dev.yml' }}
```