Repository: ocean-eddy-cpt/gcm-filters (language: python)

Issue:
Tests are failing
There seems to be an issue with conda? 

https://github.com/ocean-eddy-cpt/gcm-filters/actions/runs/6930660764/job/18850752922#step:4:2180

@jbusecke do you know what is going on?
Drop python 3.6 support
This came up in https://github.com/ocean-eddy-cpt/gcm-filters/pull/56#issuecomment-832860188, but I think the underlying problem would not be solved by dropping 3.6 (see #58).

We might still want to drop this older version. Just pinning this here. I can take care of it once we figured out what is going on with the CI.


Discussion / hints:
Short answer: Not really. 

**BUT** I believe I have encountered this before and it was fixable by using micromamba. See for example the changes made in [xgcm](https://github.com/xgcm/xgcm/commit/0a130d1e4974de70f877b6abd80934ae019a3c00)
Ok I started #164, but saw that we only deal with [pip requirements here](https://github.com/ocean-eddy-cpt/gcm-filters/blob/master/requirements.txt)? Why are we even using mamba/conda at all?
Thanks @jbusecke! I am confused about this, too. You set up the testing framework originally, so I trusted you on this. ;)

I think we want to use conda [to make contributions easier and the notebooks reproducible by users](https://gcm-filters.readthedocs.io/en/latest/how_to_contribute.html). But your question might be separate from this, namely whether we need conda for the testing framework?
@jbusecke pinging you again so we can hopefully resolve this issue. Thanks! :) 


Fix patch:
diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml
index 0ab57fd..c4efbf3 100644
--- a/.github/workflows/pre-commit.yml
+++ b/.github/workflows/pre-commit.yml
@@ -14,8 +14,6 @@ jobs:
   pre-commit:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v2.3.5
-    - uses: actions/setup-python@v2
-      with:
-        python-version: '3.8'
-    - uses: pre-commit/action@v2.0.0
+    - uses: actions/checkout@v4
+    - uses: actions/setup-python@v4
+    - uses: pre-commit/action@v3.0.1
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 207dac1..7f93d65 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -10,18 +10,10 @@ repos:
     - id: end-of-file-fixer
       exclude: tests/test_data
     - id: check-docstring-first
-#    - id: check-added-large-files
     - id: requirements-txt-fixer
     - id: file-contents-sorter
       files: requirements-dev.txt
 
-#- repo: https://gitlab.com/pycqa/flake8
-#  rev: 3.9.1
-#  hooks:
-#    - id: flake8
-#      exclude: docs/source/conf.py
-#      args: [--max-line-length=105, --ignore=E203,E501,W503, --select=select=C,E,F,W,B,B950]
-
 - repo: https://github.com/pre-commit/mirrors-isort
   rev: v5.8.0
   hooks:
@@ -34,21 +26,8 @@ repos:
   hooks:
     - id: seed-isort-config
 
-# - repo: https://github.com/psf/black
-#   rev: 21.5b0
-#   hooks:
-#   - id: black
-#     language_version: python3
-
 - repo: https://github.com/psf/black
   rev: 22.1.0
   hooks:
     - id: black
       additional_dependencies: ['click==8.0.4']
-
-#- repo: https://github.com/pre-commit/mirrors-mypy
-#  rev: v0.770
-#  hooks:
-#  - id: mypy
-#    exclude: docs/source/conf.py
-#    args: [--ignore-missing-imports]
diff --git a/ci/environment.yml b/ci/environment.yml
new file mode 100644
index 0000000..e3e9118
--- /dev/null
+++ b/ci/environment.yml
@@ -0,0 +1,11 @@
+name: gcm-filters-env
+channels:
+  - conda-forge
+  - nodefaults
+dependencies:
+  - python>=3.9
+  - zarr
+  - pytest
+  - pytest-lazy-fixtures
+  - pre-commit
+  - coverage
diff --git a/ci/environment_docs.yml b/ci/environment_docs.yml
new file mode 100644
index 0000000..7225735
--- /dev/null
+++ b/ci/environment_docs.yml
@@ -0,0 +1,16 @@
+name: gcm-filters-docs
+channels:
+  - conda-forge
+dependencies:
+  - python>=3.9
+  - numpy
+  - scipy
+  - xarray
+  - matplotlib
+  - numpydoc
+  - sphinx
+  - sphinx-book-theme
+  - sphinx-copybutton
+  - myst-parser
+  - myst-nb
+  - sphinxcontrib-srclinks
diff --git a/coverage.toml b/coverage.toml
new file mode 100644
index 0000000..62ecd2f
--- /dev/null
+++ b/coverage.toml
@@ -0,0 +1,5 @@
+[tool.coverage.run]
+omit = [
+    "tests/*",
+    "**/__init__.py",
+]
diff --git a/docs/conf.py b/docs/conf.py
index a275bac..9d43f7c 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -84,3 +84,9 @@
 # -- nbsphinx specific options ----------------------------------------------
 # this allows notebooks to be run even if they produce errors.
 nbsphinx_allow_errors = True
+
+
+html_theme_options = {
+    "repository_url": "https://github.com/ocean-eddy-cpt/gcm-filters",
+    "use_repository_button": True,
+}
diff --git a/docs/environment.yml b/docs/environment.yml
index f280d52..51a4879 100644
--- a/docs/environment.yml
+++ b/docs/environment.yml
@@ -1,4 +1,4 @@
-name: gcm-filters-docs
+name: gcm-filters-docs-and-notebooks
 channels:
   - conda-forge
 dependencies:
@@ -13,7 +13,7 @@ dependencies:
   - ipython
   - nbconvert
   - nbformat
-  - ipykernel # not strictly necessary but this is nice to run notebooks in this env to test
+  - ipykernel
   - pandoc
   - jupyter_client
   - sphinx-book-theme
diff --git a/docs/how_to_contribute.rst b/docs/how_to_contribute.rst
index b8ec005..4bf50f5 100644
--- a/docs/how_to_contribute.rst
+++ b/docs/how_to_contribute.rst
@@ -15,21 +15,15 @@ Fork the main repository, and clone your fork onto the machine of your choice.
 
 Make sure to install and activate the environment with::
 
-   conda create -n gcm-filters-env -c conda-forge --file requirements.txt --file requirements-dev.txt
+   conda env create -f ci/environment.yml
 
-If you find that `conda` takes a long time, try to install `mamba` with ``conda install mamba`` and then
-and do::
-
-   mamba create -n gcm-filters-env -c conda-forge --file requirements.txt --file requirements-dev.txt
-
-
-and finally activate the environment::
+and activate the environment::
 
    conda activate gcm-filters-env
 
 Before you can efficiently test your code you need to install this package in the now activated environment::
 
-   pip install -e . --no-deps
+   pip install -e .
 
 For the linting also install `pre-commit <https://pre-commit.com>`_ now::
 
@@ -62,12 +56,13 @@ until all checks pass.
 
 Once you got everything to pass, you can stage and commit your changes and push them to the remote github repository.
 
-How to change the documentation
--------------------------------
+
+How to build the documentation locally
+---------------------------------------
 
 In order to build the documentation locally you should build and activate the docs environment::
 
-   mamba env create -f docs/environment.yml
+   conda env create -f ci/environment_docs.yml
 
    conda activate gcm-filters-docs
 
diff --git a/docs/installation.rst b/docs/installation.rst
index ee8f8d1..41d19ae 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -57,4 +57,4 @@ If you want to run the example notebooks in this documentation, you will need a
 
    conda env create -f docs/environment.yml
 
-   conda activate gcm-filters-docs
+   conda activate gcm-filters-docs-and-notebooks
diff --git a/pyproject.toml b/pyproject.toml
index 98fa29f..c93287e 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -3,7 +3,7 @@ requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4"]
 build-backend = "setuptools.build_meta"
 
 [tool.isort]
-known_third_party = ["numpy", "pytest", "pytest_lazyfixture", "scipy", "setuptools", "xarray", "zarr"]
+known_third_party = ["numpy", "pytest", "pytest_lazy_fixtures", "scipy", "setuptools", "xarray", "zarr"]
 
 [tool.pytest.ini_options]
 minversion = "6.0"
diff --git a/readthedocs.yml b/readthedocs.yml
index fd67cf1..37e0638 100644
--- a/readthedocs.yml
+++ b/readthedocs.yml
@@ -11,7 +11,4 @@ python:
          path: .
 
 conda:
-  environment: docs/environment.yml
-
-# sphinx:
-#   fail_on_warning: true
+  environment: ci/environment_docs.yml
diff --git a/requirements-dev.txt b/requirements-dev.txt
deleted file mode 100644
index ad5c7ed..0000000
--- a/requirements-dev.txt
+++ /dev/null
@@ -1,25 +0,0 @@
-black
-check-manifest
-doctr
-flake8
-flake8-builtins
-flake8-comprehensions
-flake8-mutable
-flake8-print
-interrogate
-isort
-nbsphinx
-numcodecs
-pre-commit
-pylint
-pytest
-pytest-cov
-pytest-flake8
-pytest-lazy-fixture
-pytest-xdist
-recommonmark
-setuptools_scm
-sphinx
-twine
-wheel
-zarr
diff --git a/requirements.txt b/requirements.txt
deleted file mode 100644
index a0da9b8..0000000
--- a/requirements.txt
+++ /dev/null
@@ -1,4 +0,0 @@
-dask[array]
-matplotlib
-scipy
-xarray
diff --git a/setup.cfg b/setup.cfg
index fdf20d0..28d560b 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -15,19 +15,20 @@ classifiers =
     License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
     Programming Language :: Python
     Programming Language :: Python :: 3
-    Programming Language :: Python :: 3.6
-    Programming Language :: Python :: 3.7
-    Programming Language :: Python :: 3.8
+    Programming Language :: Python :: 3.10
+    Programming Language :: Python :: 3.11
+    Programming Language :: Python :: 3.12
     Topic :: Scientific/Engineering
 
 [options]
 zip_safe = False
 install_requires =
-    numpy
     dask
-    scipy
+    numpy
     xarray
-python_requires = >=3.6
+    scipy
+    matplotlib
+python_requires = >=3.9
 packages = find:
 
 [sdist]


Test patch:
diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml
index 4264d6a..922f9b2 100644
--- a/.github/workflows/tests.yaml
+++ b/.github/workflows/tests.yaml
@@ -17,37 +17,44 @@ jobs:
       fail-fast: false
       matrix:
         os: ["ubuntu-latest"]
-        python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]
+        python-version: ["3.10", "3.11", "3.12"]
     steps:
-      - uses: actions/checkout@v2
-      - name: Cache conda
-        uses: actions/cache@v1
-        env:
-          # Increase this value to reset cache if ci/environment.yml has not changed
-          CACHE_NUMBER: 0
-        with:
-          path: ~/conda_pkgs_dir
-          key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ hashFiles('requirements.txt', 'requirements-dev.txt') }}
-      - uses: conda-incubator/setup-miniconda@v2
+      - uses: actions/checkout@v4
+
+      - name: Create conda environment
+        uses: mamba-org/setup-micromamba@v1
         with:
-          mamba-version: "*"
-          channels: conda-forge, defaults
-          channel-priority: true
-          python-version: ${{ matrix.python-version }}
-          activate-environment: gcm-filters-env
-          use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
-      - shell: bash -l {0}
-        run: mamba install --yes --file requirements.txt --file requirements-dev.txt
-      - name: Set up conda environment
-        shell: bash -l {0}
+          cache-downloads: true
+          cache-environment: true
+          micromamba-version: 'latest'
+          environment-file: ci/environment.yml
+          create-args: |
+            python=${{ matrix.python-version }}
+          cache-key: micromamba-${{ matrix.python-version }}-${{ runner.os }}
+
+      - name: Conda info
+        run: conda info
+
+      - name: Install gcm-filters
+        shell: micromamba-shell {0}
         run: |
-          python -m pip install -e . --no-deps --force-reinstall
+           python -V
+           python -m pip install -e . --force-reinstall
+
       - name: Run Tests
         shell: bash -l {0}
         run: |
-          pytest --cov=./ --cov-report=xml
+           python -V
+           coverage run --rcfile=coverage.toml -m pytest --verbose tests/*
+
+      - name: Get coverage report
+        shell: bash -l {0}
+        run: |
+          coverage report -m
+          coverage xml
+
       - name: Upload code coverage to Codecov
-        uses: codecov/codecov-action@v1
+        uses: codecov/codecov-action@v4.0.1
         with:
           file: ./coverage.xml
           flags: unittests
diff --git a/tests/test_filter_validation.py b/tests/test_filter_validation.py
index 1c7ff88..43fd79d 100644
--- a/tests/test_filter_validation.py
+++ b/tests/test_filter_validation.py
@@ -5,7 +5,7 @@
 import xarray as xr
 import zarr
 
-from pytest_lazyfixture import lazy_fixture
+from pytest_lazy_fixtures import lf
 
 from gcm_filters import Filter, FilterShape
 
@@ -20,8 +20,8 @@ def requires_env(varname, value):
 
 
 all_grids_data_and_input_ds = [
-    lazy_fixture("grid_type_and_input_ds"),
-    lazy_fixture("vector_grid_type_and_input_ds"),
+    lf("grid_type_and_input_ds"),
+    lf("vector_grid_type_and_input_ds"),
 ]
 
 
diff --git a/tests/test_kernels_validation.py b/tests/test_kernels_validation.py
index b59fde2..f810606 100644
--- a/tests/test_kernels_validation.py
+++ b/tests/test_kernels_validation.py
@@ -5,7 +5,7 @@
 import xarray as xr
 import zarr
 
-from pytest_lazyfixture import lazy_fixture
+from pytest_lazy_fixtures import lf
 
 from gcm_filters.kernels import ALL_KERNELS
 
@@ -20,8 +20,8 @@ def requires_env(varname, value):
 
 
 all_grids_data_and_extra_kwargs = [
-    lazy_fixture("scalar_grid_type_data_and_extra_kwargs"),
-    lazy_fixture("vector_grid_type_data_and_extra_kwargs"),
+    lf("scalar_grid_type_data_and_extra_kwargs"),
+    lf("vector_grid_type_data_and_extra_kwargs"),
 ]