Repository: geoschem/gcpy (language: python)

Issue:
Problems creating virtual environment compatible with GCPy 1.4.1
### Name and Institution (Required)

Name: Lizzie Lundgren
Institution: Harvard University

### Description of your issue or question

I am creating python virtual environments for use with GCPy 1.4.1 on two different compute clusters following instructions on ReadTheDocs to install Mamba 23.3.1. On both clusters I ran into dependency issues preventing installation. Specifying library version for matplotlib enabled creating an environment but GCPy then failed due to incompatibility with other library versions (not sure exactly which).

I solved this problem by creating an environment file that specifies exactly what version for every package used. I got the version numbers from a functional python environment on the Harvard Cannon cluster. Since this environment is known to work with GCPy I suggest we make it available within the repository. There could also be an open-ended environment file that does not specify version number but pulls the latest version. That one could be used for GCST testing to upgrade libraries periodically.

Here is the list of package versions that work.

```
    - awscli==2.13.39                 # Utilities for AWS cloud
    - cartopy==0.22.0                 # Geospatial data processing
    - cf_xarray==0.8.4                # CF conventions for xarray
    - dask==2023.9.2                  # Parallel library; backend for xarray
    - gridspec==0.1.0                 # Define Earth System Model grids
    - ipython==8.15.0                 # Interactive Python (used by Jupyter)
    - joblib==1.3.2                   # Parallelize python code
    - jupyter==1.0.0                  # Jupyter Notebook
    - matplotlib==3.8.0               # Creates plots and visualizations
    - netcdf4==1.6.0                  # Python wrapper for netCDF
    - netcdf-fortran==4.5.4           # Python wrapper for netCDF-Fortran
    - numpy==1.26.0                   # Optimized mathematical functions
    - pandas==2.1.1                   # Tables/timeseries manipulation
    - pip==23.2.1                     # Install packages from PyPi
    - pylint==2.17.5                   # Python linter
    - pyproj==3.6.1                   # Python map projections library
    - python==3.9.18                          # Any python version prior to 3.10
    - pypdf==3.16.1                    # PDF utilities (bookmarks, etc.)
    - recommonmark==0.7.1             # Dependency for Sphinx
    - requests==2.31.0                 # HTTP library
    - scipy==1.11.2                   # Scientific python package
    - sparselt==0.1.3                 # Regridding earth system model data
    - tabulate==0.9.0                 # Pretty-printing for column data
    - tk==8.6.12                      # Tcl/tk library
    - xarray==2023.8.0                # Read data from netCDF etc files
    - esmf==8.1.1                     # Earth system modeling framework
    - esmpy==8.1.1                    # Python wrapper for ESMF
    - xesmf==0.5.1                    # Universal regridder
    - docutils==0.16                  # Convert text to other formats
    - jinja2==3.0.3                   # Dependency for Sphinx
    - sphinx==3.5.4                   # Generate ReadTheDocs output
    - sphinx-autoapi==1.9.0           # Sphinx autodoc style documentation
    - sphinx-autobuild==2021.3.14     # Build ReadTheDos live in browser
    - sphinxcontrib-bibtex==2.2.0     # ReadTheDocs bibliography style
    - sphinx_rtd_theme==0.5.2         # ReadTheDocs HTML theme files
```



Discussion / hints:

Fix patch:
diff --git a/.github/workflows/build-gcpy-environment.yml b/.github/workflows/build-gcpy-environment.yml
index 35b8fb19..765a18a7 100644
--- a/.github/workflows/build-gcpy-environment.yml
+++ b/.github/workflows/build-gcpy-environment.yml
@@ -1,6 +1,6 @@
 ---
 #
-# GitHub action to build the GCPy environment with micromamba
+# GitHub action to build the GCPy production environment with micromamba
 # See: https://github.com/marketplace/actions/setup-micromamba
 #
 name: build-gcpy-environment
@@ -17,16 +17,15 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.9"]
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
     steps:
       - name: Checkout the GCPy repository
         uses: actions/checkout@v2
-      - name: Create the GCPy environment
+      - name: Create "gcpy_env" environment
         uses: mamba-org/setup-micromamba@v1
         with:
           micromamba-version: 'latest'
-          environment-file: environment.yml
-          environment-name: gcpy-test-env
+          environment-file: docs/environment_files/environment.yml
           init-shell: bash
           cache-environment: true
           generate-run-shell: true
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7c68b041..ad30b2e0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,7 +7,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 ## [Unreleased] - TBD
 ### Added
 - Example script `create_test_plot.py`, which can be used to check that GCPy has been installed properly
-- GitHub action `build-gcpy-environment` which tests installation of the mamba environment specified in in `docs/source/environment.yml`
+- GitHub action `build-gcpy-environment` which tests installation of the mamba environment specified in in `docs/environment_files/environment.yml`
+- YAML file`docs/environment_files/testing.yml` for building an environment without pegged package versions (for testing)
+- GitHub action `build-test-environment` to test the environment specified in `testing.yml`
+
+### Changed
+- `build-gcpy-environment`
+
 
 ## [1.4.1] - 2023-12-08
 ### Fixed
@@ -162,7 +168,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
   - Now use restarts_subdir tag from YAML file for paths to restart files (@yantosca)
   - GCPy now uses proper year for dev in 1-yr benchmarks (@laestrada)
   - Fixed date string issue in benchmarking scripts (@lizziel)
-  - Updates for new GCHP restart file format (@lizziel)  
+  - Updates for new GCHP restart file format (@lizziel)
 - Updated environment.yml with package versions that work together (@yantosca)
 - Updated the AUTHORS.txt and LICENSE.txt files (@yantosca)
 
diff --git a/docs/environment_files/environment.yml b/docs/environment_files/environment.yml
index f5fc704d..994d13e9 100644
--- a/docs/environment_files/environment.yml
+++ b/docs/environment_files/environment.yml
@@ -1,59 +1,50 @@
+---
 # ======================================================================
 # GCPy environment file
 #
 # Recommended installation: with Mambaforge
 # $ mamba env create -n gcpy_env --file=/path/to/gcpy/environment.yml
 #
-# Some package versions are not the most recent, but these
-# have been proven to work together. (Bob Yantosca, 14 Aug 2023)
+# These package versions have been proven to work together.
+# See: https://github.com/geoschem/gcpy/issues/284
 # ======================================================================
 name: gcpy_env
 channels:
-    - conda-forge
-    - nodefaults    
+  - conda-forge
+  - nodefaults
 dependencies:
-    - awscli                          # Utilities for AWS cloud
-    - cartopy                         # Geospatial data processing
-    - cf_xarray                       # CF conventions for xarray
-    - dask                            # Parallel library; backend for xarray
-    - gridspec                        # Define Earth System Model grids
-    - ipython                         # Interactive Python (used by Jupyter)
-    - joblib                          # Parallelize python code
-    - jupyter                         # Jupyter Notebook
-    - matplotlib                      # Creates plots and visualizations
-    - netcdf4                         # Python wrapper for netCDF
-    - netcdf-fortran                  # Python wrapper for netCDF-Fortran
-    - numpy                           # Optimized mathematical functions
-    - pandas                          # Tables/timeseries manipulation
-    - pip                             # Install packages from PyPi
-    - pylint                          # Python linter
-    - pyproj                          # Python map projections library
-    - python<3.10                     # Any python version prior to 3.10
-    - pypdf                           # PDF utilities (bookmarks, etc.)
-    - recommonmark                    # Dependency for Sphinx
-    - requests                        # HTTP library
-    - scipy                           # Scientific python package
-    - sparselt>=0.1.3                 # Regridding earth system model data
-    - tabulate                        # Pretty-printing for column data
-    - tk                              # Tcl/tk library
-    - xarray                          # Read data from netCDF etc files
-    #
-    # NOTE: These packages need to be pegged at specific versions
-    # in order to avoid an ImportError.
-    #  -- Bob Yantosca (14 Aug 2023)
-    #
-    - esmf==8.1.1                     # Earth system modeling framework
-    - esmpy==8.1.1                    # Python wrapper for ESMF
-    - xesmf==0.5.1                    # Universal regridder
-    # 
-    # NOTE: These packages need to be pegged at specific versions
-    # or else the ReadTheDocs output won't render properly.
-    #  -- Bob Yantosca (14 Aug 2023)
-    #
-    - docutils==0.16                  # Convert text to other formats
-    - jinja2==3.0.3                   # Dependency for Sphinx
-    - sphinx==3.5.4                   # Generate ReadTheDocs output
-    - sphinx-autoapi==1.9.0           # Sphinx autodoc style documentation
-    - sphinx-autobuild==2021.3.14     # Build ReadTheDos live in browser
-    - sphinxcontrib-bibtex==2.2.0     # ReadTheDocs bibliography style
-    - sphinx_rtd_theme==0.5.2         # ReadTheDocs HTML theme files
+  - awscli==2.13.39                 # Utilities for AWS cloud
+  - cartopy==0.22.0                 # Geospatial data processing
+  - cf_xarray==0.8.4                # CF conventions for xarray
+  - dask==2023.9.2                  # Parallel library; backend for xarray
+  - gridspec==0.1.0                 # Define Earth System Model grids
+  - ipython==8.15.0                 # Interactive Python (used by Jupyter)
+  - joblib==1.3.2                   # Parallelize python code
+  - jupyter==1.0.0                  # Jupyter Notebook
+  - matplotlib==3.8.0               # Creates plots and visualizations
+  - netcdf4==1.6.0                  # Python wrapper for netCDF
+  - netcdf-fortran==4.5.4           # Python wrapper for netCDF-Fortran
+  - numpy==1.26.0                   # Optimized mathematical functions
+  - pandas==2.1.1                   # Tables/timeseries manipulation
+  - pip==23.2.1                     # Install packages from PyPi
+  - pylint==2.17.5                  # Python linter
+  - pyproj==3.6.1                   # Python map projections library
+  - python==3.9.18                  # Any python version prior to 3.10
+  - pypdf==3.16.1                   # PDF utilities (bookmarks, etc.)
+  - recommonmark==0.7.1             # Dependency for Sphinx
+  - requests==2.31.0                # HTTP library
+  - scipy==1.11.2                   # Scientific python package
+  - sparselt==0.1.3                 # Regridding earth system model data
+  - tabulate==0.9.0                 # Pretty-printing for column data
+  - tk==8.6.12                      # Tcl/tk library
+  - xarray==2023.8.0                # Read data from netCDF etc files
+  - esmf==8.1.1                     # Earth system modeling framework
+  - esmpy==8.1.1                    # Python wrapper for ESMF
+  - xesmf==0.5.1                    # Universal regridder
+  - docutils==0.16                  # Convert text to other formats
+  - jinja2==3.0.3                   # Dependency for Sphinx
+  - sphinx==3.5.4                   # Generate ReadTheDocs output
+  - sphinx-autoapi==1.9.0           # Sphinx autodoc style documentation
+  - sphinx-autobuild==2021.3.14     # Build ReadTheDos live in browser
+  - sphinxcontrib-bibtex==2.2.0     # ReadTheDocs bibliography style
+  - sphinx_rtd_theme==0.5.2         # ReadTheDocs HTML theme files


Test patch:
diff --git a/.github/workflows/build-test-environment.yml b/.github/workflows/build-test-environment.yml
new file mode 100644
index 00000000..42fdb1c4
--- /dev/null
+++ b/.github/workflows/build-test-environment.yml
@@ -0,0 +1,38 @@
+---
+#
+# GitHub action to build the GCPy test environment with micromamba
+# See: https://github.com/marketplace/actions/setup-micromamba
+#
+name: build-test-environment
+
+on:
+  push:
+    branches: [ "main", "dev" ]
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: [ "main", "dev" ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.9"]
+    steps:
+      - name: Checkout the GCPy repository
+        uses: actions/checkout@v2
+      - name: Create "testing" environment
+        uses: mamba-org/setup-micromamba@v1
+        with:
+          micromamba-version: 'latest'
+          environment-file: docs/environment_files/testing.yml
+          init-shell: bash
+          cache-environment: true
+          generate-run-shell: true
+          post-cleanup: 'all'
+      - name: Test if "import gcpy" works
+        run: python -c "import gcpy"
+        shell: micromamba-shell {0}
+      - name: Test if we can create a plot
+        run: python -m gcpy.examples.plotting.create_test_plot
+        shell: micromamba-shell {0}
diff --git a/docs/environment_files/testing.yml b/docs/environment_files/testing.yml
new file mode 100644
index 00000000..3c6c9de0
--- /dev/null
+++ b/docs/environment_files/testing.yml
@@ -0,0 +1,52 @@
+---
+# ======================================================================
+# GCPy testing environment file.
+# Intended to be used by the GCST for testing purposes.
+#
+# GCPy users should build the environment from "environment.yml",
+# as this will specify package versions that are known to work.
+# See: https://github.com/geoschem/gcpy/issues/284
+# ======================================================================
+name: testing
+channels:
+  - conda-forge
+  - nodefaults
+dependencies:
+  - awscli                          # Utilities for AWS cloud
+  - cartopy                         # Geospatial data processing
+  - cf_xarray                       # CF conventions for xarray
+  - dask                            # Parallel library; backend for xarray
+  - gridspec                        # Define Earth System Model grids
+  - ipython                         # Interactive Python (used by Jupyter)
+  - joblib                          # Parallelize python code
+  - jupyter                         # Jupyter Notebook
+  - matplotlib>=3.8                 # Creates plots and visualizations
+  - netcdf4                         # Python wrapper for netCDF
+  - netcdf-fortran                  # Python wrapper for netCDF-Fortran
+  - numpy                           # Optimized mathematical functions
+  - pandas                          # Tables/timeseries manipulation
+  - pip                             # Install packages from PyPi
+  - pylint                          # Python linter
+  - pyproj                          # Python map projections library
+  - python                          # Any python version prior to 3.10
+  - pypdf                           # PDF utilities (bookmarks, etc.)
+  - recommonmark                    # Dependency for Sphinx
+  - requests                        # HTTP library
+  - scipy                           # Scientific python package
+  - sparselt                        # Regridding earth system model data
+  - tabulate                        # Pretty-printing for column data
+  - tk                              # Tcl/tk library
+  - xarray                          # Read data from netCDF etc files
+  #
+  # NOTE: These packages are known to work, so peg these versions
+  #
+  - esmf==8.1.1                     # Earth system modeling framework
+  - esmpy==8.1.1                    # Python wrapper for ESMF
+  - xesmf==0.5.1                    # Universal regridder
+  - docutils==0.16                  # Convert text to other formats
+  - jinja2==3.0.3                   # Dependency for Sphinx
+  - sphinx==3.5.4                   # Generate ReadTheDocs output
+  - sphinx-autoapi==1.9.0           # Sphinx autodoc style documentation
+  - sphinx-autobuild==2021.3.14     # Build ReadTheDos live in browser
+  - sphinxcontrib-bibtex==2.2.0     # ReadTheDocs bibliography style
+  - sphinx_rtd_theme==0.5.2         # ReadTheDocs HTML theme files