Repository: sandialabs/pyGSTi (language: python)

Issue:
Idea: Switch macOS tests to run against beta branch
Recently it has been taking forever (hours) for the standard unit test suite to get picked up by the macOS github runners. We also recently learned that the macOS runners use up our free computational credits at 10X the linux rate and 5X the Windows rate. Both of these things are kind of annoying. Question for discussion:

Should we change our CI workflow so that the tests in `unit` only get run on the macOS runners upon pushing to beta (i.e. when merged into develop), rather than against all pushes to any branch?
Idea: Switch macOS tests to run against beta branch
Recently it has been taking forever (hours) for the standard unit test suite to get picked up by the macOS github runners. We also recently learned that the macOS runners use up our free computational credits at 10X the linux rate and 5X the Windows rate. Both of these things are kind of annoying. Question for discussion:

Should we change our CI workflow so that the tests in `unit` only get run on the macOS runners upon pushing to beta (i.e. when merged into develop), rather than against all pushes to any branch?


Discussion / hints:
I would be for this. The Mac tests rarely fail due to OS specific issues, so only checking on beta seems like a completely good solution to me.

A follow-up idea: We sometimes but rarely have Python version problems. Thoughts on only testing the lowest and highest versions of Python we support on most branches, and then testing all versions on either develop/beta or just beta?
Testing only the newest and oldest supported versions of python for most branches sounds good to me as well. I'll add these changes to my to-do list and update the workflows.
I would be for this. The Mac tests rarely fail due to OS specific issues, so only checking on beta seems like a completely good solution to me.

A follow-up idea: We sometimes but rarely have Python version problems. Thoughts on only testing the lowest and highest versions of Python we support on most branches, and then testing all versions on either develop/beta or just beta?
Testing only the newest and oldest supported versions of python for most branches sounds good to me as well. I'll add these changes to my to-do list and update the workflows.

Fix patch:
diff --git a/.github/workflows/autodeploy.yml b/.github/workflows/autodeploy.yml
index 5ad63e206..2d44439ab 100644
--- a/.github/workflows/autodeploy.yml
+++ b/.github/workflows/autodeploy.yml
@@ -23,11 +23,11 @@ jobs:
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0  # to fetch all branches and *tags* (needed to get version number correctly)
   
-      - uses: actions/setup-python@v2
+      - uses: actions/setup-python@v4
         name: Install Python
         with:
           python-version: '3.10'
@@ -39,7 +39,7 @@ jobs:
           CIBW_BUILD_VERBOSITY: 1
           CIBW_BEFORE_ALL_LINUX: ./.github/ci-scripts/before_install.sh
 
-      - uses: actions/upload-artifact@v2
+      - uses: actions/upload-artifact@v4
         with:
           path: ./wheelhouse/*.whl
 
@@ -49,11 +49,11 @@ jobs:
     #if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')   # doesn't work -- try using tags: above
     
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0  # to fetch all branches and *tags* (needed to get version number correctly)
 
-      - uses: actions/setup-python@v2
+      - uses: actions/setup-python@v4
         name: Install Python
         with:
           python-version: '3.10'
@@ -61,7 +61,7 @@ jobs:
       - name: Build sdist
         run: python setup.py sdist
 
-      - uses: actions/upload-artifact@v2
+      - uses: actions/upload-artifact@v4
         with:
           path: dist/*.tar.gz
 
@@ -73,7 +73,7 @@ jobs:
     # alternatively, to publish when a GitHub Release is created, use the following rule:
     # if: github.event_name == 'release' && github.event.action == 'published'
     steps:
-      - uses: actions/download-artifact@v2
+      - uses: actions/download-artifact@v4
         with:
           name: artifact
           path: dist
diff --git a/.github/workflows/extras.yml b/.github/workflows/extras.yml
index 1e7de0c29..d078237af 100644
--- a/.github/workflows/extras.yml
+++ b/.github/workflows/extras.yml
@@ -21,25 +21,25 @@ jobs:
     strategy:
       fail-fast: false # Finish all tests even if one fails
       matrix:
-        os: [ubuntu-20.04, windows-2019, macos-11]
+        os: [ubuntu-latest, windows-latest, macos-latest]
         python-version: [3.8, 3.9, '3.10', '3.11']
         
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Set up installation environment (Ubuntu or Windows)
-      if: ${{matrix.os == 'ubuntu-20.04' || matrix.os == 'windows-2019'}}
+      if: ${{matrix.os == 'ubuntu-latest' || matrix.os == 'windows-latest'}}
       run: |
         ./.github/ci-scripts/before_install.sh
     - name: Set up installation environment (MacOS)
-      if: ${{matrix.os == 'macos-11'}}
+      if: ${{matrix.os == 'macos-latest'}}
       run: |
         ./.github/ci-scripts/before_install_macos.sh
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v4
       with:
         python-version: ${{ matrix.python-version }}
     - name: Cache pip packages
-      uses: actions/cache@v2
+      uses: actions/cache@v3
       with:
         path: ~/.cache/pip
         key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}-${{ hashFiles('**/*requirements.txt') }}
@@ -51,17 +51,17 @@ jobs:
         python -m pip install -e .[testing]
         python setup.py build_ext --inplace
     - name: Run test_packages Ubuntu
-      if: ${{matrix.os == 'ubuntu-20.04'}}
+      if: ${{matrix.os == 'ubuntu-latest'}}
       run: |
         python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
         python -m pytest -v -n auto --dist loadscope --ignore=test/test_packages/mpi --ignore=test/test_packages/notebooks test/test_packages
     - name: Run test_packages Windows
-      if: ${{matrix.os == 'windows-2019'}}
+      if: ${{matrix.os == 'windows-latest'}}
       run: |
         python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
         python -m pytest -v -n auto --dist loadscope --ignore=test/test_packages/mpi --ignore=test/test_packages/notebooks test/test_packages
     - name: Run test_packages MacOS
-      if: ${{matrix.os == 'macos-11'}}
+      if: ${{matrix.os == 'macos-latest'}}
 
       run: |
         python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
diff --git a/.github/workflows/main-mac.yml b/.github/workflows/main-mac.yml
new file mode 100644
index 000000000..9e4b0290f
--- /dev/null
+++ b/.github/workflows/main-mac.yml
@@ -0,0 +1,61 @@
+# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
+
+name: Build and run tests (MacOS only, all Python versions)
+
+on:
+  push:
+    # Mac runners are expensive and oversubscribed. Only run on beta and master
+    branches: [ "beta", "master" ]
+  # Allow running manually from Actions tab
+  workflow_dispatch:
+
+env:
+  SKIP_DEAP: 1
+
+jobs:
+  build: # Main build + unit test check
+
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [macos-latest]
+        python-version: [3.8, 3.9, '3.10', '3.11']
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up installation environment (MacOS)
+      run: |
+        ./.github/ci-scripts/before_install_macos.sh
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Cache pip packages
+      uses: actions/cache@v3
+      with:
+        path: ~/.cache/pip
+        key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}-${{ hashFiles('**/*requirements.txt') }}
+    - name: Install package
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install wheel
+        python -m pip install flake8
+        python -m pip install -e .[testing]
+        python setup.py build_ext --inplace
+        # python -m pip freeze  # this isn't relevant anymore since pip install builds a wheel separately
+    - name: Lint with flake8
+      run: |
+        # Critical errors, exit on failure
+        flake8 . --count --show-source --statistics --config=.flake8-critical
+        # Standard PEP8, allowed to fail since exit-zero treats all errors as warnings
+        flake8 . --exit-zero --statistics
+    - name: Run unit tests MacOS  
+      if: ${{matrix.os == 'macos-latest'}}
+      run: |
+       python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
+       python -m pytest -n auto --dist loadscope --cov=pygsti test/unit
+
+
+
+
diff --git a/.github/workflows/main-minimal.yml b/.github/workflows/main-minimal.yml
new file mode 100644
index 000000000..6760a49a1
--- /dev/null
+++ b/.github/workflows/main-minimal.yml
@@ -0,0 +1,72 @@
+# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
+
+name: Build and run tests (no MacOS, low/high Python versions only)
+
+on:
+  push:
+    # Intended to be fast checks on non-main branches
+    branches-ignore: [ "beta", "develop", "master" ]
+  # Hacky way to only run pull requests from forked repositories (assumes : is not used in branch names unless forked)
+  # https://github.community/t/how-to-trigger-an-action-on-push-or-pull-request-but-not-both/16662/10
+  pull_request:
+    branches: [ "**:**" ]
+  # Allow running manually from Actions tab
+  workflow_dispatch:
+
+env:
+  SKIP_DEAP: 1
+
+jobs:
+  build: # Main build + unit test check
+
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+        python-version: [3.8,'3.11']
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up installation environment (Ubuntu or Windows)
+      if: ${{matrix.os == 'ubuntu-latest' || matrix.os == 'windows-latest'}}
+      run: |
+        ./.github/ci-scripts/before_install.sh
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Cache pip packages
+      uses: actions/cache@v3
+      with:
+        path: ~/.cache/pip
+        key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}-${{ hashFiles('**/*requirements.txt') }}
+    - name: Install package
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install wheel
+        python -m pip install flake8
+        python -m pip install -e .[testing]
+        python setup.py build_ext --inplace
+        # python -m pip freeze  # this isn't relevant anymore since pip install builds a wheel separately
+    - name: Lint with flake8
+      if: ${{matrix.os != 'windows-latest'}}
+      run: |
+        # Critical errors, exit on failure
+        flake8 . --count --show-source --statistics --config=.flake8-critical
+        # Standard PEP8, allowed to fail since exit-zero treats all errors as warnings
+        flake8 . --exit-zero --statistics
+    - name: Run unit tests ubuntu
+      if: ${{matrix.os == 'ubuntu-latest'}}
+      run: |
+       python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
+       python -m pytest -n auto --dist loadscope --cov=pygsti test/unit
+    - name: Run unit tests windows  
+      if: ${{matrix.os == 'windows-latest'}}
+      run: |
+       python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
+       python -m pytest -n auto --dist loadscope --cov=pygsti test/unit
+
+
+
+
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index f5361f6c8..d7134c893 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -5,9 +5,7 @@ name: Build and run tests
 
 on:
   push:
-    branches-ignore: [ "beta" ]
-  # Hacky way to only run pull requests from forked repositories (assumes : is not used in branch names unless forked)
-  # https://github.community/t/how-to-trigger-an-action-on-push-or-pull-request-but-not-both/16662/10
+    branches: [ "develop", "master" ]
   pull_request:
     branches: [ "**:**" ]
   # Allow running manually from Actions tab
@@ -22,25 +20,21 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        os: [ubuntu-20.04, windows-2019, macos-11]
+        os: [ubuntu-latest, windows-latest]
         python-version: [3.8, 3.9, '3.10', '3.11']
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Set up installation environment (Ubuntu or Windows)
-      if: ${{matrix.os == 'ubuntu-20.04' || matrix.os == 'windows-2019'}}
+      if: ${{matrix.os == 'ubuntu-latest' || matrix.os == 'windows-latest'}}
       run: |
         ./.github/ci-scripts/before_install.sh
-    - name: Set up installation environment (MacOS)
-      if: ${{matrix.os == 'macos-11'}}
-      run: |
-        ./.github/ci-scripts/before_install_macos.sh
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v4
       with:
         python-version: ${{ matrix.python-version }}
     - name: Cache pip packages
-      uses: actions/cache@v2
+      uses: actions/cache@v3
       with:
         path: ~/.cache/pip
         key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}-${{ hashFiles('**/*requirements.txt') }}
@@ -53,24 +47,19 @@ jobs:
         python setup.py build_ext --inplace
         # python -m pip freeze  # this isn't relevant anymore since pip install builds a wheel separately
     - name: Lint with flake8
-      if: ${{matrix.os != 'windows-2019'}}
+      if: ${{matrix.os != 'windows-latest'}}
       run: |
         # Critical errors, exit on failure
         flake8 . --count --show-source --statistics --config=.flake8-critical
         # Standard PEP8, allowed to fail since exit-zero treats all errors as warnings
         flake8 . --exit-zero --statistics
     - name: Run unit tests ubuntu
-      if: ${{matrix.os == 'ubuntu-20.04'}}
+      if: ${{matrix.os == 'ubuntu-latest'}}
       run: |
        python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
        python -m pytest -n auto --dist loadscope --cov=pygsti test/unit
     - name: Run unit tests windows  
-      if: ${{matrix.os == 'windows-2019'}}
-      run: |
-       python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
-       python -m pytest -n auto --dist loadscope --cov=pygsti test/unit
-    - name: Run unit tests MacOS  
-      if: ${{matrix.os == 'macos-11'}}
+      if: ${{matrix.os == 'windows-latest'}}
       run: |
        python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
        python -m pytest -n auto --dist loadscope --cov=pygsti test/unit
@@ -78,14 +67,14 @@ jobs:
   
   push: # Push to stable "beta" branch on successful build
 
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
 
     # Only run on "develop" branch if tests pass
     needs: build
     if: github.ref == 'refs/heads/develop' && github.event_name == 'push'
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
       with:
         fetch-depth: 0
         token: ${{ secrets.PYGSTI_TOKEN }}
diff --git a/.github/workflows/manualdeploy.yml b/.github/workflows/manualdeploy.yml
index b54972b15..15519b59f 100644
--- a/.github/workflows/manualdeploy.yml
+++ b/.github/workflows/manualdeploy.yml
@@ -16,11 +16,11 @@ jobs:
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0  # to fetch all branches and *tags* (needed to get version number correctly)
   
-      - uses: actions/setup-python@v2
+      - uses: actions/setup-python@v4
         name: Install Python
         with:
           python-version: '3.10'
@@ -32,7 +32,7 @@ jobs:
           CIBW_BUILD_VERBOSITY: 1
           CIBW_BEFORE_ALL_LINUX: ./.github/ci-scripts/before_install.sh
 
-      - uses: actions/upload-artifact@v2
+      - uses: actions/upload-artifact@v4
         with:
           path: ./wheelhouse/*.whl
 
@@ -41,11 +41,11 @@ jobs:
     runs-on: ubuntu-latest
     
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0  # to fetch all branches and *tags* (needed to get version number correctly)
 
-      - uses: actions/setup-python@v2
+      - uses: actions/setup-python@v4
         name: Install Python
         with:
           python-version: '3.10'
@@ -53,7 +53,7 @@ jobs:
       - name: Build sdist
         run: python setup.py sdist
 
-      - uses: actions/upload-artifact@v2
+      - uses: actions/upload-artifact@v4
         with:
           path: dist/*.tar.gz
 
@@ -61,7 +61,7 @@ jobs:
     needs: [build_wheels, build_sdist]
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/download-artifact@v2
+      - uses: actions/download-artifact@v4
         with:
           name: artifact
           path: dist
diff --git a/.github/workflows/notebook.yml b/.github/workflows/notebook.yml
index c3b3e3204..ac7dc36be 100644
--- a/.github/workflows/notebook.yml
+++ b/.github/workflows/notebook.yml
@@ -19,13 +19,13 @@ jobs:
     strategy:
       fail-fast: false # Finish all tests even if one fails
       matrix:
-        os: [ubuntu-20.04, windows-2019, macos-11]
+        os: [ubuntu-latest, windows-latest, macos-latest]
         python-version: [3.8, 3.9, '3.10', '3.11']
         
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Set up installation environment (Ubuntu or Windows)
-      if: ${{matrix.os == 'ubuntu-20.04' || matrix.os == 'windows-2019'}}
+      if: ${{matrix.os == 'ubuntu-latest' || matrix.os == 'windows-latest'}}
       run: |
         ./.github/ci-scripts/before_install.sh
         #download chp source code
@@ -33,7 +33,7 @@ jobs:
         #compile chp
         gcc -o ./jupyter_notebooks/Tutorials/algorithms/advanced/chp ./jupyter_notebooks/Tutorials/algorithms/advanced/chp.c 
     - name: Set up installation environment (MacOS)
-      if: ${{matrix.os == 'macos-11'}}
+      if: ${{matrix.os == 'macos-latest'}}
       run: |
         ./.github/ci-scripts/before_install_macos.sh
         #download chp source code
@@ -41,11 +41,11 @@ jobs:
         #compile chp source code
         gcc -o ./jupyter_notebooks/Tutorials/algorithms/advanced/chp ./jupyter_notebooks/Tutorials/algorithms/advanced/chp.c 
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v4
       with:
         python-version: ${{ matrix.python-version }}
     - name: Cache pip packages
-      uses: actions/cache@v2
+      uses: actions/cache@v3
       with:
         path: ~/.cache/pip
         key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}-${{ hashFiles('**/*requirements.txt') }}
@@ -57,17 +57,17 @@ jobs:
         python -m pip install -e .[testing]
         python setup.py build_ext --inplace
     - name: Run notebook regression ubuntu
-      if: ${{matrix.os == 'ubuntu-20.04'}}
+      if: ${{matrix.os == 'ubuntu-latest'}}
       run: |
        python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
        python -m pytest -n auto --nbval-lax --dist loadscope --nbval-current-env jupyter_notebooks
     - name: Run notebook regression windows 
-      if: ${{matrix.os == 'windows-2019'}}
+      if: ${{matrix.os == 'windows-latest'}}
       run: |
        python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
        python -m pytest -n auto --nbval-lax --dist loadscope --nbval-current-env jupyter_notebooks
     - name: Run notebook regression MacOS  
-      if: ${{matrix.os == 'macos-11'}}
+      if: ${{matrix.os == 'macos-latest'}}
       run: |
        python -Ic "import pygsti; print(pygsti.__version__); print(pygsti.__path__)"
        python -m pytest -n auto --nbval-lax --dist loadscope --nbval-current-env jupyter_notebooks


Test patch:
diff --git a/.github/workflows/testdeploy.yml b/.github/workflows/testdeploy.yml
index 2926f62eb..48badd5e9 100644
--- a/.github/workflows/testdeploy.yml
+++ b/.github/workflows/testdeploy.yml
@@ -19,11 +19,11 @@ jobs:
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0  # to fetch all branches and *tags* (needed to get version number correctly)
   
-      - uses: actions/setup-python@v2
+      - uses: actions/setup-python@v4
         name: Install Python
         with:
           python-version: '3.10'
@@ -41,7 +41,7 @@ jobs:
           CIBW_BUILD_VERBOSITY: 1
           CIBW_BEFORE_ALL_LINUX: ./.github/ci-scripts/before_install.sh
 
-      - uses: actions/upload-artifact@v2
+      - uses: actions/upload-artifact@v4
         with:
           path: ./wheelhouse/*.whl
 
@@ -49,11 +49,11 @@ jobs:
     name: Build source distribution
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0  # to fetch all branches and *tags* (needed to get version number correctly)
 
-      - uses: actions/setup-python@v2
+      - uses: actions/setup-python@v4
         name: Install Python
         with:
           python-version: '3.10'
@@ -61,7 +61,7 @@ jobs:
       - name: Build sdist
         run: python setup.py sdist
 
-      - uses: actions/upload-artifact@v2
+      - uses: actions/upload-artifact@v4
         with:
           path: dist/*.tar.gz
 
@@ -73,7 +73,7 @@ jobs:
     # alternatively, to publish when a GitHub Release is created, use the following rule:
     # if: github.event_name == 'release' && github.event.action == 'published'
     steps:
-      - uses: actions/download-artifact@v2
+      - uses: actions/download-artifact@v4
         with:
           name: artifact
           path: dist