Repository: tenpy/tenpy (language: python)

Issue:
Review MANIFEST.in
The contents look outdated.
- `setup.cfg` and `pytest.ini` no longer exist
- `pyproject.toml` should probably make this list

Should also double check if the file is actually needed / used by the build system.
If it is used, I am confused that the non-existing files have not caused any problems yet since #237.


Discussion / hints:
It should be used by setuptools to decide what's included in the pip package.
The reason why it didn't make issues yet is that we didn't have official releases since #237... 

Fix patch:
diff --git a/.github/workflows/publish-pypi.yml b/.github/workflows/publish-pypi.yml
index 73e6645cc..728beb7b3 100644
--- a/.github/workflows/publish-pypi.yml
+++ b/.github/workflows/publish-pypi.yml
@@ -6,29 +6,86 @@ on:
     types: [created]
 
 jobs:
-  build-n-publish:
+  build-native-wheels:
+    name: Build wheels on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-13, macos-14]
+
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v4
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.x'
+
+    - name: Build wheels
+      uses: pypa/cibuildwheel@v2.16.5
+      with:
+        output-dir: dist
+      env:
+          # Disable building PyPy wheels on all platforms
+          # Disable building 32bit windows wheels (h5py does not support it)
+          # Diable musllinux : would need to install BLAS/LAPACK
+          CIBW_SKIP: pp* *-win32 *-musllinux_*
+          # Build separate wheels for macOS's different architectures.
+          CIBW_ARCHS_MACOS: "x86_64 arm64"
+          # Build only on Linux architectures that don't need qemu emulation.
+          CIBW_ARCHS_LINUX: "x86_64"
+          # Include latest Python beta.
+          CIBW_PRERELEASE_PYTHONS: True
+          # Run the test suite after each build.
+          CIBW_TEST_REQUIRES: pytest>=0.6 numpy>=1.13 scipy>=0.18 bottleneck h5py pyyaml
+          CIBW_TEST_COMMAND: pytest {project}/tests
+    
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: wheels-${{ matrix.os }}
+        path: dist/*.whl
+  
+  build-sdist-and-upload:
     runs-on: ubuntu-latest
+    needs: 'build-native-wheels'
 
     steps:
     - name: Checkout
       uses: actions/checkout@v4
+
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
-        python-version: '3.7'  # use minimum version that we support
-    - name: Install build dependencies
+        python-version: '3.x'
+
+    - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install --upgrade setuptools wheel build
-    - name: Build
-      run: python -m build .
-    - name: Install for test
-      run: python -m pip install .
-    - name: quicktest
+        python -m pip install --upgrade build twine
+    
+    - name: Download wheels from build artifacts
+      uses: actions/download-artifact@v4
+      with:
+        pattern: wheels-*
+        merge-multiple: true
+        path: dist-wheels/
+    
+    - name: Build package
       run: |
-        python -m pip install pytest
-        pytest -m "not slow"
-    - name: Publish distribution to PyPI
+        python -m build --sdist
+        python -m twine check --strict dist/*
+        python -m twine check --strict dist-wheels/*
+
+    - name: Publish wheels to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        password: ${{ secrets.pypi_password }}
+        packages-dir: dist-wheels/
+
+    - name: Publish sdist to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
         password: ${{ secrets.pypi_password }}
diff --git a/MANIFEST.in b/MANIFEST.in
index 58dde750c..50b87470d 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -4,8 +4,7 @@ recursive-include tenpy *.cpp
 recursive-include tests *.py
 
 include setup.py
-include setup.cfg
-include pytest.ini
+include pyproject.toml
 include requirements.txt
 include environment.yml
 include AUTHORS.txt
diff --git a/doc/contr/release.rst b/doc/contr/release.rst
index 5d5820c7a..c73721e44 100644
--- a/doc/contr/release.rst
+++ b/doc/contr/release.rst
@@ -8,12 +8,16 @@ Follow this checklist when creating a new release, i.e. updating the version num
    A convenient way to check this is to re-run the github.com actions on the latest commit.
 
 2. Update the changelog and release notes.
-   - Make sure all important changes are listed in ``docs/changelog/latest.rst``.
+
+   - Make sure all important changes are listed in ``doc/changelog/latest.rst``.
    - Rename that file to the next version number.
-   - Add it to the toctree in ``docs/releases.rst``.
-   - Create a new ``docs/changelog/latest.rst`` from the template.
+   - Add it to the toctree in ``doc/releases.rst``.
+   - Create a new ``doc/changelog/latest.rst`` from the template.
 
-3. In the files ``setup.py`` and ``tenpy/version.py``, update the release number and set released=True.
+3. Update ``tenpy/version.py``
+  
+   - Update the release number.
+   - set ``released=True``.
 
 4. Create the version commit and a tag::
     
@@ -23,25 +27,52 @@ Follow this checklist when creating a new release, i.e. updating the version num
    Change the version number appropriately.
    You should GPG sign the commit.
 
-5. Reset the released=False flag in ``setup.py`` and ``tenpy/version.py``.
+5. Reset the ``released=False`` flag in ``tenpy/version.py``.
    Commit and push these changes::
    
      git add setup.py tenpy/version.py
      git commit -m "reset released=False"
      git push
-     git push origin v0.42.1 # also push the tag
+     git push origin v0.42.1  # also push the tag
 
 6. Pushing a tag should cause a github action to publish to test.pypi.org.
    It executes ``.github/workflows/publish-test-pypi.yml``.
    Double check that everything looks ok there.
-   Perform a test installation with ``pip install -i https://test.pypi.org/simple/ physics-tenpy``
-   and check it by running the tests.
 
-7. Create a release with release notes on github.
+7. Perform a test installation.
+
+   This is best done in a dedicated fresh environment.
+   Since the dependencies (numpy, scipy) are not available on TestPyPI, we need to pre-install them.
+   On live PyPI we could omit them here and ``pip install physics-tenpy`` would install them for us::
+   
+     conda create -n tenpytest python=3.12 pip numpy scipy h5py pyyaml pytest ipython
+     conda activate tenpytest
+     pip install -i https://test.pypi.org/simple/ physics-tenpy==0.42.1
+   
+   Make sure your working directory is *not* the root folder of a tenpy repo.
+   That would cause the tenpy version from the repo to be in the path and probably take
+   precedence over the tenpy version that we just installed.
+   In e.g. an ``ipython`` console, check that tenpy can be imported without errors and warnings and
+   check that you are importing the correct version::
+   
+     In [1] import tenpy as tp
+     In [2]: print(tp.__file__)  # make sure the following path contains 'site-packages'.
+     /Users/jakobunfried/anaconda3/envs/tenpytest/lib/python3.12/site-packages/tenpy/__init__.py
+     In [3]: tp.show_config()  # check the version number
+     tenpy 0.11.0.dev0+unknown (compiled without HAVE_MKL),
+     git revision unknown using
+     python 3.12.1 | packaged by Anaconda, Inc. | (main, Jan 19 2024, 09:45:58) [Clang 14.0.6 ]
+     numpy 1.26.3, scipy 1.11.4
+   
+   Run the test suite. Again, ensure that you are not in the root directory of the repo::
+
+      pytest path/to/local/repo/tests
+
+8. Create a release with release notes on github.
    The release triggers the github action for uploading the package to PyPI.
    It executes ``.github/workflows/publish-pypi.yml``.
 
-8. Wait for conda-forge bot to create a pull request in the `feedstock repo <https://github.com/conda-forge/physics-tenpy-feedstock>`_
+9. Wait for conda-forge bot to create a pull request in the `feedstock repo <https://github.com/conda-forge/physics-tenpy-feedstock>`_
    and merge it.
 
 
diff --git a/environment.yml b/environment.yml
index 00d79e81f..5bea0cc0d 100644
--- a/environment.yml
+++ b/environment.yml
@@ -11,7 +11,7 @@ dependencies:
   - libblas=*=*mkl  # pin BLAS to use MKL
   - bottleneck
   - pip
-  - setuptools>=30.3.0
+  - setuptools>=68.0.0
   - h5py
   - pyyaml
   - pytest>=0.6
diff --git a/environment_other.yml b/environment_other.yml
index 62439c64f..90e1c27f7 100644
--- a/environment_other.yml
+++ b/environment_other.yml
@@ -8,7 +8,7 @@ dependencies:
   - mkl-devel
   - bottleneck
   - pip
-  - setuptools>=30.3.0
+  - setuptools>=68.0.0
   - h5py
   - pyyaml
   - pytest>=0.6
diff --git a/pyproject.toml b/pyproject.toml
index dbc647031..b62636c18 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,5 +1,5 @@
 [build-system]
-requires = ["setuptools>=30.3.0", "numpy>1.13", "scipy>=0.18", "Cython>0.29"]
+requires = ["setuptools>=68.0.0", "numpy>1.13", "scipy>=0.18", "Cython>0.29"]
 build-backend = "setuptools.build_meta"
 
 [project]
@@ -7,7 +7,7 @@ name = "physics-tenpy"
 dynamic = ["version"]
 description = "Simulation of quantum many-body systems with tensor networks in Python"
 readme = {file = "README.rst", content-type = "text/x-rst"}
-requires-python = ">=3.6"
+requires-python = ">=3.7"
 license = {text = "GPLv3"}
 authors = [
     {name = "TeNPy Developer Team"},
@@ -26,7 +26,6 @@ classifiers = [
     "Programming Language :: C",
     "Programming Language :: Python",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.6",
     "Programming Language :: Python :: 3.7",
     "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",


Test patch:
diff --git a/.github/workflows/publish-test-pypi.yml b/.github/workflows/publish-test-pypi.yml
index 9d0722bf5..c1505fe38 100644
--- a/.github/workflows/publish-test-pypi.yml
+++ b/.github/workflows/publish-test-pypi.yml
@@ -1,4 +1,4 @@
-name: Upload Python Package to test.pypi
+name: Build wheels and Upload Python Package to test.pypi
 
 on:
   # when a release is created
@@ -11,30 +11,106 @@ on:
       - '**'  # on any tag
 
 jobs:
-  build-n-publish:
+
+  build-native-wheels:
+    name: Build wheels on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-13, macos-14]
+
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v4
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.x'
+
+    - name: Build wheels
+      uses: pypa/cibuildwheel@v2.16.5
+      with:
+        output-dir: dist
+      env:
+          # Disable building PyPy wheels on all platforms
+          # Disable building 32bit windows wheels (h5py does not support it)
+          # Diable musllinux : would need to install BLAS/LAPACK
+          CIBW_SKIP: pp* *-win32 *-musllinux_*
+          # Build separate wheels for macOS's different architectures.
+          CIBW_ARCHS_MACOS: "x86_64 arm64"
+          # Build only on Linux architectures that don't need qemu emulation.
+          CIBW_ARCHS_LINUX: "x86_64"
+          # Include latest Python beta.
+          CIBW_PRERELEASE_PYTHONS: True
+          # Run the test suite after each build.
+          CIBW_TEST_REQUIRES: pytest>=0.6 numpy>=1.13 scipy>=0.18 bottleneck h5py pyyaml
+          CIBW_TEST_COMMAND: pytest {project}/tests
+    
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: wheels-${{ matrix.os }}
+        path: dist/*.whl
+  
+  build-sdist-and-upload:
     runs-on: ubuntu-latest
+    needs: 'build-native-wheels'
 
     steps:
     - name: Checkout
       uses: actions/checkout@v4
+
     - name: Set up Python
       uses: actions/setup-python@v5
       with:
-        python-version: '3.7'  # use minimum version that we support
-    - name: Install build tools
+        python-version: '3.x'
+
+    - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install --upgrade setuptools wheel build
-    - name: Build
-      run: python -m build .
-    - name: Install for test
-      run: python -m pip install .
-    - name: quicktest
+        python -m pip install --upgrade build twine
+    
+    - name: Download wheels from build artifacts
+      uses: actions/download-artifact@v4
+      with:
+        pattern: wheels-*
+        merge-multiple: true
+        path: dist-wheels/
+    
+    - name: Build package
       run: |
-        python -m pip install pytest
-        pytest -m "not slow"
-    - name: Publish distribution to Test PyPI
+        python -m build --sdist
+        python -m twine check --strict dist/*
+        python -m twine check --strict dist-wheels/*
+
+    - name: Publish wheels to TestPyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        password: ${{ secrets.test_pypi_password }}
+        repository_url: https://test.pypi.org/legacy/
+        packages-dir: dist-wheels/
+
+    - name: Publish sdist to TestPyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
         password: ${{ secrets.test_pypi_password }}
         repository_url: https://test.pypi.org/legacy/
+    
+    - name: Temporary Check
+      # If this behaves as expected, we can unify the two publish actions.
+      # The idea would be to uncomment the PyPI publish below and add the same if clause.
+      if: github.event_name == 'release' && github.event.action == 'created'
+      run: echo "This is a release"
+
+    # - name: Publish wheels to PyPI
+    #   uses: pypa/gh-action-pypi-publish@release/v1
+    #   with:
+    #     password: ${{ secrets.pypi_password }}
+    #     packages-dir: dist-wheels/
+
+    # - name: Publish sdist to PyPI
+    #   uses: pypa/gh-action-pypi-publish@release/v1
+    #   with:
+    #     password: ${{ secrets.pypi_password }}