Repository: SNEWS2/sntools (language: python)

Issue:
Packaging improvements
Prompted by @benjaminrose’s suggestions in [this comment](https://github.com/openjournals/joss-reviews/issues/2877#issuecomment-742051700) on the JOSS review:

> Tests don't install the package via pip. They currently test the source code, good. But don't test what will be sent installed from pypi, less good. This explains how you are testing your project directory and not how it will be installed by users, https://hynek.me/articles/testing-packaging/. This is a valid test suite, but may not be doing what you expect.

Some background:
Since I didn’t have experience with Python packaging before working on sntools, I tried to stick close to the official recommendations ([1](https://packaging.python.org/guides/distributing-packages-using-setuptools/#your-package), [2](https://packaging.python.org/tutorials/packaging-projects/#creating-the-package-files)), which did not mention the `src` folder structure recommended in the linked blog post. (Though now that I’ve looked into it, it seems to me that this is [hotly debated](https://github.com/pypa/packaging.python.org/issues/320).)


I think my first step to improve packaging should be to set up a GitHub Action that automatically packages sntools and publishes it to PyPI whenever there’s a new release. (That will largely eliminate human error, which is probably the main source of packaging issues.) I’ll check how it’s possible to install & test as part of that action and will update this issue accordingly.
Packaging improvements
Prompted by @benjaminrose’s suggestions in [this comment](https://github.com/openjournals/joss-reviews/issues/2877#issuecomment-742051700) on the JOSS review:

> Tests don't install the package via pip. They currently test the source code, good. But don't test what will be sent installed from pypi, less good. This explains how you are testing your project directory and not how it will be installed by users, https://hynek.me/articles/testing-packaging/. This is a valid test suite, but may not be doing what you expect.

Some background:
Since I didn’t have experience with Python packaging before working on sntools, I tried to stick close to the official recommendations ([1](https://packaging.python.org/guides/distributing-packages-using-setuptools/#your-package), [2](https://packaging.python.org/tutorials/packaging-projects/#creating-the-package-files)), which did not mention the `src` folder structure recommended in the linked blog post. (Though now that I’ve looked into it, it seems to me that this is [hotly debated](https://github.com/pypa/packaging.python.org/issues/320).)


I think my first step to improve packaging should be to set up a GitHub Action that automatically packages sntools and publishes it to PyPI whenever there’s a new release. (That will largely eliminate human error, which is probably the main source of packaging issues.) I’ll check how it’s possible to install & test as part of that action and will update this issue accordingly.


Discussion / hints:
Yes, this is a hotly debated topic. As such it should not hold up the acceptance to JOSS, but it is something to understand and document. Code always does what you say rather than what you intend.
I’ve [added an action](https://github.com/JostMigenda/sntools/commit/ca1f22b1390aeb06e1c37169d9aa87f340667208) to package up any new release and publish it to PyPI. Made a new release (v0.7.1) a few hours ago to test this and it appears to work well. This action will largely eliminate the possibility of human error & will work without modification for typical changes to sntools (i.e. new detectors, input formats, etc.).

I’ve made a private note to revisit this if there are major changes to the structure of sntools & will also leave this issue open for visibility.
Yes, this is a hotly debated topic. As such it should not hold up the acceptance to JOSS, but it is something to understand and document. Code always does what you say rather than what you intend.
I’ve [added an action](https://github.com/JostMigenda/sntools/commit/ca1f22b1390aeb06e1c37169d9aa87f340667208) to package up any new release and publish it to PyPI. Made a new release (v0.7.1) a few hours ago to test this and it appears to work well. This action will largely eliminate the possibility of human error & will work without modification for typical changes to sntools (i.e. new detectors, input formats, etc.).

I’ve made a private note to revisit this if there are major changes to the structure of sntools & will also leave this issue open for visibility.

Fix patch:
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
index fa8b43c..d310228 100644
--- a/.github/workflows/publish.yml
+++ b/.github/workflows/publish.yml
@@ -21,11 +21,11 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install setuptools wheel twine
+        pip install build twine
     - name: Build and publish
       env:
         TWINE_USERNAME: __token__
         TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
       run: |
-        python setup.py sdist bdist_wheel
+        python -m build
         twine upload dist/*
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 0000000..486de6d
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,64 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+
+[project]
+name = "sntools"
+dynamic = ["version"]
+authors = [
+    {name="Jost Migenda"},
+    {name="sntools Contributors"},
+]
+description = "Event generator for supernova burst neutrinos"
+readme = "README.md"
+license = {file = "LICENSE"}
+
+requires-python = ">=3.8"  # TODO: Also update classifiers below!
+dependencies = [
+    "numpy",
+    "scipy",
+    "h5py >= 2.10",
+    "snewpy ~=1.3.0",
+    "uproot"
+]
+
+keywords = ["astrophysics", "supernova", "neutrino", "event generator"]
+classifiers = [
+    'Private :: Do Not Upload',  # TODO: only for testing
+    'License :: OSI Approved :: BSD License',
+    'Development Status :: 5 - Production/Stable',
+    'Intended Audience :: Science/Research',
+    'Topic :: Scientific/Engineering',
+    'Topic :: Scientific/Engineering :: Astronomy',
+    'Topic :: Scientific/Engineering :: Physics',
+    'Natural Language :: English',
+    'Operating System :: OS Independent',
+    'Programming Language :: Python :: 3',
+    'Programming Language :: Python :: 3 :: Only',
+    'Programming Language :: Python :: 3.8',
+    'Programming Language :: Python :: 3.9',
+    'Programming Language :: Python :: 3.10',
+    'Programming Language :: Python :: 3.11',
+]
+
+
+[project.scripts]
+sntools = "sntools.genevts:main"
+
+
+[project.urls]
+"Homepage" = "https://github.com/JostMigenda/sntools"
+"Bug Tracker" = "https://github.com/JostMigenda/sntools/issues"
+
+
+[tool.setuptools.dynamic]
+version = {attr = "sntools.__version__"}
+
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = [
+    "sntools",
+    "sntools.*",
+]
diff --git a/requirements.txt b/requirements.txt
deleted file mode 100644
index 017a2a3..0000000
--- a/requirements.txt
+++ /dev/null
@@ -1,5 +0,0 @@
-numpy
-scipy
-h5py >= 2.10
-snewpy ~=1.3.0
-uproot
diff --git a/setup.py b/setup.py
deleted file mode 100644
index 86c69c1..0000000
--- a/setup.py
+++ /dev/null
@@ -1,132 +0,0 @@
-"""A setuptools based setup module.
-
-Derived from the template in
-https://packaging.python.org/guides/distributing-packages-using-setuptools/
-https://github.com/pypa/sampleproject
-"""
-
-from setuptools import setup, find_packages
-
-import sntools
-
-setup(
-    # The name determines how to install the project (`pip install sntools`)
-    # and where it lives on PyPI: https://pypi.org/project/sntools/
-    #
-    # For restrictions on valid project names, see
-    # https://packaging.python.org/specifications/core-metadata/#name
-    name='sntools',  # Required
-
-    # Versions should comply with PEP 440:
-    # https://www.python.org/dev/peps/pep-0440/
-    #
-    # To single-source the version across setup.py and the project code, see
-    # https://packaging.python.org/en/latest/single_source_version.html
-    version=sntools.__version__,  # Required
-
-    # A one-line description or tagline of what this project does.
-    # https://packaging.python.org/specifications/core-metadata/#summary
-    description='Event generator for supernova burst neutrinos',  # Optional
-
-    # An optional longer description of the project that represents
-    # the body of text which users will see when they visit PyPI.
-    # https://packaging.python.org/specifications/core-metadata/#description-optional
-    long_description=open('README.md', 'rb').read().decode('utf-8'),  # Optional
-
-    # long_description format: text/x-rst (default), text/plain, text/markdown
-    # https://packaging.python.org/specifications/core-metadata/#description-content-type-optional
-    long_description_content_type='text/markdown',  # Optional
-
-    # Link to your project's main homepage.
-    # https://packaging.python.org/specifications/core-metadata/#home-page-optional
-    # url='https://github.com/JostMigenda/sntools',  # Optional
-
-    # Author name or name of the organization which owns the project.
-    author='Jost Migenda',  # Optional
-
-    # Email address corresponding to the author listed above.
-    # author_email='author@example.com',  # Optional
-
-    # Classifiers help users find your project by categorizing it.
-    # For a list of valid classifiers, see https://pypi.org/classifiers/
-    classifiers=[  # Optional
-        'License :: OSI Approved :: BSD License',
-        'Development Status :: 5 - Production/Stable',
-        'Intended Audience :: Science/Research',
-        'Topic :: Scientific/Engineering',
-        'Topic :: Scientific/Engineering :: Astronomy',
-        'Topic :: Scientific/Engineering :: Physics',
-        'Natural Language :: English',
-        'Operating System :: OS Independent',
-
-        # Supported Python versions. These classifiers are *not*
-        # checked by 'pip install'. See instead 'python_requires' below.
-        'Programming Language :: Python :: 3',
-        'Programming Language :: Python :: 3 :: Only',
-        'Programming Language :: Python :: 3.8',
-        'Programming Language :: Python :: 3.9',
-        'Programming Language :: Python :: 3.10',
-        'Programming Language :: Python :: 3.11',
-    ],
-
-    # Comma-separated list of keywords which will appear on the PyPI project
-    # page. Used to assist searching for the distribution in a larger catalog.
-    keywords='astrophysics, supernova, neutrino, event generator',  # Optional
-
-    # Specify if source code is in a subdirectory under the project root.
-    # package_dir={'': 'src'},  # Optional
-
-    # Specify a list of package directories manually or use find_packages().
-    # To distribute a single Python file (`foo.py`), use the `py_modules`
-    # argument instead: `py_modules=["foo"],`
-    packages=find_packages(include=('sntools', 'sntools.*')),  # Required
-
-    # Supported Python versions. 'pip install' will check this
-    # and refuse to install the project if the version does not match. See
-    # https://packaging.python.org/guides/distributing-packages-using-setuptools/#python-requires
-    # When the supported version changes, remember to also update `classifiers` above!
-    python_requires='>=3.8, <4',
-
-    # This field lists other packages that your project depends on to run.
-    # These packages will be installed by pip when your project is installed.
-    #
-    # For an analysis of "install_requires" vs pip's requirements files see:
-    # https://packaging.python.org/en/latest/requirements.html
-    install_requires=[line.strip() for line in open("requirements.txt")],  # Optional
-
-    # Additional groups of dependencies (e.g. for development).
-    # Users can install these using the "extras" syntax, for example:
-    #   $ pip install sntools[dev]
-    extras_require={  # Optional
-        # 'dev': ['black', 'flake8'],
-    },
-
-    # If there are data files included in your packages that need to be
-    # installed, specify them here.
-    # package_data={  # Optional
-    #     'sample': ['package_data.dat'],  # TODO: Include Nakazato flux files here?
-    # },
-
-    # Although 'package_data' is the preferred approach, in some case you may
-    # need to place data files outside of your packages. See:
-    # http://docs.python.org/distutils/setupscript.html#installing-additional-files
-    #
-    # In this case, 'data_file' will be installed into '<sys.prefix>/my_data'
-    # data_files=[('my_data', ['data/data_file'])],  # Optional
-
-    # `pip` can create executable scripts for each target platform that can be
-    # called as `sntools [args]` instead of `python genevts.py [args]`:
-    entry_points={  # Optional
-        'console_scripts': [
-            'sntools = sntools.genevts:main',
-        ],
-    },
-
-    # List additional URLs that are relevant to your project as a dict.
-    # The key is used as the link text on the PyPI project page.
-    # https://packaging.python.org/specifications/core-metadata/#project-url-multiple-use
-    project_urls={  # Optional
-        'Bug Reports': 'https://github.com/JostMigenda/sntools/issues',
-        'Source': 'https://github.com/JostMigenda/sntools',
-    },
-)
diff --git a/sntools/__init__.py b/src/sntools/__init__.py
similarity index 98%
rename from sntools/__init__.py
rename to src/sntools/__init__.py
index a5c23e1..72baa60 100644
--- a/sntools/__init__.py
+++ b/src/sntools/__init__.py
@@ -28,7 +28,7 @@ def setup():
 
     flux_dir = 'fluxes/'
     flux_file = flux_dir + 'intp2001.data'
-    flux_url = 'https://raw.githubusercontent.com/JostMigenda/sntools/master/fluxes/intp2001.data'
+    flux_url = 'https://raw.githubusercontent.com/JostMigenda/sntools/main/fluxes/intp2001.data'
     if os.path.exists(flux_file):
         tryprint(u"\u2705", "[SUCCESS]")
         print("Using sample flux file at " + flux_file)
diff --git a/sntools/channel.py b/src/sntools/channel.py
similarity index 100%
rename from sntools/channel.py
rename to src/sntools/channel.py
diff --git a/sntools/detectors.py b/src/sntools/detectors.py
similarity index 100%
rename from sntools/detectors.py
rename to src/sntools/detectors.py
diff --git a/sntools/event.py b/src/sntools/event.py
similarity index 100%
rename from sntools/event.py
rename to src/sntools/event.py
diff --git a/sntools/formats/__init__.py b/src/sntools/formats/__init__.py
similarity index 100%
rename from sntools/formats/__init__.py
rename to src/sntools/formats/__init__.py
diff --git a/sntools/formats/gamma.py b/src/sntools/formats/gamma.py
similarity index 100%
rename from sntools/formats/gamma.py
rename to src/sntools/formats/gamma.py
diff --git a/sntools/formats/nakazato.py b/src/sntools/formats/nakazato.py
similarity index 100%
rename from sntools/formats/nakazato.py
rename to src/sntools/formats/nakazato.py
diff --git a/sntools/formats/princeton.py b/src/sntools/formats/princeton.py
similarity index 100%
rename from sntools/formats/princeton.py
rename to src/sntools/formats/princeton.py
diff --git a/sntools/formats/totani.py b/src/sntools/formats/totani.py
similarity index 100%
rename from sntools/formats/totani.py
rename to src/sntools/formats/totani.py
diff --git a/sntools/formats/warren2020.py b/src/sntools/formats/warren2020.py
similarity index 100%
rename from sntools/formats/warren2020.py
rename to src/sntools/formats/warren2020.py
diff --git a/sntools/genevts.py b/src/sntools/genevts.py
similarity index 100%
rename from sntools/genevts.py
rename to src/sntools/genevts.py
diff --git a/sntools/interaction_channels/__init__.py b/src/sntools/interaction_channels/__init__.py
similarity index 100%
rename from sntools/interaction_channels/__init__.py
rename to src/sntools/interaction_channels/__init__.py
diff --git a/sntools/interaction_channels/c12e.py b/src/sntools/interaction_channels/c12e.py
similarity index 100%
rename from sntools/interaction_channels/c12e.py
rename to src/sntools/interaction_channels/c12e.py
diff --git a/sntools/interaction_channels/c12eb.py b/src/sntools/interaction_channels/c12eb.py
similarity index 100%
rename from sntools/interaction_channels/c12eb.py
rename to src/sntools/interaction_channels/c12eb.py
diff --git a/sntools/interaction_channels/c12nc.py b/src/sntools/interaction_channels/c12nc.py
similarity index 100%
rename from sntools/interaction_channels/c12nc.py
rename to src/sntools/interaction_channels/c12nc.py
diff --git a/sntools/interaction_channels/es.py b/src/sntools/interaction_channels/es.py
similarity index 100%
rename from sntools/interaction_channels/es.py
rename to src/sntools/interaction_channels/es.py
diff --git a/sntools/interaction_channels/ibd.py b/src/sntools/interaction_channels/ibd.py
similarity index 100%
rename from sntools/interaction_channels/ibd.py
rename to src/sntools/interaction_channels/ibd.py
diff --git a/sntools/interaction_channels/o16e.py b/src/sntools/interaction_channels/o16e.py
similarity index 100%
rename from sntools/interaction_channels/o16e.py
rename to src/sntools/interaction_channels/o16e.py
diff --git a/sntools/interaction_channels/o16eb.py b/src/sntools/interaction_channels/o16eb.py
similarity index 100%
rename from sntools/interaction_channels/o16eb.py
rename to src/sntools/interaction_channels/o16eb.py
diff --git a/sntools/interaction_channels/o16nc_n.py b/src/sntools/interaction_channels/o16nc_n.py
similarity index 100%
rename from sntools/interaction_channels/o16nc_n.py
rename to src/sntools/interaction_channels/o16nc_n.py
diff --git a/sntools/interaction_channels/o16nc_p.py b/src/sntools/interaction_channels/o16nc_p.py
similarity index 100%
rename from sntools/interaction_channels/o16nc_p.py
rename to src/sntools/interaction_channels/o16nc_p.py
diff --git a/sntools/interaction_channels/ps.py b/src/sntools/interaction_channels/ps.py
similarity index 100%
rename from sntools/interaction_channels/ps.py
rename to src/sntools/interaction_channels/ps.py
diff --git a/sntools/transformation.py b/src/sntools/transformation.py
similarity index 100%
rename from sntools/transformation.py
rename to src/sntools/transformation.py


Test patch:
diff --git a/.github/workflows/test-and-lint.yml b/.github/workflows/test-and-lint.yml
index 3e38215..b50923e 100644
--- a/.github/workflows/test-and-lint.yml
+++ b/.github/workflows/test-and-lint.yml
@@ -32,13 +32,10 @@ jobs:
       run: python -m unittest discover
     - name: Run integration test
       run: |
-        # Change into sub-directory, so we import the installed version and not the local directory
-        mkdir foo
-        cd foo
         # Run setup function
         python -c 'import sntools; sntools.setup()'
         # Compare with sample output
-        diff outfile.kin ../tests/sample_outfile.kin
+        diff outfile.kin tests/sample_outfile.kin
 
   lint:
     runs-on: 'ubuntu-latest'