Repository: WenjieDu/PyPOTS (language: python)

Issue:
pyproject.toml not work with flake8
### Issue description

configs for flake8 in pyproject.toml are not working

https://github.com/WenjieDu/PyPOTS/actions/runs/10111818488/job/27964535454
Add toml config and gather dependency files
<!-- 🚨Please create your PR to merge code from your branch to `dev` branch here, rather than `main`. -->

# What does this PR do?

<!--
Congrats! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution 😉.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, I will review your PR shortly. I may suggest changes to make the code even better 🤝.
-->

<!-- Remove if not applicable -->

1. fixing #476;
2. put all dependency files in one dir;


## Before submitting

<!-- You can remove checks that are not relevant to this PR. -->

- [x] Was this discussed/approved via a GitHub issue? Please add a link to it if that's the case.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have written necessary tests and already run them locally.



Discussion / hints:

Fix patch:
diff --git a/.github/workflows/linting.yml b/.github/workflows/linting.yml
index 4aca2b57..341a5934 100644
--- a/.github/workflows/linting.yml
+++ b/.github/workflows/linting.yml
@@ -23,8 +23,8 @@ jobs:
 
             - name: Install Flake8
               run: |
-                pip install flake8
+                pip install flake8 flake8-pyproject
 
             - name: Run linting
               run: |
-                flake8 .
\ No newline at end of file
+                flake8 .
diff --git a/.github/workflows/publish_to_PyPI.yml b/.github/workflows/publish_to_PyPI.yml
index ed05f120..6b2bffe2 100644
--- a/.github/workflows/publish_to_PyPI.yml
+++ b/.github/workflows/publish_to_PyPI.yml
@@ -39,7 +39,7 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         pip install build wheel
-        pip install -r requirements.txt
+        pip install -r requirements/requirements.txt
 
     - name: Fetch the test environment details
       run: |
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
index bba6bafd..5b812779 100644
--- a/.readthedocs.yaml
+++ b/.readthedocs.yaml
@@ -23,7 +23,7 @@ build:
             - python -m pip install --upgrade pip
             - pip install torch==1.13.1 -f https://download.pytorch.org/whl/cpu
             - pip install torch-geometric==2.3.1 torch-scatter==2.1.1 torch-sparse==0.6.17 -f "https://data.pyg.org/whl/torch-1.13.1+cpu.html"
-            - pip install -r requirements.txt
+            - pip install -r requirements/requirements.txt
             - pip install pypots
             - pip install sphinx==6.2.1 docutils==0.19 sphinxcontrib-bibtex==2.1.4 sphinxcontrib-gtagjs sphinx-autodoc-typehints furo==2023.07.26
             - git clone https://github.com/WenjieDu/TSDB TSDB_repo --depth 1 && pip install ./TSDB_repo
diff --git a/MANIFEST.in b/MANIFEST.in
deleted file mode 100644
index a65e0605..00000000
--- a/MANIFEST.in
+++ /dev/null
@@ -1,3 +0,0 @@
-prune tests
-prune pypots/*/template
-prune pypots/*/README.md
diff --git a/pypots/__init__.py b/pypots/__init__.py
index c19292a5..f76e968d 100644
--- a/pypots/__init__.py
+++ b/pypots/__init__.py
@@ -6,26 +6,16 @@
 # License: BSD-3-Clause
 
 
-# PyPOTS version
-#
-# PEP0440 compatible formatted version, see:
-# https://www.python.org/dev/peps/pep-0440/
-# Generic release markers:
-# X.Y
-# X.Y.Z # For bugfix releases
-#
-# Admissible pre-release markers:
-# X.YaN # Alpha release
-# X.YbN # Beta release
-# X.YrcN # Release Candidate
-# X.Y # Final release
-#
-# Dev branch marker is: 'X.Y.dev' or 'X.Y.devN' where N is an integer.
-# 'X.Y.dev0' is the canonical version of 'X.Y.dev'
-__version__ = "0.7"
-
-
-from . import imputation, classification, clustering, forecasting, optim, data, utils
+from . import (
+    imputation,
+    classification,
+    clustering,
+    forecasting,
+    optim,
+    data,
+    utils,
+)
+from .version import __version__
 
 __all__ = [
     "imputation",
diff --git a/pypots/version.py b/pypots/version.py
new file mode 100644
index 00000000..582517f6
--- /dev/null
+++ b/pypots/version.py
@@ -0,0 +1,25 @@
+"""
+
+"""
+
+# Created by Wenjie Du <wenjay.du@gmail.com>
+# License: BSD-3-Clause
+
+
+# PyPOTS version
+#
+# PEP0440 compatible formatted version, see:
+# https://www.python.org/dev/peps/pep-0440/
+# Generic release markers:
+# X.Y
+# X.Y.Z # For bugfix releases
+#
+# Admissible pre-release markers:
+# X.YaN # Alpha release
+# X.YbN # Beta release
+# X.YrcN # Release Candidate
+# X.Y # Final release
+#
+# Dev branch marker is: 'X.Y.dev' or 'X.Y.devN' where N is an integer.
+# 'X.Y.dev0' is the canonical version of 'X.Y.dev'
+__version__ = "0.7"
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 00000000..75c7d043
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,91 @@
+[build-system]
+requires = ["setuptools>=71"]
+
+[project]
+name = "pypots"
+description = "A Python Toolbox for Machine Learning on Partially-Observed Time Series"
+authors = [{ name = "Wenjie Du", email = "wenjay.du@gmail.com" }]
+dynamic = ["version", "readme", "dependencies", "optional-dependencies"]
+license = { file = "LICENSE" }
+requires-python = ">=3.8"
+classifiers = [
+    "Development Status :: 5 - Production/Stable",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Education",
+    "Intended Audience :: Science/Research",
+    "Intended Audience :: Healthcare Industry",
+    "License :: OSI Approved :: BSD License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries :: Application Frameworks",
+]
+keywords = [
+    "data science",
+    "data mining",
+    "neural networks",
+    "machine learning",
+    "deep learning",
+    "artificial intelligence",
+    "time-series analysis",
+    "time series",
+    "imputation",
+    "interpolation",
+    "classification",
+    "clustering",
+    "forecasting",
+    "partially observed",
+    "irregular sampled",
+    "partially-observed time series",
+    "incomplete time series",
+    "missing data",
+    "missing values",
+]
+
+[project.scripts]
+pypots-cli = "pypots.cli.pypots_cli:main"
+
+[project.urls]
+Source = "https://github.com/WenjieDu/PyPOTS"
+Homepage = "https://pypots.com"
+Documentation = "https://docs.pypots.com"
+"Bug Tracker" = "https://github.com/WenjieDu/PyPOTS/issues"
+Download = "https://github.com/WenjieDu/PyPOTS/archive/main.zip"
+
+[tool.setuptools.packages.find]
+exclude = [
+    "*template",
+    "*README.md",
+    "docs*",
+    "test*",
+    "requirements*",
+]
+
+[tool.setuptools.dynamic]
+version = { attr = "pypots.version.__version__" }
+readme = { file = "README.md", content-type = "text/markdown" }
+dependencies = { file = "requirements/requirements.txt" }
+optional-dependencies.dev = { file = "requirements/requirements_dev.txt" }
+
+[tool.flake8]
+# People may argue that coding style is personal. This may be true if the project is personal and one works like a
+# hermit, but to PyPOTS and its community, the answer is NO.
+# We use Black and Flake8 to lint code style and keep the style consistent across all commits and pull requests.
+# Black only reformats the code, and Flake8 is necessary for checking for some other issues not covered by Black.
+
+# The Black line length is default as 88, while the default of Flake8 is 79. However, considering our monitors are
+# much more advanced nowadays, I extend the maximum line length to 120, like other project e.g. transformers. People
+# who prefer the default setting can keep using 88 or 79 while coding. Please ensure your code lines not exceeding 120.
+max-line-length = 120
+# why ignore E203? Refer to https://github.com/PyCQA/pycodestyle/issues/373
+extend-ignore = """
+    E203
+"""
+# ignore some errors that are not important in template files
+exclude = [
+    "*/template"
+]
diff --git a/requirements.txt b/requirements.txt
deleted file mode 100644
index 6f4d8112..00000000
--- a/requirements.txt
+++ /dev/null
@@ -1,16 +0,0 @@
-# This requirements.txt file only include the basic dependencies for PyPOTS.
-# Please refer to setup.cfg for more dependency details.
-
-h5py
-numpy
-scipy
-sympy
-einops
-pandas
-matplotlib
-tensorboard
-scikit-learn
-torch>=1.10.0
-tsdb>=0.4
-pygrinder>=0.6
-benchpots>=0.2
diff --git a/environment-dev.yml b/requirements/conda_env.yml
similarity index 91%
rename from environment-dev.yml
rename to requirements/conda_env.yml
index 2ebf6ea7..86a0926a 100644
--- a/environment-dev.yml
+++ b/requirements/conda_env.yml
@@ -1,4 +1,4 @@
-name: pypots-dev
+name: pypots
 
 channels:
     - conda-forge
@@ -16,11 +16,12 @@ dependencies:
     - conda-forge::python
     - conda-forge::einops
     - conda-forge::pandas
+    - conda-forge::seaborn
     - conda-forge::matplotlib
     - conda-forge::tensorboard
     - conda-forge::scikit-learn
-    - conda-forge::tsdb >=0.4
     - conda-forge::pygrinder >=0.6
+    - conda-forge::tsdb >=0.6
     - conda-forge::benchpots >=0.2
     - pytorch::pytorch >=1.10.0
     ## Below we install the latest pypots because we need pypots-cli in it for development.
@@ -49,7 +50,7 @@ dependencies:
     - conda-forge::jupyterlab
 
     - pip:
+        # building
+        - build
         # doc
         - sphinxcontrib-gtagjs
-        # hyperparameter tuning
-        - nni
diff --git a/requirements/requirements.txt b/requirements/requirements.txt
new file mode 100644
index 00000000..99f7a6a0
--- /dev/null
+++ b/requirements/requirements.txt
@@ -0,0 +1,14 @@
+h5py
+numpy
+scipy
+sympy
+einops
+pandas
+seaborn
+matplotlib
+tensorboard
+scikit-learn
+torch>=1.10.0
+tsdb>=0.6
+pygrinder>=0.6
+benchpots>=0.2
\ No newline at end of file
diff --git a/requirements/requirements_dev.txt b/requirements/requirements_dev.txt
new file mode 100644
index 00000000..be8bffc2
--- /dev/null
+++ b/requirements/requirements_dev.txt
@@ -0,0 +1,16 @@
+build
+torch-sparse
+torch-scatter
+torch-geometric
+pre-commit
+jupyterlab
+black
+flake8
+flake8-pyproject
+pytest-cov
+pytest-xdist
+furo
+sphinx
+sphinxcontrib-bibtex
+sphinxcontrib-gtagjs
+sphinx-autodoc-typehints
diff --git a/setup.cfg b/setup.cfg
deleted file mode 100644
index b027a9db..00000000
--- a/setup.cfg
+++ /dev/null
@@ -1,76 +0,0 @@
-# This file stores some meta configurations for project PyPOTS.
-
-# Created by Wenjie Du <wenjay.du@gmail.com>
-# License: BSD-3-Clause
-
-[flake8]
-# People may argue that coding style is personal. This may be true if the project is personal and one works like a
-# hermit, but to PyPOTS and its community, the answer is NO.
-# We use Black and Flake8 to lint code style and keep the style consistent across all commits and pull requests.
-# Black only reformats the code, and Flake8 is necessary for checking for some other issues not covered by Black.
-
-# The Black line length is default as 88, while the default of Flake8 is 79. However, considering our monitors are
-# much more advanced nowadays, I extend the maximum line length to 120, like other project e.g. transformers. People
-# who prefer the default setting can keep using 88 or 79 while coding. Please ensure your code lines not exceeding 120.
-max-line-length = 120
-extend-ignore =
-    # why ignore E203? Refer to https://github.com/PyCQA/pycodestyle/issues/373
-    E203,
-# ignore some errors that are not important in template files
-exclude = pypots/*/template
-
-
-[options.extras_require]
-# add dependencies for different usages below
-
-# basic dependencies
-basic =
-    h5py
-    numpy
-    scipy
-    sympy
-    einops
-    pandas
-    matplotlib
-    tensorboard
-    scikit-learn
-    torch>=1.10.0
-    tsdb>=0.4
-    pygrinder>=0.6
-    benchpots>=0.2
-
-# dependencies that are optional, torch-geometric are only needed for model Raindrop
-# but its installation takes too much time
-optional =
-    torch-geometric
-    torch-scatter
-    torch-sparse
-    nni
-
-# dependencies for run all functions and models in PyPOTS =  `basic` dependencies + `optional` dependencies
-full =
-    %(basic)s
-    %(optional)s
-
-# dependencies to run tests
-test =
-    pytest-xdist
-    pytest-cov
-
-# dependencies for documentation generating
-doc =
-    furo
-    sphinx
-    sphinxcontrib-bibtex
-    sphinxcontrib-gtagjs
-    sphinx-autodoc-typehints
-
-# dependencies for development = `full` dependencies + `test` dependencies
-dev =
-    black
-    flake8
-    pre-commit
-    jupyterlab
-    %(full)s
-    %(test)s
-    %(doc)s
diff --git a/setup.py b/setup.py
deleted file mode 100644
index 37f3be90..00000000
--- a/setup.py
+++ /dev/null
@@ -1,82 +0,0 @@
-from setuptools import find_packages
-from setuptools import setup
-
-from pypots import __version__
-
-with open("./README.md", encoding="utf-8") as f:
-    README = f.read()
-
-setup(
-    name="pypots",
-    version=__version__,
-    description="A Python Toolbox for Machine Learning on Partially-Observed Time Series",
-    long_description=README,
-    long_description_content_type="text/markdown",
-    license="BSD-3-Clause",
-    author="Wenjie Du",
-    author_email="wenjay.du@gmail.com",
-    url="https://pypots.com/",
-    project_urls={
-        "Documentation": "https://docs.pypots.com/",
-        "Source": "https://github.com/WenjieDu/PyPOTS/",
-        "Tracker": "https://github.com/WenjieDu/PyPOTS/issues/",
-        "Download": "https://github.com/WenjieDu/PyPOTS/archive/main.zip",
-    },
-    keywords=[
-        "data science",
-        "data mining",
-        "neural networks",
-        "machine learning",
-        "deep learning",
-        "artificial intelligence",
-        "time-series analysis",
-        "time series",
-        "imputation",
-        "interpolation",
-        "classification",
-        "clustering",
-        "forecasting",
-        "partially observed",
-        "irregular sampled",
-        "partially-observed time series",
-        "incomplete time series",
-        "missing data",
-        "missing values",
-    ],
-    packages=find_packages(exclude=["tests"]),
-    include_package_data=True,
-    install_requires=[
-        "h5py",
-        "numpy",
-        "scipy",
-        "sympy",
-        "einops",
-        "pandas",
-        "matplotlib",
-        "tensorboard",
-        "scikit-learn",
-        "torch>=1.10.0",
-        "tsdb>=0.4",
-        "pygrinder>=0.6",
-        "benchpots>=0.2",
-    ],
-    python_requires=">=3.8.0",
-    setup_requires=["setuptools>=38.6.0"],
-    entry_points={"console_scripts": ["pypots-cli=pypots.cli.pypots_cli:main"]},
-    classifiers=[
-        "Development Status :: 5 - Production/Stable",
-        "Intended Audience :: Developers",
-        "Intended Audience :: Education",
-        "Intended Audience :: Science/Research",
-        "Intended Audience :: Healthcare Industry",
-        "License :: OSI Approved :: BSD License",
-        "Operating System :: OS Independent",
-        "Programming Language :: Python :: 3",
-        "Programming Language :: Python :: 3.8",
-        "Programming Language :: Python :: 3.9",
-        "Programming Language :: Python :: 3.10",
-        "Programming Language :: Python :: 3.11",
-        "Topic :: Scientific/Engineering :: Artificial Intelligence",
-        "Topic :: Software Development :: Libraries :: Application Frameworks",
-    ],
-)


Test patch:
diff --git a/.github/workflows/testing_ci.yml b/.github/workflows/testing_ci.yml
index a43660f4..16ecb5d2 100644
--- a/.github/workflows/testing_ci.yml
+++ b/.github/workflows/testing_ci.yml
@@ -34,7 +34,8 @@ jobs:
                 check-latest: true
                 cache: pip
                 cache-dependency-path: |
-                  setup.cfg
+                  requirements/requirements.txt
+                  requirements/requirements_dev.txt
 
             - name: Install PyTorch ${{ matrix.pytorch-version }}+cpu
               # we have to install torch in advance because torch_sparse needs it for compilation,
@@ -42,18 +43,20 @@ jobs:
               run: |
                   which python
                   which pip
-                  pip install numpy==1.24 torch==${{ matrix.pytorch-version }} -f https://download.pytorch.org/whl/cpu
+                  pip install torch==${{ matrix.pytorch-version }} -f https://download.pytorch.org/whl/cpu
                   python -c "import torch; print('PyTorch:', torch.__version__)"
 
             - name: Install other dependencies
               run: |
-                  pip install -r requirements.txt
+                  pip install -r requirements/requirements.txt
+                  pip install numpy==1.24  # many libs not compatible with numpy 2.0. Note 3.12 requests for numpy>=2.0
+                  pip install pandas==1.5  # fix pandas version to avoid installing pandas 2.0, the same reason with numpy
 
             - name: Test building package
               # we need to know if the package can be built successfully without optional dependencies
               run: |
                   pip install build wheel
-                  python -m build --no-isolation
+                  python -m build
 
             - name: Continue to install torch-geometric dependencies
               run: |
diff --git a/tests/environment_for_conda_test.yml b/requirements/environment_for_conda_test.yml
similarity index 90%
rename from tests/environment_for_conda_test.yml
rename to requirements/environment_for_conda_test.yml
index d7878f97..d5d31283 100644
--- a/tests/environment_for_conda_test.yml
+++ b/requirements/environment_for_conda_test.yml
@@ -16,10 +16,11 @@ dependencies:
     - conda-forge::numpy
     - conda-forge::einops
     - conda-forge::pandas
+    - conda-forge::seaborn
     - conda-forge::matplotlib
     - conda-forge::tensorboard
     - conda-forge::scikit-learn
-    - conda-forge::tsdb >=0.4
+    - conda-forge::tsdb >=0.6
     - conda-forge::pygrinder >=0.6
     - conda-forge::benchpots >=0.2
     - conda-forge::protobuf <=4.21.12
@@ -46,5 +47,5 @@ dependencies:
     - conda-forge::flake8
 
     - pip:
-          # doc
-          - sphinxcontrib-gtagjs
+        # doc
+        - sphinxcontrib-gtagjs