Repository: CSchoel/nolds (language: python)

Issue:
Introduce regression tests with deterministic values
Since we already know that the output of some metrics changes slightly when we switch to numpy 2.0 (see #42), we should protect against this early on. The best way I see to catch this is to have regression tests and check for _exact_ results on some sample data.

- [x] Create a regression test branching out from main. Just use random data for each of the main algorithms.
- [x] (Optional) check if there are any major branches in any of the algorithms that only trigger under specific conditions that are not met with the random data. (We can use the debugger for that.)
- [x] (Optional) also test with algorithms that include randomness by keeping seed fixed.
- [x] Store the exact values produced by the algorithms on this data.
- [x] Document the tests.
- [x] Merge tests into dev branch.
- [x] Switch back to the poetry branch (using updated scikit-learn) and check if regression tests still run.


Discussion / hints:

Fix patch:
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
index 429bb80..bdc6637 100644
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -1,6 +1,7 @@
 on:
   push:
     branches: ["main", "dev", "v*"]
+  pull_request:
 
 name: build
 
@@ -9,14 +10,17 @@ jobs:
     strategy:
       matrix:
         python: ["3.7", "3.10"]
+        extras: ["", "[RANSAC, qrandom, plots]"]
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python }}
-      - run: pip install .
+      - run: pip install ".${{ matrix.extras }}"
       - run: pip install codecov .
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
       - run: coverage run -m unittest nolds.test_measures
       - run: codecov
-        if: ${{ matrix.python == '3.10' }}
+        if: ${{ matrix.python == '3.10' && matrix.extras != '' }}
diff --git a/setup.py b/setup.py
index 9bd360a..ac929d4 100644
--- a/setup.py
+++ b/setup.py
@@ -62,7 +62,7 @@ def run(self):
         'setuptools'
     ],
     extras_require={
-        'RANSAC': ['sklearn>=0.19'],
+        'RANSAC': ['scikit-learn>=0.19'],
         'qrandom': ['quantumrandom'],
         'plots': ['matplotlib']
     },


Test patch:
diff --git a/nolds/test_measures.py b/nolds/test_measures.py
index 2777dde..914e785 100644
--- a/nolds/test_measures.py
+++ b/nolds/test_measures.py
@@ -563,6 +563,7 @@ def test_corr_dim(self):
     cd = nolds.corr_dim(data, emb_dim=5, lag=1, rvals=None, dist=nolds.rowwise_euclidean, fit="poly")
     self.assertAlmostEqual(1.303252839255068, cd, places=15)
 
+  @unittest.skipUnless(SCIPY_AVAILABLE, "Tests with RANSAC require scipy.")
   def test_corr_dim_RANSAC(self):
     """Test hypothesis: The exact output of corr_dim() with `fit=RANSAC` on random data hasn't changed since the last version."""
     data = datasets.load_qrandom()[:1000]
@@ -589,6 +590,7 @@ def test_lyap_r(self):
     expected = 0.094715945307378
     self.assertAlmostEqual(expected, le, places=15)
 
+  @unittest.skipUnless(SCIPY_AVAILABLE, "Tests with RANSAC require scipy.")
   def test_lyap_r_RANSAC(self):
     """Test hypothesis: The exact output of lyap_r() with `fit=RANSAC` on random data hasn't changed since the last version."""
     data = datasets.load_qrandom()[:1000]
@@ -606,6 +608,7 @@ def test_hurst_rs(self):
     expected = 0.5123887964986258
     self.assertAlmostEqual(expected, rs, places=15)
 
+  @unittest.skipUnless(SCIPY_AVAILABLE, "Tests with RANSAC require scipy.")
   def test_hurst_rs_RANSAC(self):
     """Test hypothesis: The exact output of hurst_rs() with `fit=RANSAC` on random data hasn't changed since the last version."""
     data = datasets.load_qrandom()[:1000]
@@ -623,6 +626,7 @@ def test_dfa(self):
     expected = 0.5450874638765073
     self.assertAlmostEqual(expected, h, places=15)
 
+  @unittest.skipUnless(SCIPY_AVAILABLE, "Tests with RANSAC require scipy.")
   def test_dfa_RANSAC(self):
     """Test hypothesis: The exact output of dfa() with `fit_exp=RANSAC` on random data hasn't changed since the last version."""
     # adds trend to data to introduce a less clear line for fitting
@@ -641,6 +645,7 @@ def test_mfhurst_b(self):
     expected = [-0.00559398934417339]
     self.assertAlmostEqual(expected[0], h[0], places=15)
 
+  @unittest.skipUnless(SCIPY_AVAILABLE, "Tests with RANSAC require scipy.")
   def test_mfhurst_b_RANSAC(self):
     """Test hypothesis: The exact output of mfhurst_b() with `fit=RANSAC` on random data hasn't changed since the last version."""
     data = datasets.load_qrandom()[:1000]
@@ -656,6 +661,7 @@ def test_mfhurst_dm(self):
     expected = [0.008762803881203145]
     self.assertAlmostEqual(expected[0], h[0], places=15)
 
+  @unittest.skipUnless(SCIPY_AVAILABLE, "Tests with RANSAC require scipy.")
   def test_mfhurst_dm_RANSAC(self):
     """Test hypothesis: The exact output of mfhurst_dm() with `fit=RANSAC` on random data hasn't changed since the last version."""
     data = datasets.load_qrandom()[:1000]