Repository: sktime/sktime

Issue:
[BUG] Irreproducible results with `MultiRocketMultivariate`
`random_state` does guarantee the same results for each run.

```python
rng = np.random.default_rng()

X = pd.DataFrame([
    pd.Series([
        pd.Series(rng.integers(0, 10, 100)).astype(float),
        pd.Series(rng.integers(0, 10, 100)).astype(float),
    ]),
    pd.Series([
        pd.Series(rng.integers(0, 10, 100)).astype(float),
        pd.Series(rng.integers(0, 10, 100)).astype(float),
    ]),
])

MultiRocketMultivariate(random_state=42, num_kernels=84).fit_transform(X) - MultiRocketMultivariate(random_state=42, num_kernels=84).fit_transform(X)
```

The output should always be a `DataFrame` of zeros, but this is not the case.

<details>

<!--
Please run the following code snippet and paste the output here:

from sktime import show_versions; show_versions()
-->

System:
    python: 3.9.6 (default, Aug 11 2023, 19:44:49)  [Clang 15.0.0 (clang-1500.0.40.1)]
executable: /Users/temp/sktime/venv/bin/python
   machine: macOS-14.1.2-arm64-arm-64bit

Python dependencies:
          pip: 23.3.1
       sktime: 0.25.0
      sklearn: 1.3.2
       skbase: 0.4.6
        numpy: 1.26.1
        scipy: 1.11.3
       pandas: 2.1.4
   matplotlib: 3.8.1
       joblib: 1.3.2
        numba: 0.58.1
  statsmodels: 0.14.0
     pmdarima: None
statsforecast: None
      tsfresh: 0.20.1
      tslearn: 0.6.3
        torch: 2.1.0
   tensorflow: None
tensorflow_probability: None

</details>

<!-- Thanks for contributing! -->



Discussion:
FYI @ChangWeiTan @fstinner @angus924
@chrico-bu-uab, just to clarify, you are saying the result varies and is not actually fixed by fixing `random_state`, am I understanding correctly?
Yes, I had to add lines to _get_parameter to set the random state as the seed.

From: Franz Király ***@***.***>
Date: Thursday, January 4, 2024 at 5:38 AM
To: sktime/sktime ***@***.***>
Cc: Coffee, Christopher (Campus) ***@***.***>, Mention ***@***.***>
Subject: Re: [sktime/sktime] [BUG] Irreproducible results with `MultiRocketMultivariate` (Issue #5696)

@chrico-bu-uab<https://github.com/chrico-bu-uab>, just to clarify, you are saying the result varies and is not actually fixed by fixing random_state, am I understanding correctly?

—
Reply to this email directly, view it on GitHub<https://github.com/sktime/sktime/issues/5696#issuecomment-1876955229>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHEYGI2BIO4HAVJYNSGHXPLYM2IEJAVCNFSM6AAAAABBME4KG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZWHE2TKMRSHE>.
You are receiving this because you were mentioned.Message ID: ***@***.***>

> Yes, I had to add lines to _get_parameter to set the random state as the seed.

Sounds like you fixed it - would you kindly be able to contribute the fix via pull request?
I tried to, but I think I did it wrong. Should I create a fork or a new branch? Sorry, new to pull requests.
> I tried to, but I think I did it wrong. Should I create a fork or a new branch? Sorry, new to pull requests.

No worries, happy to help!

Short instructions:
1. create a fork
2. create a branch on your fork
3. add the fix to the branch on your fork. Ensure it is present on GitHub (push)
4. make a PR to the upstream, if you go on the [sktime GitHub PR tracker](https://github.com/sktime/sktime/pulls) (not the fork), a green button should appear at the top to open a PR, if you made changes to a branch on your fork!

More detailed instructions:
https://www.sktime.net/en/latest/developer_guide/git_workflow.html

Resolving patch:
diff --git a/sktime/transformations/panel/rocket/_multirocket_multivariate.py b/sktime/transformations/panel/rocket/_multirocket_multivariate.py
--- a/sktime/transformations/panel/rocket/_multirocket_multivariate.py
+++ b/sktime/transformations/panel/rocket/_multirocket_multivariate.py
@@ -198,6 +198,9 @@ def _get_parameter(self, X):
             _quantiles,
         )
 
+        if self.random_state is not None:
+            np.random.seed(self.random_state)
+
         _, num_channels, input_length = X.shape
 
         num_kernels = 84


Test patch: