Repository: nilearn/nilearn

Issue:
signal clean should not use population standard deviation
Nilearn version: 0.9.2

In signal.clean, the calculation simply uses numpy.std, this method by default uses N / ddf=0 for calculation, indicating population std. For fMRI studies, especially when std are calculated along time axis, the data is usually a sample of the population. The method should use sample std instead.

### Expected behavior
Given dataframe

1  4  7
2  5  8
3  6  9

z-scored sample should be:

(anp - anp.mean(axis=0))/anp.std(axis=0, ddof=1)
array([[-1., -1., -1.],
       [ 0.,  0.,  0.],
       [ 1.,  1.,  1.]])

### Actual behavior
nilearn.signal.clean(anp)
array([[-1.2247448, -1.2247448, -1.2247448],
       [ 0.       ,  0.       ,  0.       ],
       [ 1.2247448,  1.2247448,  1.2247448]], dtype=float32)

### Steps and code to reproduce bug

I think it would be beneficial if we change the default behavior.
However, given the scope of this change, I think it's worth adding the ddof as an option instead of changing the default action.


Discussion:
Thx. The proposition is sound to me. 
@bthirion , what would you think about decorating `nilearn.signal:clean` to warn users that the new keyword argument `ddof=0` will be updated to `ddof=1` in a later version so that the default behavior is eventually correct? There is no decorator in `nilearn._utils.helpers` for this currently but I could volunteer to add one.

@DasDominus , do you think this new `ddof` argument would have to percolate to callers of `nilearn.signal:clean` like in `nilearn.image.image:clean_img` for example?
You mean that this entails a deprecation warning ? We don't use decorators for that.
@bthirion yes, a deprecation warning. I assumed you use decorators for that from `nilearn._utils.helpers:rename_parameters`:
```
def rename_parameters(replacement_params,
                      end_version='future',
                      lib_name='Nilearn',
                      ):
    """Decorator to deprecate & replace specified parameters
    in the decorated functions and methods without changing
    function definition or signature.
```
but I now see that you mostly just call `warnings.warn(category=DeprecationWarning, ...)`.

In the case of this `nilearn.signal:clean`, do you think it makes sense to show a deprecation warning so that the default behavior would eventually be as described by @DasDominus ? If so, this will be delicate to implement so that we only warn when the caller does not pass the `ddof` keyword argument and therefore relies on the current behavior of `ddof=0`. I could volunteer to help with that if needed :)
Thanks for looking! @cr3 Yeah, it might be beneficial to percolate. An additional **kwargs for passing params might be useful
I like the warning idea also : D thanks again!
@cr3 and @DasDominus thanks for looking into this! Instead of adding a new keyword argument, would it be better to just add a new strategy value to the already existing parameter `standardize` in the function `signal._standardize` that would implement the new behavior? Then simply use `warnings.warn(category=DeprecationWarning, ...)` to deprecate the default strategy in favor of the new one. Let me know what you think.
I think that's a very good suggestion! now we just need to pick a good name for the new strategy
Hi @DasDominus and @cr3, are either of you interested in working on this? A pull request is very welcome! 
I'll pick this one up next week if no one else is interested!

Resolving patch:
diff --git a/nilearn/signal.py b/nilearn/signal.py
--- a/nilearn/signal.py
+++ b/nilearn/signal.py
@@ -37,11 +37,14 @@ def _standardize(signals, detrend=False, standardize='zscore'):
         If detrending of timeseries is requested.
         Default=False.
 
-    standardize : {'zscore', 'psc', True, False}, optional
+    standardize : {'zscore_sample', 'zscore', 'psc', True, False}, optional
         Strategy to standardize the signal:
 
+            - 'zscore_sample': The signal is z-scored. Timeseries are shifted
+              to zero mean and scaled to unit variance. Uses sample std.
             - 'zscore': The signal is z-scored. Timeseries are shifted
-              to zero mean and scaled to unit variance.
+              to zero mean and scaled to unit variance. Uses population std
+              by calling default :obj:`numpy.std` with N - ``ddof=0``.
             - 'psc':  Timeseries are shifted to zero mean value and scaled
               to percent signal change (as compared to original mean signal).
             - True: The signal is z-scored (same as option `zscore`).
@@ -55,7 +58,7 @@ def _standardize(signals, detrend=False, standardize='zscore'):
     std_signals : :class:`numpy.ndarray`
         Copy of signals, standardized.
     """
-    if standardize not in [True, False, 'psc', 'zscore']:
+    if standardize not in [True, False, 'psc', 'zscore', 'zscore_sample']:
         raise ValueError('{} is no valid standardize strategy.'
                          .format(standardize))
 
@@ -70,13 +73,36 @@ def _standardize(signals, detrend=False, standardize='zscore'):
                           'would lead to zero values. Skipping.')
             return signals
 
+        elif (standardize == 'zscore_sample'):
+            if not detrend:
+                # remove mean if not already detrended
+                signals = signals - signals.mean(axis=0)
+
+            std = signals.std(axis=0, ddof=1)
+            # avoid numerical problems
+            std[std < np.finfo(np.float64).eps] = 1.
+            signals /= std
+
         elif (standardize == 'zscore') or (standardize is True):
+            std_strategy_default = (
+                "The default strategy for standardize is currently 'zscore' "
+                "which incorrectly uses population std to calculate sample "
+                "zscores. The new strategy 'zscore_sample' corrects this "
+                "behavior by using the sample std. In release 0.13, the "
+                "default strategy will be replaced by the new strategy and "
+                "the 'zscore' option will be removed. Please use "
+                "'zscore_sample' instead."
+            )
+            warnings.warn(category=FutureWarning,
+                          message=std_strategy_default,
+                          stacklevel=3)
             if not detrend:
                 # remove mean if not already detrended
                 signals = signals - signals.mean(axis=0)
 
             std = signals.std(axis=0)
-            std[std < np.finfo(np.float64).eps] = 1.  # avoid numerical problems
+            # avoid numerical problems
+            std[std < np.finfo(np.float64).eps] = 1.
             signals /= std
 
         elif standardize == 'psc':
@@ -609,11 +635,14 @@ def clean(signals, runs=None, detrend=True, standardize='zscore',
 
     %(high_pass)s
     %(detrend)s
-    standardize : {'zscore', 'psc', False}, optional
+    standardize : {'zscore_sample', 'zscore', 'psc', True, False}, optional
         Strategy to standardize the signal:
 
+            - 'zscore_sample': The signal is z-scored. Timeseries are shifted
+              to zero mean and scaled to unit variance. Uses sample std.
             - 'zscore': The signal is z-scored. Timeseries are shifted
-              to zero mean and scaled to unit variance.
+              to zero mean and scaled to unit variance. Uses population std
+              by calling default :obj:`numpy.std` with N - ``ddof=0``.
             - 'psc':  Timeseries are shifted to zero mean value and scaled
               to percent signal change (as compared to original mean signal).
             - True: The signal is z-scored (same as option `zscore`).


Test patch:
diff --git a/nilearn/tests/test_signal.py b/nilearn/tests/test_signal.py
--- a/nilearn/tests/test_signal.py
+++ b/nilearn/tests/test_signal.py
@@ -251,6 +251,14 @@ def test_standardize():
     a = rng.random_sample((n_samples, n_features))
     a += np.linspace(0, 2., n_features)
 
+    # Test raise error when strategy is not valid option
+    with pytest.raises(ValueError, match="no valid standardize strategy"):
+        nisignal._standardize(a, standardize="foo")
+
+    # test warning for strategy that will be removed
+    with pytest.warns(FutureWarning, match="default strategy for standardize"):
+        nisignal._standardize(a, standardize="zscore")
+
     # transpose array to fit _standardize input.
     # Without trend removal
     b = nisignal._standardize(a, standardize='zscore')
@@ -258,16 +266,32 @@ def test_standardize():
     np.testing.assert_almost_equal(stds, np.ones(n_features))
     np.testing.assert_almost_equal(b.sum(axis=0), np.zeros(n_features))
 
+    # Repeating test above but for new correct strategy
+    b = nisignal._standardize(a, standardize='zscore_sample')
+    stds = np.std(b)
+    np.testing.assert_almost_equal(stds, np.ones(n_features), decimal=1)
+    np.testing.assert_almost_equal(b.sum(axis=0), np.zeros(n_features))
+
     # With trend removal
     a = np.atleast_2d(np.linspace(0, 2., n_features)).T
     b = nisignal._standardize(a, detrend=True, standardize=False)
     np.testing.assert_almost_equal(b, np.zeros(b.shape))
 
+    b = nisignal._standardize(a, detrend=True, standardize="zscore_sample")
+    np.testing.assert_almost_equal(b, np.zeros(b.shape))
+
     length_1_signal = np.atleast_2d(np.linspace(0, 2., n_features))
     np.testing.assert_array_equal(length_1_signal,
                                   nisignal._standardize(length_1_signal,
                                                         standardize='zscore'))
 
+    # Repeating test above but for new correct strategy
+    length_1_signal = np.atleast_2d(np.linspace(0, 2., n_features))
+    np.testing.assert_array_equal(
+        length_1_signal,
+        nisignal._standardize(length_1_signal, standardize="zscore_sample")
+    )
+
 
 def test_detrend():
     """Test custom detrend implementation."""
@@ -827,9 +851,18 @@ def test_clean_zscore():
                                      length=n_samples)
 
     signals += rng.standard_normal(size=(1, n_features))
-    cleaned_signals = clean(signals, standardize='zscore')
+    cleaned_signals_ = clean(signals, standardize='zscore')
+    np.testing.assert_almost_equal(cleaned_signals_.mean(0), 0)
+    np.testing.assert_almost_equal(cleaned_signals_.std(0), 1)
+
+    # Repeating test above but for new correct strategy
+    cleaned_signals = clean(signals, standardize='zscore_sample')
     np.testing.assert_almost_equal(cleaned_signals.mean(0), 0)
-    np.testing.assert_almost_equal(cleaned_signals.std(0), 1)
+    np.testing.assert_almost_equal(cleaned_signals.std(0), 1, decimal=3)
+
+    # Show outcome from two zscore strategies is not equal
+    with pytest.raises(AssertionError):
+        np.testing.assert_array_equal(cleaned_signals_, cleaned_signals)
 
 
 def test_create_cosine_drift_terms():