Repository: nilearn/nilearn

Issue:
nilearn.signal.clean modifies input when "session" parameter used.
Hi guys, 

[docs](https://nilearn.github.io/modules/generated/nilearn.signal.clean.html) for nilearn.signal.clean say: 

"signals: numpy.ndarray
Timeseries. Must have shape (instant number, features number). **This array is not modified**."

However, I find that if the session parameter is used the input array **is** modified. 

Code to reproduce:
```
# make random data
np.random.seed(1)
nsamp = 1000
data_source= np.random.rand(nsamp)
confound= np.random.rand(nsamp)
data_observed_orig = data_source + confound
data_observed_input = data_observed_orig.copy()
print(np.corrcoef(data_observed_orig,confound)[0,1]**2) # theoretically 0.5
sessions_desc = np.ones(nsamp)
sessions_desc[0:nsamp//2]=0

data_cleaned_concat = nil.signal.clean(data_observed_input[:,np.newaxis],
                                       confounds=confound[:,np.newaxis],sessions=sessions_desc)
#should be True
np.all(data_observed_input == data_observed_orig)
```
output:
0.5228211029311932
False

Only happens if the session parameter is provided. In fact, the input data seems to change to something like the output, but not exactly. 

nil version 0.6.0
clean function modifies the input time series object if performing pass-filtering without detrending
Hi,

I am using version 0.6.0. I've found that function **clean** within the module signal modifies the input time series object if one performs a pass-fitering when _detrend_ option is False. 

For example, if you run the following code (test time series borrowed from [here](https://gist.github.com/miykael/4bb887029cba3926902feb2e4afe190f)) and then plot both the original and clean time series:

```
import numpy as np
import matplotlib.pylab as plt
from nilearn.signal import clean

original_sx = np.array([np.sin(np.linspace(0, 100, 100) * 1.5),
                        np.sin(np.linspace(0, 100, 100) * 3.),
                        np.sin(np.linspace(0, 100, 100) / 8.),
              ]).T

# Clean with detrending and high pass filtering
clean_sx = clean(original_sx, 
                   standardize=False, 
                   t_r=1.0, 
                   detrend=True, 
                   high_pass=0.2)

# Plot both time series
fig, axs = plt.subplots(ncols=2, figsize=(10, 5))  
axs[0].plot(original_sx)
axs[0].set_title("Original Time Series")
axs[1].plot(clean_sx)
axs[1].set_title("Clean Time Series")
plt.show()
```

You should see how "green" signal is totally filtered out (its frequency is ~ 0.02):

![example_detrend_filt](https://user-images.githubusercontent.com/25054369/82581329-6f043800-9b5e-11ea-8c81-0db4642d8c9c.png)

However, if you now try to run the same code but without detrending the time series:

```
import numpy as np
import matplotlib.pylab as plt
from nilearn.signal import clean

original_sx = np.array([np.sin(np.linspace(0, 100, 100) * 1.5),
                        np.sin(np.linspace(0, 100, 100) * 3.),
                        np.sin(np.linspace(0, 100, 100) / 8.),
              ]).T

# Clean with only high pass filtering (Note that now detrend is set to False)
clean_sx = clean(original_sx, 
                   standardize=False, 
                   t_r=1.0, 
                   detrend=False, 
                   high_pass=0.2)

# Plot both time series
fig, axs = plt.subplots(ncols=2, figsize=(10, 5))  
axs[0].plot(original_sx)
axs[0].set_title("Original Time Series")
axs[1].plot(clean_sx)
axs[1].set_title("Clean Time Series")
plt.show()
```

The original time series object is modified as well

![example_only_filt](https://user-images.githubusercontent.com/25054369/82581328-6f043800-9b5e-11ea-8584-df17b808808e.png)

When __detrend__ option is set to True, it first calls the function **_standardize**, which creates a copy of the original time series, and then carries on to do the filtering by calling the  function **butterworth**. However, when __detrend__ is False, which skips the first step, since **butterworth** is alway called with its  _copy_ argument set to False (the default value), the original time series are modified too. 

Is this the expected behaviour? I would expect the input time series to remain always the same, as inline modifications can lead to unwanted results if the original time series need to be reused again at some point.

Cheers


Discussion:
Sorry, we haven't been able to look at that.
Would you be kind enough to localize the issue and propose a PR ?
Thx a lot,
I think the issue is to do with the recursion when using the sessions parameter, since the output of the recursion is the original input, not a copy.

An easy fix would be to copy the input early on.

Currently, the input is copied if ensure_finite=True, but this is not the default behaviour. So another solution is just to make this default.

Alternatively, we could just change the description and remove the "This array is not modified". For me the issue is not the inplace modification, but the inconsistency with the docs.

I'd rather go for keeping the input array untouched. This is not a strong opinion though.
ok then easiest thing is just to copy it early on. If you have no other requests/comments then I can get on to the PR
+1 on my side.
You'll have to add a test that fails with current implementation, but works with the new one.
This could be related to #2281 . I think that the idea was that the input should never be modified.
Thx for reporting !
Would you issue a PR ?
I've never issued a pull request, but I can try it. As it is suggested in #2281, the safest thing would be just to create a copy of the input time series right at the beginning of the function.
... only if necessary. 
We want to avoid copies as much as possible, because we may have to process large chunks of data. 

Resolving patch:
diff --git a/nilearn/signal.py b/nilearn/signal.py
--- a/nilearn/signal.py
+++ b/nilearn/signal.py
@@ -482,13 +482,13 @@ def clean(signals, sessions=None, detrend=True, standardize='zscore',
                          "but you provided ensure_finite={0}"
                          .format(ensure_finite))
 
+    signals = signals.copy()
     if not isinstance(signals, np.ndarray):
         signals = as_ndarray(signals)
 
     if ensure_finite:
         mask = np.logical_not(np.isfinite(signals))
         if mask.any():
-            signals = signals.copy()
             signals[mask] = 0
 
     # Read confounds


Test patch:
diff --git a/nilearn/tests/test_signal.py b/nilearn/tests/test_signal.py
--- a/nilearn/tests/test_signal.py
+++ b/nilearn/tests/test_signal.py
@@ -275,14 +275,20 @@ def test_clean_detrending():
     trends = generate_trends(n_features=n_features,
                              length=n_samples)
     x = signals + trends
+    x_orig = x.copy()
 
     # if NANs, data out should be False with ensure_finite=True
     y = signals + trends
     y[20, 150] = np.nan
     y[5, 500] = np.nan
     y[15, 14] = np.inf
-    y = nisignal.clean(y, ensure_finite=True)
-    assert np.any(np.isfinite(y)), True
+    y_orig = y.copy()
+
+    y_clean = nisignal.clean(y, ensure_finite=True)
+    assert np.any(np.isfinite(y_clean)), True
+    # clean should not modify inputs
+    # using assert_almost_equal instead of array_equal due to NaNs
+    np.testing.assert_almost_equal(y_orig, y, decimal=13)
 
     # test boolean is not given to signal.clean
     pytest.raises(TypeError, nisignal.clean, x, low_pass=False)
@@ -292,11 +298,15 @@ def test_clean_detrending():
     x_detrended = nisignal.clean(x, standardize=False, detrend=True,
                                  low_pass=None, high_pass=None)
     np.testing.assert_almost_equal(x_detrended, signals, decimal=13)
+    # clean should not modify inputs
+    assert np.array_equal(x_orig, x)
 
     # This should do nothing
     x_undetrended = nisignal.clean(x, standardize=False, detrend=False,
                                    low_pass=None, high_pass=None)
     assert not abs(x_undetrended - signals).max() < 0.06
+    # clean should not modify inputs
+    assert np.array_equal(x_orig, x)
 
 
 def test_clean_t_r():
@@ -335,6 +345,7 @@ def test_clean_frequencies():
     sx1 = np.sin(np.linspace(0, 100, 2000))
     sx2 = np.sin(np.linspace(0, 100, 2000))
     sx = np.vstack((sx1, sx2)).T
+    sx_orig = sx.copy()
     assert clean(sx, standardize=False, high_pass=0.002, low_pass=None,
                       t_r=2.5).max() > 0.1
     assert clean(sx, standardize=False, high_pass=0.2, low_pass=None,
@@ -342,6 +353,29 @@ def test_clean_frequencies():
     assert clean(sx, standardize=False, low_pass=0.01, t_r=2.5).max() > 0.9
     pytest.raises(ValueError, clean, sx, low_pass=0.4, high_pass=0.5, t_r=2.5)
 
+    # clean should not modify inputs
+    sx_cleaned = clean(sx, standardize=False, detrend=False, low_pass=0.2, t_r=2.5)
+    assert np.array_equal(sx_orig, sx)
+
+
+def test_clean_sessions():
+    n_samples = 21
+    n_features = 501  # Must be higher than 500
+    signals, _, _ = generate_signals(n_features=n_features,
+                                     length=n_samples)
+    trends = generate_trends(n_features=n_features,
+                             length=n_samples)
+    x = signals + trends
+    x_orig = x.copy()
+    # Create session info
+    sessions = np.ones(n_samples)
+    sessions[0:n_samples // 2] = 0
+    x_detrended = nisignal.clean(x, standardize=False, detrend=True,
+                                 low_pass=None, high_pass=None,
+                                 sessions=sessions)
+    # clean should not modify inputs
+    assert np.array_equal(x_orig, x)
+
 
 def test_clean_confounds():
     signals, noises, confounds = generate_signals(n_features=41,
@@ -352,7 +386,8 @@ def test_clean_confounds():
     cleaned_signals = nisignal.clean(noises, confounds=confounds,
                                      detrend=True, standardize=False)
     assert abs(cleaned_signals).max() < 100. * eps
-    np.testing.assert_almost_equal(noises, noises1, decimal=12)
+    # clean should not modify inputs
+    assert np.array_equal(noises, noises1)
 
     # With signal: output must be orthogonal to confounds
     cleaned_signals = nisignal.clean(signals + noises, confounds=confounds,