Repository: mne-tools/mne-python

Issue:
Q? decimation copy in ICA
@dengemann Why do we `copy` data when decimating? https://github.com/mne-tools/mne-python/blob/master/mne/preprocessing/ica.py#L367



Discussion:
Have you followed `data` in the subsequent calls to see if it gets modified in-place anywhere? Might find the answer that way...

> ? Might find the answer that way...

Yes this is the reason.

FYI the only other reasons (beyond avoiding in-place modifications) I can think of would be 1) to get data aligned for SIMD operations, but I doubt that's what it's fore, or 2) to save memory if the "new" version is to be stored, but it doesn't look like that's the case either

But if it's modified in place, it should be the same whether it's decimated or not, shouldn't it? Or is it because having adjacent columns speeds up the computation?

> But if it's modified in place, it should be the same whether it's decimated or not, shouldn't it? Or is it because having adjacent columns speeds up the computation?

thinking about it we could actually avoid it, I guess we did it in the first place assuming that we would update the data but this is actually not the case. It's only used for fitting. Maybe worth testing, we could gain memory performance.

> But if it's modified in place, it should be the same whether it's decimated or not, shouldn't it

Oh... right.

> Or is it because having adjacent columns speeds up the computation

This can matter for SIMD instructions via OpenBLAS or MKL but I doubt it's really at play here.

> thinking about it we could actually avoid it ... Maybe worth testing

Agreed. @kingjr if you follow the code and verify that it doesn't modify in place (sounds like it won't) then we should be able to get rid of the copy.

I've only followed a part of the code, I'd be reassured if @dengemann validated it.

@dengemann do you have time? If not I can take a look and verify

@Eric89GXL @kingjr I am pretty sure that it does not get modified, at least not intentionally.

As long as this overwrites a copy: https://github.com/mne-tools/mne-python/blob/master/mne/preprocessing/ica.py#L371

the returned data does not have the bad segments. Yes could also be improved with nan-based routine or an annotation based API ...


Resolving patch:
diff --git a/mne/preprocessing/ica.py b/mne/preprocessing/ica.py
--- a/mne/preprocessing/ica.py
+++ b/mne/preprocessing/ica.py
@@ -363,17 +363,20 @@ def _fit_raw(self, raw, picks, start, stop, decim, reject, flat, tstep,
         self.ch_names = self.info['ch_names']
         start, stop = _check_start_stop(raw, start, stop)
 
+        # this will be a copy
         data = raw[picks, start:stop][0]
+        # this will be a view
         if decim is not None:
-            data = data[:, ::decim].copy()
+            data = data[:, ::decim]
 
+        # this will make a copy
         if (reject is not None) or (flat is not None):
             data, self.drop_inds_ = _reject_data_segments(data, reject, flat,
                                                           decim, self.info,
                                                           tstep)
 
         self.n_samples_ = data.shape[1]
-
+        # this may operate inplace or make a copy
         data, self._pre_whitener = self._pre_whiten(data,
                                                     raw.info, picks)
 
@@ -403,12 +406,16 @@ def _fit_epochs(self, epochs, picks, decim, verbose):
             self.max_pca_components = len(picks)
             logger.info('Inferring max_pca_components from picks.')
 
+        # this should be a copy (picks a list of int)
         data = epochs.get_data()[:, picks]
+        # this will be a view
         if decim is not None:
-            data = data[:, :, ::decim].copy()
+            data = data[:, :, ::decim]
 
         self.n_samples_ = np.prod(data[:, 0, :].shape)
 
+        # This will make at least one copy (one from hstack, maybe one
+        # more from _pre_whiten)
         data, self._pre_whitener = \
             self._pre_whiten(np.hstack(data), epochs.info, picks)
 


Test patch: