Repository: mne-tools/mne-python

Issue:
Run corrmap with different amount of ICs
I have several datasets with 14 channels ( ~ 14 ICs), for one proband there was an error with one channel. So I had to mark it as bad and only calculate 13 ICs.

When running `corrmap()` I get this error:

	 File "E:\current-adas\project\code\src\util\mne_util.py", line 147, in labelArtefact
		return corrmap(icas, template=template, threshold=0.85, label=label, show=False, ch_type='eeg', verbose=True)
	  File "<string>", line 2, in corrmap
	  File "C:\Python27\lib\site-packages\mne-0.14.dev0-py2.7.egg\mne\utils.py", line 708, in verbose
		return function(*args, **kwargs)
	  File "C:\Python27\lib\site-packages\mne-0.14.dev0-py2.7.egg\mne\preprocessing\ica.py", line 2389, in corrmap
		nt, mt, s, mx = _find_max_corrs(all_maps, target, threshold)
	  File "C:\Python27\lib\site-packages\mne-0.14.dev0-py2.7.egg\mne\preprocessing\ica.py", line 2162, in _find_max_corrs
		all_corrs = [compute_corr(target, subj.T) for subj in all_maps]
	  File "C:\Python27\lib\site-packages\mne-0.14.dev0-py2.7.egg\mne\utils.py", line 2382, in compute_corr
		return (fast_dot(X.T, Y) / float(len(X) - 1)) / (x_sd * y_sd)
	ValueError: shapes (14,) and (13,13) not aligned: 14 (dim 0) != 13 (dim 0)

I wonder if this is normal and how I could fix this?

A workaround could be to add a dummy IC (with 0) to have 14 or calculate only 13 for the other probands. Which both seems not to be stable.


Discussion:
Isn't the problem with the channel number rather than the component number?

I forgot if the original corrmap interpolates the maps, but this version doesn't (for now).
I thought the amount of components shouldn't be greater than the amount of channels, so it depends on each other.

I am running 0.14dev, is there any fix in the pipeline?
>I thought the amount of components shouldn't be greater than the amount of channels, so it depends on each other.

No, but they can be fewer - you an estimate 13 components for each subject.

Really, I think the problem is with channel numbers, not component numbers.

>I am running 0.14dev, is there any fix in the pipeline?

I'm not planning on working on interpolation any time soon. You can interpolate your channels before fitting ICA.
Thanks for your thoughts.

So you suggesting to estimate 13 components for every proband. 
Isn't there any other way? Assume I'll get another proband and only 12 channels are good, then I have to run the ICA for all other probands with 12 components again.
Couldn't I fill the missing component with zeros? Would this have an impact on the result?

Sure the problem are the channel numbers, but they affect the max. number of components.

How would I interpolate my channels?
>Couldn't I fill the missing component with zeros? 

This would make the data rank deficient and mean you could also only estimate fewer components. Same with interpolation actually.

Personally, I just round down to the nearest nice number: if I have 64 channels, I do 60 components.
There is also an option for selecting enough components to get x% variance explained.

You can interpolate channels with an instance method, I think it's Epochs.interpolate_bads.
> Personally, I just round down to the nearest nice number: if I have 64 channels, I do 60 components.

Alright, maybe I'll just try with less components. I thought the more components the better :)
With 14 channels, how many components would you use. My major goal right now, is to extract EOG artifacts.

> There is also an option for selecting enough components to get x% variance explained.

I don't get it, what do you mean?

Do you know if `interpolate_bads` works for channels that are zero all the time (there was a bug within the recording software).
I'm pretty sure it completely ignores the channels to be interpolated.

@jona-sassenhagen you were right, the number of channels is the problem - now I know what you meant, during the last 2 posts. Reducing the component number does not solve the issue, as the shape still differs.
Can we close this issue @ppasler ?
Unfortunately I am currently running into the same problem. I want to run Corrmap ... but to get good ICA decompositions I want to reject a channel here and there throughout the dataset so that I ent up with participants having between 60 and 64 channels with different electrodes missing. Thus, the IC maps are also different and corrmap will not be able to do proprt correlations.

From the [Viola et al. 2009](https://www.ncbi.nlm.nih.gov/pubmed/19345611) paper alone, it is not clear how the original `corrmap` deals with differing numbers of channels across data sets:

> The default ICA algorithm used by EEGLAB (Delorme and Makeig, 2004) is Infomax ICA, where the number of ICs is usually equal to the number of EEG channels, normally corresponding to the rank of the data. However, CORRMAP also accepts a different number of ICs per dataset (in case of rank-deficiency or prior dimensionality reduction), thus providing greater flexibility. CORRMAP can also deal with variations in EEG channel numbers within a dataset, such as happens in the case of defective channels. In this case, CORRMAP requests a channel configuration file, and the inverse weights for missing electrodes are then automatically replaced using a modified version of the EEGLAB function eeg_interp().

However, if they interpolate the EEG channels before doing the ICA, that would mean that they do ICA on rank-deficient data ... so they must have a different fix ... maybe someone can find out by looking at [the source (zip file)](http://www.debener.de/corrmap/corrmap1.03.zip).

For now, the only solution I see is keeping the bad channels and doing my ICA decomposition with the bad data, rejecting only particularly bad segments. This suffices for the components I am interested in (blink, horizontal eye movement, ecg) ... but it's not optimal.
> However, if they interpolate the EEG channels before doing the ICA, that would mean that they do ICA on rank-deficient data

This is a case we need to support, both for EEG reference (which reduces rank) and MEG processing such as `maxwell_filter` (which reduces from 306 to 80). So we should support it properly, documenting any necessary caveats.

Then you can maybe use interpolation -- though we'd need to check that downstream processes such as source estimation don't mis-estimate the rank.
It's perfectly fine to run ICA on rank deficient data by just estimating fewer components.
@jona-sassenhagen, by "estimating fewer components" do you mean performing a dimension reduction via PCA before the ICA? Or what else do you mean?

In case of the PCA reduction, I see how this applies to ICA and rank deficient data in general, but how would it help for the case of `corrmap`? I thought the issue was that `corrmap` is correlating the "scalp maps" of the ICs and that hence, the mapping between an IC and its corresponding "electrode position" is important. Imagine the case where we have two participants A and B:

- Participant A has electrodes Cz, Pz, ..., Oz (64 electrodes)
- Participant B has the same electrodes, however Cz was bad so it is removed (63 electrodes)

We do an ICA on the pca-reduced data (let's say, 50 components). So although the dimensions of the unmixing matrices would be comparable ( shape 50,50), the IC "scalp maps" that are used as input in `corrmap` are not comparable (shape 64,64 vs. shape 63,63).

Or is `corrmap` doing the correlations on the unmixing matrices instead of the scalp maps? That would surprise me, because the intuition is that scalp maps (e.g., for eye blinks) are stable because they get their physiological grounding from the mapping between electrode position and IC weight ... whereas if the correlation is done on unmixing matrices that have been PCA-reduced before, each row of the unmixing matrix does no longer map to an electrode position (but rather, to a ranking in terms of variance explained), and if we then correlate across participants, an eye blink component for example could have completely different topographies.

Note, that I also have some trouble understanding the `n_components` `max_pca_components` and `n_pca_components` parameters in the ICA object, see my last comment in #4856. What would be really helpful for me to understand this is a list of steps from raw to scalp maps. Something in this form (but complete and correct and not just based on my assumptions):
1. Input data X of shape (n, t)
2. apply PCA to X ... we get weights W_pca of shape(max_pca_components, n)
3. apply W_pca to X ... we get X_pca of shape (max_pca_components, t)
4. apply ICA to X_pca, selecting only the first n_components of X_pca ... we get weights W_ica of shape (n_components, n_components)
5. ...?
6. ...? 

is this correct so far? If yes, what are the steps to go from there back to scalp maps?


Sorry @sappelhoff , I missed this. Is the question still unresolved?
@jona-sassenhagen the questions is still unresolved. In my work, I decided to not perform any dimension reduction and I did not reject channels prior to using corrmap ... thus I had a workaround for this problem.

The API for ICA/PCA is still kind of confusing and not perfectly well documented. In conjunction with using corrmap, this is additionally hard to wrap the head around ;-)
Is there any development regarding this issue? as unfortunately I'm running into the same problem. 


The issue is you have rejected channels, so your IC maps have different dimensionalities between subjects?
Based on the [corrmap/Artifact Correction with ICA tutorial](https://mne-tools.github.io/dev/auto_tutorials/plot_artifacts_correction_ica.html#sphx-glr-auto-tutorials-plot-artifacts-correction-ica-py) I'm doing this:
`this_ica = ICA(n_components=epochs.info["nchan"] - 1, method="fastica", random_state=42, max_iter=500).fit(epochs)`

`epochs` is my [`autoreject`](http://autoreject.github.io/index.html) corrected epochs after removal of bad channels (bridged/high percentage of interpolated data or bad data) and was average referenced. Therefore, `n_components=epochs.info["nchan"] - 1` will get different number of channels/components. 

The actual error that I get is: 

```
  File "<ipython-input-37-c3bcced40347>", line 4, in <module>
    show=False, threshold = 0.7)

  File "<string>", line 2, in corrmap

  File "C:\Users\Amir\Anaconda3\envs\mne\lib\site-packages\mne\utils.py", line 952, in verbose
    return function(*args, **kwargs)

  File "C:\Users\Amir\Anaconda3\envs\mne\lib\site-packages\mne\preprocessing\ica.py", line 2495, in corrmap
    nt, mt, s, mx = _find_max_corrs(all_maps, target, threshold)

  File "C:\Users\Amir\Anaconda3\envs\mne\lib\site-packages\mne\preprocessing\ica.py", line 2319, in _find_max_corrs
    all_corrs = [compute_corr(target, subj.T) for subj in all_maps]

  File "C:\Users\Amir\Anaconda3\envs\mne\lib\site-packages\mne\preprocessing\ica.py", line 2319, in <listcomp>
    all_corrs = [compute_corr(target, subj.T) for subj in all_maps]

  File "C:\Users\Amir\Anaconda3\envs\mne\lib\site-packages\mne\utils.py", line 2818, in compute_corr
    return (np.dot(X.T, Y) / float(len(X) - 1)) / (x_sd * y_sd)

ValueError: shapes (31,) and (30,29) not aligned: 31 (dim 0) != 30 (dim 0)
```
Maybe someone else will, but I will not write code to handle ICA objects with different montages ... you will have to use the same montage for all. 

I will however write a better error message. 
did you consider interpolating the missing channel?

>

That would be my preferred way of handling this, too.
Even though you know that the channel/s is/are almost completely interpolated? 
You would run the ICA -> exclude ICs -> remove the channel/s? 
I would interpolate -> run ICA.

Resolving patch:
diff --git a/mne/preprocessing/ica.py b/mne/preprocessing/ica.py
--- a/mne/preprocessing/ica.py
+++ b/mne/preprocessing/ica.py
@@ -50,6 +50,7 @@
                      copy_function_doc_to_method_doc, _pl, warn,
                      _check_preload, _check_compensation_grade, fill_doc,
                      _check_option)
+from ..utils.check import _check_all_same_channel_names
 
 from ..fixes import _get_args
 from ..filter import filter_data
@@ -2516,6 +2517,13 @@ def corrmap(icas, template, threshold="auto", label=None, ch_type="eeg",
     if not isinstance(plot, bool):
         raise ValueError("`plot` must be of type `bool`")
 
+    same_chans = _check_all_same_channel_names(icas)
+    if same_chans is False:
+        raise ValueError("Not all ICA instances have the same channel names. "
+                         "Corrmap requires all instances to have the same "
+                         "montage. Consider interpolating bad channels before "
+                         "running ICA.")
+
     if threshold == 'auto':
         threshold = np.arange(60, 95, dtype=np.float64) / 100.
 
diff --git a/mne/utils/check.py b/mne/utils/check.py
--- a/mne/utils/check.py
+++ b/mne/utils/check.py
@@ -447,3 +447,12 @@ def _check_option(parameter, value, allowed_values):
         options += ' and %r' % allowed_values[-1]
     raise ValueError(msg.format(parameter=parameter, options=options,
                                 value=value))
+
+
+def _check_all_same_channel_names(instances):
+    """Check if a collection of instances all have the same channels."""
+    ch_names = instances[0].info["ch_names"]
+    for inst in instances:
+        if ch_names != inst.info["ch_names"]:
+            return False
+    return True


Test patch:
diff --git a/mne/preprocessing/tests/test_ica.py b/mne/preprocessing/tests/test_ica.py
--- a/mne/preprocessing/tests/test_ica.py
+++ b/mne/preprocessing/tests/test_ica.py
@@ -383,6 +383,10 @@ def test_ica_additional(method):
 
     plt.close('all')
 
+    ica_different_channels = ICA(n_components=2, random_state=0).fit(
+        raw, picks=[2, 3, 4, 5])
+    pytest.raises(ValueError, corrmap, [ica_different_channels, ica], (0, 0))
+
     # test warnings on bad filenames
     ica_badname = op.join(op.dirname(tempdir), 'test-bad-name.fif.gz')
     with pytest.warns(RuntimeWarning, match='-ica.fif'):