Repository: mne-tools/mne-python

Issue:
ICA questions
I don't understand some choices in the ICA calculation process, maybe someone could answer these questions:

1. Why is a so-called pre-whitener (by default channels are divided by their standard deviations) used before PCA/ICA? I think this is not necessary because the next step is PCA with `whiten=True`.
2. Speaking of PCA and whitening, sphering seems to be more optimal for ICA, and sphering is also used in EEGLAB by default. AFAIK the only change necessary would be another rotation by the eigenvector matrix to convert the PCA whitening transform to a sphering transform. Should we care to implement it?
3. I don't understand [this line](https://github.com/mne-tools/mne-python/blob/master/mne/preprocessing/ica.py#L618) - why are the elements of the unmixing matrix divided by the square root of the explained variances?


Discussion:
> Why is a so-called pre-whitener (by default channels are divided by their standard deviations) used before PCA/ICA? I think this is not necessary because the next step is PCA with whiten=True.

My guess is that it's for numerical stability. If you have channels on drastically different scales then even PCA can fail.

> Speaking of PCA and whitening, sphering seems to be more optimal for ICA, and sphering is also used in EEGLAB by default. AFAIK the only change necessary would be another rotation by the eigenvector matrix to convert the PCA whitening transform to a sphering transform. Should we care to implement it?

I have no problem with `sphering=False` default (for backward compat) but I'm guessing @agramfort or  Pierre (can't remember GitHub name) might have thoughts.

> I don't understand this line - why are the elements of the unmixing matrix divided by the square root of the explained variances?

This one I do not know, except to say that it has been there since 2014 (0.8) and was added by @dengemann
>
> Why is a so-called pre-whitener (by default channels are divided by their
> standard deviations) used before PCA/ICA? I think this is not necessary
> because the next step is PCA with whiten=True.
>
> My guess is that it's for numerical stability. If you have channels on
> drastically different scales then even PCA can fail.
>

if you do PCA dim reduction and you combine grad and mag then you will only
use the channel type with highest absolute amplitude unless you prewhiten.

> Speaking of PCA and whitening, sphering seems to be more optimal for ICA,
> and sphering is also used in EEGLAB by default. AFAIK the only change
> necessary would be another rotation by the eigenvector matrix to convert
> the PCA whitening transform to a sphering transform. Should we care to
> implement it?
>
> I have no problem with sphering=False default (for backward compat) but
> I'm guessing @agramfort <https://github.com/agramfort> or Pierre (can't
> remember GitHub name) might have thoughts.
>

Infomax does not require sphering but FastICA does. So you need to do:

prewhiten -> PCA for dim red with whiten=True -> FastICA (or Infomax)

clear?

> if you do PCA dim reduction and you combine grad and mag then you will only
use the channel type with highest absolute amplitude unless you prewhiten.

Of course! Feature normalization prior to PCA is essential - somehow I didn't make the connection (prewhitening, z-score - we should update the docs and just call ~~normalization~~ standardization by its name).

> Infomax does not require sphering but FastICA does. So you need to do:
>
> prewhiten -> PCA for dim red with whiten=True -> FastICA (or Infomax)
>
> clear?

No 😄  - `white=True` is not sphering, it's whitening. Or are you saying that normalization (aka pre-whitening) plus whitening equals sphering?
>
> if you do PCA dim reduction and you combine grad and mag then you will only
> use the channel type with highest absolute amplitude unless you prewhiten.
>
> Of course! Feature normalization prior to PCA is essential - somehow I
> didn't make the connection (prewhitening, z-score - we should update the
> docs and just call normalization by its name).
>
sounds good. PR welcome :)

> Infomax does not require sphering but FastICA does. So you need to do:
>
> prewhiten -> PCA for dim red with whiten=True -> FastICA (or Infomax)
>
> clear?
>
> No 😄 - white=True is not sphering, it's whitening. Or are you saying
> that normalization (aka pre-whitening) plus whitening equals sphering?
>

let's say data are X shape (n_channels, n,_times)
you want to whitened the data with W such that cov(WX) = np.eye(n_channels)

to do this you need W to be the inverse of a square root of cov(X). Let's
call the svd of this covariance U S^2 U^T then W can be S^{-1} U^T or
U S^{-1} U^T. What you call (I think) sphering is U S^{-1} U^T and whitening
S^{-1} U^T but they are just 2 choices to whiten the data.

clear?

> to do this you need W to be the inverse of a square root of cov(X). Let's
call the svd of this covariance U S^2 U^T then W can be S^{-1} U^T or
U S^{-1} U^T. What you call (I think) sphering is U S^{-1} U^T and whitening
S^{-1} U^T but they are just 2 choices to whiten the data.

Yes, the terminology seems to be inconsistent. Whitening is a transform that decorrelates the data (i.e. the covariance matrix becomes the identity matrix), e.g. via PCA. There are infinitely many ways to whiten due to rotation. Sphering is a special kind of whitening that rotates the data back to the original space (some call that ZCA). So exactly like you say, but currently we (and scikit-learn) use PCA whitening, but EEGLAB uses sphering (and I remember having read somewhere that ZCA is more optimal for Infomax than PCA whitening, but I can't find that anymore).

In any case, since I can't find relevant references, any kind of whitening seems to work just fine, so if I find more on that topic I'll create a new issue.

So only the last question left - @dengemann?
Improved docstring in #4886.
OK, I figured it out myself. The line

    self.unmixing_matrix_ /= np.sqrt(exp_var[sel])[None, :]

does the whitening, which is not part of the PCA transform matrix `self.pca_components_`.

Kind of confusing, I think it would be much easier if all transforms were accessible as matrices, i.e. (1) the pre-whitening (standardization) (a diagonal matrix), (2) the PCA transform matrix, (3) the whitening (again a diagonal matrix), and (4) the ICA unmixing matrix. Oh, and the mean of the data should also be removed prior to PCA I guess. Currently, the matrices accessible as `ica.unmixing_matrix_` and `ica.pca_components_` are not really that useful, because it is not obvious how they should be used. Instead, the user has to rely on a transform method, which confusingly is called `get_sources`.
why storing a diagonal matrix as full?

what do you suggest in terms of API/code changes?

It's not about how to store the matrices, but how to get them. My problem with the current approach is that currently you get `ica.pca_components_` and `ica.unmixing_matrix_` - but you cannot apply these matrices to the original data to do the transform, because you have to scale to unit variance, remove the mean, and whiten in addition. Ideally, I would like to replicate the transform that I get via `get_sources` with the transform matrices. Strangely, this is not possible with `sklearn.decomposition.PCA` either, because the whitening is not part of the PCA transform matrix. Of course this information is available with the `explained_variance_` attribute, but I think it would be more convenient to get it in matrix form.

What about adding a new method `get_transform_matrices` (is there a better/shorter name?) which returns all matrices required for the ICA transform?

Alternatively, the user might only be interested in getting the final ICA transform matrix, so this might be another option (I would call this `get_unmixing_matrix`, but this would clash with `ica.unmixing_matrix_`).

Finally, I suggest the following API changes.

1. Let's rename `get_sources` to `transform`. This is more intuitive and in line with the sklearn API.
2. Let's rename `ica.unmixing_matrix_` to `ica.ica_unmixing_matrix_`. Then we could add the final unmixing matrix as `ica.unmixing_matrix_` instead of a method `get_unmixing_matrix`. Similarly, we could add the other transform matrices as attributes.

Either way, this needs to be documented so that it is clear what is going on. Once we've decided what to do I'll be happy to take care of that.



how about we add get_unmixing_matrix method and add a param
with_withening=True | False and if True the get the matrix you want?

I am -0.5 on changing yet again the ICA API...

regarding sklearn API I would argue that we enforce this on the decoding
module that works mostly with plain numpy arrays.
get_sources is I think more explicit for non sklearn users.

> how about we add get_unmixing_matrix method and add a param
with_withening=True | False and if True the get the matrix you want?

No need for a parameter, we're always whitening. I'm OK with adding a method `get_unmixing_matrix` if it returns the complete transformation matrix which takes you from the raw data to the sources (i.e. this unmixing matrix contains all transforms).

> I am -0.5 on changing yet again the ICA API...

Yes, I don't want to do that either, so let's clearly document it.

> regarding sklearn API I would argue that we enforce this on the decoding
module that works mostly with plain numpy arrays.
get_sources is I think more explicit for non sklearn users.

Well, ICA transforms the data too. What about adding a new alias method `transform` that just calls `get_sources`?
from a user perspective why do you what to have the full matrix?
you expect users to want to use directly this numpy array rather than
calling get_sources?

regarding the transform alias I would argue:

```
$ python -c "import this" | grep "only one"
There should be one-- and preferably only one --obvious way to do it.
```

> from a user perspective why do you what to have the full matrix?
you expect users to want to use directly this numpy array rather than
calling get_sources?

Yes. Partly also because you can get the full transform matrix in EEGLAB (where it is documented to be `weights * sphere`). Since you ask this question, I think the full matrix is much more useful for users than what MNE currently offers (`ica.unmixing_matrix_` and `ica.pca_components_`). However, even better would be what I suggested previously, namely to have all matrices that compose the transform.

I also think that we should not hide the full unmixing matrix from the user - what if someone wants to analyze its structure etc.?

> regarding the transform alias I would argue:

```
$ python -c "import this" | grep "only one"
There should be one-- and preferably only one --obvious way to do it.
```

Right, but I think consistency is also important (the zen should really be updated to include that). But OK, I'm not Dutch, so `get_sources` was not really obvious to me since I was looking for `transform`.
Oh, and also, `get_sources` works with a `raw` instance, so yes, I think it would be very useful to be able to apply the ICA transform to a NumPy array.
> I think it would be very useful to be able to apply the ICA transform to a NumPy array.

A side note FYI -- this is a hack but for any of these linear transforms we have keep in mind you can hack it with something like:
```
trans_mat = thing.transform(RawArray(np.eye(len(raw.ch_names)), raw.info)
```
It sounds like you're saying that we want people to be able to do this without a hack workaround, though, so I'm fine with exposing the necessary parts and/or a method.

Given that we already expose some of the components (but not all?), maybe we should just expose any remaining necessary components and tell people how to compose the transformation matrix in the docs (e.g., `t = np.dot(a, b * c[:, np.newaxis)` rather than just giving a method to obtain it. I think we should have all of these components exposed to the user in any case, so maybe we don't need to add a new method for such an advanced application.
Oh yes, this is definitely nice but hacky (i.e. no one without some experience with MNE will be able to come up with it in less than 5 minutes).

+100 for exposing the remaining parts, this is still my preferred option. I propose the following components:

- `ica.unmixing_matrix_` (already exists)
- `ica.pca_components_` (already exists)
- `ica.standardization_` (removes mean and scales variance to 1) - but we have to think about how the `noise_cov` argument influences this pre-whitening to return the right thing
- `ica.whitening_`

Then the docs should say that the final unmixing matrix, which transforms the MEEG data into sources, can be composed as `ica.unmixing_matrix_ @ ica.whitening_ @ ica.pca_components_ @ ica.standardization_` (ignoring any necessary transpose operation or dimensionality rectification right now).
(And also, there is no `ica.transform` because it is called `ica.get_sources` - that's why I suggested to add a `transform` method.)
I am fine exposing what you suggest as attribute and documenting what you
need to do
but how about doing

ica.unmixing_matrix_ @ ica.whitening_ @ ica.standardization_

ie exposing whitening as one matrix which would contain the scaled
ica.pca_components_?

I'd prefer having separate whitening and PCA matrices, because if we combine these two people might wonder why they don't need the PCA matrix.
ok then. let's do it as you proposed 2 posts above then.

If I transform the signals to sources with the computed unmixing matrix, I get slightly different results compared to `get_sources`. The difference is not large, but large enough so that `np.allclose` returns `False`.

Here's how I build the final unmixing matrix:

```
U = ica.unmixing_matrix_ @ ica.pca_components_[:ica.n_components_] @ np.diag(1 / ica.pre_whitener_.squeeze()) @ center
```

Where `center = np.eye(n_chans) - 1/n_chans * np.ones((n_chans, n_chans))` (`n_chans` is the number of channels in the original data, I'm not sure which value to use here).

For example, here are the first 10 values of source IC001 computed with `get_sources`:
```
array([-7.70700725e-05, -2.52720611e-01, -4.37112044e-01, -4.45954244e-01,
       -2.83206876e-01, -2.46589467e-01, -5.05721721e-01, -6.42498052e-01,
       -5.47709280e-01, -3.98918470e-01])
```

And here are the values computed by `U @ raw._data`:
```
array([ 1.81307766e-13, -2.52643541e-01, -4.37034974e-01, -4.45877174e-01,
       -2.83129806e-01, -2.46512397e-01, -5.05644651e-01, -6.42420982e-01,
       -5.47632210e-01, -3.98841400e-01])
```

I have no idea why the first value differs that much, but even though all remaining values are much closer, they are still different. Any ideas?

Here's how `get_sources` works internally:
```
ica.unmixing_matrix_ @ ica.pca_components_[:n_chans] @ ((raw._data / ica.pre_whitener_).T - ica.pca_mean_).T
```

This should do the same thing, the only difference here is that `get_sources` first scales the channels and then subtracts the mean, which is unusual (standardization first subtracts the mean and then scales), but it is not the reason for the different results here.
You have `n` operands in the top and `m` in the bottom. Usually in cases like this I'd compare each step by step to see where the difference comes in. At least the leftmost one is the same so it can't be the problem. But then the question is why does this:
```
ica.pca_components_[:ica.n_components_] @ np.diag(1 / ica.pre_whitener_.squeeze()) @ center
```
differ from this:
```
ica.pca_components_[:n_chans] @ ((raw._data / ica.pre_whitener_).T - ica.pca_mean_).T
```
Is `ica.n_components_ == n_chans`? If so we know the problem is not `ica.pca_components_` either.

I'd then continue to swap in bits of one equation for the other where you think they should be equivalent, and see where it breaks.
The issue is the mean here: `get_sources` subtracts `ica.pca_mean_`, which is the mean of the channels ICA was computed on. However, I think it would be more correct to subtract the mean of the signals the ICA is applied to.
well the EEG/MEG signals are assumed to be zero mean typically. That's why
we just do a dot with the whitener.

now this suggests that PCA should be using assume_centered option to avoid
centering again.

if the data are actually zero mean this should not make a difference anyway.

my 2c

You are right, fitting ICA assumes (and requires) the data to be zero mean. This is left to the user, and we also mention the importance of a good highpass filter in the docs. But I don't understand why the `get_sources` method the subtracts the actual mean then - if we assume the signals are zero mean there is no need to do that. But if we do it, we can't subtract the mean from the signals ICA was computed on - we'd need to use the actual mean of the signal we're applying ICA on (and these two need not be the same signals, although for the purpose of artifact rejection they usually are).

So for me there are two options: (1) Do not subtract the mean in `get_sources` or (2) subtract the mean of the current signal instead of the original signal. Also, this is not related to PCA since whatever PCA does is reflected in its transform matrix (except for whitening). There is no such option to turn off centering anyway.
I would be fine not removing any mean at all.

but I am -1 on removing the mean from transformed signals as it would mean
that
transform depends on the data so for example you would not get the same
if you do Raw -> Epochs -> ICA Sources vs Raw -> ICA Sources -> Epochs

Yes, that's what I'm saying. So basically just remove these two lines and we're good?
https://github.com/mne-tools/mne-python/blob/master/mne/preprocessing/ica.py#L614-L615
well no. If it was subtracted at fit time it should be there too.

it should be removed globally or at least we should have an
option to assume the signals are zero mean initially.

It is common practice to compute ICA on one dataset and apply it to another dataset. So in the first step, you solve `s = U @ x` for `U`. This transform now includes centering, i.e. it subtracts the channel means of `x`. But if you apply `U` to a new signal, you want to compute `s1 = U @ x1`. Therefore, `U` removes the mean of `x1` - why should it remove the mean of `x` from `x1`?

An option to assume zero-mean signals would be useful though.
>
> It is common practice to compute ICA on one dataset and apply it to
> another dataset. So in the first step, you solve s = U @ x for U. This
> transform now includes centering, i.e. it subtracts the channel means of x.
> But if you apply U to a new signal, you want to compute s1 = U @ x1.
> Therefore, U removes the mean of x1 - why should it remove the mean of x
> from x1?
>
it depends what you assume about the data. If it comes from same
distribution or not.

I don't like the idea that the size if X changes the output of transform.
It's like in sklearn.
You want that

clf.transform(np.concatenate([X1, X2], axis=0))
==
np.concatenate([clf.transform(X1), clf.transform(X2)], axis=0)

> An option to assume zero-mean signals would be useful though.
>
yes

Yes, I get the point. You compute everything in `fit` and then apply that in `transform`. But this makes it hard to write the whole transform into one unmixing matrix - or am I missing something obvious?
I think that we do currently is:

U = ica.unmixing_matrix_ @ ica.pca_components_[:ica.n_components_] @ center @

np.diag(1 / ica.pre_whitener_.squeeze())


ie centering is done after whitening/scaling of channels

Currently we subtract the mean after scaling (pre-whitening), but before whitening and before PCA (after all, we use `pca.mean_`). But you can't write this is one matrix U then, because center uses a mean from a different signal. So the best solution would be to get rid of centering and assume that it is done separately (i can look into this some more to find out how it behaves). It is the same thing with filtering: ICA requires a highpass filter, but we don't incorporate the filter into the final unmixing matrix. We don't have to integrate centering either. I think if this is documented, we don't have to change code (except for renaming `ica._pre_whitener` to `ica.pre_whitener_` for consistency), but "only" update the docs.
I don't get this. You can still take the centering matrix from the fit data
and assemble the U matrix. It will assume the you did the same highpass on
the transformed data

ok about the plan

How does the centering matrix from the previous data look like then? I was using [this definition here](https://en.wikipedia.org/wiki/Centering_matrix), and it only works on the new data.
let's see if this can help


import numpy as np
X = np.random.randn(10, 30)
X_mean = np.mean(X, axis=1, keepdims=True)
X_centered = X - X_mean

# now try to write it as dot product
A = np.eye(10) - np.linalg.pinv(X.T).dot(np.tile(X_mean, (1, 30)).T).T
X_centered2 = np.dot(A, X)

X_centered - X_centered2

Hm, that's a pretty large difference (I assume because of the pseudo-inverse calculation), even with the same data X.
yeah not sure it's great either...

@cbrnr do you have time to write a summary PR of this in the `Notes` section of the `ICA` object? 
Yes, will do!

Resolving patch:
diff --git a/mne/preprocessing/ica.py b/mne/preprocessing/ica.py
--- a/mne/preprocessing/ica.py
+++ b/mne/preprocessing/ica.py
@@ -1,3 +1,5 @@
+# -*- coding: utf-8 -*-
+#
 # Authors: Denis A. Engemann <denis.engemann@gmail.com>
 #          Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
 #          Juergen Dammers <j.dammers@fz-juelich.de>
@@ -118,132 +120,111 @@ def _check_for_unsupported_ica_channels(picks, info):
 
 
 class ICA(ContainsMixin):
-    """M/EEG signal decomposition using Independent Component Analysis (ICA).
+    u"""M/EEG signal decomposition using Independent Component Analysis (ICA).
 
-    This object can be used to estimate ICA components and then
-    remove some from Raw or Epochs for data exploration or artifact
-    correction.
+    This object can be used to estimate ICA components and then remove some
+    from Raw or Epochs for data exploration or artifact correction.
 
-    Caveat! If supplying a noise covariance keep track of the projections
+    Caveat! If supplying a noise covariance, keep track of the projections
     available in the cov or in the raw object. For example, if you are
     interested in EOG or ECG artifacts, EOG and ECG projections should be
-    temporally removed before fitting the ICA. You can say::
+    temporally removed before fitting ICA, for example::
 
         >> projs, raw.info['projs'] = raw.info['projs'], []
         >> ica.fit(raw)
         >> raw.info['projs'] = projs
 
-    .. note:: Methods implemented are FastICA (default), Infomax,
-              Extended-Infomax, and Picard. Infomax can be quite sensitive to
-              differences in floating point arithmetic. Extended-Infomax seems
+    .. note:: Methods currently implemented are FastICA (default), Infomax,
+              Extended Infomax, and Picard. Infomax can be quite sensitive to
+              differences in floating point arithmetic. Extended Infomax seems
               to be more stable in this respect enhancing reproducibility and
               stability of results.
-              The stopping criteria of FastICA, Infomax, Extended
-              Infomax and Picard differ, making it hard to compare different
-              tolerance levels, but a rule of thumb is
-              `tol_fastica = tol_picard ** 2`. Reducing the tolerance speeds up
-              estimation but the consistency of the obtained results decreases.
 
     .. warning:: ICA is sensitive to low-frequency drifts and therefore
                  requires the data to be high-pass filtered prior to fitting.
-                 Typically, a cutoff frequency of 1 Hz is recommended. Note
-                 that FIR filters prior to MNE 0.15 used the ``'firwin2'``
-                 design method, which generally produces rather shallow filters
-                 that might not work for ICA processing. Therefore, it is
-                 recommended to use IIR filters for MNE up to 0.14. In MNE
-                 0.15, FIR filters can be designed with the ``'firwin'``
-                 method, which generally produces much steeper filters. This
-                 method will be the default FIR design method in MNE 0.16. In
-                 MNE 0.15, you need to explicitly set ``fir_design='firwin'``
-                 to use this method. This is the recommended filter method for
-                 ICA preprocessing.
+                 Typically, a cutoff frequency of 1 Hz is recommended.
 
     Parameters
     ----------
     n_components : int | float | None
         The number of components used for ICA decomposition. If int, it must be
-        smaller then max_pca_components. If None, all PCA components will be
-        used. If float between 0 and 1 components will be selected by the
+        smaller than `max_pca_components`. If None, all PCA components will
+        be used. If float between 0 and 1, components will be selected by the
         cumulative percentage of explained variance.
     max_pca_components : int | None
         The number of components used for PCA decomposition. If None, no
-        dimension reduction will be applied and max_pca_components will equal
-        the number of channels supplied on decomposing data. Defaults to None.
+        dimensionality reduction will be applied and `max_pca_components` will
+        equal the number of channels supplied for decomposing data.
     n_pca_components : int | float
         The number of PCA components used after ICA recomposition. The ensuing
-        attribute allows to balance noise reduction against potential loss of
-        features due to dimensionality reduction. If greater than
-        ``self.n_components_``, the next ``n_pca_components`` minus
-        ``n_components_`` PCA components will be added before restoring the
-        sensor space data. The attribute gets updated each time the according
-        parameter for in .pick_sources_raw or .pick_sources_epochs is changed.
-        If float, the number of components selected matches the number of
-        components with a cumulative explained variance below
+        attribute `n_components_` allows to balance noise reduction against
+        potential loss of information due to dimensionality reduction. If
+        greater than `n_components_`, the next `n_pca_components` minus
+        `n_components_` PCA components will be added before restoring the
+        sensor space data. If float, the number of components selected matches
+        the number of components with a cumulative explained variance below
         `n_pca_components`.
     noise_cov : None | instance of mne.cov.Covariance
         Noise covariance used for pre-whitening. If None, channels are scaled
         to unit variance prior to whitening.
     random_state : None | int | instance of np.random.RandomState
-        np.random.RandomState to initialize the FastICA estimation.
-        As the estimation is non-deterministic it can be useful to
-        fix the seed to have reproducible results. Defaults to None.
+        Random state to initialize ICA estimation for reproducible results.
     method : {'fastica', 'infomax', 'extended-infomax', 'picard'}
         The ICA method to use. Defaults to 'fastica'. For reference, see [1]_,
         [2]_, [3]_ and [4]_.
-    fit_params : dict | None.
-        Additional parameters passed to the ICA estimator chosen by `method`.
-    max_iter : int, optional
+    fit_params : dict | None
+        Additional parameters passed to the ICA estimator as specified by
+        `method`.
+    max_iter : int
         Maximum number of iterations during fit.
-    verbose : bool, str, int, or None
-        If not None, override default verbose level (see :func:`mne.verbose`
-        and :ref:`Logging documentation <tut_logging>` for more).
+    verbose : bool | str | int | None
+        If not None, override default verbosity level (see :func:`mne.verbose`
+        and :ref:`Logging documentation <tut_logging>`).
 
     Attributes
     ----------
     current_fit : str
-        Flag informing about which data type (raw or epochs) was used for
-        the fit.
+        Flag informing about which data type (raw or epochs) was used for the
+        fit.
     ch_names : list-like
         Channel names resulting from initial picking.
-        The number of components used for ICA decomposition.
     n_components_ : int
         If fit, the actual number of components used for ICA decomposition.
-    n_pca_components : int
-        See above.
-    max_pca_components : int
-        The number of components used for PCA dimensionality reduction.
-    verbose : bool, str, int, or None
-        See above.
-    pca_components_ : ndarray
-        If fit, the PCA components
-    pca_mean_ : ndarray
+    pre_whitener_ : ndarray, shape (n_channels, 1)
+        If fit, array used to pre-whiten the data prior to PCA.
+    pca_components_ : ndarray, shape (`n_components_`, n_channels)
+        If fit, the PCA components.
+    pca_mean_ : ndarray, shape (n_channels,)
         If fit, the mean vector used to center the data before doing the PCA.
-    pca_explained_variance_ : ndarray
+    pca_explained_variance_ : ndarray, shape (`n_components_`,)
         If fit, the variance explained by each PCA component
-    mixing_matrix_ : ndarray
-        If fit, the mixing matrix to restore observed data, else None.
-    unmixing_matrix_ : ndarray
-        If fit, the matrix to unmix observed data, else None.
+    mixing_matrix_ : ndarray, shape (`n_components_`, `n_components_`)
+        If fit, the mixing matrix to restore observed data.
+    unmixing_matrix_ : ndarray, shape (`n_components_`, `n_components_`)
+        If fit, the matrix to unmix observed data.
     exclude : list
         List of sources indices to exclude, i.e. artifact components identified
-        throughout the ICA solution. Indices added to this list, will be
-        dispatched to the .pick_sources methods. Source indices passed to
-        the .pick_sources method via the 'exclude' argument are added to the
-        .exclude attribute. When saving the ICA also the indices are restored.
-        Hence, artifact components once identified don't have to be added
-        again. To dump this 'artifact memory' say: ica.exclude = []
+        throughout the ICA solution. To scrap all marked components, you can
+        set this attribute to an empty list.
     info : None | instance of Info
         The measurement info copied from the object fitted.
     n_samples_ : int
-        the number of samples used on fit.
+        The number of samples used on fit.
     labels_ : dict
         A dictionary of independent component indices, grouped by types of
         independent components. This attribute is set by some of the artifact
         detection functions.
 
+    Notes
+    -----
+    Reducing the tolerance speeds up estimation at the cost of consistency of
+    the obtained results. It is difficult to directly compare tolerance levels
+    between Infomax and Picard, but for Picard and FastICA a good rule of thumb
+    is ``tol_fastica = tol_picard ** 2``.
+
     References
     ----------
-    .. [1] Hyvarinen, A., 1999. Fast and robust fixed-point algorithms for
+    .. [1] Hyvärinen, A., 1999. Fast and robust fixed-point algorithms for
            independent component analysis. IEEE transactions on Neural
            Networks, 10(3), pp.626-634.
 
@@ -435,7 +416,7 @@ def fit(self, inst, picks=None, start=None, stop=None, decim=None,
 
     def _reset(self):
         """Aux method."""
-        del self._pre_whitener
+        del self.pre_whitener_
         del self.unmixing_matrix_
         del self.mixing_matrix_
         del self.n_components_
@@ -485,7 +466,7 @@ def _fit_raw(self, raw, picks, start, stop, decim, reject, flat, tstep,
 
         self.n_samples_ = data.shape[1]
         # this may operate inplace or make a copy
-        data, self._pre_whitener = self._pre_whiten(data, raw.info, picks)
+        data, self.pre_whitener_ = self._pre_whiten(data, raw.info, picks)
 
         self._fit(data, self.max_pca_components, 'raw')
 
@@ -522,7 +503,7 @@ def _fit_epochs(self, epochs, picks, decim, verbose):
 
         # This will make at least one copy (one from hstack, maybe one
         # more from _pre_whiten)
-        data, self._pre_whitener = \
+        data, self.pre_whitener_ = \
             self._pre_whiten(np.hstack(data), epochs.info, picks)
 
         self._fit(data, self.max_pca_components, 'epochs')
@@ -531,7 +512,7 @@ def _fit_epochs(self, epochs, picks, decim, verbose):
 
     def _pre_whiten(self, data, info, picks):
         """Aux function."""
-        has_pre_whitener = hasattr(self, '_pre_whitener')
+        has_pre_whitener = hasattr(self, 'pre_whitener_')
         if not has_pre_whitener and self.noise_cov is None:
             # use standardization as whitener
             # Scale (z-score) the data by channel type
@@ -562,11 +543,11 @@ def _pre_whiten(self, data, info, picks):
             assert data.shape[0] == pre_whitener.shape[1]
             data = np.dot(pre_whitener, data)
         elif has_pre_whitener and self.noise_cov is None:
-            data /= self._pre_whitener
-            pre_whitener = self._pre_whitener
+            data /= self.pre_whitener_
+            pre_whitener = self.pre_whitener_
         else:
-            data = np.dot(self._pre_whitener, data)
-            pre_whitener = self._pre_whitener
+            data = np.dot(self.pre_whitener_, data)
+            pre_whitener = self.pre_whitener_
 
         return data, pre_whitener
 
@@ -637,7 +618,7 @@ def _fit(self, data, max_pca_components, fit_type):
                              random_state=random_state, **self.fit_params)
             del _
             self.unmixing_matrix_ = W
-        self.unmixing_matrix_ /= np.sqrt(exp_var[sel])[None, :]
+        self.unmixing_matrix_ /= np.sqrt(exp_var[sel])[None, :]  # whitening
         self.mixing_matrix_ = linalg.pinv(self.unmixing_matrix_)
         self.current_fit = fit_type
 
@@ -1385,9 +1366,9 @@ def _pick_sources(self, data, include, exclude):
 
         # restore scaling
         if self.noise_cov is None:  # revert standardization
-            data *= self._pre_whitener
+            data *= self.pre_whitener_
         else:
-            data = np.dot(linalg.pinv(self._pre_whitener, cond=1e-14), data)
+            data = np.dot(linalg.pinv(self.pre_whitener_, cond=1e-14), data)
 
         return data
 
@@ -1879,7 +1860,7 @@ def _write_ica(fid, ica):
                  _serialize(ica_misc))
 
     #   Whitener
-    write_double_matrix(fid, FIFF.FIFF_MNE_ICA_WHITENER, ica._pre_whitener)
+    write_double_matrix(fid, FIFF.FIFF_MNE_ICA_WHITENER, ica.pre_whitener_)
 
     #   PCA components_
     write_double_matrix(fid, FIFF.FIFF_MNE_ICA_PCA_COMPONENTS,
@@ -1992,7 +1973,7 @@ def f(x):
     ica = ICA(**ica_init)
     ica.current_fit = current_fit
     ica.ch_names = ch_names.split(':')
-    ica._pre_whitener = f(pre_whitener)
+    ica.pre_whitener_ = f(pre_whitener)
     ica.pca_mean_ = f(pca_mean)
     ica.pca_components_ = f(pca_components)
     ica.n_components_ = unmixing_matrix.shape[0]


Test patch:
diff --git a/mne/preprocessing/tests/test_ica.py b/mne/preprocessing/tests/test_ica.py
--- a/mne/preprocessing/tests/test_ica.py
+++ b/mne/preprocessing/tests/test_ica.py
@@ -184,7 +184,7 @@ def test_ica_reset(method):
                        eog=False, exclude='bads')[:10]
 
     run_time_attrs = (
-        '_pre_whitener',
+        'pre_whitener_',
         'unmixing_matrix_',
         'mixing_matrix_',
         'n_components_',
@@ -297,10 +297,10 @@ def test_ica_core(method):
                       include=[1, 2])
 
     # test for bug with whitener updating
-    _pre_whitener = ica._pre_whitener.copy()
+    _pre_whitener = ica.pre_whitener_.copy()
     epochs._data[:, 0, 10:15] *= 1e12
     ica.apply(epochs.copy())
-    assert_array_equal(_pre_whitener, ica._pre_whitener)
+    assert_array_equal(_pre_whitener, ica.pre_whitener_)
 
     # test expl. var threshold leading to empty sel
     ica.n_components = 0.1
@@ -476,7 +476,7 @@ def test_ica_additional(method):
 
         # check type consistency
         attrs = ('mixing_matrix_ unmixing_matrix_ pca_components_ '
-                 'pca_explained_variance_ _pre_whitener')
+                 'pca_explained_variance_ pre_whitener_')
 
         def f(x, y):
             return getattr(x, y).dtype
@@ -492,7 +492,7 @@ def f(x, y):
         ica_read = read_ica(test_ica_fname)
         for attr in ['mixing_matrix_', 'unmixing_matrix_', 'pca_components_',
                      'pca_mean_', 'pca_explained_variance_',
-                     '_pre_whitener']:
+                     'pre_whitener_']:
             assert_array_almost_equal(getattr(ica, attr),
                                       getattr(ica_read, attr))