Repository: mne-tools/mne-python

Issue:
[MRG] Add projection argument to set_eeg_reference
Fix #4302 and #3998.

Added new argument `projection=False` to `set_eeg_reference`.


Discussion:
> In order to retain the option to reference to a channel named average, I propose that referencing to one ore more channels must be specified in a list

Okay for me

> If we reference to the average with projection=False, should we return something else than None as the second element of the tuple?

Yes, return whatever gets returned for a smaller subset of channels

> The try/except block should catch (and re-raise) a specific exception - which one?

Not sure what you mean, I'll look at the code to see if it becomes clear

> did we agree that the default of projection should be False?

Yeah I think we agreed on this but let's double-check. @agramfort okay for you? It means more pain for MEG/imaging folks but easier for EEG.
# [Codecov](https://codecov.io/gh/mne-tools/mne-python/pull/4382?src=pr&el=h1) Report
> Merging [#4382](https://codecov.io/gh/mne-tools/mne-python/pull/4382?src=pr&el=desc) into [master](https://codecov.io/gh/mne-tools/mne-python/commit/5d1fc7af7301a26f3415a876efee3c08cda28bb1?src=pr&el=desc) will **increase** coverage by `0.02%`.
> The diff coverage is `94.87%`.


```diff
@@            Coverage Diff             @@
##           master    #4382      +/-   ##
==========================================
+ Coverage   82.74%   82.77%   +0.02%     
==========================================
  Files         349      349              
  Lines       63970    64089     +119     
  Branches     9851     9869      +18     
==========================================
+ Hits        52935    53047     +112     
- Misses       8308     8310       +2     
- Partials     2727     2732       +5
```



> I still need to issue a deprecation(?) warning that the default behavior of the average reference has changed - did we agree that the default of projection should be False? When and how do we warn users that the default behavior has changed?

Yes that's what we came to in the issue but we should double-check with others (@agramfort @wmvanvliet).

Same as usual for deprecation -- `projection=None` default for one cycle, which means "emit warning; use true for 'average' and false otherwise". Next cycle it will be `projection=False`

> Should the default be [] (i.e. don't touch anything) instead of 'average' (re-reference to the average via a projection sometime later in the pipeline in some functions which the user probably isn't aware of)? Personally, I'd not re-reference to anything automatically and leave that to the user to do it explicitly.

I'd rather push people in the direction of `average` because it's what you need to get the fullest functionality, so I'm +1 for `'average'` default.

> I think it would be more consistent to set info['custom_ref_applied'] = True if we apply an average reference without projection. After all, this is just a regular re-referencing operation with all channels as reference channels.

Sure
+1 and +1 !

Ready to merge, let me know if everything is OK or if further changes are necessary.
@jaeilepp can you review and give your +1 before I look? thx
I guess syncing docstrings from a function to a method isn't that straightforward because (1) we don't document the `self` argument (which would be easy to solve), and (2) in this case we don't have the `copy` argument in the method. So for now I just copied & pasted from the function to the method, removed unused args and manually reformatted for PEP8 conformity (if anyone knows how to do this in PyCharm or an editor I'd be very happy to learn about it).
@cbrnr 
There is a function for that in mne.utils - `copy_function_doc_to_method_doc`. It is being used in the code in multiple places (see some plotting options of Raw or Epochs for example).
See [this line for an example](https://github.com/mne-tools/mne-python/blob/f6791a47f8e763bf5c35817fcf5a4c9448f9536f/mne/epochs.py#L864).
@mmagnuski yes, but this decorator only strips the first argument - there's no way to strip other args, and in this case I need to strip the `copy` argument.
> this decorator only strips the first argument - there's no way to strip other args

Eventually we should fix this -- if there is already a way to remove the first arg, it shouldn't be so terrible to optionally remove other ones, too. But probably in another PR unless you're feeling brave.
@Eric89GXL let's do this in another PR, this one is already too long 😅 
Also, can someone please restart the failed Travis jobs - the errors do not seem to be related to these changes?
done
Travis is having a hard time running all those jobs today, but now all tests passed!
Done. Although someone might want to take a look at the examples/tutorials at some point, because it seems that setting the average reference is not needed. For example, `tutorials/plot_epochs_to_data_frame.py` doesn't use EEG channels at all, and in almost all of the other plot tutorials/examples it is not necessary to set the average reference projection explicitly (because it is still done automatically when loading the data).
>  it is not necessary to set the average reference projection explicitly
(because it is still done automatically when loading the data).

when loading what data? I thought it was not done automatically for raw
files.

Really? Then I must have confused it with something else. When is the average projection added automatically then? Never?
to be checked but over the years I thought we had removed this magic
automatic addition in the IO code.

OK, I quickly loaded two files and nothing was added automatically. Maybe it is still mentioned in the docs, I'll check to see if I can find it. Otherwise, forget about my comment about the tutorials/examples (except for the first one, which sets the EEG reference but never uses any EEG channels). If nothing is added automatically - perfect!
Done.
Thanks @cbrnr
Regarding the [example page](https://martinos.org/mne/dev/auto_examples/preprocessing/plot_rereference_eeg.html#sphx-glr-auto-examples-preprocessing-plot-rereference-eeg-py), I am a bit confused. 

1. The `projection `parameter is set to `True` but `.apply_proj()` is not used. Consequently the projection's state is `False `in the `info `variable.
2. The second plot showing "EEG Average reference" does not differ from the first plot with "EEG original reference". Changing the projection's state to `True` using the `.apply_proj()` command does not change that.
3. Is it possible that the sample EEG data has already an average reference and therefore referencing against average reference a second time is redundant?

I'll take a look, but it seems like `projection` should be `False`.
This example data set already has an average reference projection after you load it. It seems like projections are automatically applied when creating `Epochs`, note the output:

```
72 matching events found
Created an SSP operator (subspace dimension = 1)
4 projection items activated
```

If I delete the existing projections right after loading the raw data the example works. Note that for the same reason I can add an average reference with `projection=True` (it will be applied when creating epochs). For consistency, I would still prefer to set `projection=False`.

Can anyone confirm if I'm correct? If so, I'd fix it in a PR.
Yep, something is broken now.

    raw.set_eeg_reference([])

should have removed the average reference projection.
for a bug fix 0.15.1 release?

@wmvanvliet but this code didn't change:

    if ref_channels == []:
        logger.info('EEG data marked as already having the desired reference. '
                    'Preventing automatic future re-referencing to an average '
                    'reference.')
        inst.info['custom_ref_applied'] = True
        return inst, None

What is supposed to happen instead?
Ah wait, can it be that the original raw FIFF file already contains the average reference projector? `raw.set_eeg_reference([])` only defends against future projectors being automatically added, not against projectors that have already been added (but may not have been applied yet).

If this is the case, the example needs an `del raw.info['projs'][-1]`
Yes, that's what I was saying - the file already contains the projection.

OK, so this was always broken - I'll quickly confirm this behavior with 0.14.1.

Is the average projection always the last projection?
However, since calling `raw.set_eeg_reference(['Cz'])` **will** remove the average reference projection, it may be neater if calling `raw.set_eeg_reference([])` would do the same.
> Is the average projection always the last projection?

Not as a rule. It is in the sample data.
Hm, in 0.14.1 the average projection gets removed automatically:

```
EEG data marked as already having the desired reference. Preventing automatic future re-referencing to an average reference.
Removing existing average EEG reference projection.
```
If I remember correctly, the programming logic of the function has changed a little since then. This is probably a bug.
Yes, I'm looking into it right now.
I keep forgetting what the function does and does not do, sorry :)

Resolving patch:
diff --git a/examples/preprocessing/plot_rereference_eeg.py b/examples/preprocessing/plot_rereference_eeg.py
--- a/examples/preprocessing/plot_rereference_eeg.py
+++ b/examples/preprocessing/plot_rereference_eeg.py
@@ -3,7 +3,7 @@
 Re-referencing the EEG signal
 =============================
 
-Load raw data and apply some EEG referencing schemes.
+This example shows how to load raw data and apply some EEG referencing schemes.
 """
 # Authors: Marijn van Vliet <w.m.vanvliet@gmail.com>
 #          Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
@@ -31,31 +31,37 @@
 picks = mne.pick_types(raw.info, meg=False, eeg=True, eog=True, exclude='bads')
 
 ###############################################################################
-# Apply different EEG referencing schemes and plot the resulting evokeds.
+# We will now apply different EEG referencing schemes and plot the resulting
+# evoked potentials. Note that when we construct epochs with ``mne.Epochs``, we
+# supply the ``proj=True`` argument. This means that any available projectors
+# are applied automatically. Specifically, if there is an average reference
+# projector set by ``raw.set_eeg_reference('average', projection=True)``, MNE
+# applies this projector when creating epochs.
 
 reject = dict(eog=150e-6)
 epochs_params = dict(events=events, event_id=event_id, tmin=tmin, tmax=tmax,
-                     picks=picks, reject=reject)
+                     picks=picks, reject=reject, proj=True)
 
 fig, (ax1, ax2, ax3) = plt.subplots(nrows=3, ncols=1, sharex=True)
 
 # No reference. This assumes that the EEG has already been referenced properly.
-# This explicitly prevents MNE from adding a default EEG reference.
+# This explicitly prevents MNE from adding a default EEG reference. Any average
+# reference projector is automatically removed.
 raw.set_eeg_reference([])
 evoked_no_ref = mne.Epochs(raw, **epochs_params).average()
 
-evoked_no_ref.plot(axes=ax1, titles=dict(eeg='EEG Original reference'))
+evoked_no_ref.plot(axes=ax1, titles=dict(eeg='Original reference'), show=False)
 
 # Average reference. This is normally added by default, but can also be added
 # explicitly.
 raw.set_eeg_reference('average', projection=True)
 evoked_car = mne.Epochs(raw, **epochs_params).average()
 
-evoked_car.plot(axes=ax2, titles=dict(eeg='EEG Average reference'))
+evoked_car.plot(axes=ax2, titles=dict(eeg='Average reference'), show=False)
 
 # Re-reference from an average reference to the mean of channels EEG 001 and
 # EEG 002.
 raw.set_eeg_reference(['EEG 001', 'EEG 002'])
 evoked_custom = mne.Epochs(raw, **epochs_params).average()
 
-evoked_custom.plot(axes=ax3, titles=dict(eeg='EEG Custom reference'))
+evoked_custom.plot(axes=ax3, titles=dict(eeg='Custom reference'))
diff --git a/mne/io/reference.py b/mne/io/reference.py
--- a/mne/io/reference.py
+++ b/mne/io/reference.py
@@ -361,6 +361,9 @@ def set_eeg_reference(inst, ref_channels='average', copy=True,
         raise ValueError('Setting a reference is only supported for instances '
                          'of Raw, Epochs or Evoked.')
 
+    if ref_channels is None:
+        ref_channels = 'average'
+
     if ref_channels != 'average' and projection:
         raise ValueError('Setting projection=True is only supported for '
                          'ref_channels="average".')
@@ -401,11 +404,10 @@ def set_eeg_reference(inst, ref_channels='average', copy=True,
         logger.info('EEG data marked as already having the desired reference. '
                     'Preventing automatic future re-referencing to an average '
                     'reference.')
-        inst.info['custom_ref_applied'] = True
-        return inst, None
     else:
         logger.info('Applying a custom EEG reference.')
-        return _apply_reference(inst, ref_channels)
+
+    return _apply_reference(inst, ref_channels)
 
 
 @verbose


Test patch:
diff --git a/mne/io/tests/test_reference.py b/mne/io/tests/test_reference.py
--- a/mne/io/tests/test_reference.py
+++ b/mne/io/tests/test_reference.py
@@ -197,9 +197,15 @@ def test_set_eeg_reference():
     assert_equal(reref.info['custom_ref_applied'], True)
 
     # Test that disabling the reference does not change anything
-    reref, ref_data = set_eeg_reference(raw, [])
+    reref, _ = set_eeg_reference(raw, [])
     assert_array_equal(raw._data, reref._data)
 
+    # make sure ref_channels=[] removes average reference projectors
+    reref, _ = set_eeg_reference(raw, 'average', projection=True)
+    assert_true(_has_eeg_average_ref_proj(reref.info['projs']))
+    reref, _ = set_eeg_reference(reref, [])
+    assert_true(not _has_eeg_average_ref_proj(reref.info['projs']))
+
     # Test that average reference gives identical results when calculated
     # via SSP projection (projection=True) or directly (projection=False)
     raw.info['projs'] = []