Repository: mne-tools/mne-python

Issue:
stim_channel="auto" in read_raw_edf renames stim channel
The new default for `mne.io.read_raw_edf` in MNE 0.16 will be `stim_channel="auto"`. Using this argument, the stim channel is called `Status` instead of the usual `STI 014`. This causes `mne.find_events` to fail because by default it looks for a channel called `STI 014`. Is this intended?
New stim_channel='auto' in read_raw_edf fails in combination with montage
I have an EDF file with a stim channel at the end. Previously, `stim_channel` would default to `-1` and the following command worked (the relevant bit is setting the montage:

```
raw = mne.io.read_raw_edf("vp1_exp_eeg.bdf",
                          exclude=["EXG" + str(i) for i in range(1, 9)],
                          montage="biosemi64", preload=True)
```

However, the new default `stim_channel='auto'` results in the following error:

```
/Users/clemens/Library/Preferences/PyCharmCE2017.2/scratches/scratch.py:6: RuntimeWarning: The following EEG sensors did not have a position specified in the selected montage: ['Status']. Their position has been left untouched.
  montage="biosemi64", preload=True)
Traceback (most recent call last):
  File "/Users/clemens/Library/Preferences/PyCharmCE2017.2/scratches/scratch.py", line 6, in <module>
    montage="biosemi64", preload=True)
  File "/Users/clemens/Repositories/mne-python/mne/io/edf/edf.py", line 1231, in read_raw_edf
    exclude=exclude, preload=preload, verbose=verbose)
  File "<string>", line 2, in __init__
  File "/Users/clemens/Repositories/mne-python/mne/utils.py", line 728, in verbose
    return function(*args, **kwargs)
  File "/Users/clemens/Repositories/mne-python/mne/io/edf/edf.py", line 160, in __init__
    _check_update_montage(info, montage)
  File "/Users/clemens/Repositories/mne-python/mne/io/base.py", line 2495, in _check_update_montage
    % str(missing_positions))
KeyError: "The following positions are missing from the montage definitions: ['Status']. If those channels lack positions because they are EOG channels use the eog parameter."
The following EEG sensors did not have a position specified in the selected montage: ['Status']. Their position has been left untouched.
```


Discussion:
Not only that, but when I use `stim_channel="auto"` the resulting stim channel in the data contains negative values whereas it looks normal with the previous default of taking the last channel.
there is maybe a bug but you can can set a default name for your stim
channels with the set_config

> This causes mne.find_events to fail because by default it looks for a channel called STI 014. Is this intended?

Different systems sometime have different names. FWIW the default for the last number of years (10 maybe?) for Neuromag has even been STI101, so all newer systems use this. We usually use STI 014 when we synthesize a new events channel only. So this part is okay I think.

> when I use stim_channel="auto" the resulting stim channel in the data contains negative values whereas it looks normal with the previous default of taking the last channel.

This sounds like a bug
Using the exact same file, the stim channel is called `STI 014` if `stim_channel=True` (the current default) and `Status` if `stim_channel="auto"` (the default in 0.16). I'm currently trying to find a public EDF file with a stim channel to reproduce this behavior.
The problem seems to occur in BDF files and not in EDF files. To reproduce, download these [BDF test files](https://www.biosemi.com/download/BDFtestfiles.zip) (unzip) and run:

```
print(mne.io.read_raw_edf("BDFtestfiles/Newtest17-256.bdf", preload=True, stim_channel=True).info["ch_names"][-1])
print(mne.io.read_raw_edf("BDFtestfiles/Newtest17-256.bdf", preload=True, stim_channel="auto").info["ch_names"][-1])
```
sorry no time to look.

@larsoner do you have time to confirm?
now on current master I get :

```
~/work/src/mne-python/mne/io/edf/edf.py in _get_info(fname, stim_channel, annot, annotmap, eog, misc, exclude, preload)
    562     # These are the conditions under which a stim channel will be interpolated
    563     if stim_channel is not None and not (annot and annotmap) and \
--> 564             tal_channel is None and n_samps[stim_channel] != int(max_samp):
    565         warn('Interpolating stim channel. Events may jitter.')
    566     info._update_redundant()

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

:( 
I think we need somebody with EDF expertise/knowledge to tackle this one. @cbrnr do you know enough to have an idea of what the right thing would be to do in each of these situations? If so, maybe a table or bulleted list would be good to guide fixes and also add to the `Notes` docstring.
No, I'm not familiar enough with the internal workings of EDF to do this quickly. I assume @jaeilepp is not available anymore, right? In the worst case, we could revert his changes regarding the stim channel.
> In the worst case, we could revert his changes regarding the stim channel.

This seems like a reasonable choice to me, but it's complicated by the changes already existing in 0.15 (and it might work properly for some users in `'auto'` mode, and we would cause them pain with removal). So it's probably better to fix it rather than remove it.
Cf #4750
The last channel in the test file I've linked above is the stim channel. Since the previous default was `-1`, this channel was correctly detected as a stim channel. Now the new default is `'auto'`, which results in the last channel being interpreted as an EEG channel (and hence the weird values).

What exactly does `stim_channel='auto'` do?
To answer my question, maybe `stim_channel='auto'` should behave differently. Currently, it adds a stim channel if EDF annotations are present. If not, no stim channel is added. Maybe it should add the last channel in the latter case instead. Thoughts?
It's a mess. I'm discovering weird behavior as I go. I thought that `stim_channel=None` wouldn't add a stim channel. Consider [this example EDF file](https://www.physionet.org/pn4/eegmmidb/S001/S001R04.edf). If I load it with `stim_channel=None`, an additional MISC channel is still appended, even though this file does not have such a stim channel (events are stored as EDF annotations).
It sounds like there are now several problematic files. If you have time, making a bulleted list (one per file) at the top of one of these issues that contains file link + existing behavior + what it should do would go a long way to setting up for a proper fix.
#4760
I think the backport label can be removed, because nothing is really broken (now that I understand the internals of EDF better), but only less ideal from a usability perspective.
cool. can you document this somewhere ? and maybe suggest API improvements?

Yeah, it's all in #4760.
@agramfort @larsoner slightly OT, when is 0.15.1 planned to be released? The viz bug in raw.plot is pretty serious and 0.15.1 fixes it. This issue is for 0.16 because everything (EDF-related stuff) is still working in 0.15.
just pushed 0.15.1 https://pypi.python.org/pypi/mne

Thanks @agramfort! So far `pip` isn't seeing the new version, but I'm hoping this won't take too long.


Resolving patch:
diff --git a/mne/io/edf/edf.py b/mne/io/edf/edf.py
--- a/mne/io/edf/edf.py
+++ b/mne/io/edf/edf.py
@@ -425,10 +425,6 @@ def _get_info(fname, stim_channel, annot, annotmap, eog, misc, exclude,
         cals[idx] = 1
     if 'stim_data' in edf_info and stim_channel == 'auto':  # For GDF events.
         cals = np.append(cals, 1)
-    # Check that stimulus channel exists in dataset
-    if not ('stim_data' in edf_info or
-            'EDF Annotations' in ch_names) and stim_channel == 'auto':
-        stim_channel = None
     if stim_channel is not None:
         stim_channel = _check_stim_channel(stim_channel, ch_names, include)
 
@@ -499,7 +495,7 @@ def _get_info(fname, stim_channel, annot, annotmap, eog, misc, exclude,
             chan_info['cal'] = 1
             chan_info['coil_type'] = FIFF.FIFFV_COIL_NONE
             chan_info['unit'] = FIFF.FIFF_UNIT_NONE
-            chan_info['kind'] = FIFF.FIFFV_MISC_CH
+            chan_info['kind'] = FIFF.FIFFV_STIM_CH
             pick_mask[idx] = False
         chs.append(chan_info)
     edf_info['stim_channel'] = stim_channel


Test patch:
diff --git a/mne/io/edf/tests/test_edf.py b/mne/io/edf/tests/test_edf.py
--- a/mne/io/edf/tests/test_edf.py
+++ b/mne/io/edf/tests/test_edf.py
@@ -24,6 +24,7 @@
 from mne.utils import run_tests_if_main, requires_pandas, _TempDir
 from mne.io import read_raw_edf
 from mne.io.tests.test_raw import _test_raw_reader
+from mne.io.pick import channel_type
 from mne.io.edf.edf import _parse_tal_channel, find_edf_events
 from mne.event import find_events
 
@@ -74,6 +75,12 @@ def test_bdf_data():
     assert_true((raw_py.info['chs'][63]['loc']).any())
 
 
+def test_bdf_stim_channel():
+    """Test if last channel is detected as STIM by default."""
+    raw_py = _test_raw_reader(read_raw_edf, input_fname=bdf_path)
+    assert_true(channel_type(raw_py.info, raw_py.info["nchan"] - 1) == 'stim')
+
+
 @testing.requires_testing_data
 def test_edf_overlapping_annotations():
     """Test EDF with overlapping annotations."""
@@ -244,6 +251,10 @@ def test_edf_annotations():
 
 def test_edf_stim_channel():
     """Test stim channel for edf file."""
+    # test if stim channel is automatically detected
+    raw = read_raw_edf(edf_path, preload=True)
+    assert_true(channel_type(raw.info, raw.info["nchan"] - 1) == 'stim')
+
     raw = read_raw_edf(edf_stim_channel_path, preload=True,
                        stim_channel=-1)
     true_data = np.loadtxt(edf_txt_stim_channel_path).T