Repository: mne-tools/mne-python

Issue:
eeglab events backward compatibility
Currently when reading eeglab files I get deprecation warnings because the mne function internally uses `read_events_eeglab` and not the function to read events as annotations and then back as events. I am sure this is temporary, but it is annoying enough that I can volunteer to fix it. :)

And since I am volunteering - some other things regarding backwards compatibility of the new system with the new one:
* I was reading a file today that gave no events when read the old way (read then `mne.find_events`) but when read with the new way `egglab events -> annotations -> events array` gave one event (which is correct). It was not on my computer so I cannot check or share the file at the moment but I should be able in a day or so.
* I think the eeglab reader originally converted integer event types (`EEG.events.type`) to the same integer values in `events` array. Now, when turning annotations to events the events are numbered according to their presentation order and the correct values are given in a separate dict. That means that I have to correct the values myself (because otherwise every subject could have different event values) or pass `event_id={str(x): x for x in range(255)}` when constructing events from annotations.


Discussion:
Marking for 0.17 since this seems like something we need to fix for release. Thanks for volunteering @mmagnuski 
oups... yes we need to fix this.

you may want to read
https://github.com/mne-tools/mne-python/pull/5460/files too to see
where we want to go.

I looked through the PR you linked, I still have some questions more specific to reading eeglab files:
* do we read as digits those descriptions that are digit-only, or that contain exactly one stream of digits (so that `S02` would be turned to `2` but `A2S3` would be ignored ie. `len(re.findall('[0-9]+', event)) == 0`) @jona-sassenhagen should remember how it originally worked.
* previously, one could pass `event_id_fun`, not only `event_id` - do we still support that?
* previously events that could not be turned to digits via default `event_id_fun` or that were not in `event_id` were dropped, now they are given rather arbitrary consequent values. I think we all agree this should still work the same (throw a warning on dropping, but not assign arbitrary event values, because otherwise these would not be compatible across subjects).
>do we read as digits those descriptions that are digit-only, or that contain exactly one stream of digits (so that S02 would be turned to 2 but A2S3 would be ignored ie. len(re.findall('[0-9]+', event)) == 0) @jona-sassenhagen should remember how it originally worked.

I forgot, but intuitively, making it the same as the Brainvision reader looks appropriate, doesn't it?

>previously, one could pass event_id_fun, not only event_id - do we still support that?

Same:  make it as BV?

>previously events that could not be turned to digits via default event_id_fun or that were not in event_id were dropped, now they are given rather arbitrary consequent values. I think we all agree this should still work the same (throw a warning on dropping, but not assign arbitrary event values, because otherwise these would not be compatible across subjects).

I strongly disprefer the idea of assigning arbitrary values ...
regarding your second point above @mmagnuski (auto convertion of integer event types) it seems we changed the behavior with @massich without noticing. Can you provide a file with a test to demonstrate the regression?
@mmagnuski I still see event_id_func in the eeglab reader. Where did we remove it?

regarding the arbitrary numbers I agree. Please see if you can fix it and go back to the previous behavior

thanks
Regarding event_id_func I think it should not be part of the new function that converts annotations to events (just in whatever io reader function must have it for backward compatibility). I think annotations should be converted to events in this function if and only if they are in event_id.

Since the new plan is to construct complete annotations on read, then allow event creation separately (rather than synthesizing an event channel during read), I don't see an advantage or reason in the long run to have event_id_func in the API, and it's not as clean an interface because it breaks the iif-in-event_id logic.
> I forgot, but intuitively, making it the same as the Brainvision reader looks appropriate, doesn't it?

@jona-sassenhagen -the  BV reader expects padded values with at most 3 digits and some special characters before the digits, so it would not work for reading eeglab files. But I'll try to compare with current and past version of `read_eeglab_events`.

> it seems we changed the behavior with @massich without noticing.

@agramfort Not exactly, `read_eeglab_events` seems to work ok (but I'll take a look comparing current and previous version on some files), but it raises warning urging the user to read eeglab events as annotations and then as events. And if one does so the problems I mentioned arise. I'll test it more thoroughly tomorrow and will see how to swap `read_eeglab_events` to the new reading scheme without changing its functionality.
read_events_eeglab is deprecated. that's why you have deprecation warnings. so this is expected. thanks for taking a look @mmagnuski 

@larsoner if you want to try what you suggest with stim channel please do. 
@agramfort I know, my point was that `read_raw_eeglab` still uses it internally (https://github.com/mne-tools/mne-python/blob/master/mne/io/eeglab/eeglab.py#L361) and that was what I originally volounteered to change. :)
that's really bad :( :(

yes please send a PR so we don't use any deprecated code.

if you have some tiny bandwidth and you work a lot with eeglab files,
can you start
drafting the PR as I did for BrainVision files (#5460)

thanks a lot

@mmagnuski 	yes, right now all the readers use `read_events_xx` still. I think that the idea is to add `read_annotaions` + deprecate, and then swap the code.

Resolving patch:
diff --git a/mne/io/eeglab/eeglab.py b/mne/io/eeglab/eeglab.py
--- a/mne/io/eeglab/eeglab.py
+++ b/mne/io/eeglab/eeglab.py
@@ -7,6 +7,7 @@
 import os.path as op
 
 import numpy as np
+from functools import partial
 
 from ..utils import (_read_segments_file, _find_channels,
                      _synthesize_stim_channel)
@@ -19,7 +20,7 @@
 from ...epochs import BaseEpochs
 from ...event import read_events
 from ...externals.six import string_types
-from ...annotations import Annotations
+from ...annotations import Annotations, events_from_annotations
 
 # just fix the scaling for now, EEGLAB doesn't seem to provide this info
 CAL = 1e-6
@@ -358,9 +359,8 @@ def __init__(self, input_fname, montage, eog=(), event_id=None,
         info['chs'].append(stim_chan)
         info._update_redundant()
 
-        events = read_events_eeglab(eeg, event_id=event_id,
-                                    event_id_func=event_id_func)
-        self._create_event_ch(events, n_samples=eeg.pnts)
+        # dummy event channel to be populated from annotations later on
+        self._create_event_ch(np.empty((0, 3)), n_samples=eeg.pnts)
 
         # read the data
         if isinstance(eeg.data, string_types):
@@ -390,6 +390,43 @@ def __init__(self, input_fname, montage, eog=(), event_id=None,
                 info, data, filenames=[input_fname], last_samps=last_samps,
                 orig_format='double', verbose=verbose)
 
+        # create event_ch from annotations
+        annot = read_annotations_eeglab(input_fname)
+        self.set_annotations(annot)
+
+        _check_boundary(annot, event_id)
+
+        latencies = np.round(annot.onset * self.info['sfreq'])
+        _check_latencies(latencies)
+
+        dropped_desc = []  # use to collect dropped descriptions
+        event_id_ = partial(_event_id_func,
+                            event_id=event_id,
+                            event_id_func=event_id_func,
+                            dropped=dropped_desc)
+        events, _ = events_from_annotations(self, event_id=event_id_)
+        annot_length = self.annotations.onset.size
+        if events.shape[0] < annot_length:
+            msg = ("{0}/{1} event codes could not be mapped to integers. Use "
+                   "the 'event_id' parameter to map such events manually.")
+            warn(msg.format(annot_length - events.shape[0], annot_length))
+        if not events.size and len(annot):  # only if some events were in file
+            logger.info('Returning empty stim channel. Some annotations were'
+                        'found but dropped during build of the raw.'
+                        'Please use `event_id` and `event_id_func` to drive'
+                        'the selection/rejection of events')
+        self._create_event_ch(events, n_samples=eeg.pnts)
+        if getattr(self, 'preload', False):
+            self._data[-1] = self._event_ch
+
+        if len(dropped_desc) > 0:
+            dropped = list(set(dropped_desc))
+            logger.info("{0} annotation(s) will be dropped, such as {1}. "
+                        .format(len(dropped), dropped[:5]))
+            warn('Events like the following will be dropped entirely: {1},'
+                 ' {0} in total'.format(len(dropped), dropped[:5]),
+                 RuntimeWarning)
+
     def _create_event_ch(self, events, n_samples=None):
         """Create the event channel."""
         n_dropped = len(events[:, 0]) - len(set(events[:, 0]))
@@ -596,6 +633,28 @@ def __init__(self, input_fname, events=None, event_id=None, tmin=0,
         logger.info('Ready.')
 
 
+def _check_boundary(annot, event_id):
+    if event_id is None:
+        event_id = dict()
+    if "boundary" in annot.description and "boundary" not in event_id:
+        warn("The data contains 'boundary' events, indicating data "
+             "discontinuities. Be cautious of filtering and epoching around "
+             "these events.")
+
+
+def _check_latencies(latencies):
+    if (latencies < -1).any():
+        raise ValueError('At least one event sample index is negative. Please'
+                         ' check if EEG.event.sample values are correct.')
+    if (latencies == -1).any():
+        warn("At least one event has a sample index of -1. This usually is "
+             "a consequence of how eeglab handles event latency after "
+             "resampling - especially when you had a boundary event at the "
+             "beginning of the file. Please make sure that the events at "
+             "the very beginning of your EEGLAB file can be safely dropped "
+             "(e.g., because they are boundary events).")
+
+
 @deprecated('read_events_eeglab is deprecated from 0.17 and will be removed'
             ' in 0.18. Please use read_annotations_eeglab and create events'
             ' using events_from_annotations.')
@@ -672,25 +731,14 @@ def read_events_eeglab(eeg, event_id=None, event_id_func='strip_to_integer',
     types = annotations.description
     latencies = annotations.onset * eeg.srate
 
-    if "boundary" in types and "boundary" not in event_id:
-        warn("The data contains 'boundary' events, indicating data "
-             "discontinuities. Be cautious of filtering and epoching around "
-             "these events.")
+    _check_boundary(annotations, event_id)
 
     if len(types) < 1:  # if there are 0 events, we can exit here
         logger.info('No events found, returning empty stim channel ...')
-        return np.zeros((0, 3))
+        return np.zeros((0, 3), dtype=int)
+
+    _check_latencies(latencies)
 
-    if (latencies < -1).any():
-        raise ValueError('At least one event sample index is negative. Please'
-                         ' check if EEG.event.sample values are correct.')
-    if (latencies == -1).any():
-        warn("At least one event has a sample index of -1. This usually is "
-             "a consequence of how eeglab handles event latency after "
-             "resampling - especially when you had a boundary event at the "
-             "beginning of the file. Please make sure that the events at "
-             "the very beginning of your EEGLAB file can be safely dropped "
-             "(e.g., because they are boundary events).")
     not_in_event_id = set(x for x in types if x not in event_id)
     not_purely_numeric = set(x for x in not_in_event_id if not x.isdigit())
     no_numbers = set([x for x in not_purely_numeric
@@ -726,7 +774,7 @@ def read_events_eeglab(eeg, event_id=None, event_id_func='strip_to_integer',
         warn(msg.format(missing, len(types)))
         if len(events) < 1:
             warn("As is, the trigger channel will consist entirely of zeros.")
-            return np.zeros((0, 3))
+            return np.zeros((0, 3), dtype=int)
 
     return np.asarray(events)
 
@@ -812,3 +860,18 @@ def read_annotations_eeglab(fname, uint16_codec=None):
 def _strip_to_integer(trigger):
     """Return only the integer part of a string."""
     return int("".join([x for x in trigger if x.isdigit()]))
+
+
+def _event_id_func(trigger, event_id, event_id_func, dropped):
+    """Mimic old behavior to be used with events_from_annotations."""
+    if event_id is not None and trigger in event_id:
+        return event_id[trigger]
+    if event_id_func == 'strip_to_integer':
+        trigger_new = "".join([x for x in trigger if x.isdigit()])
+        if trigger_new.isdigit():
+            return int(trigger_new)
+        else:
+            dropped.append(trigger)
+            return None
+    elif event_id_func is not None:
+        return event_id_func(trigger)


Test patch:
diff --git a/mne/io/eeglab/tests/test_eeglab.py b/mne/io/eeglab/tests/test_eeglab.py
--- a/mne/io/eeglab/tests/test_eeglab.py
+++ b/mne/io/eeglab/tests/test_eeglab.py
@@ -66,7 +66,7 @@ def test_io_set_raw(fnames, tmpdir):
                          montage=montage)
         _test_raw_reader(read_raw_eeglab, input_fname=raw_fname_onefile,
                          montage=montage)
-    for want in ('Events like', 'consist entirely', 'could not be mapped',
+    for want in ('Events like', 'could not be mapped',
                  'string preload is not supported'):
         assert (any(want in str(ww.message) for ww in w))
 
@@ -86,7 +86,9 @@ def test_io_set_raw(fnames, tmpdir):
         assert raw0.filenames[0].endswith('.fdt')  # .set with additional .fdt
         assert raw2.filenames[0].endswith('.set')  # standalone .set
 
-        Epochs(raw0, find_events(raw0), event_id)
+        raw0_events = find_events(raw0)
+        assert raw0_events.size
+        Epochs(raw0, raw0_events, event_id)
         epochs = Epochs(raw1, find_events(raw1), event_id)
 
         assert len(find_events(raw4)) == 0  # no events without event_id
@@ -98,16 +100,14 @@ def test_io_set_raw(fnames, tmpdir):
         raw0.filter(1, None, l_trans_bandwidth='auto', filter_length='auto',
                     phase='zero')  # test that preloading works
 
-    filter_out_warnings(w, category=DeprecationWarning)
     filter_out_warnings(w, category=FutureWarning)
     filter_out_warnings(w, category=ImportWarning)
-    assert len(w) == 4  # check `preload=False` raises RuntimeWarning
+    assert len(w) == 3  # check `preload=False` raises RuntimeWarning
 
     # test that using uint16_codec does not break stuff
-    with pytest.warns(DeprecationWarning, match="read_events_eeglab"):
-        raw0 = read_raw_eeglab(input_fname=raw_fname, montage=montage,
-                               event_id=event_id, preload=False,
-                               uint16_codec='ascii')
+    raw0 = read_raw_eeglab(input_fname=raw_fname, montage=montage,
+                           event_id=event_id, preload=False,
+                           uint16_codec='ascii')
 
     # test old EEGLAB version event import (read old version)
     eeg = io.loadmat(raw_fname_mat, struct_as_record=False,
@@ -134,10 +134,9 @@ def test_io_set_raw(fnames, tmpdir):
     shutil.copyfile(op.join(base_dir, 'test_raw.fdt'),
                     one_event_fname.replace('.set', '.fdt'))
     event_id = {eeg.event[0].type: 1}
-    with pytest.warns(DeprecationWarning, match="read_events_eeglab"):
-        test_raw = read_raw_eeglab(input_fname=one_event_fname,
-                                   montage=montage, event_id=event_id,
-                                   preload=True)
+    test_raw = read_raw_eeglab(input_fname=one_event_fname,
+                               montage=montage, event_id=event_id,
+                               preload=True)
 
     # test that sample indices are read python-wise (zero-based)
     assert find_events(test_raw)[0, 0] == round(eeg.event[0].latency) - 1
@@ -148,7 +147,7 @@ def test_io_set_raw(fnames, tmpdir):
     evnts.latency = 0
     io.savemat(negative_latency_fname, {'EEG':
                {'trials': eeg.trials, 'srate': eeg.srate,
-                'nbchan': eeg.nbchan, 'data': 'test_one_event.fdt',
+                'nbchan': eeg.nbchan, 'data': 'test_negative_latency.fdt',
                 'epoch': eeg.epoch, 'event': evnts,
                 'chanlocs': eeg.chanlocs, 'pnts': eeg.pnts}},
                appendmat=False, oned_as='row')
@@ -158,6 +157,17 @@ def test_io_set_raw(fnames, tmpdir):
     with pytest.warns(RuntimeWarning, match="has a sample index of -1."):
         read_raw_eeglab(input_fname=negative_latency_fname, preload=True,
                         event_id=event_id, montage=montage)
+    evnts.latency = -1
+    io.savemat(negative_latency_fname, {'EEG':
+               {'trials': eeg.trials, 'srate': eeg.srate,
+                'nbchan': eeg.nbchan, 'data': 'test_negative_latency.fdt',
+                'epoch': eeg.epoch, 'event': evnts,
+                'chanlocs': eeg.chanlocs, 'pnts': eeg.pnts}},
+               appendmat=False, oned_as='row')
+    with pytest.raises(ValueError, match='event sample index is negative'):
+        with pytest.warns(RuntimeWarning, match="has a sample index of -1."):
+            read_raw_eeglab(input_fname=negative_latency_fname, preload=True,
+                            event_id=event_id, montage=montage)
 
     # test overlapping events
     overlap_fname = op.join(tmpdir, 'test_overlap_event.set')
@@ -170,10 +180,12 @@ def test_io_set_raw(fnames, tmpdir):
     shutil.copyfile(op.join(base_dir, 'test_raw.fdt'),
                     overlap_fname.replace('.set', '.fdt'))
     event_id = {'rt': 1, 'square': 2}
+
     with pytest.warns(RuntimeWarning, match='will be dropped'):
         raw = read_raw_eeglab(input_fname=overlap_fname,
                               montage=montage, event_id=event_id,
                               preload=True)
+
     events_stimchan = find_events(raw)
     with pytest.warns(DeprecationWarning, match="read_events_eeglab"):
         events_read_events_eeglab = read_events_eeglab(overlap_fname, event_id)
@@ -193,7 +205,6 @@ def test_io_set_raw(fnames, tmpdir):
     with pytest.warns(None) as w:
         read_raw_eeglab(input_fname=one_chan_fname, preload=True)
     # no warning for 'no events found'
-    w.pop(DeprecationWarning)
     assert len(w) == 0
 
     # test reading file with 3 channels - one without position information
@@ -254,8 +265,8 @@ def test_io_set_raw(fnames, tmpdir):
                 'times': eeg.times[:2], 'pnts': 2}},
                appendmat=False, oned_as='row')
     # load the file
-    with pytest.warns(DeprecationWarning, match="read_events_eeglab"):
-        raw = read_raw_eeglab(input_fname=nopos_fname, preload=True)
+    raw = read_raw_eeglab(input_fname=nopos_fname, preload=True)
+
     # test that channel names have been loaded but not channel positions
     for i in range(3):
         assert_equal(raw.info['chs'][i]['ch_name'], ch_names[i])
@@ -357,7 +368,7 @@ def test_eeglab_event_from_annot(recwarn):
     events_a = find_events(raw1)
     events_b = read_events_eeglab(raw_fname, event_id=event_id)
     annotations = read_annotations_eeglab(raw_fname)
-    assert len(raw1.annotations) == 0
+    assert len(raw1.annotations) == 154
     raw1.set_annotations(annotations)
     events_c, _ = events_from_annotations(raw1, event_id=event_id)