Repository: mne-tools/mne-python

Issue:
Expose "add_offset" control in `concatenate_epochs`
https://github.com/mne-tools/mne-python/blob/5cafa0f4287ae02fb2beafa557c0f70db8b7a1ed/mne/epochs.py#L2916

I find myself frequently using `concatenate_epochs` function to unite several sets of epochs for different events, and then sorting the epochs afterwards. This proves a great pattern for working with a database-like structure for machine learning applications.

However, the function by default gives an offset to the sets of events, which means that the resulting event timestamps lose their valuable information. Examining the function I have found that it uses `_concatenate_epochs` where you can disable giving an offset to the events.

I was wondering if there was a specific reason to not expose this control in the public function, and if there wasn't, would it be ok for me to submit a pull request of the correction? Thanks.


Discussion:
you just want to expose add_offset param in concatenate_epochs right?

I am curious how you can still make sense of events and time stamps when
merging epochs from different runs/raw files.

> you just want to expose add_offset param in concatenate_epochs right?

Exactly that.

>I am curious how you can still make sense of events and time stamps when
merging epochs from different runs/raw files.

Let me try to describe, perhaps it is an interesting use case. 

I have a large EEG dataset collected in an ecological task -- driving in a simulator. I have built a system to create sets of events, sometimes overlapping, sometimes not, for example, all brakes, or all accelerations, or only those brakes which happen before a turn, etc. There are a lot of sets like that, and new ones can be added. They all happen within the same long session (at least if you're talking about 1 subject), but when I do ML on them, I just want to select a few sets. I've built a query system which gets the data around a selected set of events (a tensor n_epochs x n_channels x time) and puts it into the `mne.EpochArray` [1]. 

This way I collect epochs from different queries and concatenate them, after which I sort them chronologically. But I have to have the unaltered timestamps for each of them after I concatenate [2]. 

So in the end I can make sense of them, as they are all on the same recording.

[1] After trying at first to create epochs from a raw file using mne segmentation, I found that for what I am trying to do, it turns out more straightforward and efficient, and gives me more control.
[2] I could work around it and get sorting order form a set of queries, but it would make more sense to use `Epoch.events`.
I think I get it but I would have do it this way:
- concatenate raws to have a single raw object
- make events and associated metadata
- make one big epochs (eventually not preloaded)
- use query mechanism of epochs to use metadata.

See here for a demo of using metadata:

https://martinos.org/mne/stable/auto_tutorials/plot_metadata_epochs.html

That's an interesting workflow to consider, thank you. Although, wouldn't this step
>- concatenate raws to have a single raw object

require me to fit all of the raw recordings in RAM at the same time? It wouldn't be possible in my case, as I have over 100 GB of raw files. Getting epochs separately allows me to never load raw data at all.
maybe it will work with preload=False ?

It could work with some types of data stores, which `mne` can query in an out-of-memory mode, but I have a database, and it doesn't seem feasible.

So in the end, do you think "add_offset" would be a useful control to expose, or you think it may encourage a wrong workflow?
+0.5 to expose add_offset if it's needed for you. It's not too invasive
change.

@antopolskiy can you take a stab at exposing the param for your use case?
Sorry, caught up in stuff, doing it now

Resolving patch:
diff --git a/mne/epochs.py b/mne/epochs.py
--- a/mne/epochs.py
+++ b/mne/epochs.py
@@ -3016,13 +3016,18 @@ def _finish_concat(info, data, events, event_id, tmin, tmax, metadata,
     return out
 
 
-def concatenate_epochs(epochs_list):
+def concatenate_epochs(epochs_list, add_offset=True):
     """Concatenate a list of epochs into one epochs object.
 
     Parameters
     ----------
     epochs_list : list
         list of Epochs instances to concatenate (in order).
+    add_offset : bool
+        If True, a fixed offset is added to the event times from different
+        Epochs sets, such that they are easy to distinguish after the
+        concatenation.
+        If False, the event times are unaltered during the concatenation.
 
     Returns
     -------
@@ -3033,7 +3038,8 @@ def concatenate_epochs(epochs_list):
     -----
     .. versionadded:: 0.9.0
     """
-    return _finish_concat(*_concatenate_epochs(epochs_list))
+    return _finish_concat(*_concatenate_epochs(epochs_list,
+                                               add_offset=add_offset))
 
 
 @verbose


Test patch: