Repository: mne-tools/mne-bids

Issue:
We're not writing all annotations if events are passed to write_raw_bids() as well
### Description of the problem

This problem came up in https://mne.discourse.group/t/mne-bids-structure-one-subject-is-different/5687/

The user concatenated raws, which creates an `edge` annotation. However, after writing the data via `write_raw_bids()`, the annotation didn't show up in `events.tsv`. Turns out the user passed `events` and `event_id` to `write_raw_bids()` too, which caused this issue to surface.

### Steps to reproduce

In the following code, the concatenated raw has an edge and a bad annotation, but they don't end up in the `events.tsv` sidecar:

```Python
# %%
import pandas as pd
import mne
import mne_bids

sample_dir = mne.datasets.sample.data_path()
sample_fname = sample_dir / 'MEG' / 'sample' / 'sample_audvis_raw.fif'

raw = mne.io.read_raw_fif(sample_fname)
raw_concat = mne.concatenate_raws([raw.copy(), raw])
events = mne.find_events(raw_concat)
event_id = {
    'Auditory/Left': 1,
    'Auditory/Right': 2,
    'Visual/Left': 3,
    'Visual/Right': 4,
    'Smiley': 5,
    'Button': 32
}

# %%
bp = mne_bids.BIDSPath(
    subject='sample',
    task='sample',
    suffix='meg',
    extension='.fif',
    datatype='meg',
    root='/tmp/bids-root'
)
mne_bids.write_raw_bids(
    raw_concat,
    bids_path=bp,
    events=events,
    event_id=event_id,
    overwrite=True,
    allow_preload=True,
    format='FIF'
)

# %%
events_tsv_path = (
    bp
    .copy()
    .update(suffix='events', extension='.tsv')
)
events_tsv = pd.read_csv(events_tsv_path, sep='\t')
print(events_tsv['trial_type'].unique())
```


### Expected results

We should see 642 events in `events.tsv`, including the edge and bad annotations.

### Actual results

The edge and bad annotations are missing from the `events.tsv` sidecar.

### Additional information

The problem occurs here:
https://github.com/mne-tools/mne-bids/blob/9231836b227dceecec5f73b498a32591ef2f31ef/mne_bids/read.py#L159-L163

While in line 162 we explicitly try to include edge and bad annotations, the documentation of `events_from_annotations()` also states that any annotations that do not appear in `event_id` will simply be dropped.

IMHO the cleanest solution here would be to force users to always supply an event_id dictionary. Currently it's optional if no `events` are passed, which causes us to generate arbitrary event codes for annotations – which at least one user complained about not too long ago. Of course this would break lots of existing code.

We could of course also add special handling for all annotations for which no event_id mapping has been supplied, and we simply generate some new (arbitrary) event codes.

@sappelhoff @agramfort WDYT?


Discussion:
Another option would be to keep the behavior as-is if only annotations are present (and no `events` and `event_id` are passed), and we **only** enforce all annotations to be listed in `event_id` if `events` **and** annoations are passed.

This should break significantly less existing user code!!
Just for clarification, simply not passing `events` and `event_id` to `write_raw_bids()` **does** write the annotations. It's just the combination of the two that causes issues, as annotations whose description doesn't appear in `event_id` will not be written.
we have discussed this before and we had decided that we should write
anything that is present in the files.
so to me it's a bug fix

> Message ID: ***@***.***>
>

Yes but the question is simply **how** to fix it! It's clear that it's a bug :)
Thanks for investigating, Richard!

> We could of course also add special handling for all annotations for which no event_id mapping has been supplied, and we simply generate some new (arbitrary) event codes.

I like this fix:

- keep current behavior BUT
- generate new (arbitrary) event codes for annots that have no `event_id` mapping AND
- warn if we do that, that users please use complete their `event_id` instead of relying on the auto-procedure

> Another option would be to keep the behavior as-is if only annotations are present (and no events and event_id are passed), and we only enforce all annotations to be listed in event_id if events and annoations are passed.

I would also be fine with this fix.
> keep current behavior BUT
> generate new (arbitrary) event codes for annots that have no event_id mapping AND
> warn if we do that, that users please use complete their event_id instead of relying on the auto-procedure

I'm in favor of this one! This will make things more explicit to users 
I can try to take a look sometime later today. I think this is actually an important issue and warrants a quick patch release, as users might actually be losing data in the BIDS conversion process!
Just for the procedure of a patch release we would:

1. `git cherry-pick` commits from `main` that we want to release and put them on `maint/v0.11`
2. bump version, update changelog, etc. on `maint/v0.11`
3. build dist and push to pypi on `maint/v0.11`
4. build docs on `v0.11` and update stable docs in `gh-pages` branch

right?

Resolving patch:
diff --git a/mne_bids/read.py b/mne_bids/read.py
--- a/mne_bids/read.py
+++ b/mne_bids/read.py
@@ -124,8 +124,29 @@ def _read_events(events, event_id, raw, bids_path=None):
     else:
         events = read_events(events).astype(int)
 
+    if raw.annotations:
+        if event_id is None:
+            logger.info(
+                'The provided raw data contains annotations, but you did not '
+                'pass an "event_id" mapping from annotation descriptions to '
+                'event codes. We will generate arbitrary event codes. '
+                'To specify custom event codes, please pass "event_id".'
+            )
+        else:
+            desc_without_id = sorted(
+                set(raw.annotations.description) - set(event_id.keys())
+            )
+            if desc_without_id:
+                raise ValueError(
+                    f'The provided raw data contains annotations, but '
+                    f'"event_id" does not contain entries for all annotation '
+                    f'descriptions. The following entries are missing: '
+                    f'{", ".join(desc_without_id)}'
+                )
+
+    # If we have events, convert them to Annotations so they can be easily
+    # merged with existing Annotations.
     if events.size > 0:
-        # Only keep events for which we have an ID <> description mapping.
         ids_without_desc = set(events[:, 2]) - set(event_id.values())
         if ids_without_desc:
             raise ValueError(
@@ -134,9 +155,6 @@ def _read_events(events, event_id, raw, bids_path=None):
                 f'Please add them to the event_id dictionary, or drop them '
                 f'from the events array.'
             )
-        del ids_without_desc
-        mask = [e in list(event_id.values()) for e in events[:, 2]]
-        events = events[mask]
 
         # Append events to raw.annotations. All event onsets are relative to
         # measurement beginning.
diff --git a/mne_bids/write.py b/mne_bids/write.py
--- a/mne_bids/write.py
+++ b/mne_bids/write.py
@@ -1337,14 +1337,16 @@ def write_raw_bids(
         If an array, the MNE events array (shape: ``(n_events, 3)``).
         If a path or an array and ``raw.annotations`` exist, the union of
         ``events`` and ``raw.annotations`` will be written.
-        Corresponding descriptions for all event codes (listed in the third
+        Mappings from event names to event codes (listed in the third
         column of the MNE events array) must be specified via the ``event_id``
-        parameter; otherwise, an exception is raised.
+        parameter; otherwise, an exception is raised. If
+        :class:`~mne.Annotations` are present, their descriptions must be
+        included in ``event_id`` as well.
         If ``None``, events will only be inferred from the raw object's
         :class:`~mne.Annotations`.
 
         .. note::
-           If ``not None``, writes the union of ``events`` and
+           If specified, writes the union of ``events`` and
            ``raw.annotations``. If you wish to **only** write
            ``raw.annotations``, pass ``events=None``. If you want to
            **exclude** the events in ``raw.annotations`` from being written,
@@ -1359,7 +1361,10 @@ def write_raw_bids(
         ``events``. The descriptions will be written to the ``trial_type``
         column in ``*_events.tsv``. The dictionary keys correspond to the event
         description,s and the values to the event codes. You must specify a
-        description for all event codes appearing in ``events``.
+        description for all event codes appearing in ``events``. If your data
+        contains :class:`~mne.Annotations`, you can use this parameter to
+        assign event codes to each unique annotation description (mapping from
+        description to event code).
     anonymize : dict | None
         If `None` (default), no anonymization is performed.
         If a dictionary, data will be anonymized depending on the dictionary
@@ -1575,11 +1580,7 @@ def write_raw_bids(
 
     if events is not None and event_id is None:
         raise ValueError('You passed events, but no event_id '
-                         'dictionary. You need to pass both, or neither.')
-
-    if event_id is not None and events is None:
-        raise ValueError('You passed event_id, but no events. '
-                         'You need to pass both, or neither.')
+                         'dictionary.')
 
     _validate_type(item=empty_room, item_name='empty_room',
                    types=(mne.io.BaseRaw, BIDSPath, None))


Test patch:
diff --git a/mne_bids/tests/test_write.py b/mne_bids/tests/test_write.py
--- a/mne_bids/tests/test_write.py
+++ b/mne_bids/tests/test_write.py
@@ -1272,17 +1272,10 @@ def test_eegieeg(dir_name, fname, reader, _bids_validate, tmp_path):
                           match='Encountered data in "double" format'):
             bids_output_path = write_raw_bids(**kwargs)
 
-    event_id = {'Auditory/Left': 1, 'Auditory/Right': 2, 'Visual/Left': 3,
-                'Visual/Right': 4, 'Smiley': 5, 'Button': 32}
-
     with pytest.raises(ValueError,
                        match='You passed events, but no event_id '):
         write_raw_bids(raw, bids_path, events=events)
 
-    with pytest.raises(ValueError,
-                       match='You passed event_id, but no events'):
-        write_raw_bids(raw, bids_path, event_id=event_id)
-
     # check events.tsv is written
     events_tsv_fname = bids_output_path.copy().update(suffix='events',
                                                       extension='.tsv')
@@ -2704,6 +2697,81 @@ def test_annotations(_bids_validate, bad_segments, tmp_path):
     _bids_validate(bids_root)
 
 
+@pytest.mark.parametrize(
+    'write_events', [True, False]  # whether to pass "events" to write_raw_bids
+)
+@pytest.mark.filterwarnings(warning_str['channel_unit_changed'])
+@testing.requires_testing_data
+def test_annotations_and_events(_bids_validate, tmp_path, write_events):
+    """Test combined writing of Annotations and events."""
+    bids_root = tmp_path / 'bids'
+    bids_path = _bids_path.copy().update(root=bids_root, datatype='meg')
+    raw_fname = data_path / 'MEG' / 'sample' / 'sample_audvis_trunc_raw.fif'
+    events_fname = (
+        data_path / 'MEG' / 'sample' / 'sample_audvis_trunc_raw-eve.fif'
+    )
+    events_tsv_fname = bids_path.copy().update(
+        suffix='events',
+        extension='.tsv',
+    )
+
+    events = mne.read_events(events_fname)
+    events = events[events[:, 2] != 0]  # drop unknown "0" events
+    event_id = {'Auditory/Left': 1, 'Auditory/Right': 2, 'Visual/Left': 3,
+                'Visual/Right': 4, 'Smiley': 5, 'Button': 32}
+    raw = _read_raw_fif(raw_fname)
+    annotations = mne.Annotations(
+        # Try to avoid rounding errors.
+        onset=(
+            1 / raw.info['sfreq'] * 600,
+            1 / raw.info['sfreq'] * 600,  # intentional
+            1 / raw.info['sfreq'] * 3000
+        ),
+        duration=(
+            1 / raw.info['sfreq'],
+            1 / raw.info['sfreq'],
+            1 / raw.info['sfreq'] * 200
+        ),
+        description=('BAD_segment', 'EDGE_segment', 'custom'),
+    )
+    raw.set_annotations(annotations)
+
+    # Write annotations while passing event_id
+    # Should raise since annotations descriptions are missing from event_id
+    with pytest.raises(ValueError, match='The following entries are missing'):
+        write_raw_bids(
+            raw,
+            bids_path=bids_path,
+            event_id=event_id,
+            events=events if write_events else None,
+        )
+
+    # Passing a complete mapping should work
+    event_id_with_annots = event_id.copy()
+    event_id_with_annots.update({
+        'BAD_segment': 9999,
+        'EDGE_segment': 10000,
+        'custom': 2000
+    })
+    write_raw_bids(
+        raw,
+        bids_path=bids_path,
+        event_id=event_id_with_annots,
+        events=events if write_events else None,
+    )
+    _bids_validate(bids_root)
+
+    # Ensure all events + annotations were written
+    events_tsv = _from_tsv(events_tsv_fname)
+
+    if write_events:
+        n_events_expected = len(events) + len(raw.annotations)
+    else:
+        n_events_expected = len(raw.annotations)
+
+    assert len(events_tsv['trial_type']) == n_events_expected
+
+
 @pytest.mark.parametrize(
     'drop_undescribed_events',
     [True, False]
@@ -2711,7 +2779,7 @@ def test_annotations(_bids_validate, bad_segments, tmp_path):
 @pytest.mark.filterwarnings(warning_str['channel_unit_changed'])
 @testing.requires_testing_data
 def test_undescribed_events(_bids_validate, drop_undescribed_events, tmp_path):
-    """Test we're behaving correctly if event descriptions are missing."""
+    """Test we're raising if event descriptions are missing."""
     bids_root = tmp_path / 'bids1'
     bids_path = _bids_path.copy().update(root=bids_root, datatype='meg')
     raw_fname = op.join(data_path, 'MEG', 'sample',