Repository: mne-tools/mne-python

Issue:
Importing bad data spans from Raw neuroscan cnt files into annotation description.
### Describe the new feature or enhancement

Hello! My lab has a lot of data that was processed using neuroscan, and I am trying to import some of our raw cnt files into mne. I have run into a problem with importing event files as neuroscan uses the same stim code to mark the end and the beginning of bad blocks (data spans): 
![image](https://user-images.githubusercontent.com/20428109/232119921-617300bb-5fe7-466e-b28b-b4e1217b3fed.png)
As far as I can tell, mne only imports the event code and the onset of these events (not the duration). However, I was looking through an old manual on neuroscan and it references another parameter that describes whether you should accept or reject a block:
![image](https://user-images.githubusercontent.com/20428109/232120763-4d45d5ef-f149-4cf8-a1cf-a90bbd0d987a.png)
I was wondering if this could be implemented by appending '_bad' to the end of the blocks that are meant to be rejected and manually computing the duration of the event by finding the difference in onsets of the event codes.

Another suggestion I have is that the  _translating_function does not convert the early event byte onsets properly to seconds as it says they are outside the data range (I used it myself and got a negative time). I suggest automatically setting event onset values to 0 if they have a cnt byte onset below 3665 (calculated using the translation function).

### Describe your proposed implementation

From cnt.py:
  ```
  with open(fname, 'rb') as fid:
        fid.seek(event_table_pos + 9)  # the real table stats at +9
        buffer = fid.read(teeg.total_length)

    my_events = list(event_parser(buffer))

    if not my_events:
        return Annotations(list(), list(), list(), None)
    else:
        onset = _translating_function(np.array([e.Offset for e in my_events],
                                               dtype=float),
                                      n_channels=n_channels,
                                      event_type=type(my_events[0]),
                                      data_format=data_format)
        duration = np.array([getattr(e, 'Latency', 0.) for e in my_events],
                            dtype=float)

        description = np.array([str(e.StimType) for e in my_events])
        return Annotations(onset=onset / sfreq,
                           duration=duration,
                           description=description,
                           orig_time=None)

```

- Make a new variable:  
`accept = np.array([str(e.KeyPad_Accept) for e in my_events])`
If it contains 'xc0' append '_bad' to the event code (description of event in mne)
Manually compute durations (currently all durations are 0) by finding the difference between codes.

- Set onset to 0 if .cnt byte onset is below 3664.9999999971237.

### Describe possible alternatives

I cannot think of any. 

### Additional context

_No response_


Discussion:
Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴

what I would recommend here is to send us a pull request with code snippet
and a file so can test too and see the diff in the code.

On Fri, Apr 14, 2023 at 8:10 PM welcome[bot] ***@***.***>
wrote:

> Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get
> back to you soon. 🚴
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/mne-tools/mne-python/issues/11630#issuecomment-1509042372>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AABHKHAH45QWLILB3TIPS7TXBGHLFANCNFSM6AAAAAAW6XO4TM>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>


Resolving patch:
diff --git a/mne/datasets/config.py b/mne/datasets/config.py
--- a/mne/datasets/config.py
+++ b/mne/datasets/config.py
@@ -87,7 +87,7 @@
 # respective repos, and make a new release of the dataset on GitHub. Then
 # update the checksum in the MNE_DATASETS dict below, and change version
 # here:                  ↓↓↓↓↓         ↓↓↓
-RELEASES = dict(testing='0.145', misc='0.26')
+RELEASES = dict(testing='0.146', misc='0.26')
 TESTING_VERSIONED = f'mne-testing-data-{RELEASES["testing"]}'
 MISC_VERSIONED = f'mne-misc-data-{RELEASES["misc"]}'
 
@@ -111,7 +111,7 @@
 # Testing and misc are at the top as they're updated most often
 MNE_DATASETS['testing'] = dict(
     archive_name=f'{TESTING_VERSIONED}.tar.gz',
-    hash='md5:2036f7d7616129c624b757fbb019be24',
+    hash='md5:a2e86fe404f4321408b22f38711d11b7',
     url=('https://codeload.github.com/mne-tools/mne-testing-data/'
          f'tar.gz/{RELEASES["testing"]}'),
     # In case we ever have to resort to osf.io again...
diff --git a/mne/io/cnt/cnt.py b/mne/io/cnt/cnt.py
--- a/mne/io/cnt/cnt.py
+++ b/mne/io/cnt/cnt.py
@@ -46,6 +46,17 @@ def _read_annotations_cnt(fname, data_format='int16'):
     SETUP_NCHANNELS_OFFSET = 370
     SETUP_RATE_OFFSET = 376
 
+    def _accept_reject_function(keypad_accept):
+        accept_list = []
+        for code in keypad_accept:
+            if 'xd0' in str(code):
+                accept_list.append('good')
+            elif 'xc0' in str(code):
+                accept_list.append('bad')
+            else:
+                accept_list.append('NA')
+        return np.array(accept_list)
+
     def _translating_function(offset, n_channels, event_type,
                               data_format=data_format):
         n_bytes = 2 if data_format == 'int16' else 4
@@ -53,7 +64,47 @@ def _translating_function(offset, n_channels, event_type,
             offset *= n_bytes * n_channels
         event_time = offset - 900 - (75 * n_channels)
         event_time //= n_channels * n_bytes
-        return event_time - 1
+        event_time = event_time - 1
+        # Prevent negative event times
+        np.clip(event_time, 0, None, out=event_time)
+        return event_time
+
+    def _update_bad_span_onset(accept_reject, onset, duration, description):
+        accept_reject = accept_reject.tolist()
+        onset = onset.tolist()
+        duration = duration.tolist()
+        description = description.tolist()
+        # If there are no bad spans, return original parameters
+        if 'bad' not in accept_reject:
+            return np.array(onset), np.array(duration), np.array(description)
+        # Create lists of bad and good span markers and onset
+        bad_good_span_markers = [i for i in accept_reject
+                                 if i in ['bad', 'good']]
+        bad_good_onset = [onset[i] for i, value in enumerate(accept_reject)
+                          if value in ['bad', 'good']]
+        # Calculate duration of bad span
+        first_bad_index = bad_good_span_markers.index('bad')
+        duration_list = [bad_good_onset[i + 1] - bad_good_onset[i]
+                         for i in range(first_bad_index,
+                         len(bad_good_span_markers), 2)]
+        # Add bad event marker duration and description
+        duration_list_index = 0
+        for i in range(len(onset)):
+            if accept_reject[i] == 'bad':
+                duration[i] = duration_list[duration_list_index]
+                description[i] = 'BAD_' + description[i]
+                duration_list_index += 1
+        # Remove good span markers
+        final_onset, final_duration, final_description = [], [], []
+        for i in range(len(accept_reject)):
+            if accept_reject[i] != 'good':
+                final_onset.append(onset[i])
+                final_duration.append(duration[i])
+                final_description.append(description[i])
+        return (
+            np.array(final_onset),
+            np.array(final_duration),
+            np.array(final_description))
 
     with open(fname, 'rb') as fid:
         fid.seek(SETUP_NCHANNELS_OFFSET)
@@ -86,9 +137,16 @@ def _translating_function(offset, n_channels, event_type,
                                       data_format=data_format)
         duration = np.array([getattr(e, 'Latency', 0.) for e in my_events],
                             dtype=float)
+        accept_reject = _accept_reject_function(
+            np.array([e.KeyPad_Accept for e in my_events]))
 
         description = np.array([str(e.StimType) for e in my_events])
-        return Annotations(onset=onset / sfreq,
+
+        onset, duration, description = _update_bad_span_onset(accept_reject,
+                                                              onset / sfreq,
+                                                              duration,
+                                                              description)
+        return Annotations(onset=onset,
                            duration=duration,
                            description=description,
                            orig_time=None)


Test patch:
diff --git a/mne/io/cnt/tests/test_cnt.py b/mne/io/cnt/tests/test_cnt.py
--- a/mne/io/cnt/tests/test_cnt.py
+++ b/mne/io/cnt/tests/test_cnt.py
@@ -15,6 +15,7 @@
 
 data_path = testing.data_path(download=False)
 fname = data_path / "CNT" / "scan41_short.cnt"
+fname_bad_spans = data_path / "CNT" / "test_CNT_events_mne_JWoess_clipped.cnt"
 
 
 @testing.requires_testing_data
@@ -50,3 +51,11 @@ def test_compare_events_and_annotations():
     assert len(annot) == 6
     assert_array_equal(annot.onset[:-1], events[:, 0] / raw.info['sfreq'])
     assert 'STI 014' not in raw.info['ch_names']
+
+
+@testing.requires_testing_data
+def test_bad_spans():
+    """Test reading raw cnt files with bad spans."""
+    annot = read_annotations(fname_bad_spans)
+    temp = '\t'.join(annot.description)
+    assert 'BAD' in temp