Repository: mne-tools/mne-python

Issue:
EDF+ Annotation Timestamps missing submillisecond accuracy
It was noticed that the onset time for annotations in an EDF+ file might be missing the submillisecond accuracy accociated with the start time of the recording. This results in annotation timestamps being delayed from the true event time. 

#### Expected results

The EDF+ file used here contains patient data and cannot be shared. However, if this bug is confirmed all EDF+ files will be affected. 

If the EDF+ file is brought into [EDFbrowser](https://www.teuniz.net/edfbrowser/) it will be noticed that the millisecond aspect of the start time is displayed in the `Start` field:

![thumbnail_Outlook-ibjnd242](https://user-images.githubusercontent.com/26314366/83948860-4c288380-a7ee-11ea-8f98-9f95026da92d.png)

For instance, this results in the annotation timestamps being:

![thumbnail_Outlook-of2qriv1](https://user-images.githubusercontent.com/26314366/83948874-68c4bb80-a7ee-11ea-8e9f-6cd5969134cf.png)

#### Actual results

When loading the data into MNE the annotation timestamps return:

![thumbnail_Outlook-dnk5ugfi](https://user-images.githubusercontent.com/26314366/83948890-8abe3e00-a7ee-11ea-865b-d45f36dbff99.png)

If you take the difference between the MNE timestamp and the EDFbrowser timestamp it is equal to the submillisecond portion of the recording start time.

#### Additional information

I reached out to Teunis van Beelen, the developer of [EDFbrowser](https://www.teuniz.net/edfbrowser/), to get an explanation. The explanation provided was as follows:

> EDF does not have support for subsecond precision of the starttime but EDF+ does. Because EDF+ needed to be backwards compatible with EDF and EDF doesn't have the possibility to write subsecond precision for the starttime into the header, the subsecond part of the starttime is written outside of the EDF header. The way the subsecond part of the starttime is stored in EDF+ is described here: https://www.edfplus.info/specs/edfplus.html#timekeeping

Teunis van Beelen provided a quote from the EDF specs:

> "The startdate/time of a file is specified in the EDF+ header fields 'startdate of recording' and 'starttime of recording'. These fields must indicate the absolute second in which the start of the first data record falls. So, the first TAL in the first data record always starts with +0.X2020, indicating that the first data record starts a fraction, X, of a second after the startdate/time that is specified in the EDF+ header. If X=0, then the .X may be omitted."

In other words, the subsecond part of the starttime is written in the EDF Annotations "signal".



Discussion:
do you know any public file that would allow us to
replicate/investigate this?

otherwise you can try to hack the edf.py file. Relevant code starts here:

https://github.com/mne-tools/mne-python/blob/master/mne/io/edf/edf.py#L231

I have de-identified the file I used:
https://drive.google.com/file/d/1yql0bMGcXXrEZr1At_Cp-eMi6rTKSr_F/view?usp=sharing


Resolving patch:
diff --git a/mne/datasets/utils.py b/mne/datasets/utils.py
--- a/mne/datasets/utils.py
+++ b/mne/datasets/utils.py
@@ -239,7 +239,7 @@ def _data_path(path=None, force_update=False, update_path=True, download=True,
     path = _get_path(path, key, name)
     # To update the testing or misc dataset, push commits, then make a new
     # release on GitHub. Then update the "releases" variable:
-    releases = dict(testing='0.91', misc='0.6')
+    releases = dict(testing='0.92', misc='0.6')
     # And also update the "md5_hashes['testing']" variable below.
 
     # To update any other dataset, update the data archive itself (upload
@@ -326,7 +326,7 @@ def _data_path(path=None, force_update=False, update_path=True, download=True,
         sample='12b75d1cb7df9dfb4ad73ed82f61094f',
         somato='ea825966c0a1e9b2f84e3826c5500161',
         spm='9f43f67150e3b694b523a21eb929ea75',
-        testing='f87f04fcdf56a7a55fb73b1b260b5b5b',
+        testing='42daafd1b882da2ef041de860ca6e771',
         multimodal='26ec847ae9ab80f58f204d09e2c08367',
         fnirs_motor='c4935d19ddab35422a69f3326a01fef8',
         opm='370ad1dcfd5c47e029e692c85358a374',
diff --git a/mne/io/edf/edf.py b/mne/io/edf/edf.py
--- a/mne/io/edf/edf.py
+++ b/mne/io/edf/edf.py
@@ -1375,12 +1375,23 @@ def _read_annotations_edf(annotations):
         triggers = re.findall(pat, tals.decode('latin-1'))
 
     events = []
-    for ev in triggers:
-        onset = float(ev[0])
+    offset = 0.
+    for k, ev in enumerate(triggers):
+        onset = float(ev[0]) + offset
         duration = float(ev[2]) if ev[2] else 0
         for description in ev[3].split('\x14')[1:]:
             if description:
                 events.append([onset, duration, description])
+            elif k == 0:
+                # The startdate/time of a file is specified in the EDF+ header
+                # fields 'startdate of recording' and 'starttime of recording'.
+                # These fields must indicate the absolute second in which the
+                # start of the first data record falls. So, the first TAL in
+                # the first data record always starts with +0.X, indicating
+                # that the first data record starts a fraction, X, of a second
+                # after the startdate/time that is specified in the EDF+
+                # header. If X=0, then the .X may be omitted.
+                offset = -onset
 
     return zip(*events) if events else (list(), list(), list())
 


Test patch:
diff --git a/mne/io/edf/tests/test_edf.py b/mne/io/edf/tests/test_edf.py
--- a/mne/io/edf/tests/test_edf.py
+++ b/mne/io/edf/tests/test_edf.py
@@ -56,6 +56,7 @@
                                         'multiple_annotation_chans.bdf')
 test_generator_bdf = op.join(data_path, 'BDF', 'test_generator_2.bdf')
 test_generator_edf = op.join(data_path, 'EDF', 'test_generator_2.edf')
+edf_annot_sub_s_path = op.join(data_path, 'EDF', 'subsecond_starttime.edf')
 
 eog = ['REOG', 'LEOG', 'IEOG']
 misc = ['EXG1', 'EXG5', 'EXG8', 'M1', 'M2']
@@ -385,3 +386,10 @@ def test_edf_lowpass_zero():
     with pytest.warns(RuntimeWarning, match='too long.*truncated'):
         raw = read_raw_edf(edf_stim_resamp_path)
     assert_allclose(raw.info["lowpass"], raw.info["sfreq"] / 2)
+
+
+@testing.requires_testing_data
+def test_edf_annot_sub_s_onset():
+    """Test reading of sub-second annotation onsets."""
+    raw = read_raw_edf(edf_annot_sub_s_path)
+    assert_allclose(raw.annotations.onset, [1.951172, 3.492188])