Repository: aeye-lab/pymovements
Pull request #342 (2023-04-28 03:28:08)

feat!: Add pattern argument to parse_eyelink() to be more flexible

codecov-commenter: ## [Codecov](https://codecov.io/gh/aeye-lab/pymovements/pull/342?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
Patch coverage: **`100.00`**% and no project coverage change.
> Comparison is base [(`08d93dd`)](https://codecov.io/gh/aeye-lab/pymovements/commit/08d93dd879ec97b051f77c4935918a48cc966fc5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) 100.00% compared to head [(`73b44e2`)](https://codecov.io/gh/aeye-lab/pymovements/pull/342?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) 100.00%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main      #342   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           53        53           
  Lines         1803      1828   +25     
  Branches       386       398   +12     
=========================================
+ Hits          1803      1828   +25     
```


| [Impacted Files](https://codecov.io/gh/aeye-lab/pymovements/pull/342?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) | Coverage Δ | |
|---|---|---|
| [src/pymovements/plotting/tsplot.py](https://codecov.io/gh/aeye-lab/pymovements/pull/342?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL3Bsb3R0aW5nL3RzcGxvdC5weQ==) | `100.00% <ø> (ø)` | |
| [src/pymovements/dataset/dataset\_files.py](https://codecov.io/gh/aeye-lab/pymovements/pull/342?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2RhdGFzZXQvZGF0YXNldF9maWxlcy5weQ==) | `100.00% <100.00%> (ø)` | |
| [src/pymovements/utils/parsing.py](https://codecov.io/gh/aeye-lab/pymovements/pull/342?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL3V0aWxzL3BhcnNpbmcucHk=) | `100.00% <100.00%> (ø)` | |


</details>

[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/aeye-lab/pymovements/pull/342?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).


Patch:
diff --git a/src/pymovements/dataset/dataset_files.py b/src/pymovements/dataset/dataset_files.py
index 9602fd8..9554109 100644
--- a/src/pymovements/dataset/dataset_files.py
+++ b/src/pymovements/dataset/dataset_files.py
@@ -65,6 +65,8 @@ def scan_dataset(definition: DatasetDefinition, paths: DatasetPaths) -> pl.DataF
     )
 
     if len(fileinfo_dicts) == 0:
+        print(definition.filename_format)
+        print(curly_to_regex(definition.filename_format))
         raise RuntimeError(f'no matching files found in {paths.raw}')
 
     # Create dataframe from all fileinfo records.
diff --git a/src/pymovements/plotting/tsplot.py b/src/pymovements/plotting/tsplot.py
index 406f854..31cb163 100644
--- a/src/pymovements/plotting/tsplot.py
+++ b/src/pymovements/plotting/tsplot.py
@@ -98,8 +98,6 @@ def tsplot(
 
     if arr.ndim == 1:
         arr = np.expand_dims(arr, axis=0)
-    elif arr.ndim == 2:
-        pass
 
     channel_axis = 0
     sample_axis = 1
diff --git a/src/pymovements/utils/parsing.py b/src/pymovements/utils/parsing.py
index 3942c79..9cb9be2 100755
--- a/src/pymovements/utils/parsing.py
+++ b/src/pymovements/utils/parsing.py
@@ -22,17 +22,18 @@ from __future__ import annotations
 
 import re
 from pathlib import Path
+from typing import Any
 
 import numpy as np
 import polars as pl
 
 EYE_TRACKING_SAMPLE = re.compile(
-    r'(?P<time>(\d{7}|\d{8}|\d{9}|\d{10}))\s+'
+    r'(?P<time>(\d{7,10}))\s+'
     r'(?P<x_pix>[-]?\d*[.]\d*)\s+'
     r'(?P<y_pix>[-]?\d*[.]\d*)\s+'
     r'(?P<pupil>\d*[.]\d*)\s+'
     r'(?P<dummy>\d*[.]\d*)\s+'
-    r'(?P<dots>[A-Za-z.]{5})?\s*',
+    r'(?P<dots>[A-Za-z.]{3,5})?\s*',
 )
 
 
@@ -52,96 +53,146 @@ def check_nan(sample_location: str) -> float:
     return ret
 
 
+def compile_patterns(patterns: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    """Compile patterns from strings.
+
+    Parameters
+    ----------
+    patterns:
+        The list of patterns to compile.
+    """
+    msg_prefix = r'MSG\s+\d+\s+'
+
+    compiled_patterns = []
+
+    for pattern in patterns:
+        if isinstance(pattern, str):
+            compiled_pattern = {'pattern': re.compile(msg_prefix + pattern)}
+            compiled_patterns.append(compiled_pattern)
+            continue
+
+        if isinstance(pattern, dict):
+            if isinstance(pattern['pattern'], str):
+                compiled_patterns.append({
+                    **pattern,
+                    'pattern': re.compile(msg_prefix + pattern['pattern']),
+                })
+                continue
+
+            if isinstance(pattern['pattern'], tuple):
+                for single_pattern in pattern['pattern']:
+                    compiled_patterns.append({
+                        **pattern,
+                        'pattern': re.compile(msg_prefix + single_pattern),
+                    })
+                continue
+
+            raise ValueError(f'invalid pattern: {pattern}')
+
+        raise ValueError(f'invalid pattern: {pattern}')
+
+    return compiled_patterns
+
+
+def get_additional_columns(compiled_patterns: list[dict[str, Any]]) -> set[str]:
+    """Get additionally needed columns from compiled patterns."""
+    additional_columns = set()
+
+    for compiled_pattern_dict in compiled_patterns:
+        if 'column' in compiled_pattern_dict:
+            additional_columns.add(compiled_pattern_dict['column'])
+
+        for column in compiled_pattern_dict['pattern'].groupindex.keys():
+            additional_columns.add(column)
+
+    return additional_columns
+
+
 def parse_eyelink(
         filepath: Path,
-        sync_msg_start_pattern: None | str = None,
-        sync_msg_stop_pattern: None | str = None,
+        patterns: list | None = None,
+        schema: dict | None = None,
 ) -> pl.DataFrame:
     """Processes ascii files to csv.
 
     Parameters
     ----------
-    filepath: Path
-        file name of ascii file to convert
-    sync_msg_start_pattern: str,
-        Optional starting pattern of trial as sync message,
-        if None is given reverts back to TRIALID
-    sync_msg_stop_pattern: str,
-        Optional stopping pattern of trial as sync message,
-        if None is given reverts back to previous TRIALID
+    filepath:
+        file name of ascii file to convert.
+    patterns:
+        list of patterns to match for additional columns.
+    schema:
+        Dictionary to optionally specify types of columns parsed by patterns.
+
     """
-    if sync_msg_start_pattern is None:
-        # if no specific type pattern revert to trial ids
-        _sync_msg_start_pattern = re.compile('MSG\t\\d+ TRIALID (?P<startpattern>[0-9]+)\n')
-    else:
-        _sync_msg_start_pattern = re.compile(
-            f'MSG\t\\d+ .\\d+ {sync_msg_start_pattern}(?P<startpattern>[0-9]+)',
-        )
-    if sync_msg_stop_pattern is None:
-        # if no stop pattern, stop when "next" occurance of start pattern
-        _sync_msg_stop_pattern = _sync_msg_start_pattern
-    else:
-        _sync_msg_stop_pattern = re.compile(
-            fr'MSG\t\d+\ \d+\ \S+(?P<stoppattern>{sync_msg_stop_pattern})\n',
-        )
+    if patterns is None:
+        patterns = []
+    compiled_patterns = compile_patterns(patterns)
+
+    additional_columns = get_additional_columns(compiled_patterns)
+    additional: dict[str, list] = {
+        additional_column: [] for additional_column in additional_columns
+    }
+    current_additional = {
+        additional_column: None for additional_column in additional_columns
+    }
 
     samples: dict[str, list] = {
         'time': [],
         'x_pix': [],
         'y_pix': [],
         'pupil': [],
-        'task': [],
+        **additional,
     }
 
     with open(filepath, encoding='ascii') as asc_file:
         lines = asc_file.readlines()
-    current_task = None
-    during_task_recording = False
+
     for line in lines:
+        for pattern_dict in compiled_patterns:
+            match = pattern_dict['pattern'].match(line)
 
-        if _sync_msg_stop_pattern.match(line):
-            during_task_recording = False
-        if _sync_msg_start_pattern.match(line):
-            start_matched = _sync_msg_start_pattern.match(line)
-            # mypy is unaware that 'start_matched' can never be None (l.146)
-            assert start_matched is not None
-            start_id = start_matched.group('startpattern')
-            if sync_msg_start_pattern is not None:
-                current_task = str(sync_msg_start_pattern) + start_id
-            else:
-                current_task = f'TRIALID_{start_id}'
-            during_task_recording = True
-        if during_task_recording:
-            if EYE_TRACKING_SAMPLE.match(line):
-                eye_tracking_sample_match = EYE_TRACKING_SAMPLE.match(line)
-                # mypy is unaware that 'eye_tracking_sample_match' can never be None (l.156)
-                assert eye_tracking_sample_match is not None
-
-                timestamp_s = eye_tracking_sample_match.group('time')
-                x_pix_s = eye_tracking_sample_match.group('x_pix')
-                y_pix_s = eye_tracking_sample_match.group('y_pix')
-                pupil_s = eye_tracking_sample_match.group('pupil')
-
-                timestamp = int(timestamp_s)
-                x_pix = check_nan(x_pix_s)
-                y_pix = check_nan(y_pix_s)
-                pupil = check_nan(pupil_s)
-
-                samples['time'].append(timestamp)
-                samples['x_pix'].append(x_pix)
-                samples['y_pix'].append(y_pix)
-                samples['pupil'].append(pupil)
-                samples['task'].append(current_task)
+            if match:
+                if 'value' in pattern_dict:
+                    current_column = pattern_dict['column']
+                    current_additional[current_column] = pattern_dict['value']
+
+                else:
+                    for column, value in match.groupdict().items():
+                        current_additional[column] = value
+
+        eye_tracking_sample_match = EYE_TRACKING_SAMPLE.match(line)
+        if eye_tracking_sample_match:
+
+            timestamp_s = eye_tracking_sample_match.group('time')
+            x_pix_s = eye_tracking_sample_match.group('x_pix')
+            y_pix_s = eye_tracking_sample_match.group('y_pix')
+            pupil_s = eye_tracking_sample_match.group('pupil')
+
+            timestamp = int(timestamp_s)
+            x_pix = check_nan(x_pix_s)
+            y_pix = check_nan(y_pix_s)
+            pupil = check_nan(pupil_s)
+
+            samples['time'].append(timestamp)
+            samples['x_pix'].append(x_pix)
+            samples['y_pix'].append(y_pix)
+            samples['pupil'].append(pupil)
+
+            for additional_column in additional_columns:
+                samples[additional_column].append(current_additional[additional_column])
 
     df = pl.from_dict(
         data=samples,
-        schema={
+        schema_overrides={
             'time': pl.Int64,
             'x_pix': pl.Float64,
             'y_pix': pl.Float64,
             'pupil': pl.Float64,
-            'task': pl.Utf8,
         },
     )
 
+    if schema is not None:
+        df = df.with_columns([pl.col(column).cast(dtype) for column, dtype in schema.items()])
+
     return df


Test patch:
diff --git a/tests/dataset/dataset_files_test.py b/tests/dataset/dataset_files_test.py
index d605a81..005a450 100644
--- a/tests/dataset/dataset_files_test.py
+++ b/tests/dataset/dataset_files_test.py
@@ -151,7 +151,7 @@ MSG	2154569 TRACKER_TIME 1 2222195.987
 MSG	2154570 0 READING_SCREEN_1.STOP
 """
 
-EXPECTED_DF_NO_SYNC = pl.from_dict(
+EXPECTED_DF_NO_PATTERNS = pl.from_dict(
     {
         'time': [
             2154557,
@@ -185,68 +185,36 @@ EXPECTED_DF_NO_SYNC = pl.from_dict(
             784.0,
             0.0,
             714.0,
-        ], 'task': [
-            'TRIALID_0',
-            'TRIALID_0',
-            'TRIALID_0',
-            'TRIALID_0',
-            'TRIALID_1',
-            'TRIALID_1',
-            'TRIALID_1',
         ],
     },
 )
 
-EXPECTED_DF_SYNC = pl.from_dict(
+EXPECTED_DF_PATTERNS = pl.from_dict(
     {
-        'time': [
-            2154558,
-            2154560,
-            2154561,
-            2154565,
-            2154567,
-            2154568,
-        ], 'x_pix': [
-            139.5,
-            np.nan,
-            850.7,
-            139.5,
-            np.nan,
-            850.7,
-        ], 'y_pix': [
-            131.9,
-            np.nan,
-            717.5,
-            131.9,
-            np.nan,
-            717.5,
-        ], 'pupil': [
-            784.0,
-            0.0,
-            714.0,
-            784.0,
-            0.0,
-            714.0,
-        ], 'task': [
-            'SYNCTIME_READING_SCREEN_0',
-            'SYNCTIME_READING_SCREEN_0',
-            'SYNCTIME_READING_SCREEN_0',
-            'SYNCTIME_READING_SCREEN_1',
-            'SYNCTIME_READING_SCREEN_1',
-            'SYNCTIME_READING_SCREEN_1',
-        ],
+        'time': [2154557, 2154558, 2154560, 2154561, 2154565, 2154567, 2154568],
+        'x_pix': [139.6, 139.5, np.nan, 850.7, 139.5, np.nan, 850.7],
+        'y_pix': [132.1, 131.9, np.nan, 717.5, 131.9, np.nan, 717.5],
+        'pupil': [784.0, 784.0, 0.0, 714.0, 784.0, 0.0, 714.0],
+        'task': ['reading', 'reading', 'reading', 'reading', 'reading', 'reading', 'reading'],
+        'trial_id': [0, 0, 0, 0, 1, 1, 1],
     },
 )
 
+PATTERNS = [
+    {
+        'pattern': 'SYNCTIME_READING',
+        'column': 'task',
+        'value': 'reading',
+    },
+    r'TRIALID (?P<trial_id>\d+)',
+]
+
 
 @pytest.mark.parametrize(
     'read_kwargs',
     [
         pytest.param(
-            {
-                'sync_msg_start_pattern': 'SYNCTIME_READING_SCREEN_',
-                'sync_msg_stop_pattern': 'STOP',
-            },
+            {'patterns': PATTERNS, 'schema': {'trial_id': pl.Int64}},
             id='read_kwargs_dict',
         ),
         pytest.param(
@@ -262,8 +230,8 @@ def test_load_eyelink_file(tmp_path, read_kwargs):
     df = pm.dataset.dataset_files.load_gaze_file(filepath, custom_read_kwargs=read_kwargs)
 
     if read_kwargs is not None:
-        expected_df = EXPECTED_DF_SYNC
+        expected_df = EXPECTED_DF_PATTERNS
     else:
-        expected_df = EXPECTED_DF_NO_SYNC
+        expected_df = EXPECTED_DF_NO_PATTERNS
 
-    assert_frame_equal(df, expected_df)
+    assert_frame_equal(df, expected_df, check_column_order=False)
diff --git a/tests/utils/parsing_test.py b/tests/utils/parsing_test.py
index 850d699..0c13b83 100644
--- a/tests/utils/parsing_test.py
+++ b/tests/utils/parsing_test.py
@@ -18,8 +18,6 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 """ Tests pymovements asc to csv processing"""
-# flake8: noqa: E101, W191, E501
-# pylint: disable=duplicate-code
 import numpy as np
 import polars as pl
 import pytest
@@ -28,238 +26,107 @@ from polars.testing import assert_frame_equal
 import pymovements as pm
 
 
-ASC_TEXT = r"""\
-** CONVERTED FROM D:\SamplePymovements\results\sub_1\sub_1.edf using edfapi 4.2.1 Win32  EyeLink Dataviewer Sub ComponentApr 01 1990 on Wed Sep 20 13:47:57 1989
-** DATE: Wed Sep  20 13:47:20 1989
-** TYPE: EDF_FILE BINARY EVENT SAMPLE TAGGED
-** VERSION: EYELINK II 1
-** SOURCE: EYELINK CL
-** EYELINK II CL v6.12 Feb  1 2018 (EyeLink Portable Duo)
-** CAMERA: EyeLink USBCAM Version 1.01
-** SERIAL NUMBER: CLU-DAB50
-** CAMERA_CONFIG: DAB50200.SCD
-** RECORDED BY SleepAlc
-** SREB2.2.299 WIN32 LID:20A87A96 Mod:2023.03.08 11:03 MEZ
-**
-
-MSG	2091650 !CMD 1 select_parser_configuration 0
-MSG	2091659 !CMD 0 fixation_update_interval = 50
-MSG	2091659 !CMD 0 fixation_update_accumulate = 50
-MSG	2091681 !CMD 1 auto_calibration_messages = YES
-MSG	2095865 DISPLAY_COORDS 0 0 1279 1023
-MSG	2095865 RETRACE_INTERVAL  16.646125144
-MSG	2095865 ENVIRONMENT   OpenGL on Windows (6, 2, 9200, 2, '')
-MSG	2095980 TRACKER_TIME 0 2095980.470
-MSG	2096367 -4 SYNCTIME 766 0
-MSG	2100624 SYNCTIME_READING
-INPUT	2117909	0
-INPUT	2126823	0
-MSG	2135819 !CAL
->>>>>>> CALIBRATION (HV9,P-CR) FOR LEFT: <<<<<<<<<
-MSG	2135819 !CAL Calibration points:
-MSG	2135819 !CAL -32.6, -47.7        -0,    227
-MSG	2135819 !CAL -32.2, -63.6        -0,  -2267
-MSG	2135819 !CAL -32.6, -31.7        -0,   2624
-MSG	2135819 !CAL -58.0, -47.3     -3291,    227
-MSG	2135819 !CAL -7.7, -46.8      3291,    227
-MSG	2135820 !CAL -58.8, -65.2     -3358,  -2267
-MSG	2135820 !CAL -7.9, -61.4      3358,  -2267
-MSG	2135820 !CAL -55.5, -31.2     -3227,   2624
-MSG	2135820 !CAL -9.0, -31.3      3227,   2624
-MSG	2135820 !CAL  0.0,  0.0         0,      0
-MSG	2135820 !CAL eye check box: (L,R,T,B)
-	  -65     6   -72     7
-MSG	2135820 !CAL href cal range: (L,R,T,B)
-	-5037  5037 -3489  3847
-MSG	2135820 !CAL Cal coeff:(X=a+bx+cy+dxx+eyy,Y=f+gx+goaly+ixx+jyy)
-  -0.00043008  131.07  1.437  0.051949 -0.1007
-   227.35 -1.5024  153.47 -0.1679 -0.22845
-MSG	2135820 !CAL Prenormalize: offx, offy = -32.583 -47.715
-MSG	2135820 !CAL Quadrant center: centx, centy =
-  -0.00043025  227.35
-MSG	2135820 !CAL Corner correction:
-   9.5364e-06,  3.4194e-05
-  -1.6932e-05,  2.9132e-05
-   3.3933e-05,  3.5e-06
-   1.5902e-05,  8.6479e-06
-MSG	2135820 !CAL Gains: cx:152.074 lx:170.107 rx:152.936
-MSG	2135820 !CAL Gains: cy:128.550 ty:155.848 by:116.611
-MSG	2135820 !CAL Resolution (upd) at screen center: X=1.7, Y=2.0
-MSG	2135820 !CAL Gain Change Proportion: X: 0.112 Y: 0.336
-MSG	2135821 !CAL Gain Ratio (Gy/Gx) = 0.845
-MSG	2135821 !CAL Bad Y/X gain ratio: 0.845
-MSG	2135821 !CAL PCR gain ratio(x,y) = 2.507, 2.179
-MSG	2135821 !CAL CR gain match(x,y) = 1.010, 1.010
-MSG	2135821 !CAL Slip rotation correction OFF
-MSG	2135821 !CAL CALIBRATION HV9 L LEFT    GOOD
-INPUT	2137650	0
-MSG	2148587 !CAL VALIDATION HV9 L LEFT  GOOD ERROR 0.27 avg. 0.83 max  OFFSET 0.11 deg. 3.7,2.4 pix.
-MSG	2148587 VALIDATE L POINT 0  LEFT  at 640,512  OFFSET 0.19 deg.  7.2,1.0 pix.
-MSG	2148587 VALIDATE L POINT 1  LEFT  at 640,159  OFFSET 0.12 deg.  3.9,-2.2 pix.
-MSG	2148587 VALIDATE L POINT 2  LEFT  at 640,864  OFFSET 0.42 deg.  -15.8,0.9 pix.
-MSG	2148587 VALIDATE L POINT 3  LEFT  at 172,512  OFFSET 0.83 deg.  26.3,17.5 pix.
-MSG	2148587 VALIDATE L POINT 4  LEFT  at 1107,512  OFFSET 0.19 deg.  -4.3,5.7 pix.
-MSG	2148587 VALIDATE L POINT 5  LEFT  at 228,201  OFFSET 0.06 deg.  1.3,1.9 pix.
-MSG	2148587 VALIDATE L POINT 6  LEFT  at 1051,201  OFFSET 0.33 deg.  -3.0,-12.5 pix.
-MSG	2148587 VALIDATE L POINT 7  LEFT  at 228,822  OFFSET 0.18 deg.  -6.8,0.2 pix.
-MSG	2148587 VALIDATE L POINT 8  LEFT  at 1051,822  OFFSET 0.18 deg.  3.8,5.5 pix.
-INPUT	2153108	0
-MSG	2154447 DRIFTCORRECT L LEFT  at 133,133  OFFSET 0.38 deg.  12.5,7.9 pix.
-MSG	2154540 TRIALID 0
-MSG	2154555 RECCFG CR 1000 2 1 L
-MSG	2154555 ELCLCFG BTABLER
-MSG	2154555 GAZE_COORDS 0.00 0.00 1279.00 1023.00
-MSG	2154555 THRESHOLDS L 102 242
-MSG	2154555 ELCL_WINDOW_SIZES 176 188 0 0
-MSG	2154555 CAMERA_LENS_FOCAL_LENGTH 27.00
-MSG	2154555 PUPIL_DATA_TYPE RAW_AUTOSLIP
-MSG	2154555 ELCL_PROC CENTROID (3)
-MSG	2154555 ELCL_PCR_PARAM 5 3.0
-START	2154556 	LEFT	SAMPLES	EVENTS
-PRESCALER	1
-VPRESCALER	1
-PUPIL	AREA
-EVENTS	GAZE	LEFT	RATE	1000.00	TRACKING	CR	FILTER	2
-SAMPLES	GAZE	LEFT	RATE	1000.00	TRACKING	CR	FILTER	2	INPUT
-INPUT	2154556	0
-MSG	2154557 !MODE RECORD CR 1000 2 1 L
-2154557	  139.6	  132.1	  784.0	    0.0	...
-MSG	2154558 -11 SYNCTIME_READING_SCREEN_0
-2154558	  139.5	  131.9	  784.0	    0.0	...
-MSG	2154559 -11 !V DRAW_LIST ../../runtime/dataviewer/sub_1/graphics/VC_1.vcl
-2154560	   .	   .	    0.0	    0.0	...
-2154561	  850.7	  717.5	  714.0	    0.0	...
-MSG	2154562 TRACKER_TIME 1 2222195.987
-MSG	2154563 0 READING_SCREEN_0.STOP
-MSG	2154447 DRIFTCORRECT L LEFT  at 133,133  OFFSET 0.38 deg.  12.5,7.9 pix.
-MSG	2154540 TRIALID 1
-MSG	2154555 RECCFG CR 1000 2 1 L
-MSG	2154555 ELCLCFG BTABLER
-MSG	2154555 GAZE_COORDS 0.00 0.00 1279.00 1023.00
-MSG	2154555 THRESHOLDS L 102 242
-MSG	2154555 ELCL_WINDOW_SIZES 176 188 0 0
-MSG	2154555 CAMERA_LENS_FOCAL_LENGTH 27.00
-MSG	2154555 PUPIL_DATA_TYPE RAW_AUTOSLIP
-MSG	2154555 ELCL_PROC CENTROID (3)
-MSG	2154555 ELCL_PCR_PARAM 5 3.0
-MSG	2154564 -11 SYNCTIME_READING_SCREEN_1
-2154565	  139.5	  131.9	  784.0	    0.0	...
-MSG	2154566 -11 !V DRAW_LIST ../../runtime/dataviewer/sub_1/graphics/VC_2.vcl
-2154567	   .	   .	    0.0	    0.0	...
-2154568	  850.7	  717.5	  714.0	    0.0	...
-MSG	2154569 TRACKER_TIME 1 2222195.987
-MSG	2154570 0 READING_SCREEN_1.STOP
+ASC_TEXT = r"""
+some
+lines
+to
+ignore
+the next line has all additional trial columns set to None
+10000000	  850.7	  717.5	  714.0	    0.0	...
+MSG 10000001 START_A
+the next line now should have the task column set to A
+10000002	  850.7	  717.5	  714.0	    0.0	...
+MSG 10000003 STOP_A
+the task should be set to None again
+10000004	  850.7	  717.5	  714.0	    0.0	...
+MSG 10000005 START_B
+the next line now should have the task column set to B
+10000006	  850.7	  717.5	  714.0	    0.0	...
+MSG 10000007 START_TRIAL_1
+the next line now should have the trial column set to 1
+10000008	  850.7	  717.5	  714.0	    0.0	...
+MSG 10000009 STOP_TRIAL_1
+MSG 10000010 START_TRIAL_2
+the next line now should have the trial column set to 2
+10000011	  850.7	  717.5	  714.0	    0.0	...
+MSG 10000012 STOP_TRIAL_2
+MSG 10000013 START_TRIAL_3
+the next line now should have the trial column set to 3
+10000014	  850.7	  717.5	  714.0	    0.0	...
+MSG 10000015 STOP_TRIAL_3
+MSG 10000016 STOP_B
+task and trial should be set to None again
+10000017	  850.7	  717.5	  .	    0.0	...
 """
 
-EXPECTED_DF_NO_SYNC = pl.from_dict(
+PATTERNS = [
     {
-        'time': [
-            2154557,
-            2154558,
-            2154560,
-            2154561,
-            2154565,
-            2154567,
-            2154568,
-        ], 'x_pix': [
-            139.6,
-            139.5,
-            np.nan,
-            850.7,
-            139.5,
-            np.nan,
-            850.7,
-        ], 'y_pix': [
-            132.1,
-            131.9,
-            np.nan,
-            717.5,
-            131.9,
-            np.nan,
-            717.5,
-        ], 'pupil': [
-            784.0,
-            784.0,
-            0.0,
-            714.0,
-            784.0,
-            0.0,
-            714.0,
-        ], 'task': [
-            'TRIALID_0',
-            'TRIALID_0',
-            'TRIALID_0',
-            'TRIALID_0',
-            'TRIALID_1',
-            'TRIALID_1',
-            'TRIALID_1',
-        ],
+        'pattern': 'START_A',
+        'column': 'task',
+        'value': 'A',
+    },
+    {
+        'pattern': 'START_B',
+        'column': 'task',
+        'value': 'B',
+    },
+    {
+        'pattern': ('STOP_A', 'STOP_B'),
+        'column': 'task',
+        'value': None,
     },
-)
 
-EXPECTED_DF_SYNC = pl.from_dict(
+    r'START_TRIAL_(?P<trial_id>\d+)',
     {
-        'time': [
-            2154558,
-            2154560,
-            2154561,
-            2154565,
-            2154567,
-            2154568,
-        ], 'x_pix': [
-            139.5,
-            np.nan,
-            850.7,
-            139.5,
-            np.nan,
-            850.7,
-        ], 'y_pix': [
-            131.9,
-            np.nan,
-            717.5,
-            131.9,
-            np.nan,
-            717.5,
-        ], 'pupil': [
-            784.0,
-            0.0,
-            714.0,
-            784.0,
-            0.0,
-            714.0,
-        ], 'task': [
-            'SYNCTIME_READING_SCREEN_0',
-            'SYNCTIME_READING_SCREEN_0',
-            'SYNCTIME_READING_SCREEN_0',
-            'SYNCTIME_READING_SCREEN_1',
-            'SYNCTIME_READING_SCREEN_1',
-            'SYNCTIME_READING_SCREEN_1',
-        ],
+        'pattern': r'STOP_TRIAL',
+        'column': 'trial_id',
+        'value': None,
+    },
+]
+
+EXPECTED_DF = pl.from_dict(
+    {
+        'time': [10000000, 10000002, 10000004, 10000005, 10000008, 10000011, 10000014, 10000017],
+        'x_pix': [850.7, 850.7, 850.7, 850.7, 850.7, 850.7, 850.7, 850.7],
+        'y_pix': [717.5, 717.5, 717.5, 717.5, 717.5, 717.5, 717.5, 717.5],
+        'pupil': [714.0, 714.0, 714.0, 714.0, 714.0, 714.0, 714.0, np.nan],
+        'task': [None, 'A', None, 'B', 'B', 'B', 'B', None],
+        'trial_id': [None, None, None, None, '1', '2', '3', None],
     },
 )
 
 
-@pytest.mark.parametrize(
-    'sync_msg_start_pattern',
-    [None, 'SYNCTIME_READING_SCREEN_'],
-)
-@pytest.mark.parametrize(
-    'sync_msg_stop_pattern',
-    [None, 'STOP'],
-)
-def test_parse_eyelink(tmp_path, sync_msg_start_pattern, sync_msg_stop_pattern):
+def test_parse_eyelink(tmp_path):
     filepath = tmp_path / 'sub.asc'
     filepath.write_text(ASC_TEXT)
 
     df = pm.utils.parsing.parse_eyelink(
         filepath,
-        sync_msg_start_pattern,
-        sync_msg_stop_pattern,
+        patterns=PATTERNS,
     )
 
-    if sync_msg_start_pattern is not None:
-        expected_df = EXPECTED_DF_SYNC
-    else:
-        expected_df = EXPECTED_DF_NO_SYNC
+    assert_frame_equal(df, EXPECTED_DF, check_column_order=False)
+
+
+@pytest.mark.parametrize(
+    'patterns',
+    [
+        [1],
+        [{'pattern': 1}],
+    ],
+)
+def test_parse_eyelink_raises_value_error(tmp_path, patterns):
+    filepath = tmp_path / 'sub.asc'
+    filepath.write_text(ASC_TEXT)
+
+    with pytest.raises(ValueError) as excinfo:
+        pm.utils.parsing.parse_eyelink(
+            filepath,
+            patterns=patterns,
+        )
+
+    msg, = excinfo.value.args
 
-    assert_frame_equal(df, expected_df)
+    expected_substrings = ['invalid pattern', '1']
+    for substring in expected_substrings:
+        assert substring in msg