Repository: aeye-lab/pymovements (python)

Issue:
Metadata Documentation
## Description of the problem
Currently, the metadata structure we receive after recording is poorly organized, making it difficult to interpret the various pieces of information.


## Description of a solution
We aim to document this information in a clear, structured format to ensure it is easy to read and understand 

Include steps of the proposed solution.
 -Create a class that will organize the metadata in a more structured and meticulous manner

What are the expected benefits/improvements from working on this new feature?
This feature will centralize metadata information, making it easier to access and manage.


## Sample Code

```py
>>> gazeframe, metadata = from_asc(...)
>>> metadata.tracked_eye
'right'
>>> metadata.calibrations
[...]
```

The metadata could also be part of the gazeframe (gazeframe.metadata).

Pull request:
feat: Store metadata from ASC in experiment metadata
## Description

Put the metadata parsed in `parse_eyelink()` into the `Experiment`, `Screen`, and `EyeTracker` classes, and check for mismatches compared to user-provided metadata. Create additional attributes for metadata that is not exposed yet (e.g., calibrations and validations).

Fixes #868.

Related to #875.

## Implemented changes

- [x] Store metadata from ASC file in corresponding attributes of `Experiment` object
- [x] Raise error if there are conflicts with existing values in `Experiment`
- [x] Store remaining metadata in private attribute `GazeDataFrame._metadata` to revert breaking change introduced in #767

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change is or requires a documentation update

## How Has This Been Tested?

Please describe the tests that you ran to verify your changes.
Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

- [x] `test_from_asc_fills_in_experiment_metadata`
- [x] `test_from_asc_detects_mismatches_in_experiment_metadata`

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream modules
- [x] I have checked my code and corrected any misspellings


Patch:
diff --git a/src/pymovements/__init__.py b/src/pymovements/__init__.py
index 8cf4f93..cb204c4 100644
--- a/src/pymovements/__init__.py
+++ b/src/pymovements/__init__.py
@@ -38,6 +38,7 @@ from pymovements.events import EventDataFrame
 from pymovements.events import EventGazeProcessor
 from pymovements.events import EventProcessor
 from pymovements.gaze import Experiment
+from pymovements.gaze import EyeTracker
 from pymovements.gaze import GazeDataFrame
 from pymovements.gaze import Screen
 from pymovements.measure import register_sample_measure
@@ -60,6 +61,7 @@ __all__ = [
 
     'gaze',
     'Experiment',
+    'EyeTracker',
     'Screen',
     'GazeDataFrame',
 
diff --git a/src/pymovements/dataset/dataset_files.py b/src/pymovements/dataset/dataset_files.py
index 1553fdd..dfc4c40 100644
--- a/src/pymovements/dataset/dataset_files.py
+++ b/src/pymovements/dataset/dataset_files.py
@@ -377,7 +377,7 @@ def load_gaze_file(
             column_schema_overrides=definition.filename_format_schema_overrides['gaze'],
         )
     elif filepath.suffix == '.asc':
-        gaze_df, _ = from_asc(
+        gaze_df = from_asc(
             filepath,
             experiment=definition.experiment,
             add_columns=add_columns,
diff --git a/src/pymovements/datasets/toy_dataset_eyelink.py b/src/pymovements/datasets/toy_dataset_eyelink.py
index 4d70501..488fb5b 100644
--- a/src/pymovements/datasets/toy_dataset_eyelink.py
+++ b/src/pymovements/datasets/toy_dataset_eyelink.py
@@ -166,8 +166,8 @@ class ToyDatasetEyeLink(DatasetDefinition):
         origin='upper left',
         eyetracker=EyeTracker(
             sampling_rate=1000.0,
-            left=False,
-            right=True,
+            left=True,
+            right=False,
             model='EyeLink Portable Duo',
             vendor='EyeLink',
         ),
diff --git a/src/pymovements/gaze/gaze_dataframe.py b/src/pymovements/gaze/gaze_dataframe.py
index 10f3e8b..7d81c96 100644
--- a/src/pymovements/gaze/gaze_dataframe.py
+++ b/src/pymovements/gaze/gaze_dataframe.py
@@ -283,6 +283,9 @@ class GazeDataFrame:
         else:
             self.events = events.copy()
 
+        # Remove this attribute once #893 is fixed
+        self._metadata: dict[str, Any] | None = None
+
     def apply(
             self,
             function: str,
diff --git a/src/pymovements/gaze/io.py b/src/pymovements/gaze/io.py
index 18868e3..e68d46f 100644
--- a/src/pymovements/gaze/io.py
+++ b/src/pymovements/gaze/io.py
@@ -25,7 +25,8 @@ from typing import Any
 
 import polars as pl
 
-from pymovements.gaze import Experiment  # pylint: disable=cyclic-import
+from pymovements.gaze.experiment import Experiment
+from pymovements.gaze.eyetracker import EyeTracker
 from pymovements.gaze.gaze_dataframe import GazeDataFrame  # pylint: disable=cyclic-import
 from pymovements.utils.parsing import parse_eyelink
 
@@ -277,7 +278,7 @@ def from_asc(
         experiment: Experiment | None = None,
         add_columns: dict[str, str] | None = None,
         column_schema_overrides: dict[str, Any] | None = None,
-) -> tuple[GazeDataFrame, dict[str, Any]]:
+) -> GazeDataFrame:
     """Initialize a :py:class:`pymovements.gaze.gaze_dataframe.GazeDataFrame`.
 
     Parameters
@@ -303,8 +304,8 @@ def from_asc(
 
     Returns
     -------
-    tuple[GazeDataFrame, dict[str, Any]]
-        The gaze data frame and a metadata dictionary read from the asc file.
+    GazeDataFrame
+        The gaze data frame read from the asc file.
 
     Examples
     --------
@@ -312,7 +313,7 @@ def from_asc(
     We can then load the data into a ``GazeDataFrame``:
 
     >>> from pymovements.gaze.io import from_asc
-    >>> gaze, metadata = from_asc(file='tests/files/eyelink_monocular_example.asc')
+    >>> gaze = from_asc(file='tests/files/eyelink_monocular_example.asc')
     >>> gaze.frame
     shape: (16, 3)
     ┌─────────┬───────┬────────────────┐
@@ -332,7 +333,7 @@ def from_asc(
     │ 2339290 ┆ 618.0 ┆ [637.6, 531.4] │
     │ 2339291 ┆ 618.0 ┆ [637.3, 531.2] │
     └─────────┴───────┴────────────────┘
-    >>> metadata['sampling_rate']
+    >>> gaze.experiment.eyetracker.sampling_rate
     1000.0
     """
     if isinstance(patterns, str):
@@ -360,6 +361,75 @@ def from_asc(
             for fileinfo_key, fileinfo_dtype in column_schema_overrides.items()
         ])
 
+    if experiment is None:
+        experiment = Experiment(sampling_rate=metadata['sampling_rate'])
+    if experiment.eyetracker is None:
+        experiment.eyetracker = EyeTracker()
+
+    # Compare metadata from experiment definition with metadata from ASC file.
+    # Fill in missing metadata in experiment definition and raise an error if there are conflicts
+    issues = []
+
+    # Screen resolution (assuming that width and height will always be missing or set together)
+    experiment_resolution = (experiment.screen.width_px, experiment.screen.height_px)
+    if experiment_resolution == (None, None):
+        experiment.screen.width_px, experiment.screen.height_px = metadata['resolution']
+    elif experiment_resolution != metadata['resolution']:
+        issues.append(f"Screen resolution: {experiment_resolution} vs. {metadata['resolution']}")
+
+    # Sampling rate
+    if experiment.eyetracker.sampling_rate is None:
+        experiment.eyetracker.sampling_rate = metadata['sampling_rate']
+    elif experiment.eyetracker.sampling_rate != metadata['sampling_rate']:
+        issues.append(
+            f"Sampling rate: {experiment.eyetracker.sampling_rate} vs. {metadata['sampling_rate']}",
+        )
+
+    # Tracked eye
+    asc_left_eye = 'L' in metadata['tracked_eye']
+    asc_right_eye = 'R' in metadata['tracked_eye']
+    if experiment.eyetracker.left is None:
+        experiment.eyetracker.left = asc_left_eye
+    elif experiment.eyetracker.left != asc_left_eye:
+        issues.append(f"Left eye tracked: {experiment.eyetracker.left} vs. {asc_left_eye}")
+    if experiment.eyetracker.right is None:
+        experiment.eyetracker.right = asc_right_eye
+    elif experiment.eyetracker.right != asc_right_eye:
+        issues.append(f"Right eye tracked: {experiment.eyetracker.right} vs. {asc_right_eye}")
+
+    # Mount configuration
+    if experiment.eyetracker.mount is None:
+        experiment.eyetracker.mount = metadata['mount_configuration']['mount_type']
+    elif experiment.eyetracker.mount != metadata['mount_configuration']['mount_type']:
+        issues.append(f"Mount configuration: {experiment.eyetracker.mount} vs. "
+                      f"{metadata['mount_configuration']['mount_type']}")
+
+    # Eye tracker vendor
+    asc_vendor = 'EyeLink' if 'EyeLink' in metadata['model'] else None
+    if experiment.eyetracker.vendor is None:
+        experiment.eyetracker.vendor = asc_vendor
+    elif experiment.eyetracker.vendor != asc_vendor:
+        issues.append(f"Eye tracker vendor: {experiment.eyetracker.vendor} vs. {asc_vendor}")
+
+    # Eye tracker model
+    if experiment.eyetracker.model is None:
+        experiment.eyetracker.model = metadata['model']
+    elif experiment.eyetracker.model != metadata['model']:
+        issues.append(f"Eye tracker model: {experiment.eyetracker.model} vs. {metadata['model']}")
+
+    # Eye tracker software version
+    if experiment.eyetracker.version is None:
+        experiment.eyetracker.version = metadata['version_number']
+    elif experiment.eyetracker.version != metadata['version_number']:
+        issues.append(f"Eye tracker software version: {experiment.eyetracker.version} vs. "
+                      f"{metadata['version_number']}")
+
+    if issues:
+        raise ValueError(
+            'Experiment metadata does not match the metadata in the ASC file:\n'
+            + '\n'.join(f'- {issue}' for issue in issues),
+        )
+
     # Create gaze data frame.
     gaze_df = GazeDataFrame(
         gaze_data,
@@ -368,7 +438,8 @@ def from_asc(
         time_unit='ms',
         pixel_columns=['x_pix', 'y_pix'],
     )
-    return gaze_df, metadata
+    gaze_df._metadata = metadata  # pylint: disable=protected-access
+    return gaze_df
 
 
 def from_ipc(


Test patch:
diff --git a/tests/functional/gaze_file_processing_test.py b/tests/functional/gaze_file_processing_test.py
index 7a0e844..88e57c5 100644
--- a/tests/functional/gaze_file_processing_test.py
+++ b/tests/functional/gaze_file_processing_test.py
@@ -71,7 +71,7 @@ def fixture_gaze_init_kwargs(request):
         },
         'eyelink_monocular': {
             'file': 'tests/files/eyelink_monocular_example.asc',
-            'experiment': pm.datasets.ToyDatasetEyeLink().experiment,
+            'experiment': pm.Experiment(1280, 1024, 38, 30, 60, 'upper left', 1000),
         },
         'didec': {
             'file': 'tests/files/didec_example.txt',
@@ -157,7 +157,7 @@ def test_gaze_file_processing(gaze_from_kwargs):
     elif file_extension in {'.feather', '.ipc'}:
         gaze = pm.gaze.from_ipc(**gaze_from_kwargs)
     elif file_extension == '.asc':
-        gaze, _ = pm.gaze.from_asc(**gaze_from_kwargs)
+        gaze = pm.gaze.from_asc(**gaze_from_kwargs)
 
     assert gaze is not None
 
diff --git a/tests/unit/dataset/dataset_files_test.py b/tests/unit/dataset/dataset_files_test.py
index 80af569..61101ea 100644
--- a/tests/unit/dataset/dataset_files_test.py
+++ b/tests/unit/dataset/dataset_files_test.py
@@ -206,7 +206,7 @@ def test_load_eyelink_file(tmp_path, read_kwargs):
         filepath,
         fileinfo_row={},
         definition=DatasetDefinition(
-            experiment=pm.Experiment(1024, 768, 38, 30, None, 'center', 100),
+            experiment=pm.Experiment(1280, 1024, 38, 30, None, 'center', 100),
             filename_format_schema_overrides={'gaze': {}, 'precomputed_events': {}},
         ),
         custom_read_kwargs=read_kwargs,
diff --git a/tests/unit/gaze/io/asc_test.py b/tests/unit/gaze/io/asc_test.py
index 40afb2b..7c2af7f 100644
--- a/tests/unit/gaze/io/asc_test.py
+++ b/tests/unit/gaze/io/asc_test.py
@@ -131,64 +131,128 @@ import pymovements as pm
     ],
 )
 def test_from_asc_has_shape_and_schema(kwargs, expected_frame):
-    gaze, _ = pm.gaze.from_asc(**kwargs)
+    gaze = pm.gaze.from_asc(**kwargs)
 
     assert_frame_equal(gaze.frame, expected_frame, check_column_order=False)
 
 
 @pytest.mark.parametrize(
-    ('kwargs', 'expected_metadata'),
+    ('kwargs', 'exception', 'message'),
     [
         pytest.param(
             {
                 'file': 'tests/files/eyelink_monocular_example.asc',
-                'metadata_patterns': [
-                    {'pattern': r'!V TRIAL_VAR SUBJECT_ID (?P<subject_id>-?\d+)'},
-                    r'!V TRIAL_VAR STIMULUS_COMBINATION_ID (?P<stimulus_combination_id>.+)',
-                ],
+                'patterns': 'foobar',
             },
+            ValueError,
+            "unknown pattern key 'foobar'. Supported keys are: eyelink",
+            id='unknown_pattern',
+        ),
+    ],
+)
+def test_from_asc_raises_exception(kwargs, exception, message):
+    with pytest.raises(exception) as excinfo:
+        pm.gaze.from_asc(**kwargs)
+
+    msg, = excinfo.value.args
+    assert msg == message
+
+
+@pytest.mark.parametrize(
+    ('file', 'sampling_rate'),
+    [
+        pytest.param('tests/files/eyelink_monocular_example.asc', 1000.0, id='1khz'),
+        pytest.param('tests/files/eyelink_monocular_2khz_example.asc', 2000.0, id='2khz'),
+    ],
+)
+def test_from_asc_fills_in_experiment_metadata(file, sampling_rate):
+    gaze = pm.gaze.from_asc(file, experiment=None)
+    assert gaze.experiment.screen.width_px == 1280
+    assert gaze.experiment.screen.height_px == 1024
+    assert gaze.experiment.eyetracker.sampling_rate == sampling_rate
+    assert gaze.experiment.eyetracker.left is True
+    assert gaze.experiment.eyetracker.right is False
+    assert gaze.experiment.eyetracker.model == 'EyeLink Portable Duo'
+    assert gaze.experiment.eyetracker.version == '6.12'
+    assert gaze.experiment.eyetracker.vendor == 'EyeLink'
+    assert gaze.experiment.eyetracker.mount == 'Desktop'
+
+
+@pytest.mark.parametrize(
+    ('experiment_kwargs', 'issues'),
+    [
+        pytest.param(
             {
-                'subject_id': '-1',
-                'stimulus_combination_id': 'start',
+                'screen_width_px': 1920,
+                'screen_height_px': 1080,
+                'sampling_rate': 1000,
             },
-            id='eyelink_asc_metadata_patterns',
+            ['Screen resolution: (1920, 1080) vs. (1280, 1024)'],
+            id='screen_resolution',
         ),
         pytest.param(
             {
-                'file': 'tests/files/eyelink_monocular_example.asc',
-                'metadata_patterns': [r'inexistent pattern (?P<value>-?\d+)'],
+                'eyetracker': pm.EyeTracker(sampling_rate=500),
             },
+            ['Sampling rate: 500 vs. 1000.0'],
+            id='eyetracker_sampling_rate',
+        ),
+        pytest.param(
             {
-                'value': None,
+                'eyetracker': pm.EyeTracker(
+                    left=False,
+                    right=True,
+                    sampling_rate=1000,
+                    mount='Desktop',
+                ),
             },
-            id='eyelink_asc_metadata_pattern_not_found',
+            [
+                'Left eye tracked: False vs. True',
+                'Right eye tracked: True vs. False',
+            ],
+            id='eyetracker_tracked_eye',
         ),
-    ],
-)
-def test_from_asc_metadata_patterns(kwargs, expected_metadata):
-    _, metadata = pm.gaze.from_asc(**kwargs)
-
-    for key, value in expected_metadata.items():
-        assert metadata[key] == value
-
-
-@pytest.mark.parametrize(
-    ('kwargs', 'exception', 'message'),
-    [
         pytest.param(
             {
-                'file': 'tests/files/eyelink_monocular_example.asc',
-                'patterns': 'foobar',
+                'eyetracker': pm.EyeTracker(
+                    vendor='Tobii',
+                    model='Tobii Pro Spectrum',
+                    version='1.0',
+                    sampling_rate=1000,
+                    left=True,
+                    right=False,
+                ),
             },
-            ValueError,
-            "unknown pattern key 'foobar'. Supported keys are: eyelink",
-            id='unknown_pattern',
+            [
+                'Eye tracker vendor: Tobii vs. EyeLink',
+                'Eye tracker model: Tobii Pro Spectrum vs. EyeLink Portable Duo',
+                'Eye tracker software version: 1.0 vs. 6.12',
+            ],
+            id='eyetracker_vendor_model_version',
+        ),
+        pytest.param(
+            {
+                'eyetracker': pm.EyeTracker(
+                    mount='Remote',
+                    sampling_rate=1000,
+                    vendor='EyeLink',
+                    model='EyeLink Portable Duo',
+                    version='6.12',
+                ),
+            },
+            ['Mount configuration: Remote vs. Desktop'],
+            id='eyetracker_mount',
         ),
     ],
 )
-def test_from_asc_raises_exception(kwargs, exception, message):
-    with pytest.raises(exception) as excinfo:
-        pm.gaze.from_asc(**kwargs)
+def test_from_asc_detects_mismatches_in_experiment_metadata(experiment_kwargs, issues):
+    with pytest.raises(ValueError) as excinfo:
+        pm.gaze.from_asc(
+            'tests/files/eyelink_monocular_example.asc',
+            experiment=pm.Experiment(**experiment_kwargs),
+        )
 
     msg, = excinfo.value.args
-    assert msg == message
+    expected_msg = 'Experiment metadata does not match the metadata in the ASC file:\n'
+    expected_msg += '\n'.join(f'- {issue}' for issue in issues)
+    assert msg == expected_msg
diff --git a/tests/unit/utils/parsing_test.py b/tests/unit/utils/parsing_test.py
index f33e89f..13dc2dd 100644
--- a/tests/unit/utils/parsing_test.py
+++ b/tests/unit/utils/parsing_test.py
@@ -206,6 +206,42 @@ def test_parse_eyelink(tmp_path):
     assert metadata == EXPECTED_METADATA
 
 
+@pytest.mark.parametrize(
+    ('kwargs', 'expected_metadata'),
+    [
+        pytest.param(
+            {
+                'filepath': 'tests/files/eyelink_monocular_example.asc',
+                'metadata_patterns': [
+                    {'pattern': r'!V TRIAL_VAR SUBJECT_ID (?P<subject_id>-?\d+)'},
+                    r'!V TRIAL_VAR STIMULUS_COMBINATION_ID (?P<stimulus_combination_id>.+)',
+                ],
+            },
+            {
+                'subject_id': '-1',
+                'stimulus_combination_id': 'start',
+            },
+            id='eyelink_asc_metadata_patterns',
+        ),
+        pytest.param(
+            {
+                'filepath': 'tests/files/eyelink_monocular_example.asc',
+                'metadata_patterns': [r'inexistent pattern (?P<value>-?\d+)'],
+            },
+            {
+                'value': None,
+            },
+            id='eyelink_asc_metadata_pattern_not_found',
+        ),
+    ],
+)
+def test_from_asc_metadata_patterns(kwargs, expected_metadata):
+    _, metadata = pm.utils.parsing.parse_eyelink(**kwargs)
+
+    for key, value in expected_metadata.items():
+        assert metadata[key] == value
+
+
 @pytest.mark.parametrize(
     'patterns',
     [