Repository: aeye-lab/pymovements
Pull request #862 (2024-10-23 08:03:09)

dataset: add InteRead dataset
add [InteRead](https://aclanthology.org/2024.lrec-main.802/) dataset

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/aeye-lab/pymovements/pull/862?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`a8caf95`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/a8caf9505a715e2b797cad4cb1de5fabf02e2db8?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) to head [(`91e6601`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/91e66010f3bc3fe350fb513b2f8048cb88d1e9fc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main      #862   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           72        73    +1     
  Lines         3321      3346   +25     
  Branches       594       594           
=========================================
+ Hits          3321      3346   +25     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/aeye-lab/pymovements/pull/862?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).


Patch:
diff --git a/docs/source/bibliography.bib b/docs/source/bibliography.bib
index 01d32dc..f223262 100644
--- a/docs/source/bibliography.bib
+++ b/docs/source/bibliography.bib
@@ -148,6 +148,30 @@ keywords = {code comprehension, code-fixation attention, eye-tracking, lab exper
     doi = {10.1038/s41597-021-00959-y},
 }
 
+@inproceedings{InteRead,
+    title = "{I}nte{R}ead: An Eye Tracking Dataset of Interrupted Reading",
+    author = {Zermiani, Francesca  and
+      Dhar, Prajit  and
+      Sood, Ekta  and
+      K{\"o}gel, Fabian  and
+      Bulling, Andreas  and
+      Wirzberger, Maria},
+    editor = "Calzolari, Nicoletta  and
+      Kan, Min-Yen  and
+      Hoste, Veronique  and
+      Lenci, Alessandro  and
+      Sakti, Sakriani  and
+      Xue, Nianwen",
+    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
+    month = may,
+    year = "2024",
+    address = "Torino, Italia",
+    publisher = "ELRA and ICCL",
+    url = "https://aclanthology.org/2024.lrec-main.802",
+    pages = "9154--9169",
+    abstract = "Eye movements during reading offer a window into cognitive processes and language comprehension, but the scarcity of reading data with interruptions {--} which learners frequently encounter in their everyday learning environments {--} hampers advances in the development of intelligent learning technologies. We introduce InteRead {--} a novel 50-participant dataset of gaze data recorded during self-paced reading of real-world text. InteRead further offers fine-grained annotations of interruptions interspersed throughout the text as well as resumption lags incurred by these interruptions. Interruptions were triggered automatically once readers reached predefined target words. We validate our dataset by reporting interdisciplinary analyses on different measures of gaze behavior. In line with prior research, our analyses show that the interruptions as well as word length and word frequency effects significantly impact eye movements during reading. We also explore individual differences within our dataset, shedding light on the potential for tailored educational solutions. InteRead is accessible from our datasets web-page: https://www.ife.uni-stuttgart.de/en/llis/research/datasets/.",
+ }
+
 @misc{JuDo1000,
     author = {Makowski, Silvia and Jäger, Lena A. and Prasse, Paul and Scheffer, Tobias},
     title = {{JuDo1000} Eye Tracking Data Set},
diff --git a/src/pymovements/datasets/__init__.py b/src/pymovements/datasets/__init__.py
index e98f29c..bd02aed 100644
--- a/src/pymovements/datasets/__init__.py
+++ b/src/pymovements/datasets/__init__.py
@@ -35,6 +35,7 @@
     pymovements.datasets.GazeGraph
     pymovements.datasets.GazeOnFaces
     pymovements.datasets.HBN
+    pymovements.datasets.InteRead
     pymovements.datasets.JuDo1000
     pymovements.datasets.PoTeC
     pymovements.datasets.SBSAT
@@ -60,6 +61,7 @@ from pymovements.datasets.gaze_on_faces import GazeOnFaces
 from pymovements.datasets.gazebase import GazeBase
 from pymovements.datasets.gazebasevr import GazeBaseVR
 from pymovements.datasets.hbn import HBN
+from pymovements.datasets.interead import InteRead
 from pymovements.datasets.judo1000 import JuDo1000
 from pymovements.datasets.potec import PoTeC
 from pymovements.datasets.sb_sat import SBSAT
@@ -79,6 +81,7 @@ __all__ = [
     'GazeGraph',
     'GazeOnFaces',
     'HBN',
+    'InteRead',
     'JuDo1000',
     'PoTeC',
     'SBSAT',
diff --git a/src/pymovements/datasets/interead.py b/src/pymovements/datasets/interead.py
new file mode 100644
index 0000000..27d3a94
--- /dev/null
+++ b/src/pymovements/datasets/interead.py
@@ -0,0 +1,222 @@
+# Copyright (c) 2022-2024 The pymovements Project Authors
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Provides a definition for the InteRead dataset."""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from dataclasses import field
+from typing import Any
+
+from pymovements.dataset.dataset_definition import DatasetDefinition
+from pymovements.dataset.dataset_library import register_dataset
+from pymovements.gaze.experiment import Experiment
+
+
+@dataclass
+@register_dataset
+class InteRead(DatasetDefinition):
+    """InteRead dataset :cite:p:`InteRead`.
+
+    This dataset includes monocular eye tracking data in an interrupted reading task.
+    Automatic interruption occured during a reading task and participants continued
+    reading after the pause.
+    Eye movements are recorded at a sampling frequency of 1200Hz with video-based eye tracker.
+    Provided data are raw gaze samples and precomputed event files both in pixel coordinates.
+
+    For more details, check the paper :cite:p:`InteRead`.
+
+    Attributes
+    ----------
+    name: str
+        The name of the dataset.
+
+    has_files: dict[str, bool]
+        Indicate whether the dataset contains 'gaze', 'precomputed_events', and
+        'precomputed_reading_measures'.
+
+    mirrors: dict[str, tuple[str, ...]]
+        A tuple of mirrors of the dataset. Each entry must be of type `str` and end with a '/'.
+
+    resources: dict[str, tuple[dict[str, str], ...]]
+        A tuple of dataset gaze_resources. Each list entry must be a dictionary with the following
+        keys:
+        - `resource`: The url suffix of the resource. This will be concatenated with the mirror.
+        - `filename`: The filename under which the file is saved as.
+        - `md5`: The MD5 checksum of the respective file.
+
+    extract: dict[str, bool]
+        Decide whether to extract the data.
+
+    experiment: Experiment
+        The experiment definition.
+
+    filename_format: dict[str, str]
+        Regular expression which will be matched before trying to load the file. Namedgroups will
+        appear in the `fileinfo` dataframe.
+
+    filename_format_schema_overrides: dict[str, dict[str, type]]
+        If named groups are present in the `filename_format`, this makes it possible to cast
+        specific named groups to a particular datatype.
+
+    trial_columns: list[str]
+            The name of the trial columns in the input data frame. If the list is empty or None,
+            the input data frame is assumed to contain only one trial. If the list is not empty,
+            the input data frame is assumed to contain multiple trials and the transformation
+            methods will be applied to each trial separately.
+
+    time_column: str
+        The name of the timestamp column in the input data frame. This column will be renamed to
+        ``time``.
+
+    time_unit: str
+        The unit of the timestamps in the timestamp column in the input data frame. Supported
+        units are 's' for seconds, 'ms' for milliseconds and 'step' for steps. If the unit is
+        'step' the experiment definition must be specified. All timestamps will be converted to
+        milliseconds.
+
+    pixel_columns: list[str]
+        The name of the pixel position columns in the input data frame. These columns will be
+        nested into the column ``pixel``. If the list is empty or None, the nested ``pixel``
+        column will not be created.
+
+    column_map: dict[str, str]
+        The keys are the columns to read, the values are the names to which they should be renamed.
+
+    custom_read_kwargs: dict[str, dict[str, Any]]
+        If specified, these keyword arguments will be passed to the file reading function.
+
+
+    Examples
+    --------
+    Initialize your :py:class:`~pymovements.PublicDataset` object with the
+    :py:class:`~pymovements.InteRead` definition:
+
+    >>> import pymovements as pm
+    >>>
+    >>> dataset = pm.Dataset("InteRead", path='data/InteRead')
+
+    Download the dataset resources:
+
+    >>> dataset.download()# doctest: +SKIP
+
+    Load the data into memory:
+
+    >>> dataset.load()# doctest: +SKIP
+    """
+
+    # pylint: disable=similarities
+    # The PublicDatasetDefinition child classes potentially share code chunks for definitions.
+
+    name: str = 'InteRead'
+
+    has_files: dict[str, bool] = field(
+        default_factory=lambda: {
+            'gaze': True,
+            'precomputed_events': True,
+            'precomputed_reading_measures': False,
+        },
+    )
+    mirrors: dict[str, tuple[str, ...]] = field(
+        default_factory=lambda: {
+            'gaze': (
+                'https://osf.io/download/',
+            ),
+            'precomputed_events': (
+                'https://osf.io/download/',
+            ),
+        },
+    )
+
+    resources: dict[str, tuple[dict[str, str], ...]] = field(
+        default_factory=lambda: {
+            'gaze': (
+                {
+                    'resource': '6ju3x/',
+                    'filename': 'resampled_gaze.csv.zip',
+                    'md5': '06b2cdff1827086fa125a703ee9d4324',
+                },
+            ),
+            'precomputed_events': (
+                {
+                    'resource': '85ckh/',
+                    'filename': 'resumption_fixation.csv',
+                    'md5': '44edb7c58318ad76af1fa6f1bc1f1ceb',
+                },
+            ),
+        },
+    )
+    extract: dict[str, bool] = field(
+        default_factory=lambda: {
+            'gaze': True,
+            'precomputed_events': False,
+        },
+    )
+
+    experiment: Experiment = Experiment(
+        screen_width_px=1920,
+        screen_height_px=1080,
+        screen_width_cm=52.8,
+        screen_height_cm=29.7,
+        distance_cm=57,
+        origin='center',
+        sampling_rate=1200,
+    )
+
+    filename_format: dict[str, str] = field(
+        default_factory=lambda: {
+            'gaze': r'resampled_gaze.csv',
+            'precomputed_events': r'resumption_fixation.csv',
+        },
+    )
+
+    filename_format_schema_overrides: dict[str, dict[str, type]] = field(
+        default_factory=lambda: {
+            'gaze': {},
+            'precomputed_events': {},
+        },
+    )
+
+    trial_columns: list[str] = field(
+        default_factory=lambda: [
+            'participant_id',
+            'page_id',
+            'interruption_state',
+        ],
+    )
+
+    time_column: str = ''
+
+    time_unit: str = 'ms'
+
+    pixel_columns: list[str] = field(
+        default_factory=lambda: [
+            'x',
+            'y',
+        ],
+    )
+
+    column_map: dict[str, str] = field(default_factory=lambda: {})
+
+    custom_read_kwargs: dict[str, dict[str, Any]] = field(
+        default_factory=lambda: {
+            'gaze': {},
+            'precomputed_events': {},
+        },
+    )


Test patch:
diff --git a/tests/unit/datasets/datasets_test.py b/tests/unit/datasets/datasets_test.py
index 6c6bc30..35fd354 100644
--- a/tests/unit/datasets/datasets_test.py
+++ b/tests/unit/datasets/datasets_test.py
@@ -41,6 +41,7 @@ import pymovements as pm
         pytest.param(pm.datasets.GazeBaseVR, 'GazeBaseVR', id='GazeBaseVR'),
         pytest.param(pm.datasets.GazeOnFaces, 'GazeOnFaces', id='GazeOnFaces'),
         pytest.param(pm.datasets.HBN, 'HBN', id='HBN'),
+        pytest.param(pm.datasets.InteRead, 'InteRead', id='InteRead'),
         pytest.param(pm.datasets.JuDo1000, 'JuDo1000', id='JuDo1000'),
         pytest.param(pm.datasets.PoTeC, 'PoTeC', id='PoTeC'),
         pytest.param(pm.datasets.SBSAT, 'SBSAT', id='SBSAT'),