Repository: aeye-lab/pymovements
Pull request #1009: dataset: add mousecursor dataset by Pantanowitz et al.
Author: login: SiQube
Labels: name: dataset
Created: 2025-03-07T10:42:16Z | Merged: 2025-03-25T17:06:26Z

to download the dataset we need to add a `User-Agent` to the opener.

part of #978 


Diff:
diff --git a/docs/source/bibliography.bib b/docs/source/bibliography.bib
index 01cf434be..6fda5f674 100644
--- a/docs/source/bibliography.bib
+++ b/docs/source/bibliography.bib
@@ -391,3 +391,17 @@ @article{UCL
     issue = {4},
     doi = {10.3758/s13428-012-0313-y},
 }
+
+@article{MouseCursor,
+    title = {Gaze tracking dataset for comparison of smooth and saccadic eye tracking},
+    journal = {Data in Brief},
+    volume = {34},
+    pages = {106730},
+    year = {2021},
+    issn = {2352-3409},
+    doi = {https://doi.org/10.1016/j.dib.2021.106730},
+    url = {https://www.sciencedirect.com/science/article/pii/S2352340921000160},
+    author = {Adam Pantanowitz and Kimoon Kim and Chelsey Chewins and David M Rubin},
+    keywords = {Eye movement, Saccadic eye tracking, Smooth eye tracking, Image processing, Pupil detection, Vestibulo-ocular reflex},
+    abstract = {Pupil tracking data are collected through the use of an infrared camera, and a head-mounted system [1]. The head-mounted system detects the relative pupil position and adjusts the mouse cursor position accordingly. The data are available for comparison of eye tracking with saccadic movements (with the head fixed in space) versus those from smooth movements (with the head moving in space). The analysis comprises two experiments for both types of eye tracking, which are performed with ten trials each for two participants. In the first experiment, the participant attempts to place the cursor into a target boundary of varying sizes. In the second experiment, the participant attempts to move the cursor to a target location within the shortest time.}
+}
diff --git a/docs/source/datasets/public_datasets.csv b/docs/source/datasets/public_datasets.csv
index b4985bdc7..bbc824bcd 100644
--- a/docs/source/datasets/public_datasets.csv
+++ b/docs/source/datasets/public_datasets.csv
@@ -14,6 +14,7 @@ Dataset,Citation,Samples,Events,Reading Measures,# Subjects,# Sessions per Subje
 :py:class:`~pymovements.datasets.HBN`,:cite:p:`HBN`,X,,,1246,1,120 Hz,Video
 :py:class:`~pymovements.datasets.InteRead`,:cite:p:`InteRead`,X,X,,50,1,1200 Hz,Text (documents)
 :py:class:`~pymovements.datasets.JuDo1000`,:cite:p:`JuDo1000`,X,,,150,4,1000 Hz,Random Dots
+:py:class:`~pymovements.datasets.MouseCursor`,:cite:p:`MouseCursor`,X,,,2,1,3 Hz,Mouse Cursor
 :py:class:`~pymovements.datasets.PoTeC`,:cite:p:`PoTeC`,X,,,75,1,1000 Hz,Text (documents)
 :py:class:`~pymovements.datasets.Provo`,:cite:p:`Provo`,,X,,470,1,1000 Hz,Text (documents)
 :py:class:`~pymovements.datasets.SBSAT`,:cite:p:`SB-SAT`,X,X,,95,1,1000 Hz,Text (documents)
diff --git a/src/pymovements/datasets/__init__.py b/src/pymovements/datasets/__init__.py
index 3bb09d1d5..9fc72b288 100644
--- a/src/pymovements/datasets/__init__.py
+++ b/src/pymovements/datasets/__init__.py
@@ -40,6 +40,7 @@
     HBN
     InteRead
     JuDo1000
+    MouseCursor
     PoTeC
     Provo
     SBSAT
@@ -70,6 +71,7 @@
 from pymovements.datasets.hbn import HBN
 from pymovements.datasets.interead import InteRead
 from pymovements.datasets.judo1000 import JuDo1000
+from pymovements.datasets.mousecursor import MouseCursor
 from pymovements.datasets.potec import PoTeC
 from pymovements.datasets.provo import Provo
 from pymovements.datasets.sb_sat import SBSAT
@@ -94,6 +96,7 @@
     'HBN',
     'InteRead',
     'JuDo1000',
+    'MouseCursor',
     'PoTeC',
     'Provo',
     'SBSAT',
diff --git a/src/pymovements/datasets/datasets.yaml b/src/pymovements/datasets/datasets.yaml
index 92a6e94c2..af3467136 100644
--- a/src/pymovements/datasets/datasets.yaml
+++ b/src/pymovements/datasets/datasets.yaml
@@ -13,6 +13,7 @@
 - hbn
 - interead
 - judo1000
+- mousecursor
 - potec
 - provo
 - sb_sat
diff --git a/src/pymovements/datasets/mousecursor.py b/src/pymovements/datasets/mousecursor.py
new file mode 100644
index 000000000..2d771229b
--- /dev/null
+++ b/src/pymovements/datasets/mousecursor.py
@@ -0,0 +1,212 @@
+# Copyright (c) 2022-2025 The pymovements Project Authors
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Provides a definition for the MouseCursor dataset."""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from dataclasses import field
+from typing import Any
+
+import polars as pl
+
+from pymovements.dataset.dataset_definition import DatasetDefinition
+from pymovements.gaze.experiment import Experiment
+
+
+@dataclass
+class MouseCursor(DatasetDefinition):
+    """MouseCursor dataset :cite:p:`MouseCursor`.
+
+    The paper presents a dataset for comparing two types of eye tracking:
+    smooth (vestibulo-ocular reflex or VOR-based) and saccadic eye tracking.
+    Data were collected using a head-mounted infrared camera system that adjusted
+    the mouse cursor based on pupil position.
+    The experiments involved two participants completing tasks to either position the
+    cursor within target areas or move it to a target location as quickly as possible.
+    The dataset is intended to help researchers evaluate and compare eye tracking methods
+    without needing to build the system themselves.
+
+    Attributes
+    ----------
+    name: str
+        The name of the dataset.
+
+    has_files: dict[str, bool]
+        Indicate whether the dataset contains 'gaze', 'precomputed_events', and
+        'precomputed_reading_measures'.
+
+    mirrors: dict[str, list[str]]
+        A tuple of mirrors of the dataset. Each entry must be of type `str` and end with a '/'.
+
+    resources: dict[str, list[dict[str, str]]]
+        A tuple of dataset gaze_resources. Each list entry must be a dictionary with the following
+        keys:
+        - `resource`: The url suffix of the resource. This will be concatenated with the mirror.
+        - `filename`: The filename under which the file is saved as.
+        - `md5`: The MD5 checksum of the respective file.
+
+    extract: dict[str, bool]
+        Decide whether to extract the data.
+
+    experiment: Experiment
+        The experiment definition.
+
+    filename_format: dict[str, str]
+        Regular expression which will be matched before trying to load the file. Namedgroups will
+        appear in the `fileinfo` dataframe.
+
+    filename_format_schema_overrides: dict[str, dict[str, type]]
+        If named groups are present in the `filename_format`, this makes it possible to cast
+        specific named groups to a particular datatype.
+
+    trial_columns: list[str]
+            The name of the trial columns in the input data frame. If the list is empty or None,
+            the input data frame is assumed to contain only one trial. If the list is not empty,
+            the input data frame is assumed to contain multiple trials and the transformation
+            methods will be applied to each trial separately.
+
+    time_column: str
+        The name of the timestamp column in the input data frame. This column will be renamed to
+        ``time``.
+
+    time_unit: str
+        The unit of the timestamps in the timestamp column in the input data frame. Supported
+        units are 's' for seconds, 'ms' for milliseconds and 'step' for steps. If the unit is
+        'step' the experiment definition must be specified. All timestamps will be converted to
+        milliseconds.
+
+    pixel_columns: list[str]
+        The name of the pixel position columns in the input data frame. These columns will be
+        nested into the column ``pixel``. If the list is empty or None, the nested ``pixel``
+        column will not be created.
+
+    column_map: dict[str, str]
+        The keys are the columns to read, the values are the names to which they should be renamed.
+
+    custom_read_kwargs: dict[str, dict[str, Any]]
+        If specified, these keyword arguments will be passed to the file reading function.
+
+
+    Examples
+    --------
+    Initialize your :py:class:`~pymovements.datasets.Dataset` object with the
+    :py:class:`~pymovements.datasets.MouseCursor` definition:
+
+    >>> import pymovements as pm
+    >>>
+    >>> dataset = pm.Dataset("MouseCursor", path='data/MouseCursor')
+
+    Download the dataset resources:
+
+    >>> dataset.download()# doctest: +SKIP
+
+    Load the data into memory:
+
+    >>> dataset.load()# doctest: +SKIP
+    """
+
+    # pylint: disable=similarities
+    # All DatasetDefinition classes potentially share large code chunks.
+
+    name: str = 'MouseCursor'
+
+    has_files: dict[str, bool] = field(
+        default_factory=lambda: {
+            'gaze': True,
+            'precomputed_events': False,
+            'precomputed_reading_measures': False,
+        },
+    )
+    mirrors: dict[str, list[str]] = field(
+        default_factory=lambda: {
+            'gaze': [
+                'https://ars.els-cdn.com/content/image/',
+            ],
+        },
+    )
+
+    resources: dict[str, list[dict[str, str]]] = field(
+        default_factory=lambda: {
+            'gaze': [
+                {
+                    'resource': '1-s2.0-S2352340921000160-mmc1.zip',
+                    'filename': 'mousecursor.zip',
+                    'md5': '7885e8fd44f14f02f60e9f62431aea63',
+                },
+            ],
+        },
+    )
+    extract: dict[str, bool] = field(default_factory=lambda: {'gaze': True})
+
+    experiment: Experiment = field(
+        default_factory=lambda: Experiment(
+            # page 4: middle of screen is 960/540 => 1920/1080
+            screen_width_px=1920,
+            screen_height_px=1080,
+            # page 5: calculation
+            screen_width_cm=52.99,
+            screen_height_cm=29.81,
+            distance_cm=50,
+            origin='upper left',
+            # page 5: 334 ms intervals of tracking gaze (PlayStation-Eye Camera)
+            sampling_rate=3,
+        ),
+    )
+
+    filename_format: dict[str, str] = field(
+        default_factory=lambda: {
+            'gaze': r'Experiment {experiment_id:d}.csv',
+        },
+    )
+
+    filename_format_schema_overrides: dict[str, dict[str, type]] = field(
+        default_factory=lambda: {
+            'gaze': {
+                'experiment_id': int,
+            },
+        },
+    )
+
+    trial_columns: list[str] = field(default_factory=lambda: ['Trial', 'Participant'])
+
+    time_column: str = 'Time'
+
+    time_unit: str = 'ms'
+
+    pixel_columns: list[str] = field(default_factory=lambda: ['x', 'y'])
+
+    column_map: dict[str, str] = field(default_factory=lambda: {})
+
+    custom_read_kwargs: dict[str, dict[str, Any]] = field(
+        default_factory=lambda: {
+            'gaze': {
+                'schema_overrides': {
+                    'Tracking': pl.Utf8,
+                    'Trial': pl.Int64,
+                    'Measurement': pl.Int64,
+                    'ExactTime': pl.Utf8,
+                    'Time': pl.Float32,
+                    'x': pl.Float32,
+                    'y': pl.Float32,
+                    'Participant': pl.Int64,
+                },
+            },
+        },
+    )
diff --git a/src/pymovements/datasets/mousecursor.yaml b/src/pymovements/datasets/mousecursor.yaml
new file mode 100644
index 000000000..ac8776d7a
--- /dev/null
+++ b/src/pymovements/datasets/mousecursor.yaml
@@ -0,0 +1,61 @@
+name: "MouseCursor"
+
+has_files:
+  gaze: true
+  precomputed_events: false
+  precomputed_reading_measures: false
+
+mirrors:
+  gaze:
+    - "https://ars.els-cdn.com/content/image/"
+
+resources:
+  gaze:
+    - resource: "1-s2.0-S2352340921000160-mmc1.zip"
+      filename: "mousecursor.zip"
+      md5: "7885e8fd44f14f02f60e9f62431aea63"
+
+extract:
+  gaze: true
+
+experiment:
+  screen_width_px: 1920
+  screen_height_px: 1080
+  screen_width_cm: 52.99
+  screen_height_cm: 29.81
+  distance_cm: 50
+  origin: "upper left"
+  sampling_rate: 3
+
+filename_format:
+  gaze: 'Experiment {experiment_id:d}.csv'
+
+filename_format_schema_overrides:
+  gaze:
+    experiment_id: !int
+
+trial_columns:
+  - "Trial"
+  - "Participant"
+
+time_column: "Time"
+
+time_unit: "ms"
+
+pixel_columns:
+  - "x"
+  - "y"
+
+column_map: {}
+
+custom_read_kwargs:
+  gaze:
+    schema_overrides:
+      Tracking: !polars.Utf8
+      Trial: !polars.Int64
+      Measurement: !polars.Int64
+      ExactTime: !polars.Utf8
+      Time: !polars.Float32
+      x: !polars.Float32
+      y: !polars.Float32
+      Participant: !polars.Int64
diff --git a/src/pymovements/utils/downloads.py b/src/pymovements/utils/downloads.py
index 63fdecc68..ea19db3e1 100644
--- a/src/pymovements/utils/downloads.py
+++ b/src/pymovements/utils/downloads.py
@@ -194,6 +194,9 @@ def _get_redirected_url(url: str, max_hops: int = 3) -> str:
     """
     initial_url = url
     headers = {'Method': 'HEAD', 'User-Agent': USER_AGENT}
+    opener = urllib.request.build_opener()
+    opener.addheaders = [('User-Agent', USER_AGENT)]
+    urllib.request.install_opener(opener)
 
     for _ in range(max_hops + 1):
         with urllib.request.urlopen(urllib.request.Request(url, headers=headers)) as response:
diff --git a/tests/unit/datasets/datasets_test.py b/tests/unit/datasets/datasets_test.py
index 9e10766ac..e3b8d5530 100644
--- a/tests/unit/datasets/datasets_test.py
+++ b/tests/unit/datasets/datasets_test.py
@@ -44,6 +44,7 @@
         pytest.param(pm.datasets.HBN, 'HBN', id='HBN'),
         pytest.param(pm.datasets.InteRead, 'InteRead', id='InteRead'),
         pytest.param(pm.datasets.JuDo1000, 'JuDo1000', id='JuDo1000'),
+        pytest.param(pm.datasets.MouseCursor, 'MouseCursor', id='MouseCursor'),
         pytest.param(pm.datasets.PoTeC, 'PoTeC', id='PoTeC'),
         pytest.param(pm.datasets.Provo, 'Provo', id='Provo'),
         pytest.param(pm.datasets.SBSAT, 'SBSAT', id='SBSAT'),
@@ -77,6 +78,7 @@ def test_public_dataset_registered(definition, dataset_name):
         pytest.param(pm.datasets.HBN, 'HBN', id='HBN'),
         pytest.param(pm.datasets.InteRead, 'InteRead', id='InteRead'),
         pytest.param(pm.datasets.JuDo1000, 'JuDo1000', id='JuDo1000'),
+        pytest.param(pm.datasets.MouseCursor, 'MouseCursor', id='MouseCursor'),
         pytest.param(pm.datasets.PoTeC, 'PoTeC', id='PoTeC'),
         pytest.param(pm.datasets.Provo, 'Provo', id='Provo'),
         pytest.param(pm.datasets.SBSAT, 'SBSAT', id='SBSAT'),


Reviews:
user: login: dkrako
body: This is a dataset which provides gaze samples but has no screen or sampling rate specified. Can you add this? If there's some more info on the used eyetrackers it would be great if you could fill the respective fields in the eyetracker definition.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-07T18:56:53Z
user: login: dkrako
body: just minor change requests in the csv and maybe add the smooth pursuit info to the docstring instead.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-19T18:56:34Z
user: login: SiQube
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-19T19:19:03Z
user: login: dkrako
body: Great, thanks for the addition!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-25T16:54:33Z

Review comments:
diff_hunk: @@ -14,6 +14,7 @@ Dataset,Citation,Samples,Events,Reading Measures,# Subjects,# Sessions per Subje
 :py:class:`~pymovements.datasets.HBN`,:cite:p:`HBN`,X,,,1246,1,120 Hz,Video
 :py:class:`~pymovements.datasets.InteRead`,:cite:p:`InteRead`,X,X,,50,1,1200 Hz,Text (documents)
 :py:class:`~pymovements.datasets.JuDo1000`,:cite:p:`JuDo1000`,X,,,150,4,1000 Hz,Random Dots
+:py:class:`~pymovements.datasets.JuDo1000`,:cite:p:`MouseCursor`,X,,,2,1,3 Hz, Mouse Cursor (saccadic vs smooth pursuit)
path: docs/source/datasets/public_datasets.csv
user: login: dkrako
body: There's a copypaste error in the first column.

In the last column I would just list it as `Mouse Cursor` and specify the details in the docstring, as the `saccadic vs smooth pursuit` is missing there.
created_at: 2025-03-19T18:55:06Z
author_association: MEMBER
diff_hunk: @@ -14,6 +14,7 @@ Dataset,Citation,Samples,Events,Reading Measures,# Subjects,# Sessions per Subje
 :py:class:`~pymovements.datasets.HBN`,:cite:p:`HBN`,X,,,1246,1,120 Hz,Video
 :py:class:`~pymovements.datasets.InteRead`,:cite:p:`InteRead`,X,X,,50,1,1200 Hz,Text (documents)
 :py:class:`~pymovements.datasets.JuDo1000`,:cite:p:`JuDo1000`,X,,,150,4,1000 Hz,Random Dots
+:py:class:`~pymovements.datasets.JuDo1000`,:cite:p:`MouseCursor`,X,,,2,1,3 Hz, Mouse Cursor (saccadic vs smooth pursuit)
path: docs/source/datasets/public_datasets.csv
user: login: SiQube
body: done
created_at: 2025-03-19T19:19:03Z
author_association: CONTRIBUTOR

PR comments:
user: login: SiQube
created_at: 2025-03-07T10:42:31Z
body: I'll adjust if #914 is merged first
author_association: CONTRIBUTOR
user: login: codecov[bot]
created_at: 2025-03-07T10:46:54Z
body: ## [Codecov](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1009?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`35f6b88`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/35f6b88e333f05d68afff03b4ede243b489c283a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) to head [(`d86ef3f`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/d86ef3fc789175f5c7da1a5403a5a2e805373a28?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).
> Report is 86 commits behind head on main.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main     #1009   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           80        81    +1     
  Lines         3602      3629   +27     
  Branches       646       646           
=========================================
+ Hits          3602      3629   +27     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1009?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE
user: login: SiQube
created_at: 2025-03-19T18:22:53Z
body: @dkrako 
author_association: CONTRIBUTOR