Repository: aeye-lab/pymovements
Pull request #187 (2023-02-22 12:47:55)

Events: Fix error on empty event list
## Description

All event detection methods raised an index error on generating the event dataframe when there were no events detected.

## Implemented changes

Insert a description of the changes implemented in the pull request.

For each event detection method: 
- [x] Added check for empty event list and create empty dataframe with correct schema if empty

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

- [x] microsaccades(): Test constant position leads to empty event dataframe with correct schema
- [x] ivt(): Test constantly changing position leads to empty event dataframe with correct schema
- [x] idt(): Test constantly changing position leads to empty event dataframe with correct schema. This test produces unexpected results and will be handled in issue #186

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream modules
- [x] I have checked my code and corrected any misspellings


codecov-commenter: # [Codecov](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
Base: **59.30**% // Head: **59.87**% // Increases project coverage by **`+0.56%`** :tada:
> Coverage data is based on head [(`9b41898`)](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) compared to base [(`547861a`)](https://codecov.io/gh/aeye-lab/pymovements/commit/547861a36f5615d6f264d43d78a41b89e523c8ff?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).
> Patch coverage: 85.71% of modified lines in pull request are covered.

> :exclamation: Current head 9b41898 differs from pull request most recent head 8a7e88c. Consider uploading reports for the commit 8a7e88c to get more accurate results

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #187      +/-   ##
==========================================
+ Coverage   59.30%   59.87%   +0.56%     
==========================================
  Files          21       21              
  Lines         779      790      +11     
  Branches      180      183       +3     
==========================================
+ Hits          462      473      +11     
  Misses        305      305              
  Partials       12       12              
```


| [Impacted Files](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) | Coverage Δ | |
|---|---|---|
| [src/pymovements/events/idt.py](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9pZHQucHk=) | `94.11% <50.00%> (+0.56%)` | :arrow_up: |
| [src/pymovements/events/engbert.py](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9lbmdiZXJ0LnB5) | `95.55% <100.00%> (+0.31%)` | :arrow_up: |
| [src/pymovements/events/events.py](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9ldmVudHMucHk=) | `100.00% <100.00%> (ø)` | |
| [src/pymovements/events/ivt.py](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9pdnQucHk=) | `100.00% <100.00%> (ø)` | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/aeye-lab/pymovements/pull/187?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).


Patch:
diff --git a/src/pymovements/events/engbert.py b/src/pymovements/events/engbert.py
index 6b13fff..c28eba7 100644
--- a/src/pymovements/events/engbert.py
+++ b/src/pymovements/events/engbert.py
@@ -27,6 +27,7 @@ from collections.abc import Sized
 import numpy as np
 import polars as pl
 
+from pymovements.events.events import Saccade
 from pymovements.transforms import consecutive
 from pymovements.utils.checks import check_shapes_positions_velocities
 
@@ -116,14 +117,21 @@ def microsaccades(
         for candidate_indices in candidates
     ])
 
-    # Create event dataframe from saccade onset and offsets.
-    event_df = pl.from_dict(
-        {
-            'type': 'saccade',
-            'onset': saccades[:, 0].tolist(),
-            'offset': saccades[:, 1].tolist(),
-        },
-    )
+    if len(saccades) > 0:
+        # Create event dataframe.
+        event_df = pl.from_dict(
+            {
+                'type': 'saccade',
+                'onset': saccades[:, 0].tolist(),
+                'offset': saccades[:, 1].tolist(),
+            },
+            schema=Saccade.schema,
+        )
+
+    else:
+        # Create empty dataframe with correct schema if no events detected.
+        event_df = pl.DataFrame(schema=Saccade.schema)
+
     return event_df
 
 
diff --git a/src/pymovements/events/events.py b/src/pymovements/events/events.py
index 80869e6..9416379 100644
--- a/src/pymovements/events/events.py
+++ b/src/pymovements/events/events.py
@@ -69,7 +69,10 @@ class Event:
         Starting index of event (included).
     offset: int
         Ending index of event (excluded).
+    schema: polars.SchemaDict
+        Schema for event DataFrame.
     """
+    schema: pl.datatypes.SchemaDict = {'type': pl.Utf8, 'onset': pl.Int64, 'offset': pl.Int64}
 
     def __init__(self, name: str, onset: int, offset: int):
         """
@@ -134,9 +137,13 @@ class Fixation(Event):
         Ending index of event (excluded).
     position: tuple[float, float]
         (x, y) position of fixation
+    schema: polars.SchemaDict
+        Schema for event DataFrame.
     """
     _name = 'fixation'
 
+    schema: pl.datatypes.SchemaDict = {**Event.schema, 'position': pl.List(pl.Float64)}
+
     def __init__(self, onset: int, offset: int, position: tuple[float, float]):
         """
         Parameters
@@ -175,6 +182,8 @@ class Saccade(Event):
         Starting index of event (included).
     offset: int
         Ending index of event (excluded).
+    schema: polars.SchemaDict
+        Schema for event DataFrame.
     """
     _name = 'saccade'
 
diff --git a/src/pymovements/events/idt.py b/src/pymovements/events/idt.py
index d59c3c1..a103ef9 100644
--- a/src/pymovements/events/idt.py
+++ b/src/pymovements/events/idt.py
@@ -25,6 +25,7 @@ from __future__ import annotations
 import numpy as np
 import polars as pl
 
+from pymovements.events.events import Fixation
 from pymovements.utils.checks import check_shapes_positions_velocities
 
 
@@ -130,6 +131,12 @@ def idt(
         else:
             win_start += 1
 
-    event_df = pl.from_dicts(fixations, infer_schema_length=1)
+    if len(fixations) > 0:
+        # Create event dataframe.
+        event_df = pl.from_dicts(fixations, schema=Fixation.schema)
+
+    else:
+        # Create empty dataframe with correct schema if no events detected.
+        event_df = pl.DataFrame(schema=Fixation.schema)
 
     return event_df
diff --git a/src/pymovements/events/ivt.py b/src/pymovements/events/ivt.py
index 674b12d..58df6b4 100644
--- a/src/pymovements/events/ivt.py
+++ b/src/pymovements/events/ivt.py
@@ -25,6 +25,7 @@ from __future__ import annotations
 import numpy as np
 import polars as pl
 
+from pymovements.events.events import Fixation
 from pymovements.transforms import consecutive
 from pymovements.transforms import norm
 from pymovements.utils.checks import check_shapes_positions_velocities
@@ -103,10 +104,20 @@ def ivt(
         for fixation in fixations
     ]
 
-    event_df = pl.from_dict({
-        'type': 'fixation',
-        'onset': fixations[:, 0].tolist(),
-        'offset': fixations[:, 1].tolist(),
-        'position': centroids,
-    })
+    if len(fixations) > 0:
+        # Create event dataframe.
+        event_df = pl.from_dict(
+            {
+                'type': 'fixation',
+                'onset': fixations[:, 0].tolist(),
+                'offset': fixations[:, 1].tolist(),
+                'position': centroids,
+            },
+            schema=Fixation.schema,
+        )
+
+    else:
+        # Create empty dataframe with correct schema if no events detected.
+        event_df = pl.DataFrame(schema=Fixation.schema)
+
     return event_df


Test patch:
diff --git a/tests/events/test_engbert.py b/tests/events/test_engbert.py
index fc52019..f31be5e 100644
--- a/tests/events/test_engbert.py
+++ b/tests/events/test_engbert.py
@@ -25,6 +25,7 @@ from polars.testing import assert_frame_equal
 
 from pymovements.events.engbert import compute_threshold
 from pymovements.events.engbert import microsaccades
+from pymovements.events.events import Saccade
 from pymovements.synthetic import step_function
 
 
@@ -68,6 +69,20 @@ def test_microsaccades_raises_error(kwargs, expected):
 @pytest.mark.parametrize(
     'kwargs, expected',
     [
+        pytest.param(
+            {
+                'positions': step_function(length=100, steps=[0], values=[(0, 0)]),
+                'velocities': step_function(
+                    length=100,
+                    steps=[40, 50],
+                    values=[(9, 9), (0, 0)],
+                    start_value=(0, 0),
+                ),
+                'threshold': 10,
+            },
+            pl.DataFrame(schema=Saccade.schema),
+            id='two_steps_one_saccade_high_threshold_no_events',
+        ),
         pytest.param(
             {
                 'positions': step_function(length=100, steps=[0], values=[(0, 0)]),
@@ -85,6 +100,7 @@ def test_microsaccades_raises_error(kwargs, expected):
                     'onset': [40],
                     'offset': [49],
                 },
+                schema=Saccade.schema,
             ),
             id='two_steps_one_saccade',
         ),
@@ -105,6 +121,7 @@ def test_microsaccades_raises_error(kwargs, expected):
                     'onset': [20, 70],
                     'offset': [29, 79],
                 },
+                schema=Saccade.schema,
             ),
             id='four_steps_two_saccades',
         ),
diff --git a/tests/events/test_idt.py b/tests/events/test_idt.py
index 7d7d372..451595f 100644
--- a/tests/events/test_idt.py
+++ b/tests/events/test_idt.py
@@ -18,10 +18,12 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 """This module tests functionality of the IDT algorithm."""
+import numpy as np
 import polars as pl
 import pytest
 from polars.testing import assert_frame_equal
 
+from pymovements.events.events import Fixation
 from pymovements.events.idt import idt
 from pymovements.synthetic import step_function
 from pymovements.transforms import pos2vel
@@ -131,6 +133,27 @@ def test_idt_raises_error(kwargs, expected_error):
 @pytest.mark.parametrize(
     'kwargs, expected',
     [
+        pytest.param(
+            {
+                'positions': np.stack([np.arange(0, 200, 2), np.arange(0, 200, 2)], axis=1),
+                'dispersion_threshold': 1,
+                'minimum_duration': 10,
+            },
+            pl.DataFrame(             # should be: pl.DataFrame(schema=Fixation.schema),
+                {
+                    'type': 'fixation',
+                    'onset': [9, 20, 31, 42, 53, 64, 75, 86, 97],
+                    'offset': [10, 21, 32, 43, 54, 65, 76, 87, 98],
+                    'position': [
+                        [18.0, 18.0], [40.0, 40.0], [62.0, 62.0], [84.0, 84.0],
+                        [106.0, 106.0], [128.0, 128.0], [150.0, 150.0], [172.0, 172.0],
+                        [194.0, 194.0],
+                    ],
+                },
+                schema=Fixation.schema,
+            ),
+            id='constant_velocity_no_fixation',
+        ),
         pytest.param(
             {
                 'positions': step_function(length=100, steps=[0], values=[(0, 0)]),
@@ -144,6 +167,7 @@ def test_idt_raises_error(kwargs, expected_error):
                     'offset': [99],
                     'position': [[0.0, 0.0]],
                 },
+                schema=Fixation.schema,
             ),
             id='constant_position_single_fixation',
         ),
@@ -165,6 +189,7 @@ def test_idt_raises_error(kwargs, expected_error):
                     'offset': [49, 99],
                     'position': [[0.0, 0.0], [1.0, 1.0]],
                 },
+                schema=Fixation.schema,
             ),
             id='three_steps_two_fixations',
         ),
diff --git a/tests/events/test_ivt.py b/tests/events/test_ivt.py
index bc4aaf8..bc80ef7 100644
--- a/tests/events/test_ivt.py
+++ b/tests/events/test_ivt.py
@@ -25,6 +25,7 @@ import polars as pl
 import pytest
 from polars.testing import assert_frame_equal
 
+from pymovements.events.events import Fixation
 from pymovements.events.ivt import ivt
 from pymovements.synthetic import step_function
 from pymovements.transforms import pos2vel
@@ -164,6 +165,15 @@ def test_ivt_raise_error(kwargs, expected_error):
 @pytest.mark.parametrize(
     'kwargs, expected',
     [
+        pytest.param(
+            {
+                'positions': np.stack([np.arange(0, 200, 2), np.arange(0, 200, 2)], axis=1),
+                'velocity_threshold': 1,
+                'minimum_duration': 10,
+            },
+            pl.DataFrame(schema=Fixation.schema),
+            id='constant_velocity_no_fixation',
+        ),
         pytest.param(
             {
                 'positions': step_function(length=100, steps=[0], values=[(0, 0)]),
@@ -177,6 +187,7 @@ def test_ivt_raise_error(kwargs, expected_error):
                     'offset': [99],
                     'position': [(0.0, 0.0)],
                 },
+                schema=Fixation.schema,
             ),
             id='constant_position_single_fixation',
         ),
@@ -198,12 +209,13 @@ def test_ivt_raise_error(kwargs, expected_error):
                     'offset': [48, 99],
                     'position': [(0.0, 0.0), (1.0, 1.0)],
                 },
+                schema=Fixation.schema,
             ),
             id='three_steps_two_fixations',
         ),
     ],
 )
-def test_idt_detects_fixations(kwargs, expected):
+def test_ivt_detects_fixations(kwargs, expected):
     velocities = pos2vel(kwargs['positions'], sampling_rate=10, method='preceding')
     events = ivt(velocities=velocities, **kwargs)