Repository: aeye-lab/pymovements
Pull request #1145 (2025-05-09 22:12:09)

ci: add local pre-commit hook to write datasets.yaml
## Description

Building up on @SiQube's work in #914 this PR adds a pre-commit hook to autowrite the required `src/pymovements/datasets/datasets.yaml`.

## Implemented changes

- [x] add `write_datasets_yaml.py` including a `main()` function
- [x] add script as local pre-commit hook in `.pre-commit-config.yaml`

## How Has This Been Tested?

Basic tests which compare the resulting `datasets.yaml` against the expected one:

- [x] empty datasets directory
- [x] single yaml definition
- [x] two yaml definitions

## Context

#### related issues:
- #1021

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream modules
- [x] I have checked my code and corrected any misspellings


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1145?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`232c552`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/232c55231ce2954dc3e73ca5c92be38a27f887c6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) to head [(`5e71967`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/5e71967f24b6e8fccb852513afe486291b7fcbe4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).
> Report is 13 commits behind head on main.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##              main     #1145    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           87        90     +3     
  Lines         3818      3950   +132     
  Branches       679       695    +16     
==========================================
+ Hits          3818      3950   +132     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1145?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>

Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 3a26cfc..84b2168 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -149,3 +149,12 @@ repos:
             '--disable=protected-access',
             '--rcfile=pylintrc',
           ]
+-   repo: local
+    hooks:
+      - id: write-datasets-yaml
+        name: write-datasets-yaml
+        entry: python src/pymovements/_scripts/write_datasets_yaml.py
+        language: python
+        verbose: true
+        additional_dependencies:
+        - pyyaml
diff --git a/src/pymovements/_scripts/__init__.py b/src/pymovements/_scripts/__init__.py
new file mode 100644
index 0000000..6786db2
--- /dev/null
+++ b/src/pymovements/_scripts/__init__.py
@@ -0,0 +1,19 @@
+# Copyright (c) 2025 The pymovements Project Authors
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
diff --git a/src/pymovements/_scripts/write_datasets_yaml.py b/src/pymovements/_scripts/write_datasets_yaml.py
new file mode 100644
index 0000000..aa099c4
--- /dev/null
+++ b/src/pymovements/_scripts/write_datasets_yaml.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2025 The pymovements Project Authors
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Write datasets.yaml for DatasetLibrary."""
+from __future__ import annotations
+
+from pathlib import Path
+
+import yaml
+
+
+def main(
+        datasets_dirpath: str | Path = './src/pymovements/datasets',
+        datasets_yaml_filename: str = 'datasets.yaml',
+) -> int:
+    """Write datasets yaml file for DatasetLibrary.
+
+    Parameters
+    ----------
+    datasets_dirpath: str | Path
+        The path to the directory containing dataset definition yaml files.
+        (default: './src/pymovements/datasets')
+    datasets_yaml_filename: str
+        The filename of the datasets yaml file. (default: 'datasets.yaml')
+
+    Returns
+    -------
+    int
+        ``0`` if no changes needed, ``1`` otherwise.
+    """
+    datasets_dirpath = Path(datasets_dirpath)
+
+    dataset_filename_stems = sorted(
+        [
+            filepath.stem for filepath in datasets_dirpath.glob('*.yaml')
+            if filepath.name != datasets_yaml_filename  # Ignore datasets.yaml file.
+        ],
+    )
+
+    try:
+        with open(datasets_dirpath / datasets_yaml_filename, encoding='utf-8') as f:
+            dataset_yaml_content = yaml.safe_load(f)
+    except FileNotFoundError:
+        dataset_yaml_content = None
+
+    # File content matches. Exit successfully.
+    if dataset_filename_stems == dataset_yaml_content:
+        return 0
+
+    # We have some updates in the datasets directory. Update the datasets yaml file.
+    with open(datasets_dirpath / datasets_yaml_filename, 'w', encoding='utf-8') as f:
+        yaml.dump(dataset_filename_stems, f)
+
+    return 1
+
+
+if __name__ == '__main__':  # pragma: no cover
+    raise SystemExit(main())


Test patch:
diff --git a/tests/unit/_scripts/write_datasets_yaml_test.py b/tests/unit/_scripts/write_datasets_yaml_test.py
new file mode 100644
index 0000000..b54934a
--- /dev/null
+++ b/tests/unit/_scripts/write_datasets_yaml_test.py
@@ -0,0 +1,110 @@
+# Copyright (c) 2025 The pymovements Project Authors
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+"""Test write_datasets_yaml script."""
+from pathlib import Path
+
+import pytest
+import yaml
+
+from pymovements._scripts import write_datasets_yaml
+
+
+@pytest.fixture(name='make_datasets_directory')
+def make_datasets_directory_fixture(tmp_path):
+    def _make_datasets_directory(param: str, make_yaml: bool) -> Path:
+        datasets = []
+        if param in {'single', 'two'}:
+            filepath = tmp_path / 'first.yaml'
+            filepath.touch()
+            datasets.append('first')
+        if param == 'two':
+            filepath = tmp_path / 'second.yaml'
+            filepath.touch()
+            datasets.append('second')
+        if make_yaml:
+            with open(tmp_path / 'datasets.yaml', 'w', encoding='utf-8') as f:
+                yaml.dump(datasets, f)
+        return tmp_path
+
+    yield _make_datasets_directory
+
+
+@pytest.mark.parametrize(
+    ('fixture_args', 'expected_return', 'expected_yaml'),
+    [
+        pytest.param(
+            ['empty', False],
+            1,
+            [],
+            id='empty_new',
+        ),
+
+        pytest.param(
+            ['empty', True],
+            0,
+            [],
+            id='empty_exist',
+        ),
+
+        pytest.param(
+            ['single', False],
+            1,
+            ['first'],
+            id='single_new',
+        ),
+
+        pytest.param(
+            ['single', True],
+            0,
+            ['first'],
+            id='single_exist',
+        ),
+
+        pytest.param(
+            ['two', False],
+            1,
+            ['first', 'second'],
+            id='two_new',
+        ),
+
+        pytest.param(
+            ['two', True],
+            0,
+            ['first', 'second'],
+            id='two_exist',
+        ),
+    ],
+)
+def test_write_datasets_yaml(
+        fixture_args, expected_return, expected_yaml, make_datasets_directory,
+):
+    datasets_dirpath = make_datasets_directory(*fixture_args)
+    datasets_yaml_filename = 'datasets.yaml'
+
+    return_value = write_datasets_yaml.main(
+        datasets_dirpath=datasets_dirpath,
+        datasets_yaml_filename=datasets_yaml_filename,
+    )
+
+    with open(datasets_dirpath / datasets_yaml_filename, encoding='utf-8') as f:
+        yaml_content = yaml.safe_load(f)
+
+    assert return_value == expected_return
+    assert yaml_content == expected_yaml