Repository: aeye-lab/pymovements
Pull request #1034 (2025-03-13 16:37:52)

perf: resume archive extraction by default
## Description

Take use of  #786 by default as I don't see any actual harm in this just benefits.

This is blocked by #1030 to prevent a breaking change.

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1034?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
Attention: Patch coverage is `93.22034%` with `16 lines` in your changes missing coverage. Please review.
> Project coverage is 99.55%. Comparing base [(`592ff7b`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/592ff7bbecace5b4547714fcc5392458e608b443?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) to head [(`41f2cba`)](https://app.codecov.io/gh/aeye-lab/pymovements/commit/41f2cba1cb01063651b685a280fd8701791c08c5?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).

| [Files with missing lines](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1034?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) | Patch % | Lines |
|---|---|---|
| [src/pymovements/dataset/\_utils/\_downloads.py](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1034?src=pr&el=tree&filepath=src%2Fpymovements%2Fdataset%2F_utils%2F_downloads.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2RhdGFzZXQvX3V0aWxzL19kb3dubG9hZHMucHk=) | 80.55% | [9 Missing and 5 partials :warning: ](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1034?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) |
| [src/pymovements/dataset/\_utils/\_archives.py](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1034?src=pr&el=tree&filepath=src%2Fpymovements%2Fdataset%2F_utils%2F_archives.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2RhdGFzZXQvX3V0aWxzL19hcmNoaXZlcy5weQ==) | 97.97% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1034?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) |

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##              main    #1034      +/-   ##
===========================================
- Coverage   100.00%   99.55%   -0.45%     
===========================================
  Files           78       81       +3     
  Lines         3561     3591      +30     
  Branches       629      629              
===========================================
+ Hits          3561     3575      +14     
- Misses           0       10      +10     
- Partials         0        6       +6     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/aeye-lab/pymovements/pull/1034?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).

<details><summary>🚀 New features to boost your workflow: </summary>

- ❄ [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
dkrako: Thanks, I'll take care of the merge conflicts.

In this case it was not possible to `git checkout origin/main -b NEWBRANCH` as this PR depends on #1030 (which depends on #1028 which depends on #1029) otherwise it would introduce breaking changes to the utils interface (as `resume` was missing from `download_and_extract_archive()`). 
SiQube: then let's rebase #1030 into this branch after #1030 is merged

Patch:
diff --git a/src/pymovements/dataset/_utils/_archives.py b/src/pymovements/dataset/_utils/_archives.py
index bbc3745..9e970ba 100644
--- a/src/pymovements/dataset/_utils/_archives.py
+++ b/src/pymovements/dataset/_utils/_archives.py
@@ -44,7 +44,7 @@ def extract_archive(
         recursive: bool = True,
         remove_finished: bool = False,
         remove_top_level: bool = True,
-        resume: bool = False,
+        resume: bool = True,
         verbose: int = 1,
 ) -> Path:
     """Extract an archive.
@@ -67,7 +67,7 @@ def extract_archive(
         If ``True``, remove the top-level directory if it has only one child. (default: True)
     resume: bool
         Resume previous extraction by skipping existing files.
-        Checks for correct size of existing files but not integrity. (default: False)
+        Checks for correct size of existing files but not integrity. (default: True)
     verbose: int
         Verbosity levels: (1) Print messages for extracting each dataset resource without printing
         messages for recursive archives. (2) Print additional messages for each recursive archive
diff --git a/src/pymovements/dataset/_utils/_downloads.py b/src/pymovements/dataset/_utils/_downloads.py
index 4216f7f..a915fc8 100644
--- a/src/pymovements/dataset/_utils/_downloads.py
+++ b/src/pymovements/dataset/_utils/_downloads.py
@@ -38,9 +38,11 @@ def download_and_extract_archive(
         download_filename: str,
         extract_dirpath: Path | None = None,
         md5: str | None = None,
+        *,
         recursive: bool = True,
         remove_finished: bool = False,
         remove_top_level: bool = True,
+        resume: bool = True,
         verbose: int = 1,
 ) -> None:
     """Download and extract archive file.
@@ -63,6 +65,9 @@ def download_and_extract_archive(
         Remove downloaded file after successful extraction or decompression. (default: False)
     remove_top_level: bool
         If ``True``, remove the top-level directory if it has only one child. (default: True)
+    resume: bool
+        Resume previous extraction by skipping existing files.
+        Checks for correct size of existing files but not integrity. (default: True)
     verbose: int
         Verbosity levels: (1) Show download progress bar and print info messages on downloading
         and extracting archive files without printing messages for recursive archive extraction.
@@ -91,6 +96,7 @@ def download_and_extract_archive(
         recursive=recursive,
         remove_finished=remove_finished,
         remove_top_level=remove_top_level,
+        resume=resume,
         verbose=verbose,
     )
 
@@ -100,6 +106,7 @@ def download_file(
         dirpath: Path,
         filename: str,
         md5: str | None = None,
+        *,
         max_redirect_hops: int = 3,
         verbose: bool = True,
 ) -> Path:
diff --git a/src/pymovements/dataset/dataset.py b/src/pymovements/dataset/dataset.py
index 1ecd1e8..49acf60 100644
--- a/src/pymovements/dataset/dataset.py
+++ b/src/pymovements/dataset/dataset.py
@@ -966,7 +966,7 @@ class Dataset:
             *,
             extract: bool = True,
             remove_finished: bool = False,
-            resume: bool = False,
+            resume: bool = True,
             verbose: int = 1,
     ) -> Dataset:
         """Download dataset resources.
@@ -990,7 +990,7 @@ class Dataset:
             Remove archive files after extraction. (default: False)
         resume: bool
             Resume previous extraction by skipping existing files.
-            Checks for correct size of existing files but not integrity. (default: False)
+            Checks for correct size of existing files but not integrity. (default: True)
         verbose: int
             Verbosity levels: (1) Show download progress bar and print info messages on downloading
             and extracting archive files without printing messages for recursive archive extraction.
@@ -1023,7 +1023,7 @@ class Dataset:
             *,
             remove_finished: bool = False,
             remove_top_level: bool = True,
-            resume: bool = False,
+            resume: bool = True,
             verbose: int = 1,
     ) -> Dataset:
         """Extract downloaded dataset archive files.
@@ -1036,7 +1036,7 @@ class Dataset:
             If ``True``, remove the top-level directory if it has only one child. (default: True)
         resume: bool
             Resume previous extraction by skipping existing files.
-            Checks for correct size of existing files but not integrity. (default: False)
+            Checks for correct size of existing files but not integrity. (default: True)
         verbose: int
             Verbosity levels: (1) Print messages for extracting each dataset resource without
             printing messages for recursive archives. (2) Print additional messages for each
diff --git a/src/pymovements/dataset/dataset_download.py b/src/pymovements/dataset/dataset_download.py
index 4ff1c40..e80696d 100644
--- a/src/pymovements/dataset/dataset_download.py
+++ b/src/pymovements/dataset/dataset_download.py
@@ -37,7 +37,7 @@ def download_dataset(
         *,
         extract: bool = True,
         remove_finished: bool = False,
-        resume: bool = False,
+        resume: bool = True,
         verbose: bool = True,
 ) -> None:
     """Download dataset resources.
@@ -65,7 +65,7 @@ def download_dataset(
         Remove archive files after extraction. (default: False)
     resume: bool
         Resume previous extraction by skipping existing files.
-        Checks for correct size of existing files but not integrity. (default: False)
+        Checks for correct size of existing files but not integrity. (default: True)
     verbose: bool
         If True, show progress of download and print status messages for integrity checking and
         file extraction. (default: True)
@@ -145,7 +145,7 @@ def extract_dataset(
         *,
         remove_finished: bool = False,
         remove_top_level: bool = True,
-        resume: bool = False,
+        resume: bool = True,
         verbose: int = 1,
 ) -> None:
     """Extract downloaded dataset archive files.
@@ -162,7 +162,7 @@ def extract_dataset(
         If ``True``, remove the top-level directory if it has only one child. (default: True)
     resume: bool
         Resume previous extraction by skipping existing files.
-        Checks for correct size of existing files but not integrity. (default: False)
+        Checks for correct size of existing files but not integrity. (default: True)
     verbose: int
         Verbosity levels: (1) Print messages for extracting each dataset resource without printing
         messages for recursive archives. (2) Print messages for extracting each dataset resource and
diff --git a/src/pymovements/utils/archives.py b/src/pymovements/utils/archives.py
index 2e9bdcd..0eed4eb 100644
--- a/src/pymovements/utils/archives.py
+++ b/src/pymovements/utils/archives.py
@@ -44,7 +44,7 @@ def extract_archive(
         recursive: bool = True,
         remove_finished: bool = False,
         remove_top_level: bool = True,
-        resume: bool = False,
+        resume: bool = True,
         verbose: int = 1,
 ) -> Path:
     """Extract an archive.


Test patch:
diff --git a/tests/unit/dataset/dataset_download_test.py b/tests/unit/dataset/dataset_download_test.py
index afac3db..b212d41 100644
--- a/tests/unit/dataset/dataset_download_test.py
+++ b/tests/unit/dataset/dataset_download_test.py
@@ -949,7 +949,7 @@ def test_dataset_extract_remove_finished_true_gaze(
             recursive=True,
             remove_finished=True,
             remove_top_level=False,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
     ])
@@ -979,7 +979,7 @@ def test_dataset_extract_rm(
             recursive=True,
             remove_finished=False,
             remove_top_level=True,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
     ])
@@ -1009,7 +1009,7 @@ def test_dataset_extract_remove_finished_true_both(
             recursive=True,
             remove_finished=True,
             remove_top_level=False,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
         mock.call(
@@ -1018,7 +1018,7 @@ def test_dataset_extract_remove_finished_true_both(
             recursive=True,
             remove_finished=True,
             remove_top_level=False,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
     ])
@@ -1048,7 +1048,7 @@ def test_dataset_extract_remove_finished_true_precomputed(
             recursive=True,
             remove_finished=True,
             remove_top_level=False,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
     ])
@@ -1078,7 +1078,7 @@ def test_dataset_extract_remove_finished_false_both(
             recursive=True,
             remove_finished=False,
             remove_top_level=True,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
         mock.call(
@@ -1087,7 +1087,7 @@ def test_dataset_extract_remove_finished_false_both(
             recursive=True,
             remove_finished=False,
             remove_top_level=True,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
     ])
@@ -1117,7 +1117,7 @@ def test_dataset_extract_remove_finished_false_gaze(
             recursive=True,
             remove_finished=False,
             remove_top_level=True,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
     ])
@@ -1147,7 +1147,7 @@ def test_dataset_extract_remove_finished_false_precomputed(
             recursive=True,
             remove_finished=False,
             remove_top_level=True,
-            resume=False,
+            resume=True,
             verbose=1,
         ),
     ])
@@ -1332,7 +1332,8 @@ def test_public_dataset_registered_correct_attributes(tmp_path, dataset_definiti
     assert dataset.definition.resources == dataset_definition.resources
     assert dataset.definition.experiment == dataset_definition.experiment
     assert dataset.definition.filename_format == dataset_definition.filename_format
-    assert dataset.definition.filename_format_schema_overrides == dataset_definition.filename_format_schema_overrides  # noqa: E501 # pylint: disable=line-too-long
+    assert dataset.definition.filename_format_schema_overrides \
+        == dataset_definition.filename_format_schema_overrides
     assert dataset.definition.has_files == dataset_definition.has_files