Repository: aeye-lab/pymovements
Pull request #264 (2023-03-08 13:51:05)

tests for public dataset
## Description

I added tests for public dataset + a small change in the public dataset when the dataset download fails. We officially reach 100% coverage with this 👯 

Fixes issue #184 

## Implemented changes

Insert a description of the changes implemented in the pull request.

- [x] implemented test cases for public dataset
- [x] changed error message in public dataset if download fails

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change is or requires a documentation update

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream modules
- [x] I have checked my code and corrected any misspellings


codecov-commenter: # [Codecov](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
Patch coverage: **`100.00`**% and project coverage change: **`+2.10`** :tada:
> Comparison is base [(`2609840`)](https://codecov.io/gh/aeye-lab/pymovements/commit/2609840ca6ac60c1b0443583c77d4f06a198b9a1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) 97.89% compared to head [(`3077ecd`)](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) 100.00%.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##             main      #264      +/-   ##
===========================================
+ Coverage   97.89%   100.00%   +2.10%     
===========================================
  Files          24        28       +4     
  Lines         997      1080      +83     
  Branches      233       248      +15     
===========================================
+ Hits          976      1080     +104     
+ Misses         19         0      -19     
+ Partials        2         0       -2     
```


| [Impacted Files](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) | Coverage Δ | |
|---|---|---|
| [src/pymovements/datasets/public\_dataset.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2RhdGFzZXRzL3B1YmxpY19kYXRhc2V0LnB5) | `100.00% <100.00%> (+40.38%)` | :arrow_up: |
| [src/pymovements/\_\_init\_\_.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL19faW5pdF9fLnB5) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/events/idt.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9pZHQucHk=) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/events/ivt.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9pdnQucHk=) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/exceptions.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V4Y2VwdGlvbnMucHk=) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/utils/checks.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL3V0aWxzL2NoZWNrcy5weQ==) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/events/events.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9ldmVudHMucHk=) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/events/engbert.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2V2ZW50cy9lbmdiZXJ0LnB5) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/datasets/dataset.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2RhdGFzZXRzL2RhdGFzZXQucHk=) | `100.00% <0.00%> (ø)` | |
| [src/pymovements/transforms.py](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL3RyYW5zZm9ybXMucHk=) | | |
| ... and [8 more](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/aeye-lab/pymovements/pull/264?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).


Patch:
diff --git a/src/pymovements/datasets/public_dataset.py b/src/pymovements/datasets/public_dataset.py
index a76abd7..c8c182f 100644
--- a/src/pymovements/datasets/public_dataset.py
+++ b/src/pymovements/datasets/public_dataset.py
@@ -157,7 +157,8 @@ class PublicDataset(Dataset, metaclass=ABCMeta):
                     )
                     success = True
 
-                except URLError as error:
+                # pylint: disable=overlapping-except
+                except (URLError, OSError, RuntimeError) as error:
                     print(f'Failed to download (trying next):\n{error}')
                     # downloading the resource, try next mirror
                     continue


Test patch:
diff --git a/tests/datasets/public_dataset_test.py b/tests/datasets/public_dataset_test.py
index df19a82..93cb92d 100644
--- a/tests/datasets/public_dataset_test.py
+++ b/tests/datasets/public_dataset_test.py
@@ -19,12 +19,32 @@
 # SOFTWARE.
 """Test all functionality in pymovements.datasets.public_dataset."""
 from pathlib import Path
+from unittest import mock
 
 import pytest
 
 from pymovements.datasets.public_dataset import PublicDataset
 
 
+@pytest.fixture(name='dataset')
+def custom_dataset(tmp_path):
+    class CustomPublicDataset(PublicDataset):
+        _mirrors = ['https://example.com/', 'https://another_example.com/']
+        _resources = [
+            {
+                'resource': 'test.gz.tar',
+                'filename': 'test.gz.tar',
+                'md5': '52bbf03a7c50ee7152ccb9d357c2bb30',
+            },
+        ]
+
+    return CustomPublicDataset(
+        root=tmp_path,
+        dataset_dirname='',
+        downloads_dirname='',
+    )
+
+
 @pytest.mark.parametrize(
     'init_kwargs, expected_paths',
     [
@@ -78,3 +98,116 @@ def test_paths(init_kwargs, expected_paths):
     assert dataset.root == expected_paths['root']
     assert dataset.path == expected_paths['path']
     assert dataset.downloads_rootpath == expected_paths['download']
+
+
+@mock.patch('pymovements.datasets.public_dataset.download_file')
+def test_dataset_download_fails(mock_download_file, tmp_path, dataset):
+    mock_download_file.side_effect = OSError()
+
+    with pytest.raises(RuntimeError):
+        dataset.download()
+
+    mock_download_file.assert_has_calls([
+        mock.call(
+            url='https://example.com/test.gz.tar',
+            dirpath=tmp_path,
+            filename='test.gz.tar',
+            md5='52bbf03a7c50ee7152ccb9d357c2bb30',
+        ),
+    ])
+
+
+@mock.patch('pymovements.datasets.public_dataset.download_file')
+def test_dataset_download_file_not_found(mock_download_file, tmp_path, dataset):
+    mock_download_file.side_effect = RuntimeError()
+
+    with pytest.raises(RuntimeError):
+        dataset.download()
+
+    mock_download_file.assert_has_calls([
+        mock.call(
+            url='https://example.com/test.gz.tar',
+            dirpath=tmp_path,
+            filename='test.gz.tar',
+            md5='52bbf03a7c50ee7152ccb9d357c2bb30',
+        ),
+    ])
+
+
+@mock.patch('pymovements.datasets.public_dataset.download_file')
+def test_dataset_download(mock_download_file, tmp_path, dataset):
+    mock_download_file.return_value = 'path'
+
+    dataset.download()
+
+    mock_download_file.assert_has_calls([
+        mock.call(
+            url='https://example.com/test.gz.tar',
+            dirpath=tmp_path,
+            filename='test.gz.tar',
+            md5='52bbf03a7c50ee7152ccb9d357c2bb30',
+        ),
+    ])
+
+
+@mock.patch('pymovements.datasets.public_dataset.extract_archive')
+def test_dataset_extract_remove_finished_true(
+        mock_extract_archive,
+        tmp_path,
+        dataset,
+):
+    mock_extract_archive.return_value = 'path'
+
+    dataset.extract(remove_finished=True)
+
+    mock_extract_archive.assert_has_calls([
+        mock.call(
+            source_path=tmp_path / 'test.gz.tar',
+            destination_path=tmp_path / 'raw',
+            recursive=True,
+            remove_finished=True,
+        ),
+    ])
+
+
+@mock.patch('pymovements.datasets.public_dataset.extract_archive')
+def test_dataset_extract_remove_finished_false(
+        mock_extract_archive,
+        tmp_path,
+        dataset,
+):
+    mock_extract_archive.return_value = 'path'
+
+    dataset.extract()
+
+    mock_extract_archive.assert_has_calls([
+        mock.call(
+            source_path=tmp_path / 'test.gz.tar',
+            destination_path=tmp_path / 'raw',
+            recursive=True,
+            remove_finished=False,
+        ),
+    ])
+
+
+@mock.patch('pymovements.datasets.public_dataset.PublicDataset.download')
+@mock.patch('pymovements.datasets.public_dataset.PublicDataset.extract')
+def test_custom_dataset_download_extract(mock_extract, mock_download, tmp_path):
+    mock_extract.return_value = None
+    mock_download.return_value = None
+
+    class CustomPublicDataset(PublicDataset):
+        _mirrors = ['https://example.com/']
+        _resources = [
+            {
+                'resource': 'test.gz.tar',
+                'filename': 'test.gz.tar',
+                'md5': '52bbf03a7c50ee7152ccb9d357c2bb30',
+            },
+        ]
+
+    CustomPublicDataset(root=tmp_path, download=True)
+    CustomPublicDataset(root=tmp_path, extract=True)
+
+    mock_download.assert_called_once()
+    mock_extract.assert_called_once()