Repository: aeye-lab/pymovements
Pull request #273 (2023-03-13 14:22:01)

Datasets: Add extension argument to save() and load()
## Description

Fixes issue #267

## Implemented changes

Insert a description of the changes implemented in the pull request.

- PR to savd/load events and gaze data as csv-files (Issue #267)

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How Has This Been Tested?

- test-cases are added

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream modules
- [x] I have checked my code and corrected any misspellings


codecov-commenter: ## [Codecov](https://codecov.io/gh/aeye-lab/pymovements/pull/273?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) Report
Patch coverage: **`72.00`**% and project coverage change: **`-0.52`** :warning:
> Comparison is base [(`68c9c5c`)](https://codecov.io/gh/aeye-lab/pymovements/commit/68c9c5c5e69ec0b116f098850c0785522931b2b3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) 98.39% compared to head [(`ad52e18`)](https://codecov.io/gh/aeye-lab/pymovements/pull/273?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) 97.88%.

> :exclamation: Current head ad52e18 differs from pull request most recent head 884228d. Consider uploading reports for the commit 884228d to get more accurate results

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #273      +/-   ##
==========================================
- Coverage   98.39%   97.88%   -0.52%     
==========================================
  Files          32       32              
  Lines        1310     1321      +11     
  Branches      307      314       +7     
==========================================
+ Hits         1289     1293       +4     
- Misses         19       22       +3     
- Partials        2        6       +4     
```


| [Impacted Files](https://codecov.io/gh/aeye-lab/pymovements/pull/273?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab) | Coverage Δ | |
|---|---|---|
| [src/pymovements/datasets/dataset.py](https://codecov.io/gh/aeye-lab/pymovements/pull/273?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab#diff-c3JjL3B5bW92ZW1lbnRzL2RhdGFzZXRzL2RhdGFzZXQucHk=) | `97.03% <72.00%> (-2.97%)` | :arrow_down: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab)

</details>

[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/aeye-lab/pymovements/pull/273?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aeye-lab).


Patch:
diff --git a/src/pymovements/datasets/dataset.py b/src/pymovements/datasets/dataset.py
index 045539c..dc09da6 100644
--- a/src/pymovements/datasets/dataset.py
+++ b/src/pymovements/datasets/dataset.py
@@ -122,6 +122,7 @@ class Dataset:
             subset: None | dict[str, float | int | str | list[float | int | str]] = None,
             events_dirname: str | None = None,
             preprocessed_dirname: str | None = None,
+            extension: str = 'feather',
     ):
         """Parse file information and load all gaze files.
 
@@ -145,6 +146,9 @@ class Dataset:
             :py:meth:`pymovements.Dataset.path`.
             This argument is used only for this single call and does not alter
             :py:meth:`pymovements.Dataset.preprocessed_rootpath`.
+        extension:
+            Specifies the file format for loading data. Valid options are: `csv`, `feather`.
+            :Default: `feather`.
 
         The parsed file information is assigned to the `fileinfo` attribute.
         All gaze files will be loaded as dataframes and assigned to the `gaze` attribute.
@@ -153,10 +157,14 @@ class Dataset:
         self.fileinfo = self.take_subset(fileinfo=fileinfo, subset=subset)
         self.gaze = self.load_gaze_files(
             preprocessed=preprocessed, preprocessed_dirname=preprocessed_dirname,
+            extension=extension,
         )
 
         if events:
-            self.events = self.load_event_files(events_dirname=events_dirname)
+            self.events = self.load_event_files(
+                events_dirname=events_dirname,
+                extension=extension,
+            )
 
     def infer_fileinfo(self) -> pl.DataFrame:
         """Infer information from filepaths and filenames.
@@ -263,6 +271,7 @@ class Dataset:
             self,
             preprocessed: bool = False,
             preprocessed_dirname: str | None = None,
+            extension: str = 'feather',
     ) -> list[GazeDataFrame]:
         """Load all available gaze data files.
 
@@ -275,6 +284,9 @@ class Dataset:
             :py:meth:`pymovements.Dataset.path`.
             This argument is used only for this single call and does not alter
             :py:meth:`pymovements.Dataset.preprocessed_rootpath`.
+        extension:
+            Specifies the file format for loading data. Valid options are: `csv`, `feather`.
+            :Default: `feather`.
 
         Returns
         -------
@@ -300,10 +312,14 @@ class Dataset:
             if preprocessed:
                 filepath = self._raw_to_preprocessed_filepath(
                     filepath, preprocessed_dirname=preprocessed_dirname,
+                    extension=extension,
                 )
 
             if filepath.suffix == '.csv':
-                gaze_df = pl.read_csv(filepath, **self._custom_read_kwargs)
+                if preprocessed:
+                    gaze_df = pl.read_csv(filepath)
+                else:
+                    gaze_df = pl.read_csv(filepath, **self._custom_read_kwargs)
             elif filepath.suffix == '.feather':
                 gaze_df = pl.read_ipc(filepath)
             else:
@@ -316,7 +332,11 @@ class Dataset:
 
         return gaze_dfs
 
-    def load_event_files(self, events_dirname: str | None = None) -> list[EventDataFrame]:
+    def load_event_files(
+        self,
+        events_dirname: str | None = None,
+        extension: str = 'feather',
+    ) -> list[EventDataFrame]:
         """Load all available event files.
 
         Parameters
@@ -326,6 +346,9 @@ class Dataset:
             :py:meth:`pymovements.Dataset.path`.
             This argument is used only for this single call and does not alter
             :py:meth:`pymovements.Dataset.events_rootpath`.
+        extension:
+            Specifies the file format for loading data. Valid options are: `csv`, `feather`.
+            :Default: `feather`.
 
         Returns
         -------
@@ -336,6 +359,8 @@ class Dataset:
         ------
         AttributeError
             If `fileinfo` is None or the `fileinfo` dataframe is empty.
+        ValueError
+            If extension is not in list of valid extensions.
         """
         self._check_fileinfo()
 
@@ -346,8 +371,21 @@ class Dataset:
             filepath = Path(fileinfo['filepath'])
             filepath = self.raw_rootpath / filepath
 
-            filepath = self._raw_to_event_filepath(filepath, events_dirname=events_dirname)
-            event_df = pl.read_ipc(filepath)
+            filepath = self._raw_to_event_filepath(
+                filepath, events_dirname=events_dirname,
+                extension=extension,
+            )
+
+            if extension == 'feather':
+                event_df = pl.read_ipc(filepath)
+            elif extension == 'csv':
+                event_df = pl.read_csv(filepath)
+            else:
+                valid_extensions = ['csv', 'feather']
+                raise ValueError(
+                    f'unsupported file format "{extension}".'
+                    f'Supported formats are: {valid_extensions}',
+                )
 
             # Add fileinfo columns to dataframe.
             event_df = self._add_fileinfo(event_df, fileinfo)
@@ -544,10 +582,11 @@ class Dataset:
             events_dirname: str | None = None,
             preprocessed_dirname: str | None = None,
             verbose: int = 1,
+            extension: str = 'feather',
     ):
         """Save preprocessed gaze and event files.
 
-        Data will be saved as feather files to ``Dataset.preprocessed_roothpath`` or
+        Data will be saved as feather/csv files to ``Dataset.preprocessed_roothpath`` or
         ``Dataset.events_roothpath`` with the same directory structure as the raw data.
 
         Parameters
@@ -562,11 +601,17 @@ class Dataset:
             :py:meth:`pymovements.Dataset.preprocessed_rootpath`.
         verbose : int
             Verbosity level (0: no print output, 1: show progress bar, 2: print saved filepaths)
+        extension:
+            extension specifies the fileformat to store the data
         """
-        self.save_events(events_dirname, verbose=verbose)
-        self.save_preprocessed(preprocessed_dirname, verbose=verbose)
+        self.save_events(events_dirname, verbose=verbose, extension=extension)
+        self.save_preprocessed(preprocessed_dirname, verbose=verbose, extension=extension)
 
-    def save_events(self, events_dirname: str | None = None, verbose: int = 1):
+    def save_events(
+        self, events_dirname: str | None = None,
+        verbose: int = 1,
+        extension: str = 'feather',
+    ):
         """Save events to files.
 
         Data will be saved as feather files to ``Dataset.events_roothpath`` with the same directory
@@ -580,6 +625,14 @@ class Dataset:
             :py:meth:`pymovements.Dataset.events_rootpath`.
         verbose : int
             Verbosity level (0: no print output, 1: show progress bar, 2: print saved filepaths)
+        extension:
+            Specifies the file format for loading data. Valid options are: `csv`, `feather`.
+            :Default: `feather`.
+
+        Raises
+        ------
+        ValueError
+            If extension is not in list of valid extensions.
         """
         disable_progressbar = not verbose
 
@@ -587,6 +640,7 @@ class Dataset:
             raw_filepath = self.raw_rootpath / Path(self.fileinfo[file_id, 'filepath'])
             events_filepath = self._raw_to_event_filepath(
                 raw_filepath, events_dirname=events_dirname,
+                extension=extension,
             )
 
             event_df_out = event_df.frame.clone()
@@ -598,9 +652,22 @@ class Dataset:
                 print('Save file to', events_filepath)
 
             events_filepath.parent.mkdir(parents=True, exist_ok=True)
-            event_df_out.write_ipc(events_filepath)
+            if extension == 'feather':
+                event_df_out.write_ipc(events_filepath)
+            elif extension == 'csv':
+                event_df_out.write_csv(events_filepath)
+            else:
+                valid_extensions = ['csv', 'feather']
+                raise ValueError(
+                    f'unsupported file format "{extension}".'
+                    f'Supported formats are: {valid_extensions}',
+                )
 
-    def save_preprocessed(self, preprocessed_dirname: str | None = None, verbose: int = 1):
+    def save_preprocessed(
+        self, preprocessed_dirname: str | None = None,
+        verbose: int = 1,
+        extension: str = 'feather',
+    ):
         """Save preprocessed gaze files.
 
         Data will be saved as feather files to ``Dataset.preprocessed_roothpath`` with the same
@@ -614,6 +681,14 @@ class Dataset:
             :py:meth:`pymovements.Dataset.preprocessed_rootpath`.
         verbose : int
             Verbosity level (0: no print output, 1: show progress bar, 2: print saved filepaths)
+        extension:
+            Specifies the file format for loading data. Valid options are: `csv`, `feather`.
+            :Default: `feather`.
+
+        Raises
+        ------
+        ValueError
+            If extension is not in list of valid extensions.
         """
         disable_progressbar = not verbose
 
@@ -621,6 +696,7 @@ class Dataset:
             raw_filepath = self.raw_rootpath / Path(self.fileinfo[file_id, 'filepath'])
             preprocessed_filepath = self._raw_to_preprocessed_filepath(
                 raw_filepath, preprocessed_dirname=preprocessed_dirname,
+                extension=extension,
             )
 
             gaze_df_out = gaze_df.frame.clone()
@@ -632,7 +708,16 @@ class Dataset:
                 print('Save file to', preprocessed_filepath)
 
             preprocessed_filepath.parent.mkdir(parents=True, exist_ok=True)
-            gaze_df_out.write_ipc(preprocessed_filepath)
+            if extension == 'feather':
+                gaze_df_out.write_ipc(preprocessed_filepath)
+            elif extension == 'csv':
+                gaze_df_out.write_csv(preprocessed_filepath)
+            else:
+                valid_extensions = ['csv', 'feather']
+                raise ValueError(
+                    f'unsupported file format "{extension}".'
+                    f'Supported formats are: {valid_extensions}',
+                )
 
     @property
     def path(self) -> Path:
@@ -781,6 +866,7 @@ class Dataset:
             self,
             raw_filepath: Path,
             preprocessed_dirname: str | None = None,
+            extension: str = 'feather',
     ) -> Path:
         """Get preprocessed filepath in accordance to filepath of the raw file.
 
@@ -795,6 +881,8 @@ class Dataset:
             :py:meth:`pymovements.Dataset.path`.
             This argument is used only for this single call and does not alter
             :py:meth:`pymovements.Dataset.preprocessed_rootpath`.
+        extension:
+            extension specifies the fileformat to store the data
 
         Returns
         -------
@@ -812,14 +900,19 @@ class Dataset:
         preprocessed_file_dirpath = preprocessed_rootpath / relative_raw_dirpath
 
         # Get new filename for saved feather file.
-        preprocessed_filename = raw_filepath.stem + '.feather'
+        preprocessed_filename = raw_filepath.stem + '.' + extension
 
         return preprocessed_file_dirpath / preprocessed_filename
 
-    def _raw_to_event_filepath(self, raw_filepath: Path, events_dirname: str | None = None) -> Path:
+    def _raw_to_event_filepath(
+        self,
+        raw_filepath: Path,
+        events_dirname: str | None = None,
+        extension: str = 'feather',
+    ) -> Path:
         """Get event filepath in accordance to filepath of the raw file.
 
-        The event filepath will point to a feather file.
+        The event filepath will point to file with the specified extension.
 
         Parameters
         ----------
@@ -830,6 +923,8 @@ class Dataset:
             :py:meth:`pymovements.Dataset.path`.
             This argument is used only for this single call and does not alter
             :py:meth:`pymovements.Dataset.events_rootpath`.
+        extension:
+            extension specifies the fileformat to store the data
 
         Returns
         -------
@@ -847,7 +942,7 @@ class Dataset:
         events_file_dirpath = events_rootpath / relative_raw_dirpath
 
         # Get new filename for saved feather file.
-        events_filename = raw_filepath.stem + '.feather'
+        events_filename = raw_filepath.stem + '.' + extension
 
         return events_file_dirpath / events_filename
 


Test patch:
diff --git a/tests/datasets/dataset_test.py b/tests/datasets/dataset_test.py
index 5327820..3958718 100644
--- a/tests/datasets/dataset_test.py
+++ b/tests/datasets/dataset_test.py
@@ -18,6 +18,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 """Test all functionality in pymovements.datasets.dataset."""
+import os
 import shutil
 import unittest
 from pathlib import Path
@@ -384,6 +385,83 @@ def test_load_exceptions(init_kwargs, load_kwargs, exception, dataset_configurat
         dataset.load(**load_kwargs)
 
 
+@pytest.mark.parametrize(
+    'init_kwargs, save_kwargs, exception',
+    [
+        pytest.param(
+            {},
+            {'extension': 'invalid'},
+            ValueError,
+            id='wrong_extension_save_gaze',
+        ),
+    ],
+)
+def test_save_gaze_exceptions(init_kwargs, save_kwargs, exception, dataset_configuration):
+    init_kwargs = {**dataset_configuration['init_kwargs'], **init_kwargs}
+    dataset = Dataset(**init_kwargs)
+
+    with pytest.raises(exception):
+        dataset.load()
+        dataset.pix2deg()
+        dataset.pos2vel()
+        dataset.save_preprocessed(**save_kwargs)
+
+
+@pytest.mark.parametrize(
+    'init_kwargs, load_kwargs, exception',
+    [
+        pytest.param(
+            {},
+            {'extension': 'invalid'},
+            ValueError,
+            id='wrong_extension_load_events',
+        ),
+    ],
+)
+def test_load_events_exceptions(init_kwargs, load_kwargs, exception, dataset_configuration):
+    init_kwargs = {**dataset_configuration['init_kwargs'], **init_kwargs}
+    dataset = Dataset(**init_kwargs)
+
+    with pytest.raises(exception):
+        dataset.load()
+        dataset.pix2deg()
+        dataset.pos2vel()
+        dataset.detect_events(
+            method=ivt,
+            velocity_threshold=45,
+            minimum_duration=55,
+        )
+        dataset.save_events()
+        dataset.load_event_files(**load_kwargs)
+
+
+@pytest.mark.parametrize(
+    'init_kwargs, save_kwargs, exception',
+    [
+        pytest.param(
+            {},
+            {'extension': 'invalid'},
+            ValueError,
+            id='wrong_extension_events',
+        ),
+    ],
+)
+def test_save_events_exceptions(init_kwargs, save_kwargs, exception, dataset_configuration):
+    init_kwargs = {**dataset_configuration['init_kwargs'], **init_kwargs}
+    dataset = Dataset(**init_kwargs)
+
+    with pytest.raises(exception):
+        dataset.load()
+        dataset.pix2deg()
+        dataset.pos2vel()
+        dataset.detect_events(
+            method=ivt,
+            velocity_threshold=45,
+            minimum_duration=55,
+        )
+        dataset.save_events(**save_kwargs)
+
+
 def test_load_no_files_raises_exception(dataset_configuration):
     init_kwargs = {**dataset_configuration['init_kwargs']}
     dataset = Dataset(**init_kwargs)
@@ -653,26 +731,36 @@ def test_clear_events(events_init, events_expected):
 
 
 @pytest.mark.parametrize(
-    'detect_event_kwargs, events_dirname, expected_save_dirpath',
+    'detect_event_kwargs, events_dirname, expected_save_dirpath, save_kwargs',
     [
         pytest.param(
             {'method': microsaccades, 'threshold': 1, 'eye': 'auto'},
             None,
             'events',
+            {},
             id='none_dirname',
         ),
         pytest.param(
             {'method': microsaccades, 'threshold': 1, 'eye': 'auto'},
             'events_test',
             'events_test',
+            {},
             id='explicit_dirname',
         ),
+        pytest.param(
+            {'method': microsaccades, 'threshold': 1, 'eye': 'auto'},
+            None,
+            'events',
+            {'extension': 'csv'},
+            id='save_events_extension_csv',
+        ),
     ],
 )
 def test_save_events(
         detect_event_kwargs,
         events_dirname,
         expected_save_dirpath,
+        save_kwargs,
         dataset_configuration,
 ):
     dataset = Dataset(**dataset_configuration['init_kwargs'])
@@ -685,7 +773,7 @@ def test_save_events(
         events_dirname = 'events'
     shutil.rmtree(dataset.path / Path(events_dirname), ignore_errors=True)
     shutil.rmtree(dataset.path / Path(expected_save_dirpath), ignore_errors=True)
-    dataset.save_events(events_dirname)
+    dataset.save_events(events_dirname, **save_kwargs)
 
     assert (dataset.path / expected_save_dirpath).is_dir(), (
         f'data was not written to {dataset.path / Path(expected_save_dirpath)}'
@@ -693,26 +781,36 @@ def test_save_events(
 
 
 @pytest.mark.parametrize(
-    'detect_event_kwargs, events_dirname, expected_save_dirpath',
+    'detect_event_kwargs, events_dirname, expected_save_dirpath, load_save_kwargs',
     [
         pytest.param(
             {'method': microsaccades, 'threshold': 1, 'eye': 'auto'},
             None,
             'events',
+            {},
             id='none_dirname',
         ),
         pytest.param(
             {'method': microsaccades, 'threshold': 1, 'eye': 'auto'},
             'events_test',
             'events_test',
+            {},
             id='explicit_dirname',
         ),
+        pytest.param(
+            {'method': microsaccades, 'threshold': 1, 'eye': 'auto'},
+            None,
+            'events',
+            {'extension': 'csv'},
+            id='load_events_extension_csv',
+        ),
     ],
 )
-def test_load_previously_saved_events(
+def test_load_previously_saved_events_gaze(
         detect_event_kwargs,
         events_dirname,
         expected_save_dirpath,
+        load_save_kwargs,
         dataset_configuration,
 ):
     dataset = Dataset(**dataset_configuration['init_kwargs'])
@@ -729,11 +827,12 @@ def test_load_previously_saved_events(
 
     shutil.rmtree(dataset.path / Path(events_dirname_), ignore_errors=True)
     shutil.rmtree(dataset.path / Path(expected_save_dirpath), ignore_errors=True)
-    dataset.save_events(events_dirname)
+    dataset.save_events(events_dirname, **load_save_kwargs)
+    dataset.save_preprocessed(**load_save_kwargs)
 
     dataset.events = []
 
-    dataset.load(events=True, events_dirname=events_dirname)
+    dataset.load(events=True, preprocessed=True, events_dirname=events_dirname, **load_save_kwargs)
     assert dataset.events
 
 
@@ -788,17 +887,23 @@ def test_save_preprocessed(preprocessed_dirname, expected_save_dirpath, dataset_
             'preprocessed_test',
             'events',
             {'preprocessed_dirname': 'preprocessed_test'},
-            id='none_dirname',
+            id='explicit_prepocessed_dirname',
         ),
         pytest.param(
             'preprocessed',
             'events_test',
             {'events_dirname': 'events_test'},
-            id='none_dirname',
+            id='explicit_events_dirname',
+        ),
+        pytest.param(
+            'preprocessed',
+            'events',
+            {'extension': 'csv'},
+            id='extension_equals_csv',
         ),
     ],
 )
-def test_save(
+def test_save_creates_correct_directory(
         expected_save_preprocessed_path,
         expected_save_events_path,
         save_kwargs,
@@ -822,13 +927,71 @@ def test_save(
     dataset.save(**save_kwargs)
 
     assert (dataset.path / Path(expected_save_preprocessed_path)).is_dir(), (
-        f'data was not written to {dataset.path / Path(expected_save_events_path)}'
+        f'data was not written to {dataset.path / Path(expected_save_preprocessed_path)}'
     )
     assert (dataset.path / Path(expected_save_events_path)).is_dir(), (
         f'data was not written to {dataset.path / Path(expected_save_events_path)}'
     )
 
 
+@pytest.mark.parametrize(
+    'expected_save_preprocessed_path, expected_save_events_path, save_kwargs',
+    [
+        pytest.param(
+            'preprocessed',
+            'events',
+            {'extension': 'feather'},
+            id='extension_equals_feather',
+        ),
+        pytest.param(
+            'preprocessed',
+            'events',
+            {'extension': 'csv'},
+            id='extension_equals_csv',
+        ),
+    ],
+)
+def test_save_files_have_correct_extension(
+        expected_save_preprocessed_path,
+        expected_save_events_path,
+        save_kwargs,
+        dataset_configuration,
+):
+    dataset = Dataset(**dataset_configuration['init_kwargs'])
+    dataset.load()
+    dataset.pix2deg()
+    dataset.pos2vel()
+
+    detect_events_kwargs = {'method': microsaccades, 'threshold': 1, 'eye': 'auto'}
+    dataset.detect_events(**detect_events_kwargs)
+
+    preprocessed_dirname = save_kwargs.get('preprocessed_dirname', 'preprocessed')
+    events_dirname = save_kwargs.get('events_dirname', 'events')
+    extension = save_kwargs.get('extension', 'feather')
+
+    shutil.rmtree(dataset.path / Path(preprocessed_dirname), ignore_errors=True)
+    shutil.rmtree(dataset.path / Path(expected_save_preprocessed_path), ignore_errors=True)
+    shutil.rmtree(dataset.path / Path(events_dirname), ignore_errors=True)
+    shutil.rmtree(dataset.path / Path(expected_save_events_path), ignore_errors=True)
+    dataset.save(**save_kwargs)
+
+    preprocessed_dir = dataset.path / Path(expected_save_preprocessed_path)
+    preprocessed_file_list = os.listdir(preprocessed_dir)
+    extension_list = [a.endswith(extension) for a in preprocessed_file_list]
+    extension_sum = sum(extension_list)
+    assert extension_sum == len(preprocessed_file_list), (
+        f'not all preprocessed files created have correct extension {extension}'
+    )
+
+    events_dir = dataset.path / Path(expected_save_events_path)
+    events_file_list = os.listdir(events_dir)
+    extension_list = [a.endswith(extension) for a in events_file_list]
+    extension_sum = sum(extension_list)
+    assert extension_sum == len(events_file_list), (
+        f'not all events files created have correct extension {extension}'
+    )
+
+
 @pytest.mark.parametrize(
     'init_kwargs, expected_paths',
     [