Repository: mne-tools/mne-python

Issue:
Inconsistent splits naming between `Raw` and `Epochs` when using `BIDS` schema
### Description of the problem

Not really a bug, more of an inconsistent behaviour.
Saving large `Raw` and `Epochs` data with `split_naming='bids'` uses different naming schema. 

When I save large `Raw`, I get suffixes `split-01` and `split-02`, etc. but when saving `Epochs`, the first split is saved without suffix, while the second one has `split-01`. See the code snippet below for details. All asserts pass for me.

Is there a reason for such difference?

### Steps to reproduce

```Python
import mne
from pathlib import Path

sample_data_folder = mne.datasets.sample.data_path()
sample_data_raw_file = sample_data_folder / "MEG" / "sample" / "sample_audvis_raw.fif"
raw = mne.io.read_raw_fif(sample_data_raw_file, verbose=False).crop(tmax=60)
ev = mne.find_events(raw)
epochs = mne.Epochs(raw, ev, tmin=-0.3, tmax=0.7)

raw.save("raw_meg.fif", split_naming="bids", split_size="50MB")
epochs.save("epochs_epo.fif", split_naming="bids", split_size="50MB")

# all these asserts pass
assert not Path("raw_meg.fif").is_file()
assert Path("raw_split-01_meg.fif").is_file()
assert Path("raw_split-02_meg.fif").is_file()

assert Path("epochs_epo.fif").is_file()  # this should fail though
assert Path("epochs_split-01_epo.fif").is_file()
assert not Path("epochs_split-02_epo.fif").is_file()  # this should fail as well
```


### Link to data

_No response_

### Expected results

These two asserts should fail
```python
assert Path("epochs_epo.fif").is_file()
assert not Path("epochs_split-02_epo.fif").is_file()
```

### Actual results

All asserts pass

### Additional information

Platform             macOS-13.4.1-arm64-arm-64bit
Python               3.11.3 (main, May 15 2023, 18:01:31) [Clang 14.0.6 ]
Executable           /Users/dmalt/Applications/miniconda3/envs/mne_cli_tools/bin/python3
CPU                  arm (8 cores)
Memory               Unavailable (requires "psutil" package)
Core
├☑ mne               1.4.2
├☑ numpy             1.25.2 (OpenBLAS 0.3.23.dev with 8 threads)
├☑ scipy             1.9.3
├☑ matplotlib        3.7.2Installed osx event loop hook.
 (backend=MacOSX)
├☑ pooch             1.7.0
└☑ jinja2            3.1.2

Numerical (optional)
├☑ sklearn           1.3.0
├☑ pandas            1.5.3
└☐ unavailable       numba, nibabel, nilearn, dipy, openmeeg, cupy

Visualization (optional)
└☐ unavailable       pyvista, pyvistaqt, ipyvtklink, vtk, qtpy, ipympl, pyqtgraph, mne-qt-browser

Ecosystem (optional)
└☐ unavailable       mne-bids, mne-nirs, mne-features, mne-connectivity, mne-icalabel, mne-bids-pipeline



Discussion:
Seems like a bug to me. @sappelhoff WDYT?
Thanks for the report! I agree, this looks like a bug.

Furthermore, a minor point: the indexes (01, 02, ... ) do not really need to be prefixed with a 0 if they never exceed 9 -- and I guess we can know that before writing the files and adjust the behavior accordingly.
Thanks for such a quick response from both of you!

Frankly, now that I think of it, I like the behaviour of `Epochs` better than that of `Raw`, although I suspect it was a conscious design choice in case of `Raw`.  

Picture this. You have to write a data processing pipeline for a new experiment for which only part of the data were recorded. So far all the recorded data are under 2GB, so you don't have to worry about splits. Then the new data arrive and for one subject the file size is above 2GB, so when you save the intermediate preprocessed copy, the saved file will have the `split-` suffix.  Now all the other scripts that are trying to access this file are broken and should be changed. If the intermediate steps don't reduce the file size, like filtering, the result for those steps will also have the `split-` suffix after saving, and so on. The situation gets even worse if you're using some sort of text-based templating framework for the filenames (like hydra). This actually happened to me once and it was really annoying to fix.

One way to avoid this would be to use the old "neuromag" split naming, but in this case I think BIDS checkers start complaining.

Instead if the main split didn't change its name, like for `Epochs` now,  everything would have worked the same way.
I'm not sure though, what BIDS specification has to say about this matter and what was the original thinking process behind the splits behaviour for `Raw` as it is now.

What do you guys think?
> Instead if the main split didn't change its name, like for Epochs now, everything would have worked the same way.
I'm not sure though, what BIDS specification has to say about this matter and what was the original thinking process behind the splits behaviour for Raw as it is now.

The BIDS specification says when a file is split, all parts of that file need to contain the split suffix -- so the behavior as it is for raw currently is the BIDS conformant behavior.

> It is RECOMMENDED that .fif files with multiple parts use the split-<index> entity to indicate each part.

https://bids-specification.readthedocs.io/en/latest/glossary.html#split-entities

I understand where you are coming from, but I think you'd have to build in a little logic into your reading code to deal with this problem, and/or rely on BIDS reading software like mne-bids. Although admittedly, we *do* have open issues about split files there, see:

- https://github.com/mne-tools/mne-bids/issues/1064
> The BIDS specification says when a file is split, all parts of that file need to contain the split suffix -- so the behavior as it is for raw currently is the BIDS conformant behavior.

Ok, I see, `Epochs.save` needs to change then. I can try making a PR .

> ...and/or rely on BIDS reading software like mne-bids

I don't think mne-bids supports splits reading and writing with the same template at the moment. I would partly solve the problem. For example, this fails:
```python
from pathlib import Path
from mne_bids import BIDSPath, read_raw_bids
from mne.io import read_raw
from mne.datasets import sample

root_path = Path("./tmp")
subj_path = root_path / "sub-01" / "meg"
subj_path.mkdir(parents=True)

raw_fpath = sample.data_path() / "MEG" / "sample" / "sample_audvis_raw.fif"
raw = read_raw(raw_fpath)
raw.crop(0, 10)

bp = BIDSPath(subject="01", root=root_path, suffix="meg", extension=".fif", datatype="meg")
raw.save(bp.fpath, split_size="10MB", split_naming="bids")
# produces tmp/sub-01/meg/sub-01_split-01_meg.fif, tmp/sub-01/meg/sub-01_split-02_meg.fif

loaded_raw = read_raw_bids(bp)  # fails with
# FileNotFoundError: File does not exist:
# tmp/sub-01/meg/sub-01_meg.fif
# Did you mean one of:
# sub-01_split-02_meg.fif
# sub-01_split-01_meg.fif
# instead of:
# sub-01_meg.fif

```

Something like this should work though, both with and without splits
```python
loaded_raw = read_raw_bids(sorted(bp.match(), key=str)[0])
```
(`sorted(..., key=str)` is required, since we want to pick the first split, but `BIDSPath`s are not comparable)

> https://github.com/mne-tools/mne-bids/issues/1064

Splits are a source of never ending joy, it seems :)
Now that I remember, I actually had a splits-related issue with mne-bids two years ago:
https://github.com/mne-tools/mne-bids/issues/712
ouch. it's really bad that `raw.save(my_bidspath)` generates filenames that can't be read by `read_raw_bids(my_bidspath)`.
BTW, is it possible to modify saving raw files in `.fif.gz` so all the splits live inside the same archive to avoid having splits altogether?
I think "parse, don't validate" rule applies here: it might be better to convert to a format that doesn't use splits once than to think about this corner case each time. Just a hypothetical thought :)

EDIT.
I guess, gunzip doesn't allow merging the splits together. It would have to be a tarball, so something like `.fif.tar.gz` instead. But still worth a thought. 
>
> BTW, is it possible to modify saving raw files in .fif.gz so all the
> splits live inside the same archive to avoid this altogether?
>
it's not what .gz (gzip) does. It only compresses one file in mne (to make
it smaller).

> I think "parse, not validate" rule applies here: it might be better to
> convert to a format that doesn't use splits once than to think about this
> corner case each time. Just a thought.
>
it's a trade off. Using .fif guarantees that all measurement info /
metadata  are not lost for MEG data. I don't know any other
format that guarantees this and that already has readers for most
programming languages and will likely be readable in > 10 years.

I would say we need to fix the way we handle split files to make it
painless even if it involves a bit of magic.

> Message ID: ***@***.***>
>

> it's not what .gz (gzip) does. It only compresses one file in mne (to make it smaller).

I see. Somehow I thought gzip was like zip archives.

I'm definitely not suggesting moving away from `.fif`. Merely having an option to wrap it into archive as a potential solution to splits. You already do the compression/decompression in io functions, when filename ends in `.fif.gz`. The same way you could wrap/unwrap split files (or a single file) in a tarball, when the extension is `.fif.tar.gz` or `.fif.tar`.  It's not gonna solve the splits problem completely, because you can't really drop the support for  the vanilla `.fif`, so the splits bugs will reappear from time to time. But if tarballs are advertised in the docs as a recommended way of dealing with splits, a lot of these bugs will go away for the end user. `.fif` splits are not meant to be treated as separate files: any time you move them apart or even rename them, everything breaks. It won't happen if they are inside an archive. 

I realise there are issues with this approach but still thought it was worth suggesting.  Don't know. Maybe it's all nonsense and will only complicate things further.

Resolving patch:
diff --git a/mne/epochs.py b/mne/epochs.py
--- a/mne/epochs.py
+++ b/mne/epochs.py
@@ -18,7 +18,7 @@
 
 import numpy as np
 
-from .io.utils import _construct_bids_filename
+from .io.utils import _make_split_fnames
 from .io.write import (
     start_and_end_file,
     start_block,
@@ -119,36 +119,22 @@ def _pack_reject_params(epochs):
     return reject_params
 
 
-def _save_split(epochs, fname, part_idx, n_parts, fmt, split_naming, overwrite):
+def _save_split(epochs, split_fnames, part_idx, n_parts, fmt, overwrite):
     """Split epochs.
 
     Anything new added to this function also needs to be added to
     BaseEpochs.save to account for new file sizes.
     """
     # insert index in filename
-    base, ext = op.splitext(fname)
-    if part_idx > 0:
-        if split_naming == "neuromag":
-            fname = "%s-%d%s" % (base, part_idx, ext)
-        else:
-            assert split_naming == "bids"
-            fname = _construct_bids_filename(base, ext, part_idx, validate=False)
-            _check_fname(fname, overwrite=overwrite)
+    this_fname = split_fnames[part_idx]
+    _check_fname(this_fname, overwrite=overwrite)
 
-    next_fname = None
+    next_fname, next_idx = None, None
     if part_idx < n_parts - 1:
-        if split_naming == "neuromag":
-            next_fname = "%s-%d%s" % (base, part_idx + 1, ext)
-        else:
-            assert split_naming == "bids"
-            next_fname = _construct_bids_filename(
-                base, ext, part_idx + 1, validate=False
-            )
         next_idx = part_idx + 1
-    else:
-        next_idx = None
+        next_fname = split_fnames[next_idx]
 
-    with start_and_end_file(fname) as fid:
+    with start_and_end_file(this_fname) as fid:
         _save_part(fid, epochs, fmt, n_parts, next_fname, next_idx)
 
 
@@ -2149,13 +2135,14 @@ def save(
 
         epoch_idxs = np.array_split(np.arange(n_epochs), n_parts)
 
+        _check_option("split_naming", split_naming, ("neuromag", "bids"))
+        split_fnames = _make_split_fnames(fname, n_parts, split_naming)
         for part_idx, epoch_idx in enumerate(epoch_idxs):
             this_epochs = self[epoch_idx] if n_parts > 1 else self
             # avoid missing event_ids in splits
             this_epochs.event_id = self.event_id
-            _save_split(
-                this_epochs, fname, part_idx, n_parts, fmt, split_naming, overwrite
-            )
+
+            _save_split(this_epochs, split_fnames, part_idx, n_parts, fmt, overwrite)
 
     @verbose
     def export(self, fname, fmt="auto", *, overwrite=False, verbose=None):
diff --git a/mne/io/utils.py b/mne/io/utils.py
--- a/mne/io/utils.py
+++ b/mne/io/utils.py
@@ -342,3 +342,18 @@ def _construct_bids_filename(base, ext, part_idx, validate=True):
     if dirname:
         use_fname = op.join(dirname, use_fname)
     return use_fname
+
+
+def _make_split_fnames(fname, n_splits, split_naming):
+    """Make a list of split filenames."""
+    if n_splits == 1:
+        return [fname]
+    res = []
+    base, ext = op.splitext(fname)
+    for i in range(n_splits):
+        if split_naming == "neuromag":
+            res.append(f"{base}-{i:d}{ext}" if i else fname)
+        else:
+            assert split_naming == "bids"
+            res.append(_construct_bids_filename(base, ext, i + 1))
+    return res


Test patch:
diff --git a/mne/tests/test_epochs.py b/mne/tests/test_epochs.py
--- a/mne/tests/test_epochs.py
+++ b/mne/tests/test_epochs.py
@@ -1574,13 +1574,13 @@ def test_split_saving_and_loading_back(tmp_path, epochs_to_split, preload):
         (
             "bids",
             "test_epo.fif",
-            lambda i: f"test_split-{i:02d}_epo.fif" if i else "test_epo.fif",
+            lambda i: f"test_split-{i + 1:02d}_epo.fif",
         ),
         (
             "bids",
-            "test-epo.fif",
+            "a_b-epo.fif",
             # Merely stating the fact:
-            lambda i: f"_split-{i:02d}_test-epo.fif" if i else "test-epo.fif",
+            lambda i: f"a_split-{i + 1:02d}_b-epo.fif",
         ),
     ],
     ids=["neuromag", "bids", "mix"],
@@ -1629,6 +1629,42 @@ def test_saved_fname_no_splitting(
     assert not split_1_fpath.is_file()
 
 
+@pytest.mark.parametrize(
+    "epochs_to_split",
+    [
+        ("3MB", 18, False, False, 3),
+        pytest.param(
+            ("2GB", 18, False, False, 1),
+            marks=pytest.mark.xfail(reason="No check when not splitting"),
+        ),
+    ],
+    indirect=True,
+)
+@pytest.mark.parametrize(
+    "dst_fname",
+    [
+        "test-epo.fif",
+        pytest.param(
+            "a_b_c-epo.fif",
+            marks=pytest.mark.xfail(reason="No check for several bids clauses"),
+        ),
+    ],
+)
+def test_bids_splits_fail_for_bad_fname_ending(epochs_to_split, dst_fname, tmp_path):
+    """Make sure split_naming=bids is only used with bids endings.
+
+    Non-bids endings can cause surprising split names, e.g. test-epo.fif
+    producing splits _split-01_test-epo.fif.
+
+    """
+    epochs, split_size, _ = epochs_to_split
+    dst_fpath = tmp_path / dst_fname
+    save_kwargs = {"split_naming": "bids", "split_size": split_size}
+
+    with pytest.raises(ValueError, match=".* must end with an underscore"):
+        epochs.save(dst_fpath, verbose=True, **save_kwargs)
+
+
 @pytest.mark.parametrize(
     "epochs_to_split", [("3MB", 18, False, False, 3)], indirect=True
 )
@@ -1636,12 +1672,7 @@ def test_saved_fname_no_splitting(
     "split_naming, dst_fname, existing_fname",
     [
         ("neuromag", "test-epo.fif", "test-epo.fif"),
-        pytest.param(
-            "neuromag",
-            "test-epo.fif",
-            "test-epo-1.fif",
-            marks=pytest.mark.xfail(reason="bug"),
-        ),
+        ("neuromag", "test-epo.fif", "test-epo-1.fif"),
         ("bids", "test_epo.fif", "test_epo.fif"),
         ("bids", "test_epo.fif", "test_split-01_epo.fif"),
         ("bids", "test_epo.fif", "test_split-02_epo.fif"),