Repository: mne-tools/mne-bids (python)

Issue:
get_entity_vals should ignore sidecars
### Describe the problem

My use case for `get_entity_vals` is this: for a given subject + task, find which sessions exist where the subject performed that task. This is proving to be quite hard (impossible?) to do with `get_entity_vals`, because `get_entity_vals` doesn't seem to discriminate when it comes to data files vs sidecars. MWE:

```py
from pathlib import Path
from tempfile import TemporaryDirectory

import mne_bids as mb

# create temporary fake BIDS dataset
with TemporaryDirectory() as root:
    root = Path(root)
    subdir = root / "sub-1" / "ses-a" / "meg"
    subdir.mkdir(parents=True)
    stem = "sub-1_ses-a_{}"
    for fname in ("task-IDS_meg.fif", "coordsystem.json"):
        (subdir / stem.format(fname)).touch()
    (subdir.parent / stem.format("scans.tsv")).touch()

    mb.print_dir_tree(root)

    # which of sub-1's session(s) have data for tasks other than IDS?
    result = mb.get_entity_vals(
        root / "sub-1",
        entity_key="session",
        ignore_tasks=("IDS",),
    )
print(result)
```

Result is this:
```
|tmp39vnjgza/
|--- sub-1/
|------ ses-a/
|--------- sub-1_ses-a_scans.tsv   # <-- false positive?
|--------- meg/
|------------ sub-1_ses-a_coordsystem.json   # <-- false positive?
|------------ sub-1_ses-a_task-IDS_meg.fif   # this is ignored by `ignore_tasks`
['a']
```

Expected (or at least desired) result here is that `get_entity_vals` returns `[]` because there are no *data files* once `ignore_tasks` has taken effect. I don't care if there are `coordsystem` or `scans` files present, because they relate to task(s) that I'm ignoring.

### Describe your solution

`get_entity_vals` should (have a param that lets you easily) ignore non-data sidecars.  I'm not super clear on what the ramifications of that would be, exactly which sidecars should be ignored, or how widely used `get_entity_vals` is internally or in user code. But at least the use cases I've thought about, I can't see why non-data files would be of interest --- in theory they'll only be present if data files are *also* present, and the spirit of the params like `ignore_tasks`, `ignore_sessions` etc seems to suggest that (one of) the intended uses of this function is to check whether a given task is present for a given subject, or check which subjects have task IDS in session a, or similar such queries.

### Describe possible alternatives

Maybe there's a better way to do what I want than `get_entity_vals`, that I'm not thinking of?

### Additional context

_No response_

Pull request:
allow get_entity_vals to ignore sidecars
closes #1351 

PR Description
--------------

adds a new parameter `ignore_suffixes` to `get_entity_vals` so that sidecars (scans, coordsystem) may easily be ignored.

Merge checklist
---------------

Maintainer, please confirm the following before merging.
If applicable:

- [ ] All comments are resolved
- [ ] This is not your own PR
- [x] All CIs are happy
- [ ] PR title starts with [MRG]
- [x] [whats_new.rst](https://github.com/mne-tools/mne-bids/blob/main/doc/whats_new.rst) is updated
- [x] New contributors have been added to [CITATION.cff](https://github.com/mne-tools/mne-bids/blob/main/CITATION.cff)
- [x] PR description includes phrase "closes <#issue-number>"


Patch:
diff --git a/doc/whats_new.rst b/doc/whats_new.rst
index 5bf145c1..17039eeb 100644
--- a/doc/whats_new.rst
+++ b/doc/whats_new.rst
@@ -32,6 +32,7 @@ Detailed list of changes
 ^^^^^^^^^^^^^^^
 
 - :func:`mne_bids.write_raw_bids()` can now handle mne `Raw` objects with `eyegaze` and `pupil` channels, by `Christian O'Reilly`_ (:gh:`1344`)
+- :func:`mne_bids.get_entity_vals()` has a new parameter ``ignore_suffixes`` to easily ignore sidecar files, by `Daniel McCloy`_ (:gh:`1362`)
 
 
 🧐 API and behavior changes
diff --git a/mne_bids/path.py b/mne_bids/path.py
index 83c5ebe5..d0471a0a 100644
--- a/mne_bids/path.py
+++ b/mne_bids/path.py
@@ -4,6 +4,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 
 import glob
+import inspect
 import json
 import os
 import re
@@ -1897,6 +1898,7 @@ def get_entity_vals(
     ignore_modalities=None,
     ignore_datatypes=None,
     ignore_dirs=("derivatives", "sourcedata"),
+    ignore_suffixes=None,
     with_key=False,
     verbose=None,
 ):
@@ -1949,7 +1951,7 @@ def get_entity_vals(
 
         .. versionadded:: 0.11
     ignore_modalities : str | array-like of str | None
-        Modalities(s) to ignore. If ``None``, include all modalities.
+        Modalities to ignore. If ``None``, include all modalities.
     ignore_datatypes : str | array-like of str | None
         Datatype(s) to ignore. If ``None``, include all datatypes (i.e.
         ``anat``, ``ieeg``, ``eeg``, ``meg``, ``func``, etc.)
@@ -1958,6 +1960,11 @@ def get_entity_vals(
         include all directories in the search.
 
         .. versionadded:: 0.9
+    ignore_suffixes : str | array-like of str | None
+        Suffixes to ignore. If ``None``, include all suffixes. This can be helpful for
+        ignoring non-data sidecars such as `*_scans.tsv` or `*_coordsystem.json`.
+
+        .. versionadded:: 0.17
     with_key : bool
         If ``True``, returns the full entity with the key and the value. This
         will for example look like ``['sub-001', 'sub-002']``.
@@ -1990,6 +1997,7 @@ def get_entity_vals(
     .. [1] https://bids-specification.rtfd.io/en/latest/common-principles.html#entities
 
     """
+    params = inspect.signature(get_entity_vals).parameters  # for debug messages
     root = _check_fname(
         fname=root,
         overwrite="read",
@@ -2043,7 +2051,7 @@ def get_entity_vals(
     ignore_splits = _ensure_tuple(ignore_splits)
     ignore_descriptions = _ensure_tuple(ignore_descriptions)
     ignore_modalities = _ensure_tuple(ignore_modalities)
-
+    ignore_suffixes = _ensure_tuple(ignore_suffixes)
     ignore_dirs = _ensure_tuple(ignore_dirs)
     existing_ignore_dirs = [
         root / d for d in ignore_dirs if (root / d).exists() and (root / d).is_dir()
@@ -2061,6 +2069,10 @@ def get_entity_vals(
         ):
             continue
 
+        if ignore_suffixes and any(
+            [filename.stem.endswith(s) for s in ignore_suffixes]
+        ):
+            continue
         if ignore_datatypes and filename.parent.name in ignore_datatypes:
             continue
         if ignore_subjects and any(
@@ -2110,6 +2122,15 @@ def get_entity_vals(
             value = f"{entity_long_abbr_map[entity_key]}-{value}"
         if value not in values:
             values.append(value)
+            # display all non-default params passed into the function
+            param_string = ", ".join(
+                f"{k}={v!r}"
+                for k, v in inspect.currentframe().f_back.f_locals.items()
+                if k in params and v != params[k].default
+            )
+            logger.debug(
+                "%s matched by get_entity_vals(%s)", filename.name, param_string
+            )
     return sorted(values)
 
 


Test patch:
diff --git a/mne_bids/tests/test_path.py b/mne_bids/tests/test_path.py
index 605050f9..0eecdf49 100644
--- a/mne_bids/tests/test_path.py
+++ b/mne_bids/tests/test_path.py
@@ -100,6 +100,15 @@ def test_get_keys(return_bids_test_dir):
         ("bogus", None, None),
         ("subject", [subject_id], None),
         ("session", [session_id], None),
+        (
+            "session",
+            [],
+            dict(
+                ignore_tasks="testing",
+                ignore_acquisitions=("calibration", "crosstalk"),
+                ignore_suffixes=("scans", "coordsystem"),
+            ),
+        ),
         ("run", [run, "02"], None),
         ("acquisition", ["calibration", "crosstalk"], None),
         ("task", [task], None),
@@ -153,7 +162,6 @@ def test_get_entity_vals(entity, expected_vals, kwargs, return_bids_test_dir):
         )
         if entity not in ("acquisition", "run"):
             assert "deriv" in entities
-
     # Clean up
     shutil.rmtree(deriv_path)