Repository: mne-tools/mne-bids-pipeline

Issue:
Adding a param exclude_runs?
I am working on a study with 20 subjects with 8 runs each. But among these 20*8 runs, only 3 are faulty. So I would like to add a parameter `exclude_runs` mapping the participants to their faulty runs. For example: `exclude_runs={"215": ["03"], ...}`. I started to discuss with @hoechenberger, and he suggested me to create a method to specify the valid runs to consider instead. Here is the transcript of the conversation

Charbel 
Shall I begin to code the param `exclude_runs`?

Richard 
I thought what we wanted instead was specifying the includes for each participant individually … no?

Charbel
`include_runs` is already `runs`. And `exclude_runs` would be more symmetric because we already have a param `exclude_subjects`. wdyt ?

Richard 
I generally like the idea but I’m wondering if “bad” runs aren’t a problem that should be fixed in BIDS beforehand?
With subjects it’s different – you may want to exclude only a small number of participants from a big study without having to provide a super long “subjects” list. Whereas for runs, we rarely have more than 8, 9, maybe 10, so it’s easy to specify all that you want explicitly

Charbel

> “bad” runs aren’t a problem that should be fixed in BIDS beforehand?

Since the exclusion of whole subjects is already done in the params of the bids pipeline, I don't think it would be a considerable conceptual stretch to add the exclude_runs param?
But yes, having a long list of runs isn't a problem, and would be very easy to code.


Discussion:
Why putting in the dataset runs that are broken in the first place ?

Good point. But if so, why does the param `exclude_subject` was added in the pipeline? It seems to me that it takes time to clean perfectly a dataset, and having more malleability in the pipeline can't hurt?
That is an interesting point @crsegerie makes here. He's currently in this iterative back-and-forth process of running the pipeline, discovering issues in the BIDS data, fixing BIDS, running the pipeline again, ...

However we already figured that some runs for certain participants are problematic (and probably shouldn't have been part of our BIDS dataset to begin with); but now @crsegerie has to idle until I've fixed the data on NS (I need to do the BIDS conversion and anonymization again), .....

So from a theoretical "good science" perspective, I agree with @agramfort: the BIDs data is what needs to be fixed. But from an "I need to get a job done without constant interruptions; and sometimes I want to proceed even if part of the data is broken, if only to avoid constant task switching" I believe this feature could totally make sense. 

But maybe I've simply had too much coffee today ☕️😅
> So from a theoretical "good science" perspective, I agree with @agramfort: the BIDs data is what needs to be fixed. But from an "I need to get a job done without constant interruptions; and sometimes I want to proceed even if part of the data is broken, if only to avoid constant task switching" I believe this feature could totally make sense.

I think this summarizes the issue nicely.  To me, preparing the dataset for distribution/dissemination is a separate activity from analysis, and as you say, during prelim analysis it would be nice to be able to exclude certain runs on-the-fly when you discover that they are problematic.  I did this myself yesterday (not using the pipeline; just saying it's a need that arises often)
ok I buy the argument. Let's add this exclude_runs parameter and be
pragmatic here.

>


Resolving patch:
diff --git a/config.py b/config.py
--- a/config.py
+++ b/config.py
@@ -89,6 +89,22 @@
 The runs to process.
 """
 
+exclude_runs: Optional[Dict[str, List[str]]] = None
+"""
+Specify runs to exclude from analysis, for each participant individually.
+
+???+ example "Example"
+    ```python
+    exclude_runs = None  # Include all runs.
+    exclude_runs = {'01': ['02']}  # Exclude run 02 of subject 01.
+    ```
+
+???+ info "Good Practice / Advice"
+    Keep track of the criteria leading you to exclude
+    a run (e.g. too many movements, missing blocks, aborted experiment,
+    did not understand the instructions, etc.).
+"""
+
 crop_runs: Optional[Tuple[float, float]] = None
 """
 Crop the raw data of each run to the specified time interval ``[tmin, tmax]``,
@@ -1553,6 +1569,9 @@ def get_runs_all_subjects() -> dict:
         valid_runs_subj = get_entity_vals(
             get_bids_root(), entity_key='run',
             ignore_subjects=ignore_subjects)
+        if exclude_runs and subj in exclude_runs:
+            valid_runs_subj = [r for r in valid_runs_subj
+                               if r not in exclude_runs[subj]]
         subj_runs[subj] = valid_runs_subj
 
     return subj_runs


Test patch: