Repository: mne-tools/mne-bids-pipeline

Issue:
_bads.tsv missing for reference run (preprocessing/_03_maxfilter)
Sorry, another issue: 
It seems like the pipeline wants an extra file for the bad channels for the reference run, but in my BIDS dataset, bad channels are marked in the channels.tsv files for each run, and I don't have the _bads.tsv file
Is there a parameter I am missing to use this file?
Thanks!

```
┌────────┬ preprocessing/_03_maxfilter ─────────────────────────────────────────
│16:57:50│ ❌ sub-155 run-01 A critical error occurred. The error message was: missing in_files["raw_ref_run-bads"] = /neurospin/meg/meg_tmp/TimeInWM_Izem_2019/BIDS_anonymized/derivatives/mne-bids-pipeline/sub-155/meg/sub-155_task-tiwm_run-01_bads.tsv

Aborting pipeline run. The traceback is:

  File "/home/sh254795/Documents/REPOS/mne-study-template/mne_bids_pipeline/_run.py", line 412, in _path_to_str_hash
    assert v.exists(), f'missing {kind}_files["{k}"] = {v}'
```



Discussion:
This is on latest `main`? And on a new/clean run of the data (with derivatives deleted)?

There are some complicated conditionals about when to add bads and where they come from. It's possible there is an error in the logic somewhere. Any chance you can share one subject's data and a config file with me privately that I can use to reproduce the error?
Hi @larsoner, I confirm, no existing derivatives when starting the pipeline, and latest main. I will send you a link. Thanks!

Resolving patch:
diff --git a/mne_bids_pipeline/_import_data.py b/mne_bids_pipeline/_import_data.py
--- a/mne_bids_pipeline/_import_data.py
+++ b/mne_bids_pipeline/_import_data.py
@@ -683,10 +683,11 @@ def _get_run_rest_noise_path(
 
 
 def _get_mf_reference_run_path(
+    *,
     cfg: SimpleNamespace,
     subject: str,
     session: Optional[str],
-    add_bads: bool,
+    add_bads: Optional[bool] = None,
 ) -> dict:
     return _get_run_path(
         cfg=cfg,
diff --git a/mne_bids_pipeline/steps/preprocessing/_03_maxfilter.py b/mne_bids_pipeline/steps/preprocessing/_03_maxfilter.py
--- a/mne_bids_pipeline/steps/preprocessing/_03_maxfilter.py
+++ b/mne_bids_pipeline/steps/preprocessing/_03_maxfilter.py
@@ -64,7 +64,7 @@ def get_input_fnames_esss(
         mf_reference_run=cfg.mf_reference_run,
         **kwargs,
     )
-    in_files.update(_get_mf_reference_run_path(add_bads=True, **kwargs))
+    in_files.update(_get_mf_reference_run_path(**kwargs))
     return in_files
 
 
@@ -241,7 +241,8 @@ def get_input_fnames_maxwell_filter(
         )
 
     # reference run (used for `destination` and also bad channels for noise)
-    in_files.update(_get_mf_reference_run_path(add_bads=True, **kwargs))
+    # use add_bads=None here to mean "add if autobad is turned on"
+    in_files.update(_get_mf_reference_run_path(**kwargs))
 
     is_rest_noise = run is None and task in ("noise", "rest")
     if is_rest_noise:


Test patch:
diff --git a/mne_bids_pipeline/tests/configs/config_ds003392.py b/mne_bids_pipeline/tests/configs/config_ds003392.py
--- a/mne_bids_pipeline/tests/configs/config_ds003392.py
+++ b/mne_bids_pipeline/tests/configs/config_ds003392.py
@@ -6,8 +6,9 @@
 subjects = ["01"]
 
 task = "localizer"
-find_flat_channels_meg = True
-find_noisy_channels_meg = True
+# usually a good idea to use True, but we know no bads are detected for this dataset
+find_flat_channels_meg = False
+find_noisy_channels_meg = False
 use_maxwell_filter = True
 ch_types = ["meg"]