Repository: mne-tools/mne-bids-pipeline

Issue:
Config error when specifying no high-pass filtering on the raw data
The checks to make sure ica_l_freq < l_freq does not take into account the possibility where l_freq is None.

The code fails on these checks: 
- https://github.com/mne-tools/mne-bids-pipeline/blob/0c67af9e7587bf6639def22c553e0eb64927b371/config.py#L1306
- https://github.com/mne-tools/mne-bids-pipeline/blob/0c67af9e7587bf6639def22c553e0eb64927b371/scripts/preprocessing/04a-run_ica.py#L72
- https://github.com/mne-tools/mne-bids-pipeline/blob/0c67af9e7587bf6639def22c553e0eb64927b371/scripts/preprocessing/04a-run_ica.py#L75

both when l_freq is None and ica_l_freq > 0 (expected to raise an error but currently the error is a TypeError), and when l_freq and ica_l_freq are both None.

This can be reproduced by setting l_freq and ica_l_freq to None in the config file.

I don't know what the best practice is: I want to estimate the low-frequency power spectrum, so I can't have my data high-passed. However I'm not sure it is a good idea to run the ICA with no high-pass filter.
I was able to get the code running with a few modifications, but I don't know if I should include highpass filtering for the ICA or not.


Discussion:
Thanks for reporting this! We should check whether `config.l_freq` or `config.ica_l_freq` are `None` before doing the comparisons.



> However I'm not sure it is a good idea to run the ICA with no high-pass filter.

It's not a good idea, you need to remove those slow drifts, otherwise ICA will have a bad time.

So you'd want to e.g. use

```python
l_freq = 0.1
ica_l_freq = 1
```
I'm working on a project where I need to estimate the power spectrum well below 1 Hz so I really cannot afford to highpass filter the raw data.

Would this work?
```python
l_freq = None
ica_l_freq = 1
```
In the 'old' version, we just high-pass filtered the raw and created new epochs_for_ica, independently from the hp applied to the data: https://github.com/brainthemind/CogBrainDyn_MEG_Pipeline/blob/master/05a-run_ica.py
Would that still work, @hoechenberger?

Resolving patch:
diff --git a/config.py b/config.py
--- a/config.py
+++ b/config.py
@@ -834,7 +834,7 @@
 """
 The cutoff frequency of the high-pass filter to apply before running ICA.
 Using a relatively high cutoff like 1 Hz will remove slow drifts from the
-data, yielding improved ICA results.
+data, yielding improved ICA results. Must be set to 1 Hz or above.
 
 Set to ``None`` to not apply an additional high-pass filter.
 
@@ -842,6 +842,13 @@
       The filter will be applied to raw data which was already filtered
       according to the ``l_freq`` and ``h_freq`` settings. After filtering, the
       data will be epoched, and the epochs will be submitted to ICA.
+
+!!! info
+    The Pipeline will only allow you to perform ICA on data that has been
+    high-pass filtered with a 1 Hz cutoff or higher. This is a conscious,
+    opinionated (but partially data-driven) decision made by the developers.
+    If you have reason to challenge this behavior, please get in touch with
+    us so we can discuss.
 """
 
 ica_max_iterations: int = 500
@@ -1303,7 +1310,25 @@ def get_t1_from_meeg(bids_path):
            f"{ica_algorithm}.")
     raise ValueError(msg)
 
-if use_ica and ica_l_freq < l_freq:
+if (use_ica and ica_l_freq is None and
+        l_freq is not None and l_freq < 1):
+    msg = (f'You requested to high-pass filter your data with l_freq={l_freq} '
+           f'Hz and to perform ICA without performing any additional '
+           f'filtering first by setting ica_l_freq=None. However, ICA will '
+           f'not work reliably unless slow drifts have been removed from the '
+           f'data. Please either increase l_freq to 1 Hz or above, or enable '
+           f'additional filtering for ICA by setting ica_l_freq to 1 Hz or '
+           f'higher.')
+    raise ValueError(msg)
+
+if use_ica and ica_l_freq < 1:
+    msg = (f'You requested to high-pass filter the data before ICA with '
+           f'ica_l_freq={ica_l_freq} Hz. Please increase this setting to '
+           f'1 Hz or above to ensure reliable ICA function.')
+    raise ValueError(msg)
+
+if (use_ica and ica_l_freq is not None and l_freq is not None and
+        ica_l_freq < l_freq):
     msg = (f'You requested a lower high-pass filter cutoff frequency for ICA '
            f'than for your raw data: ica_l_freq = {ica_l_freq} < '
            f'l_freq = {l_freq}. Adjust the cutoffs such that ica_l_freq >= '
diff --git a/scripts/preprocessing/04a-run_ica.py b/scripts/preprocessing/04a-run_ica.py
--- a/scripts/preprocessing/04a-run_ica.py
+++ b/scripts/preprocessing/04a-run_ica.py
@@ -69,14 +69,9 @@ def load_and_concatenate_raws(bids_path):
 
 def filter_for_ica(raw, subject, session):
     """Apply a high-pass filter if needed."""
-    if config.ica_l_freq <= config.l_freq or config.ica_l_freq is None:
-        # Nothing to do here!
-        msg = 'Not applying high-pass filter '
-        if config.ica_l_freq <= config.l_freq:
-            msg += (f'(data is already filtered, '
-                    f'cutoff: {raw.info["highpass"]} Hz).')
-        else:
-            msg = '(no filtering requested).'
+    if config.ica_l_freq is None:
+        msg = (f'Not applying high-pass filter (data is already filtered, '
+               f'cutoff: {raw.info["highpass"]} Hz).')
         logger.info(gen_log_message(message=msg, step=4, subject=subject,
                                     session=session))
     else:


Test patch: