Repository: mne-tools/mne-bids-pipeline

Issue:
`ERROR: Could not consume arg: freesurfer` when finishing `run.py freesurfer`
Here is the output from running `run.py freesurfer`:
```
2021-04-15 17:02:00 INFO Now running: init_derivatives_dir
17:02:01 Using custom configuration: /home/merlin/hMT+/config.py
17:02:01 [Step-01] Running: Initializing output directories.
17:02:01 [Step-01] Initializing output directories.
17:02:01 [Step-01] Completed: Initializing output directories.
2021-04-15 17:02:01 INFO Successfully finished running: init_derivatives_dir
2021-04-15 17:02:01 INFO Now running: recon_all
17:02:01 Running FreeSurfer
17:02:01 Running recon-all on subject 01. This will take a LONG time – it's a good idea to let it run over night.
17:02:01 Subject 01 is already present. Please delete the directory if you want to recompute.
ERROR: Could not consume arg: freesurfer
Usage: recon_all.py freesurfer

```
Here I only ask to process subject 01, which was already done.
Even though there was no error within `recon_all.py` (which I checked using a print at the end of `main`), the script exits with an error.
The problem seems to be coming from using both Fire and runpy: https://github.com/mne-tools/mne-bids-pipeline/blob/d49d05351b2da2a4269dff5761a457e0e85964bd/run.py#L98
it seems Fire expects all arguments to be used for both `run.py` and `recon_all.py`, when they are only used for `run.py`.

Note that this is not a regression from #289, as it happens even without that modification.

I'm not sure how fix this, maybe it is simple though as it does not seem to be a problem for other scripts.


Discussion:
> 17:02:01 Running recon-all on subject 01. This will take a LONG time – it's a good idea to let it run over night.
> 17:02:01 Subject 01 is already present. Please delete the directory if you want to recompute.

Does the problem also occur if you remove the existing subjects?
If I remove the existing subject the code runs properly, but it takes a long time and I haven't had the time to run it to the end yet, as it crashed due to some problems with linking fsaverage. 
That should be solved now though, and I will report as soon as my current run is done.
I do have the same issue after deleting the existing subjects:

```
(mne-bids) merlin@Everchange:~/PhD/mne-bids-pipeline$ python run.py freesurfer --config=~/hMT+/config.py --n_jobs=4
2021-04-15 17:22:34 INFO Now running: init_derivatives_dir
17:22:34 Using custom configuration: /home/merlin/hMT+/config.py
17:22:34 [Step-01] Running: Initializing output directories.
17:22:34 [Step-01] Initializing output directories.
17:22:34 [Step-01] Completed: Initializing output directories.
2021-04-15 17:22:34 INFO Successfully finished running: init_derivatives_dir
2021-04-15 17:22:34 INFO Now running: recon_all
17:22:34 Running FreeSurfer
ERROR: Could not consume arg: freesurfer
Usage: recon_all.py freesurfer

For detailed information on this command, run:
  recon_all.py freesurfer --help
```

When setting `n_jobs` > 1 there is no output, however I know the freesurfer recon-all command was executed properly based on the log files.
Very interesting, especially since it only occurs after (successfully!) running FreeSurfer…
Yes when I wrote this file I saw that stdout from freesurfer was not shown.
But it does not mean it's not working.

>

The output not showing is not the problem here, it's that successful execution of the script results in an error, which might be confusing.

I'm guessing it's because the decorator `failsafe_run` is not used in `recon_all.py`.
> I'm guessing it's because the decorator failsafe_run is not used in recon_all.py.

After some testing, it does not appear to be the case.
Might be because the main fuction takes an argument, when for the other scripts it is not the case?
let us know if you go to the bottom of this. thx

>

I'll look into this tonight 
… I'm delayed, sorry. But this is on my agenda for today.

Resolving patch:
diff --git a/scripts/freesurfer/recon_all.py b/scripts/freesurfer/recon_all.py
--- a/scripts/freesurfer/recon_all.py
+++ b/scripts/freesurfer/recon_all.py
@@ -7,8 +7,6 @@
 import logging
 from typing import Union
 
-import fire
-
 from mne.utils import run_subprocess
 from mne.parallel import parallel_func
 
@@ -60,10 +58,10 @@ def run_recon(root_dir, subject, fs_bids_app) -> None:
     run_subprocess(cmd, env=env, verbose=logger.level)
 
 
-def main(*, n_jobs: int = 1) -> None:
+def main() -> None:
     """Run freesurfer recon-all command on BIDS dataset.
 
-    The command allows to run the freesurfer recon-all
+    The script allows to run the freesurfer recon-all
     command on all subjects of your BIDS dataset. It can
     run in parallel with the --n_jobs parameter.
 
@@ -71,26 +69,26 @@ def main(*, n_jobs: int = 1) -> None:
 
     https://github.com/BIDS-Apps/freesurfer
 
-    You must have freesurfer available on your system.
+    and the MNE BIDS Pipeline
 
-    Examples
-    --------
-    run_freesurfer.py /path/to/bids/dataset/study-template-config.py /path/to/freesurfer_bids_app/
+    https://mne.tools/mne-bids-pipeline
 
-    or to run in parallel (3 subjects at a time):
+    You must have freesurfer available on your system.
 
-    run_freesurfer.py /path/to/bids/dataset/study-template-config.py /path/to/freesurfer_bids_app/ --n_jobs=3
+    Run via the MNE BIDS Pipeline's `run.py`:
+
+    python run.py --steps=freesurfer --config=your_pipeline_config.py
 
     """  # noqa
 
     logger.info('Running FreeSurfer')
 
     subjects = config.get_subjects()
-
     root_dir = config.bids_root
     subjects_dir = _get_subjects_dir(root_dir)
     subjects_dir.mkdir(parents=True, exist_ok=True)
 
+    n_jobs = config.N_JOBS
     parallel, run_func, _ = parallel_func(run_recon, n_jobs=n_jobs)
     parallel(run_func(root_dir, subject, fs_bids_app)
              for subject in subjects)
@@ -109,4 +107,4 @@ def main(*, n_jobs: int = 1) -> None:
 
 
 if __name__ == '__main__':
-    fire.Fire(main)
+    main()


Test patch: