Repository: nipy/nipype

Issue:
afni/preprocess Volreg interface does not produce md1d_file output.
I am having an issue with Volreg in afni/preprocess. When I specify a name for a max displacement 1D file (md1dfile input) to be written out, the regular 1D file is present, but the md1d file is ultimately missing from the working directory, and any nodes downstream expecting the md1d_file output of Volreg will report it missing.

However, when I take the command line string that the interface creates (taken from command.txt) and run it in terminal, it actually works and produces both the max displacement 1D file and the regular 1D file. So, it is creating the command properly.

If it helps, I did have the working directory for the node open one time while it was running, and I noticed that it DID write the md1d file - but it then poofed when the regular 1D file was written, as if it was being overwritten and renamed (the names I specified for both were different).

Here is also the inputs/outputs list from the report.rst, which is interesting. The md1d_file output remains <undefined>:
## Original Inputs
- args : -Fourier -twopass
- basefile : /home/likewise-open/CHILDMIND/steven.giavasis/run/temptest-work/resting_preproc_2014113/func_preproc_0/_scan_rest_1_rest/func_get_mean_motion/rest_calc_resample_volreg_tstat.nii.gz
- copyorigin : <undefined>
- environ : {}
- ignore_exception : False
- in_file : /home/likewise-open/CHILDMIND/steven.giavasis/run/temptest-work/resting_preproc_2014113/func_preproc_0/_scan_rest_1_rest/func_reorient/rest_calc_resample.nii.gz
- md1dfile : max_displacement.1D
- oned_file : <undefined>
- out_file : <undefined>
- outputtype : NIFTI_GZ
- terminal_output : allatonce
- timeshift : <undefined>
- verbose : <undefined>
- zpad : 4
## Execution Inputs
- args : -Fourier -twopass
- basefile : /home/likewise-open/CHILDMIND/steven.giavasis/run/temptest-work/resting_preproc_2014113/func_preproc_0/_scan_rest_1_rest/func_get_mean_motion/rest_calc_resample_volreg_tstat.nii.gz
- copyorigin : <undefined>
- environ : {}
- ignore_exception : False
- in_file : /home/likewise-open/CHILDMIND/steven.giavasis/run/temptest-work/resting_preproc_2014113/func_preproc_0/_scan_rest_1_rest/func_reorient/rest_calc_resample.nii.gz
- md1dfile : max_displacement.1D
- oned_file : <undefined>
- out_file : <undefined>
- outputtype : NIFTI_GZ
- terminal_output : allatonce
- timeshift : <undefined>
- verbose : <undefined>
- zpad : 4
## Execution Outputs
- md1d_file : <undefined>
- oned_file : /home/likewise-open/CHILDMIND/steven.giavasis/run/temptest-work/resting_preproc_2014113/func_preproc_0/_scan_rest_1_rest/func_motion_correct_A/rest_calc_resample.1D
- out_file : /home/likewise-open/CHILDMIND/steven.giavasis/run/temptest-work/resting_preproc_2014113/func_preproc_0/_scan_rest_1_rest/func_motion_correct_A/rest_calc_resample_volreg.nii.gz

Cheers,
Steve



Discussion:
by default nipype removes any file not needed downstream. so if that output is connected to something it will remain. or you can disable the feature by using: `worklfow.config['execution']['remove_unnecessary_outputs'] = False`

I do have this output (md1d_file) connected to an output node, and then this workflow connects that output into another workflow further down, should this still be happening?

Coming back to this, to illustrate:

```
#calculate motion parameters
func_motion_correct = pe.Node(interface=preprocess.Volreg(),
                         name='func_motion_correct')
func_motion_correct.inputs.args = '-Fourier -twopass'
func_motion_correct.inputs.zpad = 4
func_motion_correct.inputs.outputtype = 'NIFTI_GZ'

....
....

func_motion_correct_A = func_motion_correct.clone('func_motion_correct_A')
func_motion_correct_A.inputs.md1dfile = 'max_displacement.1D'

preproc.connect(func_reorient, 'out_file',
                func_motion_correct_A, 'in_file')
preproc.connect(func_get_mean_motion, 'out_file',
                func_motion_correct_A, 'basefile')

preproc.connect(func_motion_correct_A, 'out_file',
                outputNode, 'motion_correct')
preproc.connect(func_motion_correct_A, 'md1d_file',
                outputNode, 'max_displacement')
preproc.connect(func_motion_correct_A, 'oned_file',
                outputNode, 'movement_parameters')
```

With these connections (the md1d_file going to the outputNode, which ultimately feeds the file to the resource pool - which then gets used later on anyway), my max_displacement.1D file still gets removed. Am I missing something?

Thanks again,

Steve

i very much suspect outputnode is an identityinterface, which all get removed before execution - since they act as passthrough nodes. so if max_displacement doesn't connect further, it would explain why your output file is getting removed. 

Yes, this is true. But it does connect further, in fact, I only found out about this because a workflow downstream was expecting the max displacement:

```
            node, out_file = strat.get_node_from_resource_pool('max_displacement')
            workflow.connect(node, out_file, gen_motion_stats, 'inputspec.max_displacement')
```

(max displacement in the resource pool coming from the workflow I posted in my last post)

And with this set up, I first stumbled on this issue due to this error, with gen_motion_parameters being a node within the gen_motion_stats workflow:

```
            TypeError: gen_motion_parameters() takes exactly 4 arguments (3 given)
            Interface Function failed to run.
```

Max displacement being the missing argument.

Would the connection being enclosed within a try block have anything to do with it maybe?


Resolving patch:
diff --git a/nipype/interfaces/afni/preprocess.py b/nipype/interfaces/afni/preprocess.py
--- a/nipype/interfaces/afni/preprocess.py
+++ b/nipype/interfaces/afni/preprocess.py
@@ -560,9 +560,9 @@ class VolregInputSpec(AFNICommandInputSpec):
                       ' by \'n\' voxels during rotations',
                       argstr='-zpad %d',
                       position=-5)
-    md1dfile = File(desc='max displacement output file',
-                    argstr='-maxdisp1D %s',
-                    position=-4)
+    md1d_file = File(name_template='%s_md.1D', desc='max displacement output file',
+                    argstr='-maxdisp1D %s', name_source="in_file",
+                    keep_extension=True, position=-4)
     oned_file = File(name_template='%s.1D', desc='1D movement parameters output file',
                      argstr='-1Dfile %s',
                      name_source="in_file",
@@ -597,7 +597,7 @@ class Volreg(AFNICommand):
     >>> volreg.inputs.zpad = 4
     >>> volreg.inputs.outputtype = "NIFTI"
     >>> volreg.cmdline #doctest: +ELLIPSIS
-    '3dvolreg -Fourier -twopass -1Dfile functional.1D -prefix functional_volreg.nii -zpad 4 functional.nii'
+    '3dvolreg -Fourier -twopass -1Dfile functional.1D -prefix functional_volreg.nii -zpad 4 -maxdisp1D functional_md.1D functional.nii'
     >>> res = volreg.run() # doctest: +SKIP
 
     """


Test patch:
diff --git a/nipype/interfaces/afni/tests/test_auto_Volreg.py b/nipype/interfaces/afni/tests/test_auto_Volreg.py
--- a/nipype/interfaces/afni/tests/test_auto_Volreg.py
+++ b/nipype/interfaces/afni/tests/test_auto_Volreg.py
@@ -20,17 +20,17 @@ def test_Volreg_inputs():
     args=dict(argstr='%s',
     ),
     outputtype=dict(),
-    md1dfile=dict(position=-4,
-    argstr='-maxdisp1D %s',
-    ),
     zpad=dict(position=-5,
     argstr='-zpad %d',
     ),
     terminal_output=dict(mandatory=True,
     nohash=True,
     ),
-    environ=dict(nohash=True,
-    usedefault=True,
+    md1d_file=dict(name_source='in_file',
+    keep_extension=True,
+    position=-4,
+    name_template='%s_md.1D',
+    argstr='-maxdisp1D %s',
     ),
     in_file=dict(position=-1,
     mandatory=True,
@@ -38,6 +38,9 @@ def test_Volreg_inputs():
     ),
     copyorigin=dict(argstr='-twodup',
     ),
+    environ=dict(nohash=True,
+    usedefault=True,
+    ),
     out_file=dict(name_source='in_file',
     name_template='%s_volreg',
     argstr='-prefix %s',