Repository: nipy/nipype

Issue:
SelectFiles doesn't accept lists as input
### Summary
The documentation for SelectFiles indicates that you should be able to pass lists as inputs for the fields defined the template string. When I try that, I get the error: `TypeError: unsupported format string passed to list.__format__` 

### Actual behavior
Crashes with the error: `TypeError: unsupported format string passed to list.__format__` 

### Expected behavior
SelectFiles accepts an subject_id as a list and pull the anatomicals from the test session for both subject 1 and subject 2. As shown in the [documentation](https://nipype.readthedocs.io/en/latest/interfaces/generated/nipype.interfaces.io.html#id8). 

### How to replicate the behavior
Run the following snippet in the Nipype Tutorial container: miykael/nipype_tutorial:latest

### Script/Workflow details
Running in the Nipype Tutorial container: miykael/nipype_tutorial:latest
```
from nipype import Node, SelectFiles
template = {'anat': 'sub-{subject_id:02d}/ses-{ses_name}/anat/*_T1w.nii.gz'}
sf = Node(SelectFiles(template, 
                      force_lists=True,
                      base_directory='/data/ds000114/',
                      sort_filelist=True),
          name='selectfile')
sf.inputs.subject_id = [1,2]
sf.inputs.ses_name = 'test'
sf.run()
```
produces:
```
181004-18:58:46,481 nipype.workflow INFO:
	 [Node] Setting-up "selectfile" in "/tmp/tmpu4a95r0i/selectfile".
181004-18:58:46,496 nipype.workflow INFO:
	 [Node] Running "selectfile" ("nipype.interfaces.io.SelectFiles")
181004-18:58:46,513 nipype.workflow WARNING:
	 [Node] Error on "selectfile" (/tmp/tmpu4a95r0i/selectfile)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-7559abb2aeb6> in <module>()
      8 sf.inputs.subject_id = [1,2]
      9 sf.inputs.ses_name = 'test'
---> 10 sf.run()

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in run(self, updatehash)
    469 
    470         try:
--> 471             result = self._run_interface(execute=True)
    472         except Exception:
    473             logger.warning('[Node] Error on "%s" (%s)', self.fullname, outdir)

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in _run_interface(self, execute, updatehash)
    553             self._update_hash()
    554             return self._load_results()
--> 555         return self._run_command(execute)
    556 
    557     def _load_results(self):

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/pipeline/engine/nodes.py in _run_command(self, execute, copyfiles)
    633         logger.info(message)
    634         try:
--> 635             result = self._interface.run(cwd=outdir)
    636         except Exception as msg:
    637             result.runtime.stderr = '%s\n\n%s'.format(

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py in run(self, cwd, ignore_exception, **inputs)
    521             runtime = self._run_interface(runtime)
    522             runtime = self._post_run_hook(runtime)
--> 523             outputs = self.aggregate_outputs(runtime)
    524         except Exception as e:
    525             import traceback

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/base/core.py in aggregate_outputs(self, runtime, needed_outputs)
    595         """
    596 
--> 597         predicted_outputs = self._list_outputs()
    598         outputs = self._outputs()
    599         if predicted_outputs:

/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype/interfaces/io.py in _list_outputs(self)
   1414 
   1415             # Fill in the template and glob for files
-> 1416             filled_template = template.format(**info)
   1417             filelist = glob.glob(filled_template)
   1418 

TypeError: unsupported format string passed to list.__format__
```
Please put URL to code or code here (if not too long).

### Platform details:

<!-- Please run the following code from your shell and place the output between the triple ticks, below.
python -c "import nipype; from pprint import pprint; pprint(nipype.get_info())"
-->

```
{'commit_hash': '%h',
 'commit_source': 'archive substitution',
 'networkx_version': '2.1',
 'nibabel_version': '2.3.0',
 'nipype_version': '1.1.3-dev',
 'numpy_version': '1.15.1',
 'pkg_path': '/opt/miniconda-latest/envs/neuro/lib/python3.6/site-packages/nipype',
 'scipy_version': '1.1.0',
 'sys_executable': '/opt/miniconda-latest/envs/neuro/bin/python',
 'sys_platform': 'linux',
 'sys_version': '3.6.6 | packaged by conda-forge | (default, Jul 26 2018, '
                '09:53:17) \n'
                '[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]',
 'traits_version': '4.6.0'}
```

### Execution environment

Choose one
- Container  miykael/nipype_tutorial:latest, environment [conda env:neuro]




Discussion:
@Shotgunosine lists should still be allowed, the problem is that you are trying to format a list with `:02d`. If you change your template to
```
template = {'anat': 'sub-{subject_id}/ses-{ses_name}/anat/*_T1w.nii.gz'}
```
it should remove the error
This is more of an enhancement suggestion, but from what I can observe, `SelectFiles` doesn't actually support list inputs like `DataGrabber`.

Given a list as argument, `DataGrabber` accesses each element in the list, puts it into the template, then matches files. Any lists of strings/numbers behaves as expected.

However, `SelectFiles` just puts the whole list's string representation into the template. Then, `glob` module's [ ] handling takes over.

The problem is that, for `glob`, [ ] defines a character set a la regex. Except in cases with single digits like `[0,1,2]`, character sets behave very differently from lists. For example, `sub-[10,11,12]*` actually matches `sub-1*`, `sub-0*`, `sub-2*` NOT `sub-10*`, `sub-11*`, `sub-12*` as most users might expect.

This could easily lead to unexpected errors and confusion. I suggest either
1. implement proper list support like `DataGrabber` or
2. change `SelectFiles` documentation to clarify that it does NOT support list input and maybe direct users to Python's `glob` pattern expansion documentation.
I think the latter would be reasonable. `SelectFiles` was written specifically to be a simpler alternative to `DataGrabber`, and changing its semantics would break things for a lot of users.

Resolving patch:
diff --git a/nipype/interfaces/io.py b/nipype/interfaces/io.py
--- a/nipype/interfaces/io.py
+++ b/nipype/interfaces/io.py
@@ -1296,13 +1296,13 @@ class SelectFilesInputSpec(DynamicTraitedSpec, BaseInterfaceInputSpec):
 class SelectFiles(IOBase):
     """Flexibly collect data from disk to feed into workflows.
 
-    This interface uses the {}-based string formatting syntax to plug
+    This interface uses Python's {}-based string formatting syntax to plug
     values (possibly known only at workflow execution time) into string
-    templates and collect files from persistant storage. These templates
-    can also be combined with glob wildcards. The field names in the
-    formatting template (i.e. the terms in braces) will become inputs
-    fields on the interface, and the keys in the templates dictionary
-    will form the output fields.
+    templates and collect files from persistant storage. These templates can
+    also be combined with glob wildcards (``*``, ``?``) and character ranges (``[...]``).
+    The field names in the formatting template (i.e. the terms in braces) will
+    become inputs fields on the interface, and the keys in the templates
+    dictionary will form the output fields.
 
     Examples
     --------
@@ -1310,18 +1310,22 @@ class SelectFiles(IOBase):
     >>> import pprint
     >>> from nipype import SelectFiles, Node
     >>> templates={"T1": "{subject_id}/struct/T1.nii",
-    ...            "epi": "{subject_id}/func/f[0, 1].nii"}
+    ...            "epi": "{subject_id}/func/f[0,1].nii"}
     >>> dg = Node(SelectFiles(templates), "selectfiles")
     >>> dg.inputs.subject_id = "subj1"
     >>> pprint.pprint(dg.outputs.get())  # doctest:
     {'T1': <undefined>, 'epi': <undefined>}
 
-    The same thing with dynamic grabbing of specific files:
+    Note that SelectFiles does not support lists as inputs for the dynamic
+    fields. Attempts to do so may lead to unexpected results because brackets
+    also express glob character ranges. For example,
 
-    >>> templates["epi"] = "{subject_id}/func/f{run!s}.nii"
+    >>> templates["epi"] = "{subject_id}/func/f{run}.nii"
     >>> dg = Node(SelectFiles(templates), "selectfiles")
     >>> dg.inputs.subject_id = "subj1"
-    >>> dg.inputs.run = [2, 4]
+    >>> dg.inputs.run = [10, 11]
+
+    would match f0.nii or f1.nii, not f10.nii or f11.nii.
 
     """
     input_spec = SelectFilesInputSpec


Test patch: