Repository: nipy/nipype

Issue:
N4BiasFieldCorrection fails for compiled ANTS
### Summary
`N4BiasFieldCorrection` throws an error

### Actual behavior
The `rescale_intensities` trait is set by default to `False` while it must not be specified for ANTS version under 2.1.0
### Expected behavior
`rescale_intensities` should be undefined by default
### How to replicate the behavior
We can replicate the behavior with compiled ANTs
### Script/Workflow details
My Ants version 
``` console
salma@salma-Latitude-E7450:~/CODE/sammba-mri $ N4BiasFieldCorrection --version
ANTs Version: 0.0.0.0
Compiled: Dec 22 2019 22:03:46
```
Ants version as read by nipype 
```console
salma@salma-Latitude-E7450:~/CODE/sammba-mri $ python -c "from nipype.interfaces import ants; print(ants.base.Info.version())"
200104-19:14:53,203 nipype.utils INFO:
	 Running nipype version 1.5.0-dev+g7f6cc2de2 (latest: 1.4.0)
0.0.0
```
```Python
from nipype.interfaces import ants

bias_correct = ants.N4BiasFieldCorrection().run
bias_correct(input_image='/tmp/func.nii.gz')
```
throws the following error
```
200104-19:20:52,963 nipype.utils INFO:
	 Running nipype version 1.5.0-dev+g7f6cc2de2 (latest: 1.4.0)
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-1-9321c82e17d6> in <module>
      2 
      3 bias_correct = ants.N4BiasFieldCorrection().run
----> 4 bias_correct(input_image='/tmp/func.nii.gz')

~/CODE/nipype/nipype/interfaces/base/core.py in run(self, cwd, ignore_exception, **inputs)
    348         self.inputs.trait_set(**inputs)
    349         self._check_mandatory_inputs()
--> 350         self._check_version_requirements(self.inputs)
    351         interface = self.__class__
    352         self._duecredit_cite()

~/CODE/nipype/nipype/interfaces/base/core.py in _check_version_requirements(self, trait_object, raise_exception)
    281                         raise Exception(
    282                             "Trait %s (%s) (version %s < required %s)"
--> 283                             % (name, self.__class__.__name__, version, min_ver)
    284                         )
    285 

Exception: Trait rescale_intensities (N4BiasFieldCorrection) (version 0.0.0 < required 2.1.0)
```
while forcing the trait `rescale_intensities` to be undefined solves the issue
```Python
from nipype.interfaces.base import traits                                                                                                                                                                                                                                 

bias_correct(input_image='/tmp/func.nii.gz', rescale_intensities=traits.Undefined)
```

### Platform details:

<!-- Please run the following code from your shell and place the output between the triple ticks, below.
python -c "import nipype; from pprint import pprint; pprint(nipype.get_info())"
-->
```console
200104-19:19:19,39 nipype.utils INFO:
	 Running nipype version 1.5.0-dev+g7f6cc2de2 (latest: 1.4.0)
{'commit_hash': '7f6cc2de2',
 'commit_source': 'repository',
 'networkx_version': '2.4',
 'nibabel_version': '3.0.0',
 'nipype_version': '1.5.0-dev+g7f6cc2de2',
 'numpy_version': '1.17.4',
 'pkg_path': '/home/salma/CODE/nipype/nipype',
 'scipy_version': '1.3.2',
 'sys_executable': '/home/salma/miniconda3/bin/python',
 'sys_platform': 'linux',
 'sys_version': '3.7.4 (default, Aug 13 2019, 20:35:49) \n[GCC 7.3.0]',
 'traits_version': '5.2.0'}```

```

```

### Execution environment
- My python environment outside container



Discussion:
this seems like an issue with ants version rather than nipype itself. nipype appears to be doing the correct thing.
@satra yes ANTs version is not correct. Yet is it expected that for versions less than `2.1.0` that the N4BiasFieldCorrection fails unless `rescale_intensities` is explicitely set to `Undefined`?
My understanding of the requirement `min_ver="2.1.0"` for the `rescale_intensities` trait is that it should not be specified for versions less than 2.1.0. I do not specify it, but here by default it is set to `False`.
thanks @salma1601 - this is then a bug in the nipype core. at present the handling of `use_default` is conflicting with version checking.

in such a situation we should either:
1. create a version dependent inputspec (this is roughly what we do with some interfaces for FSL and SPM).
2. disable it
3. keep it but not use it (this gets tricky between commandline and non-commandline interfaces).

my preference is towards 2, but this will require some modifications to how inputs are handled.

If I understand correctly, this is a more general problem than specific to N4BiasFieldCorrection.
What do you mean by disable it ? 
> this is a more general problem than specific to N4BiasFieldCorrection.

yes

> What do you mean by disable it ?

when you create an instance of an interface, the inputs that are available will depend on a scan through `min_ver`/`max_ver` and the `interface.version` property. one way would be to add a metadata property `unavailable=True` and every place where inputs are iterated would exclude any inputs with that property. 

this would allow hashing of inputs properly and not using those inputs in format_arg, parse_arg, etc.,. also when we do interface.inputs it should not list disabled inputs.

however, this would be a bigger change and we would have to ensure our tests cover the issues this may cause.
Let's try to resolve the primary issue in 1.6.0.
@satra Ran across this while thinking about pushing for a 1.6.0 release. If you get some minutes, would you mind refreshing yourself on the issue and estimating how big a lift this would be? I can probably find time to do it if the path is reasonably clear.

Resolving patch:
diff --git a/nipype/interfaces/base/core.py b/nipype/interfaces/base/core.py
--- a/nipype/interfaces/base/core.py
+++ b/nipype/interfaces/base/core.py
@@ -31,7 +31,7 @@
 
 from ...external.due import due
 
-from .traits_extension import traits, isdefined
+from .traits_extension import traits, isdefined, Undefined
 from .specs import (
     BaseInterfaceInputSpec,
     CommandLineInputSpec,
@@ -180,7 +180,16 @@ def __init__(
         if not self.input_spec:
             raise Exception("No input_spec in class: %s" % self.__class__.__name__)
 
-        self.inputs = self.input_spec(**inputs)
+        # Create input spec, disable any defaults that are unavailable due to
+        # version, and then apply the inputs that were passed.
+        self.inputs = self.input_spec()
+        unavailable_traits = self._check_version_requirements(
+            self.inputs, permissive=True
+        )
+        if unavailable_traits:
+            self.inputs.trait_set(**{k: Undefined for k in unavailable_traits})
+        self.inputs.trait_set(**inputs)
+
         self.ignore_exception = ignore_exception
 
         if resource_monitor is not None:
@@ -264,8 +273,12 @@ def _check_mandatory_inputs(self):
         ):
             self._check_requires(spec, name, getattr(self.inputs, name))
 
-    def _check_version_requirements(self, trait_object, raise_exception=True):
+    def _check_version_requirements(self, trait_object, permissive=False):
         """ Raises an exception on version mismatch
+
+        Set the ``permissive`` attribute to True to suppress warnings and exceptions.
+        This is currently only used in __init__ to silently identify unavailable
+        traits.
         """
         unavailable_traits = []
         # check minimum version
@@ -283,7 +296,8 @@ def _check_version_requirements(self, trait_object, raise_exception=True):
                         f"Nipype cannot validate the package version {version!r} for "
                         f"{self.__class__.__name__}. Trait {name} requires version >={min_ver}."
                     )
-                    iflogger.warning(f"{msg}. Please verify validity.")
+                    if not permissive:
+                        iflogger.warning(f"{msg}. Please verify validity.")
                     if config.getboolean("execution", "stop_on_unknown_version"):
                         raise ValueError(msg) from err
                     continue
@@ -291,7 +305,7 @@ def _check_version_requirements(self, trait_object, raise_exception=True):
                     unavailable_traits.append(name)
                     if not isdefined(getattr(trait_object, name)):
                         continue
-                    if raise_exception:
+                    if not permissive:
                         raise Exception(
                             "Trait %s (%s) (version %s < required %s)"
                             % (name, self.__class__.__name__, version, min_ver)
@@ -311,7 +325,8 @@ def _check_version_requirements(self, trait_object, raise_exception=True):
                         f"Nipype cannot validate the package version {version!r} for "
                         f"{self.__class__.__name__}. Trait {name} requires version <={max_ver}."
                     )
-                    iflogger.warning(f"{msg}. Please verify validity.")
+                    if not permissive:
+                        iflogger.warning(f"{msg}. Please verify validity.")
                     if config.getboolean("execution", "stop_on_unknown_version"):
                         raise ValueError(msg) from err
                     continue
@@ -319,7 +334,7 @@ def _check_version_requirements(self, trait_object, raise_exception=True):
                     unavailable_traits.append(name)
                     if not isdefined(getattr(trait_object, name)):
                         continue
-                    if raise_exception:
+                    if not permissive:
                         raise Exception(
                             "Trait %s (%s) (version %s > required %s)"
                             % (name, self.__class__.__name__, version, max_ver)


Test patch:
diff --git a/nipype/interfaces/base/tests/test_core.py b/nipype/interfaces/base/tests/test_core.py
--- a/nipype/interfaces/base/tests/test_core.py
+++ b/nipype/interfaces/base/tests/test_core.py
@@ -253,7 +253,7 @@ class input_spec(nib.TraitedSpec):
     assert len(caplog.records) == 2
 
 
-def test_input_version_missing_error():
+def test_input_version_missing_error(caplog):
     from nipype import config
 
     class DerivedInterface(nib.BaseInterface):
@@ -263,13 +263,55 @@ class input_spec(nib.TraitedSpec):
 
         _version = "misparsed-garbage"
 
-    with mock.patch.object(config, "getboolean", return_value=True):
-        obj = DerivedInterface(foo=1)
-        with pytest.raises(ValueError):
-            obj._check_version_requirements(obj.inputs)
-        obj = DerivedInterface(bar=1)
-        with pytest.raises(ValueError):
-            obj._check_version_requirements(obj.inputs)
+    obj1 = DerivedInterface(foo=1)
+    obj2 = DerivedInterface(bar=1)
+    with caplog.at_level(logging.WARNING, logger="nipype.interface"):
+        with mock.patch.object(config, "getboolean", return_value=True):
+            with pytest.raises(ValueError):
+                obj1._check_version_requirements(obj1.inputs)
+            with pytest.raises(ValueError):
+                obj2._check_version_requirements(obj2.inputs)
+    assert len(caplog.records) == 2
+
+
+def test_unavailable_input():
+    class WithInput(nib.BaseInterface):
+        class input_spec(nib.TraitedSpec):
+            foo = nib.traits.Int(3, usedefault=True, max_ver="0.5")
+
+        _version = "0.4"
+
+        def _run_interface(self, runtime):
+            return runtime
+
+    class WithoutInput(WithInput):
+        _version = "0.6"
+
+    has = WithInput()
+    hasnt = WithoutInput()
+    trying_anyway = WithoutInput(foo=3)
+    assert has.inputs.foo == 3
+    assert not nib.isdefined(hasnt.inputs.foo)
+    assert trying_anyway.inputs.foo == 3
+
+    has.run()
+    hasnt.run()
+    with pytest.raises(Exception):
+        trying_anyway.run()
+
+    # Still settable
+    has.inputs.foo = 4
+    hasnt.inputs.foo = 4
+    trying_anyway.inputs.foo = 4
+    assert has.inputs.foo == 4
+    assert hasnt.inputs.foo == 4
+    assert trying_anyway.inputs.foo == 4
+
+    has.run()
+    with pytest.raises(Exception):
+        hasnt.run()
+    with pytest.raises(Exception):
+        trying_anyway.run()
 
 
 def test_output_version():
diff --git a/nipype/interfaces/tests/test_io.py b/nipype/interfaces/tests/test_io.py
--- a/nipype/interfaces/tests/test_io.py
+++ b/nipype/interfaces/tests/test_io.py
@@ -465,7 +465,7 @@ def test_datasink_substitutions(tmpdir):
         files.append(f)
         open(f, "w")
     ds = nio.DataSink(
-        parametrization=False,
+        parameterization=False,
         base_directory=str(outdir),
         substitutions=[("ababab", "ABABAB")],
         # end archoring ($) is used to assure operation on the filename