Repository: nilearn/nilearn

Issue:
pb with plot_stat_map with MGZ T1 image
we use a plot_stat_map in MNE:

http://martinos.org/mne/dev/auto_examples/inverse/plot_lcmv_beamformer_volume.html#example-inverse-plot-lcmv-beamformer-volume-py

with the current master of nilearn it fails due to the dtype (int8) of the anat I make available at:

https://dl.dropboxusercontent.com/u/2140486/T1.mgz

error is:

```
/Users/alex/work/src/nibabel/nibabel/freesurfer/mghformat.pyc in set_data_dtype(self, datatype)
    284             code = self._data_type_codes[datatype]
    285         except KeyError:
--> 286             raise MGHError('datatype dtype "%s" not recognized' % datatype)
    287         self._header_data['type'] = code
    288

MGHError: datatype dtype "int8" not recognized
```

any help?



Discussion:
if I convert the mgz to nii it works...

I cannot right now run the full mne example, but is this a nilearn or a nibabel thing in your opinion?

Loading the T1.mgz worked transparently with `nibabel.load`, which I assume it is expected to do. It gives me dtype `uint8` though, instead of `int8`, which is weird.

Fucked up format. Stop promoting / using it and help us push for pen standards rather than lone labs feeling powerful enough to invent new file formats. 

Sent from my phone. Please forgive brevity and mis spelling

On May 8, 2015, 22:40, at 22:40, Alexandre Gramfort notifications@github.com wrote:

> we use a plot_stat_map in MNE:
> 
> http://martinos.org/mne/dev/auto_examples/inverse/plot_lcmv_beamformer_volume.html#example-inverse-plot-lcmv-beamformer-volume-py
> 
> with the current master of nilearn it fails due to the dtype (int8) of
> the anat I make available at:
> 
> https://dl.dropboxusercontent.com/u/2140486/T1.mgz
> 
> error is:
> 
> ```
> /Users/alex/work/src/nibabel/nibabel/freesurfer/mghformat.pyc in
> set_data_dtype(self, datatype)
>    284             code = self._data_type_codes[datatype]
>    285         except KeyError:
> --> 286             raise MGHError('datatype dtype "%s" not recognized'
> % datatype)
>    287         self._header_data['type'] = code
>    288
> 
> MGHError: datatype dtype "int8" not recognized
> ```
> 
> any help?
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/nilearn/nilearn/issues/577

... https://xkcd.com/927/

2015-05-08 23:17 GMT+02:00 Gael Varoquaux notifications@github.com:

> Fucked up format. Stop promoting / using it and help us push for pen
> standards rather than lone labs feeling powerful enough to invent new file
> formats.
> 
> Sent from my phone. Please forgive brevity and mis spelling
> 
> On May 8, 2015, 22:40, at 22:40, Alexandre Gramfort <
> notifications@github.com> wrote:
> 
> > we use a plot_stat_map in MNE:
> > 
> > http://martinos.org/mne/dev/auto_examples/inverse/plot_lcmv_beamformer_volume.html#example-inverse-plot-lcmv-beamformer-volume-py
> > 
> > with the current master of nilearn it fails due to the dtype (int8) of
> > the anat I make available at:
> > 
> > https://dl.dropboxusercontent.com/u/2140486/T1.mgz
> > 
> > error is:
> > 
> > ```
> > /Users/alex/work/src/nibabel/nibabel/freesurfer/mghformat.pyc in
> > set_data_dtype(self, datatype)
> > 284 code = self._data_type_codes[datatype]
> > 285 except KeyError:
> > --> 286 raise MGHError('datatype dtype "%s" not recognized'
> > % datatype)
> > 287 self._header_data['type'] = code
> > 288
> > 
> > MGHError: datatype dtype "int8" not recognized
> > ```
> > 
> > any help?
> > 
> > ---
> > 
> > Reply to this email directly or view it on GitHub:
> > https://github.com/nilearn/nilearn/issues/577
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/nilearn/nilearn/issues/577#issuecomment-100371538.

(With the difference that the proposed standards already exist)

On Fri, May 8, 2015 at 11:29 PM, Denis A. Engemann <notifications@github.com

> wrote:
> 
> ... https://xkcd.com/927/
> 
> 2015-05-08 23:17 GMT+02:00 Gael Varoquaux notifications@github.com:
> 
> > Fucked up format. Stop promoting / using it and help us push for pen
> > standards rather than lone labs feeling powerful enough to invent new
> > file
> > formats.
> > 
> > Sent from my phone. Please forgive brevity and mis spelling
> > 
> > On May 8, 2015, 22:40, at 22:40, Alexandre Gramfort <
> > notifications@github.com> wrote:
> > 
> > > we use a plot_stat_map in MNE:
> > 
> > http://martinos.org/mne/dev/auto_examples/inverse/plot_lcmv_beamformer_volume.html#example-inverse-plot-lcmv-beamformer-volume-py
> > 
> > > with the current master of nilearn it fails due to the dtype (int8) of
> > > the anat I make available at:
> > > 
> > > https://dl.dropboxusercontent.com/u/2140486/T1.mgz
> > > 
> > > error is:
> > > 
> > > ```
> > > /Users/alex/work/src/nibabel/nibabel/freesurfer/mghformat.pyc in
> > > set_data_dtype(self, datatype)
> > > 284 code = self._data_type_codes[datatype]
> > > 285 except KeyError:
> > > --> 286 raise MGHError('datatype dtype "%s" not recognized'
> > > % datatype)
> > > 287 self._header_data['type'] = code
> > > 288
> > > 
> > > MGHError: datatype dtype "int8" not recognized
> > > ```
> > > 
> > > any help?
> > > 
> > > ---
> > > 
> > > Reply to this email directly or view it on GitHub:
> > > https://github.com/nilearn/nilearn/issues/577
> > 
> > —
> > Reply to this email directly or view it on GitHub
> > https://github.com/nilearn/nilearn/issues/577#issuecomment-100371538.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/nilearn/nilearn/issues/577#issuecomment-100373622.

> I cannot right now run the full mne example, but is this a nilearn or a nibabel thing in your opinion?

A bit of both, mainly nilearn's fault.

What happens is that data is loaded in `uint8` type and then some magic happens in the code and the data is converted to `int8` that is not supported by the data format.

IMHO, converting data implictly is utterly wrong.

OK. If the data is boolean, we convert it to int8. We do that because Nifti1Image does not handle bool. The problem is that MGH format does not handle int8, only uint8. I can't find an easy way to get accepted types for each format.

But Nifti1Image can also do uint8. What is the rationale behind using int8
and not uint8 for the bool conversion?

On Fri, May 8, 2015 at 11:51 PM, Alexandre Abraham <notifications@github.com

> wrote:
> 
> OK. If the data is boolean, we convert it to int8. We do that because
> Nifti1Image does not handle bool. The problem is that MGH format does not
> handle int8, only uint8. I can't find an easy way to get accepted types for
> each format.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/nilearn/nilearn/issues/577#issuecomment-100377390.

One additional character!

We have to check if uint8 is accepted by all other formats.

No converting the data is not wrong : I've seen many bugs due to over and under flows when using the wrong type. 

Sent from my phone. Please forgive brevity and mis spelling

On May 8, 2015, 23:41, at 23:41, Alexandre Abraham notifications@github.com wrote:

> > I cannot right now run the full mne example, but is this a nilearn or
> > a nibabel thing in your opinion?
> 
> A bit of both, mainly nilearn's fault.
> 
> What happens is that data is loaded in `uint8` type and then some magic
> happens in the code and the data is converted to `int8` that is not
> supported by the data format.
> 
> In my mind, converting data implictly is utterly wrong.
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/nilearn/nilearn/issues/577#issuecomment-100375915

And readers: mricron, fslview and papaya souks be able to read uint8

Sent from my phone. Please forgive brevity and mis spelling

On May 8, 2015, 23:56, at 23:56, Alexandre Abraham notifications@github.com wrote:

> One additional character!
> 
> We have to check if uint8 is accepted by all other formats.
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/nilearn/nilearn/issues/577#issuecomment-100378384

So should we just convert to uint8 instead of int8?

First: I don't understand the problem: I cannot reproduce, calling plot_anat on the corresponding file.

Second, I am very worried about supporting mgz. I think that this is feature creep. Very few people use it, and if we try supporting it, I am afraid that it will be broken most often. Our docs clearly talk about 'niimg like', ie NiftiImage like, which mgz is.

I guess that my point of view on this problem is that we will accept a PR fixing it in a clean way, and if people join the dev team to support mgz files, its fine. But I don't feel like allocating our already sparse resources to it.


Resolving patch:
diff --git a/nilearn/_utils/niimg.py b/nilearn/_utils/niimg.py
--- a/nilearn/_utils/niimg.py
+++ b/nilearn/_utils/niimg.py
@@ -7,6 +7,7 @@
 import copy
 import gc
 import collections
+from distutils.version import LooseVersion
 
 import numpy as np
 import nibabel
@@ -79,7 +80,11 @@ def new_img_like(ref_img, data, affine, copy_header=False):
         An image which has the same type as the reference image.
     """
     if data.dtype == bool:
-        data = as_ndarray(data, dtype=np.int8)
+        default_dtype = np.int8
+        if (LooseVersion(nibabel.__version__) >= LooseVersion('1.2.0') and
+                isinstance(ref_img, nibabel.freesurfer.mghformat.MGHImage)):
+            default_dtype = np.uint8
+        data = as_ndarray(data, dtype=default_dtype)
     header = None
     if copy_header:
         header = copy.copy(ref_img.get_header())


Test patch:
diff --git a/nilearn/tests/data/test.mgz b/nilearn/tests/data/test.mgz
new file mode 100644
Binary files /dev/null and b/nilearn/tests/data/test.mgz differ
diff --git a/nilearn/tests/test_niimg.py b/nilearn/tests/test_niimg.py
--- a/nilearn/tests/test_niimg.py
+++ b/nilearn/tests/test_niimg.py
@@ -1,7 +1,32 @@
+import os
+from distutils.version import LooseVersion
+import numpy as np
+import nibabel
+from nose import SkipTest
+
 from nilearn._utils.testing import assert_raises_regex
 from nilearn._utils import niimg
 
+currdir = os.path.dirname(os.path.abspath(__file__))
+datadir = os.path.join(currdir, 'data')
+
 
 def test_copy_img():
     assert_raises_regex(ValueError, "Input value is not an image",
-        niimg.copy_img, 3)
+                        niimg.copy_img, 3)
+
+
+def test_new_img_like_mgz():
+    """Check that new images can be generated with bool MGZ type
+    This is usually when computing masks using MGZ inputs, e.g.
+    when using plot_stap_map
+    """
+
+    if not LooseVersion(nibabel.__version__) >= LooseVersion('1.2.0'):
+        # Old nibabel do not support MGZ files
+        raise SkipTest
+
+    ref_img = nibabel.load(os.path.join(datadir, 'test.mgz'))
+    data = np.ones(ref_img.get_data().shape, dtype=np.bool)
+    affine = ref_img.get_affine()
+    niimg.new_img_like(ref_img, data, affine, copy_header=False)