Repository: nilearn/nilearn

Issue:
Plotting thresholds are ignored for images whose affine contains a rotation
I've run into an issue trying to use nilearn to visualize data in native space. This came up when trying to plot ROIs:

``` python
plot_roi(roi_img, epi_img)
```

![4jxsfhcftoutz on44aeto5 amrliyfontmfmzgztu1pwycbeokebeeliotdtslqgp4eqqgghlwwnbkiiiyquguejqgghhbscngzcccgefijgaygeeeikqaobeeiiiywg0uaiiysqqtboiiqqqkghadqqqgghpba0ggghhbbscbonhbbccckejqzcccgefijgaygeeeikqaobeeiiiywg0uaiiysqqtboiiqqqkghadqqqgghpba0gggh](https://cloud.githubusercontent.com/assets/315810/7950342/2d7a5150-094e-11e5-938d-dca4053718fa.png)

(where `roi_img` and `epi_img` are `nibabel.Nifti1Image` objects).

This issue does not appear to occur with the example [haxby data](https://nilearn.github.io/auto_examples/manipulating_visualizing/plot_roi_extraction.html).

Something similar is happening with a continuous statistical map:

``` python
plot_stat_map(z_img, epi_img, threshold=9)
```

![nu3t7u8rkyrl bt89cviofaqfqvnxy6pvq4ybgqgjvlw84vps0lkjuljykgtk5 cbxv1dxvbt27p6lkycbgaas6kxsvglfy82d udhx9xexm5li9fruvxrysnj0c5otlyuvzq7ozm6ruwdxaaalac5gabaadliqabaadliqabaadliqabaadliqabaadliqabaadliqabaadliqabaadliqabaadliqabaadliqabaadliqabaadliqaba](https://cloud.githubusercontent.com/assets/315810/7950385/bd969a78-094e-11e5-9a31-d6d7939d7108.png)

I've tracked down the issue to [this line of code](https://github.com/nilearn/nilearn/blob/0.1.3/nilearn/plotting/displays.py#L532). The `img` object that goes into the `reorder_img` function has a masked data array, but the `img` that comes out of it just has a bare float array of 1s and 0s (in the case of the ROI image).

My native space images have a LPS orientation; presumably the mask  information is getting lost somewhere as the data are reordered to RAS. Presumably that step is skipped for the Haxby data, which are already oriented properly, and that is why the examples look fine.

I'm using nilearn 1.3.



Discussion:
Can you provide an example reproducing the problem? It's not entirely obvious to me what the problem is by just looking at the plots. In particular in the plot_stat_map case, it does look like the threshold=9 is applied.

I checked whether `reorder_img` was getting rid of the mask when the input image in in LPS orientation but this doesn't seem to be the case.

``` python

import numpy as np
import nibabel as nib

from nilearn import image, plotting

data = np.random.randn(20, 20, 20)
data = np.ma.masked_less(data, 0.)

# This should be LPS hopefully
affine = np.array([[-3., 0., 0., 1.],
                   [0., -3., 0., -2.],
                   [0., 0., 3., 3.],
                   [0., 0., 0., 1.]])
img = nib.Nifti1Image(data, affine)

reordered_img = image.reorder_img(img)

print('Consistent mask:',
      np.sum(data.mask) == np.sum(reordered_img.get_data().mask))

# The plotting seems fine too AFAICS
plotting.plot_stat_map(img, threshold=0.5)

```

> Can you provide an example reproducing the problem? It's not entirely obvious to me what the problem is by just looking at the plots.

Do you mean the images themselves? Sure I can do that when I get to school. 

> In particular in the plot_stat_map case, it does look like the threshold=9 is applied.

Why do you say that? I would have expected much of the overlay would be transparent (z = 9 is a high threshold) and that we would see the gray colormapped mean epi image in those spots.

> I checked whether reorder_img was getting rid of the mask when the input image in in LPS orientation but this doesn't seem to be the case.

What happens when you make an affine with rotations?

> What happens when you make an affine with rotations?

Good point, I can reproduce the problem, the reordered_img lose the mask for some reason which needs to be elucidated:

``` python
import numpy as np
import nibabel as nib

from nilearn import image, plotting

data = np.random.randn(20, 20, 20)
data = np.ma.masked_less(data, 0.)

# matrix with rotation
affine = np.array([[-3., 1., 0., 1.],
                   [-1., -3., 0., -2.],
                   [0., 0., 3., 3.],
                   [0., 0., 0., 1.]])
img = nib.Nifti1Image(data, affine)

reordered_img = image.reorder_img(img, resample='continuous')
print('img data type:', type(img.get_data()))  # numpy.ma.core.MaskedArray
print('reordered_img data type:', type(reordered_img.get_data()))  # numpy.ndarray
```

This seems more related to affine with rotations rather than LPS vs RAS by the way.

> This seems more related to affine with rotations rather than LPS vs RAS by the
> way.

It's going to be the resampling.

One thing that we could do would be to move this reordering before we
apply the threshold. The other option is to resample the mask, but that's
going to be ugly (resampling discrete info is not something that I like).

> One thing that we could do would be to move this reordering before we apply the threshold

From a quick look at the code, it does look like it is feasible.

> One thing that we could do would be to move this reordering before we
> apply the threshold.

That sounds like the right order of operations to me, although I am (much) less familiar with the codebase.

> That sounds like the right order of operations to me,

Quite clearly. This was an oversight.

G

What if the user gives us data that is already masked?


Resolving patch:
diff --git a/nilearn/plotting/displays.py b/nilearn/plotting/displays.py
--- a/nilearn/plotting/displays.py
+++ b/nilearn/plotting/displays.py
@@ -490,19 +490,10 @@ def add_overlay(self, img, threshold=1e-6, colorbar=False, **kwargs):
 
         img = _utils.check_niimg_3d(img)
 
-        if threshold is not None:
-            data = img.get_data()
-            if threshold == 0:
-                data = np.ma.masked_equal(data, 0, copy=False)
-            else:
-                data = np.ma.masked_inside(data, -threshold, threshold,
-                                           copy=False)
-            img = new_img_like(img, data, img.get_affine())
-
         # Make sure that add_overlay shows consistent default behavior
         # with plot_stat_map
         kwargs.setdefault('interpolation', 'nearest')
-        ims = self._map_show(img, type='imshow', **kwargs)
+        ims = self._map_show(img, type='imshow', threshold=threshold, **kwargs)
 
         if colorbar:
             self._colorbar_show(ims[0], threshold)
@@ -528,9 +519,20 @@ def add_contours(self, img, **kwargs):
         self._map_show(img, type='contour', **kwargs)
         plt.draw_if_interactive()
 
-    def _map_show(self, img, type='imshow', resampling_interpolation='continuous', **kwargs):
+    def _map_show(self, img, type='imshow',
+                  resampling_interpolation='continuous',
+                  threshold=None, **kwargs):
         img = reorder_img(img, resample=resampling_interpolation)
 
+        if threshold is not None:
+            data = img.get_data()
+            if threshold == 0:
+                data = np.ma.masked_equal(data, 0, copy=False)
+            else:
+                data = np.ma.masked_inside(data, -threshold, threshold,
+                                           copy=False)
+            img = new_img_like(img, data, img.get_affine())
+
         affine = img.get_affine()
         data = img.get_data()
         data_bounds = get_bounds(data.shape, affine)


Test patch:
diff --git a/nilearn/plotting/tests/test_img_plotting.py b/nilearn/plotting/tests/test_img_plotting.py
--- a/nilearn/plotting/tests/test_img_plotting.py
+++ b/nilearn/plotting/tests/test_img_plotting.py
@@ -130,6 +130,25 @@ def test_plot_stat_map():
     plot_stat_map(new_img, threshold=1000, colorbar=True)
 
 
+def test_plot_stat_map_threshold_for_affine_with_rotation():
+    # threshold was not being applied when affine has a rotation
+    # see https://github.com/nilearn/nilearn/issues/599 for more details
+    data = np.random.randn(10, 10, 10)
+    # matrix with rotation
+    affine = np.array([[-3., 1., 0., 1.],
+                       [-1., -3., 0., -2.],
+                       [0., 0., 3., 3.],
+                       [0., 0., 0., 1.]])
+    img = nibabel.Nifti1Image(data, affine)
+    display = plot_stat_map(img, bg_img=None, threshold=1e6,
+                            display_mode='z', cut_coords=1)
+    # Next two lines retrieve the numpy array from the plot
+    ax = list(display.axes.values())[0].ax
+    plotted_array = ax.images[0].get_array()
+    # Given the high threshold the array should be entirely masked
+    assert_true(plotted_array.mask.all())
+
+
 def test_save_plot():
     img = _generate_img()