Repository: nilearn/nilearn

Issue:
plot_img fails:  Cannot reorder the axes: the image affine contains rotations
Hi,

My T1 image does not plot when in scanner space.  I have to register to MNI in order to plot.  I don't want to do that specifically because I want to see activation and anatomy for each subject without registration distortion.

file_name = 'T1_001.nii.gz' #error because of rotations
nilearn.plotting.plot_img(os.path.join(file_path, file_name ))

ValueError: Cannot reorder the axes: the image affine contains rotations



Discussion:
Indeed, that's probably because we are using reoder_img without allowing
rotations. We should change this default value.

Thanks for reporting

So, is there a workaround, or should I just wait?  I'm really new at
nilearn, and haven't done too much digging.  I downloaded yesterday
specifically to help with brain visualization.

Thanks.

On Sat, Jan 24, 2015 at 9:28 AM, Gael Varoquaux notifications@github.com
wrote:

> Indeed, that's probably because we are using reoder_img without allowing
> rotations. We should change this default value.
> 
> Thanks for reporting
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/nilearn/nilearn/issues/368#issuecomment-71319381.

> So, is there a workaround, or should I just wait?

No workaround. This is a bug (or rather misfeature) from our side. I
cannot tackle it right now, but the solution is rather simple. Hopefuly
we will fix it in the next few days.

Maybe calling `resample_img` with a diagonal matrix as affine before
calling `plot_img` could do the trick.

On 1/24/15, Gael Varoquaux notifications@github.com wrote:

> > So, is there a workaround, or should I just wait?
> 
> No workaround. This is a bug (or rather misfeature) from our side. I
> cannot tackle it right now, but the solution is rather simple. Hopefuly
> we will fix it in the next few days.
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/nilearn/nilearn/issues/368#issuecomment-71319631

> Maybe calling `resample_img` with a diagonal matrix as affine before
> calling `plot_img` could do the trick.

Actually that's a good point. Inserting the following lines would do the
trick:

<pre>
from nilearn import image
file_name = 'T1_001.nii.gz'
img = image.reorder_img(os.path.join(file_path, file_name), resample='continuous')
nilearn.plotting.plot_img(img)
</pre>

Thanks!  This worked.  I do see the "resample" option.  Does this mean the
images should be taken with a grain of salt  because the values are being
changed before being plotted?  Whereas if it it just worked, then this
resampling, and therefore changed values would not be an issue?

On Sat, Jan 24, 2015 at 4:12 PM, Gael Varoquaux notifications@github.com
wrote:

> > Maybe calling `resample_img` with a diagonal matrix as affine before
> > calling `plot_img` could do the trick.
> 
> Actually that's a good point. Inserting the following lines would do the
> trick:
> 
> <pre>
> from nilearn import image
> file_name = 'T1_001.nii.gz'
> img = image.reorder_img(os.path.join(file_path, file_name),
> resample='continuous')
> nilearn.plotting.plot_img(img)
> </pre>
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/nilearn/nilearn/issues/368#issuecomment-71337522.

> Thanks! This worked. I do see the "resample" option. Does this mean the
> images should be taken with a grain of salt because the values are being
> changed before being plotted? Whereas if it it just worked, then this
> resampling, and therefore changed values would not be an issue?

You are never looking at the number of your data matrix. There is always
resampling and interpolation in visualization, unless one pixel of your
screen represents one voxel of your data matrix.

While your remark is true, I think that it is important to _always_ keep
in mind that a visualization system is not neutral to what it does to the
data.

Completely agree.  I was thinking in the vein of queryable data (e.g., if I
clicked on an element, would the displayed values be the original matrix).
These plots seem static, so then it really is the usual viewer beware
caveat.  Don't get me wrong, I appreciate the solution.  I always want to
know exactly what I'm doing, as much as possible, so I can field questions
from supervisors.

On Sun, Jan 25, 2015 at 9:27 AM, Gael Varoquaux notifications@github.com
wrote:

> > Thanks! This worked. I do see the "resample" option. Does this mean the
> > images should be taken with a grain of salt because the values are being
> > changed before being plotted? Whereas if it it just worked, then this
> > resampling, and therefore changed values would not be an issue?
> 
> You are never looking at the number of your data matrix. There is always
> resampling and interpolation in visualization, unless one pixel of your
> screen represents one voxel of your data matrix.
> 
> While your remark is true, I think that it is important to _always_ keep
> in mind that a visualization system is not neutral to what it does to the
> data.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/nilearn/nilearn/issues/368#issuecomment-71375056.

> Completely agree. I was thinking in the vein of queryable data (e.g., if I
> clicked on an element, would the displayed values be the original matrix).

For these purposes, I would advise using fslview.

> Don't get me wrong, I appreciate the solution. I always want to know
> exactly what I'm doing, as much as possible, so I can field questions
> from supervisors.

If your data is a set of labels, rather than continuous use should use
"resample='nearest'"

>  if I clicked on an element, would the displayed values be the original matrix

As said by Gaël, you can save your image into a Nifti file and visualize it with fslview. pynax (see my repos) can also be used to do that. However, it is not compatible with current plotting functions in nilearn but it shouldn't be too much work.

So what do we want to do with this one, just have a better error message mentioning to use reorder_img with the `resample` argument set in order to force resampling when the image has an affine with rotations ? 

A not so great alternative is to add the `resample` argument to all the plotting functions since we don't know which resampling between 'nearest' or 'continuous' is appropriate, unless of course we go down the way of having some shoddy logic to decide which one is the most likely, for example based on the number of unique values in the array (not 100 % convinced atm) ...

> A not so great alternative is to add the resample argument to all the
> plotting functions since we don't know which resampling between
> 'nearest' or 'continuous' is appropriate, unless of course we go down
> the way of having some shoddy logic to decide which one is the most
> likely, for example based on the number of unique values in the array

We _need_ to make this work. People don't want to have to resample, they
want things to 'just work'. In addition 99% of them won't know which
resampling method to use, so forcing make a choice is not solving the
problem. We can add a warning.

And with regards to which resampling method: plot_roi should use
'nearest', the others a continuous resampling. That's exactly the reason
why we separated these function.

"People don't want to have to resample, they
want things to 'just work'. "

YES!

On Thu, Jan 29, 2015 at 3:24 PM, Gael Varoquaux notifications@github.com
wrote:

> > A not so great alternative is to add the resample argument to all the
> > plotting functions since we don't know which resampling between
> > 'nearest' or 'continuous' is appropriate, unless of course we go down
> > the way of having some shoddy logic to decide which one is the most
> > likely, for example based on the number of unique values in the array
> 
> We _need_ to make this work. People don't want to have to resample, they
> want things to 'just work'. In addition 99% of them won't know which
> resampling method to use, so forcing make a choice is not solving the
> problem. We can add a warning.
> 
> And with regards to which resampling method: plot_roi should use
> 'nearest', the others a continuous resampling. That's exactly the reason
> why we separated these function.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/nilearn/nilearn/issues/368#issuecomment-72097422.

> And with regards to which resampling method: plot_roi should use 'nearest', the others a continuous resampling. That's exactly the reason why we separated these function.

OK that makes sense but then how do we cater for use cases like this [one](https://github.com/nilearn/nilearn/pull/270), basically using plot_img (so no reasonable default) to plot of a segmentation of the brain (data is a set of labels). Should plot_roi be used instead?

The thing is that the [doc](http://nilearn.github.io/building_blocks/plotting.html) mentions that plot_img comes with no specific presets.

I strongly believe in the 80 / 20 rule: we address most of the problems,
and we leave the others not addressed. Some people will have such a
problem, but it requires them to work with data that actually need
resampling (which is a tiny fraction), and in a very unusual situation.
And, if we don't solve the problem for them, the only consequence is that
they will have some strange aliasing in their visualization.

I don't think that, for now, such a case is worth spending too much time.
We need to keep in mind that we are working with a fixed budget, and that
we need to maximize the impact of nilearn with this fixed budget. That
means making nilearn usable for end-user problem solving for a maximum of
people.

We already have heuristics in nilearn that are quite hazardous. A `resampling='auto'` for one function seems feasible. A heuristic could be: `dtype == int and n_labels < 500 and n_connected_components < 500`. No strong feelings about this one.

500 is really arbitrary. I don't like it.

> We already have heuristics in nilearn that are quite hazardous. A
> resampling= 'auto' for one function seems feasible. A heuristic could
> be: dtype == int and n_labels < 500 and n_connected_components < 500.
> No strong feelings about this one.

That sounds like a reasonnable approach to me for a generic function like
'plot_img'.


Resolving patch:
diff --git a/nilearn/plotting/displays.py b/nilearn/plotting/displays.py
--- a/nilearn/plotting/displays.py
+++ b/nilearn/plotting/displays.py
@@ -427,8 +427,8 @@ def add_contours(self, img, **kwargs):
         self._map_show(img, type='contour', **kwargs)
         pl.draw_if_interactive()
 
-    def _map_show(self, img, type='imshow', **kwargs):
-        img = reorder_img(img)
+    def _map_show(self, img, type='imshow', resampling_interpolation='continuous', **kwargs):
+        img = reorder_img(img, resample=resampling_interpolation)
 
         affine = img.get_affine()
         data = img.get_data()
diff --git a/nilearn/plotting/img_plotting.py b/nilearn/plotting/img_plotting.py
--- a/nilearn/plotting/img_plotting.py
+++ b/nilearn/plotting/img_plotting.py
@@ -194,7 +194,7 @@ def plot_img(img, cut_coords=None, output_file=None, display_mode='ortho',
                     output_file=output_file, display_mode=display_mode,
                     figure=figure, axes=axes, title=title,
                     threshold=threshold, annotate=annotate,
-                    draw_cross=draw_cross,
+                    draw_cross=draw_cross, resampling_interpolation='continuous',
                     black_bg=black_bg, colorbar=colorbar, **kwargs)
 
     return display
@@ -528,6 +528,7 @@ def plot_roi(roi_img, bg_img=MNI152TEMPLATE, cut_coords=None,
                                annotate=annotate, draw_cross=draw_cross,
                                black_bg=black_bg, threshold=0.5,
                                bg_vmin=bg_vmin, bg_vmax=bg_vmax,
+                               resampling_interpolation='nearest',
                                alpha=alpha, cmap=cmap, **kwargs)
     return display
 
@@ -661,6 +662,7 @@ def plot_stat_map(stat_map_img, bg_img=MNI152TEMPLATE, cut_coords=None,
                                bg_vmin=bg_vmin, bg_vmax=bg_vmax, cmap=cmap,
                                vmin=vmin, vmax=vmax, colorbar=colorbar,
                                cbar_vmin=cbar_vmin, cbar_vmax=cbar_vmax,
+			       resampling_interpolation='continuous',
                                **kwargs)
 
     return display
@@ -734,6 +736,7 @@ def display_factory(display_mode):
                                 black_bg=black_bg, threshold=threshold,
                                 cmap=cmap, colorbar=False,
                                 display_factory=display_factory,
+                                resampling_interpolation='continuous',
                                 **kwargs)
 
     return display


Test patch:
diff --git a/nilearn/plotting/tests/test_img_plotting.py b/nilearn/plotting/tests/test_img_plotting.py
--- a/nilearn/plotting/tests/test_img_plotting.py
+++ b/nilearn/plotting/tests/test_img_plotting.py
@@ -124,3 +124,16 @@ def test_plot_img_with_auto_cut_coords():
     for display_mode in 'xyz':
         plot_img(img, cut_coords=None, display_mode=display_mode,
                  black_bg=True)
+
+
+def test_plot_img_with_resampling():
+    import pylab as pl
+    pl.switch_backend('svg')
+    data = MNI152TEMPLATE.get_data()[:5, :5, :5]
+    affine = np.array([[ 1., -1.,  0.,  0.],
+                       [ 1.,  1.,  0.,  0.],
+                       [ 0.,  0.,  1.,  0.],
+                       [ 0.,  0.,  0.,  1.]])
+    img = nibabel.Nifti1Image(data, affine)
+    display = plot_img(img)
+    display.add_overlay(img)
\ No newline at end of file