Repository: nilearn/nilearn

Issue:
Voxels with all-zeros invalidate permuted_ols results across all voxels
<!--

Hi! 
If you have: 
 -1 Questions about how to use Nilearn or 
 -2 Need analysis suggestions & recommendations?

A bunch of fMRI researchers hang out at Neurostars (http://neurostars.org/). 
Post those questions there. 
Add the tag `nilearn` (we get an email from Neurostars if you do).

Posting them here makes life more complicated for the Nilearn developers.  
-->

<!--

For the Bug Report, 
Include this information:
-------------------------
What version of Nilearn are you using?
What were you trying to do?
What did you expect will happen?
What actually happened?

List the steps you performed that revealed the bug to you.
Include any code samples. Enclose them in triple back-ticks (```)
Like this:

```
<code>
``` 
-->
I am using the current master branch of nilearn. When a voxel has zeros across all input maps and `permuted_ols` is used, all voxels in the results maps are invalid for permutation-based findings (i.e., the `-log(p)` and `h0` results, not the t-statistics).

This stems from https://github.com/nilearn/nilearn/issues/2554#issuecomment-672005510.

## Replicating the bug

```python
import numpy as np
from nilearn.mass_univariate import permuted_ols

n_voxels = 100  # instead of n_descriptors
n_maps = 50  # instead of n_samples
n_regressors = 1

tested_vars = np.ones((n_maps, n_regressors))
target_vars = np.random.random((n_maps, n_voxels))

# perfectly fine
p, t, h0 = permuted_ols(tested_vars, target_vars, model_intercept=False, n_perm=1000, random_state=1)

# now, one voxel will have all zeros
target_vars2 = target_vars.copy()
target_vars2[:, 0] = 0

p, t, h0 = permuted_ols(tested_vars, target_vars2, model_intercept=False, n_perm=1000, random_state=1)
```

In the second case, `h0` will be all NaNs, while `p` will be all -0.

## Fixing the bug

I am sure there are a number of ways this could be handled, from raising a warning when invalid voxels are included to working around the NaN issue. One solution I came across would be to replace [`numpy.amax()`](https://numpy.org/doc/stable/reference/generated/numpy.amax.html) (which propagates NaNs) with [`numpy.nanmax()`](https://numpy.org/doc/stable/reference/generated/numpy.nanmax.html#numpy.nanmax) here: 
https://github.com/nilearn/nilearn/blob/21fdc532dee895b5e46c3fc6a3d4f9803b247ac1/nilearn/mass_univariate/permuted_least_squares.py#L223


Discussion:
I think that issuing a warning  + using nanmax is the right thing to do.


Resolving patch:
diff --git a/nilearn/mass_univariate/permuted_least_squares.py b/nilearn/mass_univariate/permuted_least_squares.py
--- a/nilearn/mass_univariate/permuted_least_squares.py
+++ b/nilearn/mass_univariate/permuted_least_squares.py
@@ -220,7 +220,7 @@ def _permuted_ols_on_chunk(scores_original_data, tested_vars, target_vars,
                                                        confounding_vars))
         if two_sided_test:
             perm_scores = np.fabs(perm_scores)
-        h0_fmax_part[i] = np.amax(perm_scores, 0)
+        h0_fmax_part[i] = np.nanmax(perm_scores, axis=0)
         # find the rank of the original scores in h0_part
         # (when n_descriptors or n_perm are large, it can be quite long to
         #  find the rank of the original scores into the whole H0 distribution.
@@ -353,6 +353,10 @@ def permuted_ols(tested_vars, target_vars, confounding_vars=None,
                             "s" if target_vars.ndim > 1 else ""))
     target_vars = np.asfortranarray(target_vars)  # efficient for chunking
     n_descriptors = target_vars.shape[1]
+    if np.any(np.all(target_vars == 0, axis=0)):
+        warnings.warn("Some descriptors in 'target_vars' have zeros across all "
+                      "samples. These descriptors will be ignored during null "
+                      "distribution generation.")
 
     # check explanatory variates dimensions
     if tested_vars.ndim == 1:


Test patch:
diff --git a/nilearn/mass_univariate/tests/test_permuted_least_squares.py b/nilearn/mass_univariate/tests/test_permuted_least_squares.py
--- a/nilearn/mass_univariate/tests/test_permuted_least_squares.py
+++ b/nilearn/mass_univariate/tests/test_permuted_least_squares.py
@@ -9,6 +9,7 @@
 
 from numpy.testing import (assert_almost_equal, assert_array_almost_equal,
                            assert_array_less, assert_equal)
+import pytest
 
 from nilearn.mass_univariate import permuted_ols
 from nilearn.mass_univariate.permuted_least_squares import (
@@ -306,6 +307,37 @@ def test_permuted_ols_nocovar(random_state=0):
                               decimal=6)
 
 
+def test_permuted_ols_nocovar_warning(random_state=0):
+    """
+    Ensure that a warning is raised when a given voxel has all zeros.
+    """
+    rng = check_random_state(random_state)
+
+    # design parameters
+    n_samples = 50
+    n_descriptors = 10
+    n_regressors = 1
+
+    # create design
+    target_var = rng.randn(n_samples, n_descriptors)
+    tested_var = rng.randn(n_samples, n_regressors)
+
+    # permuted OLS
+    _, own_score, _ = permuted_ols(
+        tested_var, target_var, model_intercept=False,
+        n_perm=100, random_state=random_state)
+
+    # test with ravelized tested_var
+    target_var[:, 0] = 0
+
+    with pytest.warns(UserWarning):
+        _, own_score2, _ = permuted_ols(
+            np.ravel(tested_var), target_var, model_intercept=False,
+            n_perm=100, random_state=random_state)
+
+    assert np.array_equal(own_score[1:], own_score2[1:])
+
+
 def test_permuted_ols_withcovar(random_state=0):
     """