Repository: nilearn/nilearn

Issue:
nistats: adding two_sided flag to get_clusters_table would result in more consistent API?
Hello,

I've been recently using nistats to report results from fMRI data analysis. I wanted to perform two-sided test for my 2nd level spm, since I was interested in both increased and decreased brain responses. `map_threshold` function provide flag `two_sided` which nicely adjust significance level preserving clusters with both positive and negative test statistic values. 

Next I wanted to use `get_clusters_table` function to pool information about peak coordinates. However, this function can only report peaks above `stat_threshold` (basically neglecting negative test stat values). I made a workaround to make it consistent with `map_threshold` – I multiplied my spm by (-1), reapplied `get_clusters_table` and then merged two tables together. 

Do you think that adding `two_sided` flag and its functionality to `get_clusters_table` is a good idea?


Discussion:
Sorry for the late answer. Yes, I think that this is a good idea.
Do you want to take care of that ?

Is it worth it to take a cue from AfNI and support one-sided, two-sided, _and_ bi-sided?
Sorry, can you explain the diff btw two-sided and bi-sided ?  Thx
Two-sided clustering combines positive and negative values as single clusters, while bi-sided clusters positive and negative values separately. To be honest, I've never understood the purpose of two-sided clustering as they define it, but I imagine there's a use for it since it's been supported in AfNI for so long (e.g., in [`3dClusterize`](https://afni.nimh.nih.gov/pub/dist/doc/program_help/3dClusterize.html)).

It's not something I know a lot about, but it would impact this case in that it would require a parameter that accepts a string (e.g., `sided='one'`) instead of a bool (e.g., `two_sided=False`), so I thought I'd bring it up.
Well, I'm not interested in having a method for which we don't have a use case... from what I understand what Afni calls `two-sided` is not a sensible approach... the problem is that `two-sided` is used in other packages for what Afni calls `bi-sided`.
So, I'd rather not implement this feature and keep the argument as-is.
Do you have strong feelings about that ?
That sounds good to me. I just thought I'd bring it up in case it was worth the extra effort.
Is there any news on making `two-sided` an option for `get_clusters_table`? I feel like this would be a very useful enhancement (but unfortunately I don't think my Python skills suffice to contribute – not yet, at least).

Resolving patch:
diff --git a/nilearn/reporting/_get_clusters_table.py b/nilearn/reporting/_get_clusters_table.py
--- a/nilearn/reporting/_get_clusters_table.py
+++ b/nilearn/reporting/_get_clusters_table.py
@@ -131,7 +131,7 @@ def _get_val(row, input_arr):
 
 
 def get_clusters_table(stat_img, stat_threshold, cluster_threshold=None,
-                       min_distance=8.):
+                       two_sided=False, min_distance=8.):
     """Creates pandas dataframe with img cluster statistics.
 
     Parameters
@@ -146,6 +146,10 @@ def get_clusters_table(stat_img, stat_threshold, cluster_threshold=None,
     cluster_threshold : `int` or `None`, optional
         Cluster size threshold, in voxels.
 
+    two_sided : `bool`, optional
+        Whether to employ two-sided thresholding or to evaluate positive values
+        only. Default=False.
+
     min_distance : `float`, optional
         Minimum distance between subpeaks in mm. Default=8mm.
 
@@ -169,75 +173,120 @@ def get_clusters_table(stat_img, stat_threshold, cluster_threshold=None,
     else:
         stat_map = get_data(stat_img).copy()
 
-    conn_mat = np.zeros((3, 3, 3), int)  # 6-connectivity, aka NN1 or "faces"
+    # Define array for 6-connectivity, aka NN1 or "faces"
+    conn_mat = np.zeros((3, 3, 3), int)
     conn_mat[1, 1, :] = 1
     conn_mat[1, :, 1] = 1
     conn_mat[:, 1, 1] = 1
     voxel_size = np.prod(stat_img.header.get_zooms())
 
-    # Binarize using CDT
-    binarized = stat_map > stat_threshold
-    binarized = binarized.astype(int)
-
-    # If the stat threshold is too high simply return an empty dataframe
-    if np.sum(binarized) == 0:
-        warnings.warn('Attention: No clusters with stat higher than %f' %
-                      stat_threshold)
-        return pd.DataFrame(columns=cols)
-
-    # Extract connected components above cluster size threshold
-    label_map = ndimage.measurements.label(binarized, conn_mat)[0]
-    clust_ids = sorted(list(np.unique(label_map)[1:]))
-    for c_val in clust_ids:
-        if cluster_threshold is not None and np.sum(
-            label_map == c_val) < cluster_threshold:
-            stat_map[label_map == c_val] = 0
-            binarized[label_map == c_val] = 0
-
-    # If the cluster threshold is too high simply return an empty dataframe
-    # this checks for stats higher than threshold after small clusters
-    # were removed from stat_map
-    if np.sum(stat_map > stat_threshold) == 0:
-        warnings.warn('Attention: No clusters with more than %d voxels' %
-                      cluster_threshold)
-        return pd.DataFrame(columns=cols)
-
-    # Now re-label and create table
-    label_map = ndimage.measurements.label(binarized, conn_mat)[0]
-    clust_ids = sorted(list(np.unique(label_map)[1:]))
-    peak_vals = np.array(
-        [np.max(stat_map * (label_map == c)) for c in clust_ids])
-    clust_ids = [clust_ids[c] for c in
-                 (-peak_vals).argsort()]  # Sort by descending max value
-
+    signs = [1, -1] if two_sided else [1]
+    no_clusters_found = True
     rows = []
-    for c_id, c_val in enumerate(clust_ids):
-        cluster_mask = label_map == c_val
-        masked_data = stat_map * cluster_mask
-
-        cluster_size_mm = int(np.sum(cluster_mask) * voxel_size)
-
-        # Get peaks, subpeaks and associated statistics
-        subpeak_ijk, subpeak_vals = _local_max(masked_data, stat_img.affine,
-                                               min_distance=min_distance)
-        subpeak_xyz = np.asarray(coord_transform(subpeak_ijk[:, 0],
-                                                 subpeak_ijk[:, 1],
-                                                 subpeak_ijk[:, 2],
-                                                 stat_img.affine)).tolist()
-        subpeak_xyz = np.array(subpeak_xyz).T
-
-        # Only report peak and, at most, top 3 subpeaks.
-        n_subpeaks = np.min((len(subpeak_vals), 4))
-        for subpeak in range(n_subpeaks):
-            if subpeak == 0:
-                row = [c_id + 1, subpeak_xyz[subpeak, 0],
-                       subpeak_xyz[subpeak, 1], subpeak_xyz[subpeak, 2],
-                       subpeak_vals[subpeak], cluster_size_mm]
-            else:
-                # Subpeak naming convention is cluster num+letter: 1a, 1b, etc
-                sp_id = '{0}{1}'.format(c_id + 1, ascii_lowercase[subpeak - 1])
-                row = [sp_id, subpeak_xyz[subpeak, 0], subpeak_xyz[subpeak, 1],
-                       subpeak_xyz[subpeak, 2], subpeak_vals[subpeak], '']
-            rows += [row]
-    df = pd.DataFrame(columns=cols, data=rows)
+    for sign in signs:
+        # Flip map if necessary
+        temp_stat_map = stat_map * sign
+
+        # Binarize using CDT
+        binarized = temp_stat_map > stat_threshold
+        binarized = binarized.astype(int)
+
+        # If the stat threshold is too high simply return an empty dataframe
+        if np.sum(binarized) == 0:
+            warnings.warn(
+                'Attention: No clusters with stat {0} than {1}'.format(
+                    'higher' if sign == 1 else 'lower',
+                    stat_threshold * sign,
+                )
+            )
+            continue
+
+        # Extract connected components above cluster size threshold
+        label_map = ndimage.measurements.label(binarized, conn_mat)[0]
+        clust_ids = sorted(list(np.unique(label_map)[1:]))
+        for c_val in clust_ids:
+            if cluster_threshold is not None and np.sum(
+                    label_map == c_val) < cluster_threshold:
+                temp_stat_map[label_map == c_val] = 0
+                binarized[label_map == c_val] = 0
+
+        # If the cluster threshold is too high simply return an empty dataframe
+        # this checks for stats higher than threshold after small clusters
+        # were removed from temp_stat_map
+        if np.sum(temp_stat_map > stat_threshold) == 0:
+            warnings.warn(
+                'Attention: No clusters with more than {0} voxels'.format(
+                    cluster_threshold,
+                )
+            )
+            continue
+
+        # Now re-label and create table
+        label_map = ndimage.measurements.label(binarized, conn_mat)[0]
+        clust_ids = sorted(list(np.unique(label_map)[1:]))
+        peak_vals = np.array(
+            [np.max(temp_stat_map * (label_map == c)) for c in clust_ids])
+        # Sort by descending max value
+        clust_ids = [clust_ids[c] for c in (-peak_vals).argsort()]
+
+        for c_id, c_val in enumerate(clust_ids):
+            cluster_mask = label_map == c_val
+            masked_data = temp_stat_map * cluster_mask
+
+            cluster_size_mm = int(np.sum(cluster_mask) * voxel_size)
+
+            # Get peaks, subpeaks and associated statistics
+            subpeak_ijk, subpeak_vals = _local_max(
+                masked_data,
+                stat_img.affine,
+                min_distance=min_distance,
+            )
+            subpeak_vals *= sign  # flip signs if necessary
+            subpeak_xyz = np.asarray(
+                coord_transform(
+                    subpeak_ijk[:, 0],
+                    subpeak_ijk[:, 1],
+                    subpeak_ijk[:, 2],
+                    stat_img.affine,
+                )
+            ).tolist()
+            subpeak_xyz = np.array(subpeak_xyz).T
+
+            # Only report peak and, at most, top 3 subpeaks.
+            n_subpeaks = np.min((len(subpeak_vals), 4))
+            for subpeak in range(n_subpeaks):
+                if subpeak == 0:
+                    row = [
+                        c_id + 1,
+                        subpeak_xyz[subpeak, 0],
+                        subpeak_xyz[subpeak, 1],
+                        subpeak_xyz[subpeak, 2],
+                        subpeak_vals[subpeak],
+                        cluster_size_mm,
+                    ]
+                else:
+                    # Subpeak naming convention is cluster num+letter:
+                    # 1a, 1b, etc
+                    sp_id = '{0}{1}'.format(
+                        c_id + 1,
+                        ascii_lowercase[subpeak - 1],
+                    )
+                    row = [
+                        sp_id,
+                        subpeak_xyz[subpeak, 0],
+                        subpeak_xyz[subpeak, 1],
+                        subpeak_xyz[subpeak, 2],
+                        subpeak_vals[subpeak],
+                        '',
+                    ]
+                rows += [row]
+
+        # If we reach this point, there are clusters in this sign
+        no_clusters_found = False
+
+    if no_clusters_found:
+        df = pd.DataFrame(columns=cols)
+    else:
+        df = pd.DataFrame(columns=cols, data=rows)
+
     return df
diff --git a/nilearn/reporting/glm_reporter.py b/nilearn/reporting/glm_reporter.py
--- a/nilearn/reporting/glm_reporter.py
+++ b/nilearn/reporting/glm_reporter.py
@@ -54,6 +54,7 @@ def make_glm_report(model,
                     alpha=0.001,
                     cluster_threshold=0,
                     height_control='fpr',
+                    two_sided=False,
                     min_distance=8.,
                     plot_type='slice',
                     display_mode=None,
@@ -128,6 +129,10 @@ def make_glm_report(model,
         threshold: 'fpr' (default) or 'fdr' or 'bonferroni' or None.
         Default='fpr'.
 
+    two_sided : `bool`, optional
+        Whether to employ two-sided thresholding or to evaluate positive values
+        only. Default=False.
+
     min_distance : float, optional
         For display purposes only.
         Minimum distance between subpeaks in mm. Default=8mm.
@@ -220,6 +225,7 @@ def make_glm_report(model,
         alpha=alpha,
         cluster_threshold=cluster_threshold,
         height_control=height_control,
+        two_sided=two_sided,
         min_distance=min_distance,
         bg_img=bg_img,
         display_mode=display_mode,
@@ -630,6 +636,7 @@ def _mask_to_svg(mask_img, bg_img):
 def _make_stat_maps_contrast_clusters(stat_img, contrasts_plots, threshold,
                                       alpha,
                                       cluster_threshold, height_control,
+                                      two_sided,
                                       min_distance, bg_img,
                                       display_mode, plot_type):
     """Populates a smaller HTML sub-template with the proper values,
@@ -668,6 +675,10 @@ def _make_stat_maps_contrast_clusters(stat_img, contrasts_plots, threshold,
         False positive control meaning of cluster forming
         threshold: 'fpr' or 'fdr' or 'bonferroni' or None.
 
+    two_sided : `bool`, optional
+        Whether to employ two-sided thresholding or to evaluate positive values
+        only. Default=False.
+
     min_distance : float
         For display purposes only.
         Minimum distance between subpeaks in mm. Default is 8 mm.
@@ -734,6 +745,7 @@ def _make_stat_maps_contrast_clusters(stat_img, contrasts_plots, threshold,
             stat_threshold=threshold,
             cluster_threshold=cluster_threshold,
             min_distance=min_distance,
+            two_sided=two_sided,
         )
 
         cluster_table_html = _dataframe_to_html(cluster_table,


Test patch:
diff --git a/nilearn/reporting/tests/test_reporting.py b/nilearn/reporting/tests/test_reporting.py
--- a/nilearn/reporting/tests/test_reporting.py
+++ b/nilearn/reporting/tests/test_reporting.py
@@ -73,31 +73,49 @@ def test_get_clusters_table(tmp_path):
     shape = (9, 10, 11)
     data = np.zeros(shape)
     data[2:4, 5:7, 6:8] = 5.
+    data[4:6, 7:9, 8:10] = -5.
     stat_img = nib.Nifti1Image(data, np.eye(4))
 
     # test one cluster extracted
-    cluster_table = get_clusters_table(stat_img, 4, 0)
+    cluster_table = get_clusters_table(stat_img, 4, 0, two_sided=False)
     assert len(cluster_table) == 1
 
     # test empty table on high stat threshold
-    cluster_table = get_clusters_table(stat_img, 6, 0)
+    cluster_table = get_clusters_table(stat_img, 6, 0, two_sided=False)
     assert len(cluster_table) == 0
 
     # test empty table on high cluster threshold
-    cluster_table = get_clusters_table(stat_img, 4, 9)
+    cluster_table = get_clusters_table(stat_img, 4, 9, two_sided=False)
+    assert len(cluster_table) == 0
+
+    # test two clusters with different signs extracted
+    cluster_table = get_clusters_table(stat_img, 4, 0, two_sided=True)
+    assert len(cluster_table) == 2
+
+    # test empty table on high stat threshold
+    cluster_table = get_clusters_table(stat_img, 6, 0, two_sided=True)
+    assert len(cluster_table) == 0
+
+    # test empty table on high cluster threshold
+    cluster_table = get_clusters_table(stat_img, 4, 9, two_sided=True)
     assert len(cluster_table) == 0
 
     # test with filename
     fname = str(tmp_path / "stat_img.nii.gz")
     stat_img.to_filename(fname)
-    cluster_table = get_clusters_table(fname, 4, 0)
-    assert len(cluster_table) == 1
+    cluster_table = get_clusters_table(fname, 4, 0, two_sided=True)
+    assert len(cluster_table) == 2
 
     # test with extra dimension
     data_extra_dim = data[..., np.newaxis]
     stat_img_extra_dim = nib.Nifti1Image(data_extra_dim, np.eye(4))
-    cluster_table = get_clusters_table(stat_img_extra_dim, 4, 0)
-    assert len(cluster_table) == 1
+    cluster_table = get_clusters_table(
+        stat_img_extra_dim,
+        4,
+        0,
+        two_sided=True
+    )
+    assert len(cluster_table) == 2
 
 
 def test_get_clusters_table_not_modifying_stat_image():
@@ -110,18 +128,33 @@ def test_get_clusters_table_not_modifying_stat_image():
     data_orig = get_data(stat_img).copy()
 
     # test one cluster should be removed
-    clusters_table = get_clusters_table(stat_img, 4, cluster_threshold=10)
+    clusters_table = get_clusters_table(
+        stat_img,
+        4,
+        cluster_threshold=10,
+        two_sided=True
+    )
     assert np.allclose(data_orig, get_data(stat_img))
     assert len(clusters_table) == 1
 
     # test no clusters should be removed
     stat_img = nib.Nifti1Image(data, np.eye(4))
-    clusters_table = get_clusters_table(stat_img, 4, cluster_threshold=7)
+    clusters_table = get_clusters_table(
+        stat_img,
+        4,
+        cluster_threshold=7,
+        two_sided=False
+    )
     assert np.allclose(data_orig, get_data(stat_img))
     assert len(clusters_table) == 2
 
     # test cluster threshold is None
     stat_img = nib.Nifti1Image(data, np.eye(4))
-    clusters_table = get_clusters_table(stat_img, 4, cluster_threshold=None)
+    clusters_table = get_clusters_table(
+        stat_img,
+        4,
+        cluster_threshold=None,
+        two_sided=False
+    )
     assert np.allclose(data_orig, get_data(stat_img))
     assert len(clusters_table) == 2