Repository: mne-tools/mne-python

Issue:
stats.spatio_temporal_cluster_1samp_test() fails with an index error
#### Describe the bug
`stats.spatio_temporal_cluster_1samp_test()` fails for me with an index error. Unfortunately I didn't manage to put together a MWE yet, but hopefully the provided information is sufficient to track things down? Any help would be greatly appreciated. :)

#### Steps and/or code to reproduce
```python
src = mne.read_source_spaces(surface_source_space_file)
connectivity = mne.spatial_src_connectivity(src)

# Concatenate STCs from individual subjects.
# `stcs` is a list of SourceEstimates, one per subject.
# The resulting shape of the ndarray is (n_subj, n_timepoints, n_vertices)
stcs_for_stats = [s.data.T for s in stcs]
stcs_for_stats = np.array(stcs_for_stats)

mne.stats.spatio_temporal_cluster_1samp_test(
    X=stcs_for_stats,
    connectivity=connectivity,
    n_jobs=1,
    verbose=True)
```

#### Expected results
The function should return the results of the test.
#### Actual results
```python
Using a threshold of 2.109816
stat_fun(H1): min=2.351713 max=20.239683
/Users/hoechenberger/Projects/TaLa/Code/Analysis/EEG/source_70_make_inverse_solution.py:132: RuntimeWarning: Provided stat_fun does not treat variables independently. Setting buffer_size to None.
  verbose=True)
Running initial clustering
Traceback (most recent call last):
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-7331a14a6e31>", line 1, in <module>
    runfile('/Users/hoechenberger/Projects/TaLa/Code/Analysis/EEG/source_70_make_inverse_solution.py', wdir='/Users/hoechenberger/Projects/TaLa/Code/Analysis/EEG')
  File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/hoechenberger/Projects/TaLa/Code/Analysis/EEG/source_70_make_inverse_solution.py", line 132, in <module>
    verbose=True)
  File "</Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/externals/decorator.py:decorator-gen-218>", line 2, in spatio_temporal_cluster_1samp_test
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/utils/_logging.py", line 88, in wrapper
    return function(*args, **kwargs)
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/stats/cluster_level.py", line 1382, in spatio_temporal_cluster_1samp_test
    check_disjoint=check_disjoint, buffer_size=buffer_size)
  File "</Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/externals/decorator.py:decorator-gen-217>", line 2, in permutation_cluster_1samp_test
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/utils/_logging.py", line 89, in wrapper
    return function(*args, **kwargs)
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/stats/cluster_level.py", line 1264, in permutation_cluster_1samp_test
    buffer_size=buffer_size)
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/stats/cluster_level.py", line 875, in _permutation_cluster_test
    show_info=True)
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/stats/cluster_level.py", line 431, in _find_clusters
    ndimage)
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/stats/cluster_level.py", line 475, in _find_clusters_1dir_parts
    t_power, ndimage)
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/stats/cluster_level.py", line 527, in _find_clusters_1dir
    clusters = _get_clusters_st(x_in, connectivity, max_step)
  File "/Users/hoechenberger/miniconda3/envs/TaLa-source-loc/lib/python3.7/site-packages/mne/stats/cluster_level.py", line 254, in _get_clusters_st
    row, col = np.unravel_index(cl_goods, (n_times, n_src))
ValueError: index 4917600 is out of bounds for array with size 4917600
```
#### Additional information
```python
In[3]: stcs_for_stats.shape
Out[3]: (18, 601, 8195)
In[11]: connectivity.shape
Out[11]: (8196, 8196)
```

Inside `mne.stats.cluster_level._get_clusters_st()`:
```python
In[3]: len(x_in)
Out[3]: 4925195
In[4]: n_src
Out[4]: 8196
In[5]: n_times
Out[5]: 600
In[7]: len(cl_goods)
Out[7]: 4925195
```

This is with `master`.


Discussion:
Can you share `X` and `connectivity`?
@larsoner Since this is actual, unpublished data, may I share this privately with you?
Sure, feel free to email a link
Done. Thanks for your help, @larsoner!

Resolving patch:
diff --git a/mne/stats/cluster_level.py b/mne/stats/cluster_level.py
--- a/mne/stats/cluster_level.py
+++ b/mne/stats/cluster_level.py
@@ -570,15 +570,22 @@ def _pval_from_histogram(T, H0, tail):
     return pval
 
 
-def _setup_connectivity(connectivity, n_vertices, n_times):
+def _setup_connectivity(connectivity, n_tests, n_times):
     if not sparse.issparse(connectivity):
-        raise ValueError("If connectivity matrix is given, it must be a"
-                         "scipy sparse matrix.")
-    if connectivity.shape[0] == n_vertices:  # use global algorithm
+        raise ValueError("If connectivity matrix is given, it must be a "
+                         "SciPy sparse matrix.")
+    if connectivity.shape[0] == n_tests:  # use global algorithm
         connectivity = connectivity.tocoo()
     else:  # use temporal adjacency algorithm
-        if not round(n_vertices / float(connectivity.shape[0])) == n_times:
-            raise ValueError('connectivity must be of the correct size')
+        got_times, mod = divmod(n_tests, connectivity.shape[0])
+        if got_times != n_times or mod != 0:
+            raise ValueError(
+                'connectivity (len %d) must be of the correct size, i.e. be '
+                'equal to or evenly divide the number of tests (%d).\n\n'
+                'If connectivity was computed for a source space, try using '
+                'the fwd["src"] or inv["src"] as some original source space '
+                'vertices can be excluded during forward computation'
+                % (connectivity.shape[0], n_tests))
         # we claim to only use upper triangular part... not true here
         connectivity = (connectivity + connectivity.transpose()).tocsr()
         connectivity = [connectivity.indices[connectivity.indptr[i]:
@@ -810,11 +817,12 @@ def _permutation_cluster_test(X, threshold, n_permutations, tail, stat_fun,
     is elicited.
     """
     _check_option('out_type', out_type, ['mask', 'indices'])
-    if not isinstance(threshold, dict) and (tail < 0 and threshold > 0 or
-                                            tail > 0 and threshold < 0 or
-                                            tail == 0 and threshold < 0):
-        raise ValueError('incompatible tail and threshold signs, got %s and %s'
-                         % (tail, threshold))
+    if not isinstance(threshold, dict):
+        threshold = float(threshold)
+        if (tail < 0 and threshold > 0 or tail > 0 and threshold < 0 or
+                tail == 0 and threshold < 0):
+            raise ValueError('incompatible tail and threshold signs, got '
+                             '%s and %s' % (tail, threshold))
 
     # check dimensions for each group in X (a list at this stage).
     X = [x[:, np.newaxis] if x.ndim == 1 else x for x in X]


Test patch:
diff --git a/mne/stats/tests/test_cluster_level.py b/mne/stats/tests/test_cluster_level.py
--- a/mne/stats/tests/test_cluster_level.py
+++ b/mne/stats/tests/test_cluster_level.py
@@ -331,10 +331,9 @@ def test_cluster_permutation_with_connectivity(numba_conditional):
         else:
             X1d_3 = np.reshape(X1d_2, (-1, 2, n_space))
 
-        out_connectivity_3 = spatio_temporal_func(X1d_3, n_permutations=50,
-                                                  connectivity=connectivity,
-                                                  max_step=0, threshold=1.67,
-                                                  check_disjoint=True)
+        out_connectivity_3 = spatio_temporal_func(
+            X1d_3, n_permutations=50, connectivity=connectivity,
+            max_step=0, threshold=1.67, check_disjoint=True)
         # make sure we were operating on the same values
         split = len(out[0])
         assert_array_equal(out[0], out_connectivity_3[0][0])
@@ -350,12 +349,12 @@ def test_cluster_permutation_with_connectivity(numba_conditional):
         assert len(data_1.intersection(data_2)) == len(data_1)
 
         # test new versus old method
-        out_connectivity_4 = spatio_temporal_func(X1d_3, n_permutations=50,
-                                                  connectivity=connectivity,
-                                                  max_step=2, threshold=1.67)
-        out_connectivity_5 = spatio_temporal_func(X1d_3, n_permutations=50,
-                                                  connectivity=connectivity,
-                                                  max_step=1, threshold=1.67)
+        out_connectivity_4 = spatio_temporal_func(
+            X1d_3, n_permutations=50, connectivity=connectivity,
+            max_step=2, threshold=1.67)
+        out_connectivity_5 = spatio_temporal_func(
+            X1d_3, n_permutations=50, connectivity=connectivity,
+            max_step=1, threshold=1.67)
 
         # clusters could be in a different order
         sums_4 = [np.sum(out_connectivity_4[0][a])
@@ -372,8 +371,9 @@ def test_cluster_permutation_with_connectivity(numba_conditional):
                           max_step=1, threshold=1.67, n_jobs=-1000)
 
         # not enough TFCE params
-        pytest.raises(KeyError, spatio_temporal_func, X1d_3,
-                      connectivity=connectivity, threshold=dict(me='hello'))
+        with pytest.raises(KeyError, match='threshold, if dict, must have'):
+            spatio_temporal_func(
+                X1d_3, connectivity=connectivity, threshold=dict(me='hello'))
 
         # too extreme a start threshold
         with pytest.warns(None) as w:
@@ -383,26 +383,31 @@ def test_cluster_permutation_with_connectivity(numba_conditional):
             assert len(w) == 1
             did_warn = True
 
-        # too extreme a start threshold
-        pytest.raises(ValueError, spatio_temporal_func, X1d_3,
-                      connectivity=connectivity, tail=-1,
-                      threshold=dict(start=1, step=-1))
-        pytest.raises(ValueError, spatio_temporal_func, X1d_3,
-                      connectivity=connectivity, tail=-1,
-                      threshold=dict(start=-1, step=1))
-        # Make sure connectivity has to be sparse
-        pytest.raises(ValueError, spatio_temporal_func, X1d_3,
-                      n_permutations=50, connectivity=connectivity.todense(),
-                      max_step=1, threshold=1.67)
-
-        # wrong type for threshold
-        pytest.raises(TypeError, spatio_temporal_func, X1d_3,
-                      connectivity=connectivity, threshold=[])
-
-        # wrong value for tail
-        with pytest.warns(None):  # sometimes ignoring tail
-            pytest.raises(ValueError, spatio_temporal_func, X1d_3,
-                          connectivity=connectivity, tail=2)
+        with pytest.raises(ValueError, match='threshold.*<= 0 for tail == -1'):
+            spatio_temporal_func(
+                X1d_3, connectivity=connectivity, tail=-1,
+                threshold=dict(start=1, step=-1))
+        with pytest.warns(RuntimeWarning, match='threshold.* is more extreme'):
+            spatio_temporal_func(
+                X1d_3, connectivity=connectivity, tail=1,
+                threshold=dict(start=100, step=1))
+        bad_con = connectivity.todense()
+        with pytest.raises(ValueError, match='must be a SciPy sparse matrix'):
+            spatio_temporal_func(
+                X1d_3, n_permutations=50, connectivity=bad_con,
+                max_step=1, threshold=1.67)
+        bad_con = connectivity.tocsr()[:-1, :-1].tocoo()
+        with pytest.raises(ValueError, match='connectivity.*the correct size'):
+            spatio_temporal_func(
+                X1d_3, n_permutations=50, connectivity=bad_con,
+                max_step=1, threshold=1.67)
+        with pytest.raises(TypeError, match='must be a'):
+            spatio_temporal_func(
+                X1d_3, connectivity=connectivity, threshold=[])
+        with pytest.raises(ValueError, match='Invalid value for the \'tail\''):
+            with pytest.warns(None):  # sometimes ignoring tail
+                spatio_temporal_func(
+                    X1d_3, connectivity=connectivity, tail=2)
 
         # make sure it actually found a significant point
         out_connectivity_6 = spatio_temporal_func(X1d_3, n_permutations=50,