Repository: mne-tools/mne-python

Issue:
Failing test in test_source_space.py
`nosetests mne/tests/test_source_space.py:test_other_volume_source_spaces` fails for me locally with the current master. Can someone confirm?


Discussion:
It passes for me. Do you have MNE-C installed?
it does not pass for me either:

F
======================================================================
FAIL: Test setting up other volume source spaces.
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/Users/alex/anaconda/envs/py35/lib/python3.5/site-packages/nose/case.py",
line 198, in runTest
    self.test(*self.arg)
  File
"/Users/alex/anaconda/envs/py35/lib/python3.5/site-packages/numpy/testing/decorators.py",
line 147, in skipper_func
    return f(*args, **kwargs)
  File "/Users/alex/work/src/mne-python/mne/utils.py", line 843, in dec
    return function(*args, **kwargs)
  File "/Users/alex/work/src/mne-python/mne/tests/test_source_space.py",
line 257, in test_other_volume_source_spaces
    _compare_source_spaces(src, src_new, mode='approx')
  File "/Users/alex/work/src/mne-python/mne/source_space.py", line 2680, in
_compare_source_spaces
    assert_equal(s0[name], s1[name], name)
AssertionError: 8407 != 7497 : nuse
    '8407 != 7497' = '%s != %s' % _common_shorten_repr(8407, 7497)
    '8407 != 7497 : nuse' = self._formatMessage('8407 != 7497 : nuse',
'8407 != 7497')
>>  raise self.failureException('8407 != 7497 : nuse')


----------------------------------------------------------------------

do you have the same error?

@jaeilepp yes I have MNE-C
@agramfort I have the same error 
It must be some recent change. I pulled the latest and it fails for me too.
Looks like it's https://github.com/mne-tools/mne-python/pull/4355
hum. How come it is not detected on travis?

I'll try to look by early next week

So it turns out this is failing because the updated code is actually *more* accurate at determining points inside the limit than it was before.

Why? The `_filter_source_spaces` code, which operates on tesselated surfaces, first filters out points outside the surface (using sum of solid angles). Of the remaining points, it finds the closest surface vertex, and ensures that it is at least `limit` mm away from this point. When the user requested a sphere, it used to use a sphere with an ico-3 surface, which has 642 points. This puts the points roughly 1cm apart on the 90mm radius sphere used in the test. Thus you can be up to ~5mm away from the nearest tesselation point *and actually be on the surface of the sphere*, yet such a point will be kept. In this test it's not that bad -- the point closest to the surface of the sphere is 3.5mm inside. This is clearly less than the 5mm it should be, but is not so terrible.

So maybe the new values should be used as a bugfix, and we should document it as an accuracy increase for sphere-based vertex limiting in volume source spaces?
... as to why it wasn't tested, it looks like the `requires_mne` tests were skipped on that run, not sure why. Maybe in the test-split migration I accidentally got rid of those tests? I'll look.

Resolving patch:
diff --git a/mne/source_space.py b/mne/source_space.py
--- a/mne/source_space.py
+++ b/mne/source_space.py
@@ -2672,7 +2672,7 @@ def _compare_source_spaces(src0, src1, mode='exact', nearest=True,
     if mode != 'exact' and 'approx' not in mode:  # 'nointerp' can be appended
         raise RuntimeError('unknown mode %s' % mode)
 
-    for s0, s1 in zip(src0, src1):
+    for si, (s0, s1) in enumerate(zip(src0, src1)):
         # first check the keys
         a, b = set(s0.keys()), set(s1.keys())
         assert_equal(a, b, str(a ^ b))
@@ -2725,10 +2725,11 @@ def _compare_source_spaces(src0, src1, mode='exact', nearest=True,
                     assert_true(len((s0['dist'] - s1['dist']).data) == 0)
         else:  # 'approx' in mode:
             # deal with vertno, inuse, and use_tris carefully
-            assert_array_equal(s0['vertno'], np.where(s0['inuse'])[0],
-                               'left hemisphere vertices')
-            assert_array_equal(s1['vertno'], np.where(s1['inuse'])[0],
-                               'right hemisphere vertices')
+            for ii, s in enumerate((s0, s1)):
+                assert_array_equal(s['vertno'], np.where(s['inuse'])[0],
+                                   'src%s[%s]["vertno"] != '
+                                   'np.where(src%s[%s]["inuse"])[0]'
+                                   % (ii, si, ii, si))
             assert_equal(len(s0['vertno']), len(s1['vertno']))
             agreement = np.mean(s0['inuse'] == s1['inuse'])
             assert_true(agreement >= 0.99, "%s < 0.99" % agreement)


Test patch:
diff --git a/mne/tests/test_dipole.py b/mne/tests/test_dipole.py
--- a/mne/tests/test_dipole.py
+++ b/mne/tests/test_dipole.py
@@ -167,14 +167,15 @@ def test_dipole_fitting():
                                                      axis=1)))]
         amp_errs += [np.sqrt(np.mean((amp - d.amplitude) ** 2))]
         gofs += [np.mean(d.gof)]
-    assert_true(dists[0] >= dists[1], 'dists: %s' % dists)
-    assert_true(corrs[0] <= corrs[1], 'corrs: %s' % corrs)
-    assert_true(gc_dists[0] >= gc_dists[1] * 0.8,
+    factor = 0.8
+    assert_true(dists[0] / factor >= dists[1], 'dists: %s' % dists)
+    assert_true(corrs[0] * factor <= corrs[1], 'corrs: %s' % corrs)
+    assert_true(gc_dists[0] / factor >= gc_dists[1] * 0.8,
                 'gc-dists (ori): %s' % gc_dists)
-    assert_true(amp_errs[0] >= amp_errs[1],
+    assert_true(amp_errs[0] / factor >= amp_errs[1],
                 'amplitude errors: %s' % amp_errs)
     # This one is weird because our cov/sim/picking is weird
-    assert_true(gofs[0] <= gofs[1] * 2, 'gof: %s' % gofs)
+    assert_true(gofs[0] * factor <= gofs[1] * 2, 'gof: %s' % gofs)
 
 
 @testing.requires_testing_data
diff --git a/mne/tests/test_source_space.py b/mne/tests/test_source_space.py
--- a/mne/tests/test_source_space.py
+++ b/mne/tests/test_source_space.py
@@ -254,7 +254,19 @@ def test_other_volume_source_spaces():
     src = read_source_spaces(temp_name)
     src_new = setup_volume_source_space(None, pos=7.0, mri=fname_mri,
                                         subjects_dir=subjects_dir)
-    _compare_source_spaces(src, src_new, mode='approx')
+    # we use a more accurate elimination criteria, so let's fix the MNE-C
+    # source space
+    assert_equal(len(src_new[0]['vertno']), 7497)
+    assert_equal(len(src), 1)
+    assert_equal(len(src_new), 1)
+    good_mask = np.in1d(src[0]['vertno'], src_new[0]['vertno'])
+    src[0]['inuse'][src[0]['vertno'][~good_mask]] = 0
+    assert_equal(src[0]['inuse'].sum(), 7497)
+    src[0]['vertno'] = src[0]['vertno'][good_mask]
+    assert_equal(len(src[0]['vertno']), 7497)
+    src[0]['nuse'] = len(src[0]['vertno'])
+    assert_equal(src[0]['nuse'], 7497)
+    _compare_source_spaces(src_new, src, mode='approx')
     assert_true('volume, shape' in repr(src))
     del src
     del src_new