Repository: mne-tools/mne-python

Issue:
BUG: Forward code should raise an error if sensors are inside BEM
Hi,

I get strange BEM results with `sample` using the precomputed BEM. These manifest as erroneous/unexpected field values, mostly in regions below the brain. The problems seem to go away if I recreate the BEM in mne-python.

To demonstrate, I created an array of 1000 pointlike magnetometers uniformly spread around the brain:

![snapshot](https://user-images.githubusercontent.com/8491321/116250994-e97a0680-a776-11eb-84d5-21ed84dd5c1a.png)

I then computed the forward fields (for one particular source; doesn't really matter which one) using the precomputed BEM and a freshly created one:

![Figure_1](https://user-images.githubusercontent.com/8491321/116251785-aa988080-a777-11eb-892a-7cd5c4bc9c1c.png)

Note that the sensor index increases with decreasing z coordinate. It looks like the precomputed BEM starts to increasingly deviate from the newly created one as we go lower in the coordinate system. In addition to this, the values of precomputed BEM for some sensors are really strange (the spikes in the plot).

The results from a spherical model, as well as the mne-python freshly created BEM, look fine.

To recreate, please download this sensor description for the spherical array and unzip it:
[pointlike.zip](https://github.com/mne-tools/mne-python/files/6385035/pointlike.zip)

Then run the code below:

```
import mne
import pathlib
import matplotlib.pyplot as plt
import pickle

data_path = pathlib.Path(mne.datasets.sample.data_path())
subjects_dir = data_path / 'subjects'
subject = 'sample'
head_mri_trans_file = data_path / 'MEG/sample/sample_audvis_raw-trans.fif'
bem_file = subjects_dir / 'sample/bem/sample-5120-5120-5120-bem-sol.fif'
raw_file = data_path / 'MEG/sample/sample_audvis_raw.fif'
info = mne.io.read_info(raw_file)

# replace 'info' with a custom one to get pointlike sensors
alt_array_name = 'pointlike.dat'
with open(alt_array_name, 'rb') as f:
    info = pickle.load(f)

# source space
src_cort = mne.setup_source_space(
    subject, spacing='oct6', subjects_dir=subjects_dir, add_dist=False
)

# precomputed BEM
fwd_bem_precomp = mne.make_forward_solution(
    info,
    head_mri_trans_file,
    src_cort,
    str(bem_file),
    eeg=False,
)
lead_bem_precomp = fwd_bem_precomp['sol']['data']

# recompute BEM
model = mne.make_bem_model(
    subject=subject, ico=4, conductivity=[0.3], subjects_dir=subjects_dir
)
bem = mne.make_bem_solution(model)

fwd_bem = mne.make_forward_solution(
    info,
    head_mri_trans_file,
    src_cort,
    bem,
    eeg=False,
)
lead_bem = fwd_bem['sol']['data']

sphere = mne.make_sphere_model(head_radius=None)
fwd_sph = mne.make_forward_solution(
    info,
    head_mri_trans_file,
    src_cort,
    sphere,
    eeg=False,
)
lead_sph = fwd_sph['sol']['data']

# plot
src_ind = 3000  # some random source
plt.figure()
# plt.plot(lead_sph[:, src_ind], label='sphere model forward')
plt.plot(lead_bem[:, src_ind], label='BEM model')
# plt.plot(lead_bem_precomp[:, src_ind], label='BEM model, precomputed')
plt.plot(
    abs(lead_bem[:, src_ind] - lead_bem_precomp[:, src_ind]),
    label='Precomputed vs fresh BEM, absolute difference',
)
plt.ylim([-0.5e-5, 0.5e-5])
plt.legend()
plt.xlabel('Sensor index')
plt.ylabel('Field amplitude')










Discussion:
is this a pb of numerics that change between systems?
@agramfort maybe, but I find it hard to understand why the precomputed BEM blows up pretty much completely for some field points.


maybe the precomputed BEM was computed by the C code that uses float32?

>

To fix `sample`, we can of course just create and save new BEM models using mne-python. However some mne-python docs still suggest creating BEM models with the command line tools, so we might want to make sure they don't produce erroneous models.

BTW, this issue would probably go unnoticed in many cases due to 1) finite-area sensors causing the field values to be spatially averaged, reducing the error, and 2) the fact that MEG systems usually don't have sensors below the neck line. Still, it was bad enough to completely screw up some simulations I was doing :)

>
> To fix sample, we can of course just create and save new BEM models using
> mne-python. However some mne-python docs still suggest creating BEM models
> with the command line tools,
>
where?

> so we might want to make sure they don't produce erroneous models.
>

is it really erroneous or just numerics compatibility issues?


> BTW, this issue would probably go unnoticed by normal sensor setups due to
> 1) finite-area sensors cause the field values to be spatially averaged and
> reduce the error and 2) the fact that MEG systems usually don't have
> sensors below the neck line.
>

ok

> To fix sample, we can of course just create and save new BEM models using mne-python. However some mne-python docs still suggest creating BEM models with the command line tools,
> where?

At least https://mne.tools/stable/auto_tutorials/forward/30_forward.html says:

"Computing the BEM surfaces requires FreeSurfer and makes use of the command-line tools mne watershed_bem or mne flash_bem, or the related functions mne.bem.make_watershed_bem() or mne.bem.make_flash_bem()"

> is it really erroneous or just numerics compatibility issues?

Well, if the BEM example data gives results that are way off, I'd say it's erroneous. Whether it's due to numerics compability or something else is beside the point (IMO). 

Of course, one might argue that the pointlike array is just a corner case. However even with the default 306-channel array, I get 14% mean difference* between the precomputed and mne-python computed BEMs. 


<br>
*according to following definition:

    np.abs(lead_bem - lead_bem_precomp).mean() / np.abs(lead_bem_precomp).mean()


Hi MNEers!

Before going for actual bugs or smaller differences due to different numerical libraries, let's look at the model...
The precomputed MNE model is a 3-shell model, and based on what I understand from Jussi's script / its output, the new one is a single-shell model. For differences between a single- and three-shell model in a normal MEG sensor setting, please see (Stenroos et al, NeuroImage 2014). I haven't looked at MEG fields in this kind of closed sensor layout, but based on the geometry differences in head models the largest differences will be at inferior sensors. So this explains part of the difference.

Part of Jussi's inferior sensors are inside the head in the three-shell model, but outside the single-shell model. As the "helmet" intersects the scalp, some sensor may be very close to the computation points in the scalp mesh, causing potentially major numerical trouble.

So I would first move the sensors outside the scalp (say, at least 1/2 triangle side-length outside the scalp).

Hope this helps!
 Matti

![modelgeo_problem](https://user-images.githubusercontent.com/50441521/116425788-3a5b2f00-a84b-11eb-86e8-46da97de40d5.png)


Thanks @MattiStenroos! I feel dumb now. Somehow the fact that the precomputed model is a three-shell one completely escaped me.

I guess I was somewhat misled by https://mne.tools/stable/auto_tutorials/forward/30_forward.html, which uses the same dataset and  says:

"Here we’ll assume it’s already computed. It takes a few minutes per subject.
For EEG we use 3 layers (inner skull, outer skull, and skin) while for MEG 1 layer (inner skull) is enough."

So I just assumed that the precomputed model is single shell. 




Don't worry, you are not the first one to get confused with relationships between different head models and sensor or source locations...

I think it would always be a good idea to plot at least the scalp mesh and the sensor locations --- maybe add that to some MNE basic example scripts, if it is not there yet? As coregistration sanity-check, I would also check the distance from scalp to sensors. With head movement correction, I would additionally check that the reconstruction helmet is well outside the scalp.


this tells me that the forward code should crash if it detects some
channels inside the BEM surfaces...

also you say it recommends to use the command line but the command line you
refer to is just python code
so the only difference could be an IO roundtrip.


Resolving patch:
diff --git a/mne/forward/_make_forward.py b/mne/forward/_make_forward.py
--- a/mne/forward/_make_forward.py
+++ b/mne/forward/_make_forward.py
@@ -21,13 +21,13 @@
 from ..io.constants import FIFF, FWD
 from ..transforms import (_ensure_trans, transform_surface_to, apply_trans,
                           _get_trans, _print_coord_trans, _coord_frame_name,
-                          Transform)
+                          Transform, invert_transform)
 from ..utils import logger, verbose, warn, _pl
 from ..parallel import check_n_jobs
 from ..source_space import (_ensure_src, _filter_source_spaces,
                             _make_discrete_source_space, _complete_vol_src)
 from ..source_estimate import VolSourceEstimate
-from ..surface import _normalize_vectors
+from ..surface import _normalize_vectors, _CheckInside
 from ..bem import read_bem_solution, _bem_find_surface, ConductorModel
 
 from .forward import Forward, _merge_meg_eeg_fwds, convert_forward_solution
@@ -482,11 +482,38 @@ def _prepare_for_forward(src, mri_head_t, info, bem, mindist, n_jobs,
     bem = _setup_bem(bem, bem_extra, len(eegnames), mri_head_t,
                      allow_none=allow_bem_none)
 
-    # Circumvent numerical problems by excluding points too close to the skull
-    if bem is not None and not bem['is_sphere']:
-        inner_skull = _bem_find_surface(bem, 'inner_skull')
-        _filter_source_spaces(inner_skull, mindist, mri_head_t, src, n_jobs)
-        logger.info('')
+    # Circumvent numerical problems by excluding points too close to the skull,
+    # and check that sensors are not inside any BEM surface
+    if bem is not None:
+        if not bem['is_sphere']:
+            check_surface = 'inner skull surface'
+            inner_skull = _bem_find_surface(bem, 'inner_skull')
+            check_inside = _filter_source_spaces(
+                inner_skull, mindist, mri_head_t, src, n_jobs)
+            logger.info('')
+            if len(bem['surfs']) == 3:
+                check_surface = 'scalp surface'
+                check_inside = _CheckInside(
+                    _bem_find_surface(bem, 'head'))
+        else:
+            check_surface = 'outermost sphere shell'
+            if len(bem['layers']) == 0:
+                def check_inside(x):
+                    return np.zeros(len(x), bool)
+            else:
+                def check_inside(x):
+                    return (np.linalg.norm(x, axis=1) <
+                            bem['layers'][-1]['rad'])
+        if len(megcoils):
+            meg_loc = apply_trans(
+                invert_transform(mri_head_t),
+                np.array([coil['r0'] for coil in megcoils]))
+            n_inside = check_inside(meg_loc).sum()
+            if n_inside:
+                raise RuntimeError(
+                    f'Found {n_inside} MEG sensor{_pl(n_inside)} inside the '
+                    f'{check_surface}, perhaps coordinate frames and/or '
+                    'coregistration must be incorrect')
 
     rr = np.concatenate([s['rr'][s['vertno']] for s in src])
     if len(rr) < 1:
diff --git a/mne/source_space.py b/mne/source_space.py
--- a/mne/source_space.py
+++ b/mne/source_space.py
@@ -2167,6 +2167,7 @@ def _filter_source_spaces(surf, limit, mri_head_t, src, n_jobs=1,
         # Adjust the patch inds as well if necessary
         if omit_limit + omit_outside > 0:
             _adjust_patch_info(s)
+    return check_inside
 
 
 @verbose


Test patch:
diff --git a/mne/forward/tests/test_make_forward.py b/mne/forward/tests/test_make_forward.py
--- a/mne/forward/tests/test_make_forward.py
+++ b/mne/forward/tests/test_make_forward.py
@@ -16,6 +16,7 @@
                  make_sphere_model, pick_types_forward, pick_info, pick_types,
                  read_evokeds, read_cov, read_dipole,
                  get_volume_labels_from_aseg)
+from mne.surface import _get_ico_surface
 from mne.transforms import Transform
 from mne.utils import requires_mne, requires_nibabel, run_subprocess
 from mne.forward._make_forward import _create_meg_coils, make_forward_dipole
@@ -475,7 +476,7 @@ def test_use_coil_def(tmpdir):
     """Test use_coil_def."""
     info = create_info(1, 1000., 'mag')
     info['chs'][0]['coil_type'] = 9999
-    info['chs'][0]['loc'][:] = [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1]
+    info['chs'][0]['loc'][:] = [0, 0, 0.02, 1, 0, 0, 0, 1, 0, 0, 0, 1]
     sphere = make_sphere_model((0., 0., 0.), 0.01)
     src = setup_volume_source_space(pos=5, sphere=sphere)
     trans = Transform('head', 'mri', None)
@@ -502,3 +503,30 @@ def test_use_coil_def(tmpdir):
   0.1250  0.750e-03  0.750e-03  0.750e-03  0.000  0.000  1.000""")
     with use_coil_def(coil_fname):
         make_forward_solution(info, trans, src, sphere)
+
+
+@testing.requires_testing_data
+def test_sensors_inside_bem():
+    """Test that sensors inside the BEM are problematic."""
+    rr = _get_ico_surface(1)['rr']
+    rr /= np.linalg.norm(rr, axis=1, keepdims=True)
+    rr *= 0.1
+    assert len(rr) == 42
+    info = create_info(len(rr), 1000., 'mag')
+    info['dev_head_t'] = Transform('meg', 'head', np.eye(4))
+    for ii, ch in enumerate(info['chs']):
+        ch['loc'][:] = np.concatenate((rr[ii], np.eye(3).ravel()))
+    trans = Transform('head', 'mri', np.eye(4))
+    trans['trans'][2, 3] = 0.03
+    sphere_noshell = make_sphere_model((0., 0., 0.), None)
+    sphere = make_sphere_model((0., 0., 0.), 1.01)
+    with pytest.raises(RuntimeError, match='.* 15 MEG.*inside the scalp.*'):
+        make_forward_solution(info, trans, fname_src, fname_bem)
+    make_forward_solution(info, trans, fname_src, fname_bem_meg)  # okay
+    make_forward_solution(info, trans, fname_src, sphere_noshell)  # okay
+    with pytest.raises(RuntimeError, match='.* 42 MEG.*outermost sphere sh.*'):
+        make_forward_solution(info, trans, fname_src, sphere)
+    for ch in info['chs']:
+        ch['loc'][:3] *= 0.1
+    with pytest.raises(RuntimeError, match='.* 42 MEG.*the inner skull.*'):
+        make_forward_solution(info, trans, fname_src, fname_bem_meg)