Repository: mne-tools/mne-python

Issue:
mne.bem.make_watershed_bem() Coordinate System Error
#### Describe the bug
Incorrect extraction of volume info by the _extract_volume_info method in make_watershed_bem() in bem.py. If you look at the extracted BEM surfaces using the ancient mne freeview_bem_surfaces method from terminal that calls Freeview, you will see that the BEM surfaces have flipped yz coordinates. This is due to incorrect use of the header['Mdc'].T matrix. 


#### Steps to reproduce
Sorry I can't provide a snippet since this bug requires examining the BEM surfaces outside MNE-Python. But if you just run mne.bem.make_watershed_bem() on any Freesurfer T1 recon, and then use freeview to check the surfaces. For instance, you can use the MNE-C method mne freeview_bem_surfaces -s subject to check (or you can manually do it by freeview -f surfacefile) in the watershed folder, you will see the yz dimensions are flipped. 

[MWE]: https://en.wikipedia.org/wiki/Minimal_Working_Example
[built-in datasets]: https://mne.tools/dev/overview/datasets_index.html
[sample]: https://mne.tools/dev/overview/datasets_index.html#sample


#### Expected results
Well.. the BEM surface should have the same volume-geometry as the T1.mgz from Freesurfer recon. It is not in version 0.20.0. 


#### Actual results
It's flipped in yz dimensions. 


#### Additional information
Ok here's some historical information: this is an inherited bug since long time ago when initially there was the bug on the line "write_surface(s, rr, tris, volume_info=volume_info)" that should have been "write_surface(surf_ws_out, rr, tris, volume_info=volume_info)". I don't know at which version this is fixed but it is correct in 0.20.0. However the _extract_volume_info method is wrong. header['Mdc'].T is not [xras, yras, bras]. Instead, it is [rxyz, axyz, sxyz]. So you need to transpose the matrix before assigning the ras direction vectors in vol_info. 

Here is an easy fix: change line 1229 from 
vol_info['xras'], vol_info['yras'], vol_info['zras'] = header['Mdc'].T
to 
vol_info['xras'], vol_info['yras'], vol_info['zras'] = list(zip(*header['Mdc'].T))
and you will fix it. 

In theory if everyone's using freesurfer correctly then the outputted surfaces will already be in the right Freesurfer coordinates. So just like you don't update volume_geometry in mne.bem.make_flash_bem(), you don't need to update watershed surfaces in mne.bem.make_watershed_bem() either. I could see the initial thought was probably to accommodate T1.mgz beyond the default output volume from recon-all pipeline, so I guess it makes more sense to fix this transpose error than to remove the volume-geometry update codes all together. Just as a further suggestion, there is a useful freesurfer command mris_convert --vol-geom volume function that handles all these different structural scan type issues. So you might just want to call mris_convert in a subprocess.run() call that will save you a lot of trouble and avoid having to define your own _extract_volume_info() function, that is dependent on how nibable reads header files. Cheers. 


Discussion:
I just did `mne freeview_bem_surfaces -s sample` and it looks fine. Can you point to a dataset for which it's broken? It also works fine on somato dataset.
Did you re-run mne.bem.make_watershed_bem() on sample or were you viewing the generated watershed BEM surfaces? I believe the BEM surfaces generated before v0.19 were correct because of the error on the line write_surface(s, rr, tris, volume_info=volume_info) that failed to update volume geometry on the generated surface files. Since they are in the correct coordinates after generated by freesurfer watershed, not updating actually gave the right volume geometry. 
> Sorry I can't provide a snippet since this bug requires examining the BEM surfaces outside MNE-Python. 

Even something scripted that requires Freeview is fine. For example I just tried:
```
$ cd ~/mne_data/MNE-sample-data/subjects
$ export SUBJECTS_DIR=$PWD
$ export SUBJECT=sample
$ rm -Rf sample/bem/*.surf sample/bem/watershed
$ mne watershed_bem -s sample -d $PWD --verbose
$ freeview -v sample/mri/T1.mgz -f sample/bem/inner_skull.surf
```
And got:

![Screenshot from 2020-04-07 09-55-21](https://user-images.githubusercontent.com/2365790/78677585-e8e5a800-78b5-11ea-8e5b-39a811be5a8b.png)

Can you make a PR to fix the behavior? Then we can test locally and see that it works. If you want to go the extra mile, you could look into modifying `test_watershed_bem` in `mne/commands/tests/test_commands.py` to examine the volume info of the output surface. There is already a `read_surface` call at the end, I think if you modify it to do `read_metadata=True` you could come up with some check of the xras yras and/or zras that should fail on `master` but pass on this PR.
Maybe I'm missing something, but isn't the surface shown in your plot upside-down?? That's precisely the issue I had. 
Yes, I mean to say that I can replicate the issue :)

Are you up for making a PR to fix it?
Oh, I see. I read the "Even something scripted that requires Freeview is fine" as the surface looks fine. Sure... I can do that, but which fix do you want? 
1) Remove volume geometry update all together from mne.bem.make_watershed_bem()
2) Do the transpose fix of _extract_volume_info method 
3) Do a subprocess call of mris_convert --vol-geom to update volume geometry
Any of the three will fix this issue. 
As a user who just generated a ton of bems using 0.20, is this a
consistency with freeview/freesurfer visualization bug or will it also
effect MNE coregistration/source modeling/forward generation as well?

On Tue, Apr 7, 2020 at 10:10 AM Alex He <notifications@github.com> wrote:

> Oh, I see. I read the "Even something scripted that requires Freeview is
> fine" as the surface looks fine. Sure... I can do that, but which fix do
> you want?
>
>    1. Remove volume geometry update all together from
>    mne.bem.make_watershed_bem()
>    2. Do the transpose fix of _extract_volume_info method
>    3. Do a subprocess call of mris_convert --vol-geom to update volume
>    geometry
>    Any of the three will fix this issue.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/mne-tools/mne-python/issues/7570#issuecomment-610409380>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABKTXHNEBZRFVZAESZ6BJCLRLMX4BANCNFSM4MCXQR6Q>
> .
>

I'll let the experts confirm this, but I think it does. It's not a freeview visualization bug but an error on volume geometry of BEM surfaces. ~~Your forward model will be wrong since the source space is flipped upside-down from electrode registration (BEM surfaces might be fine since I don't think volume geometry alters the coordinates of triangulation surfaces).~~
Edit: Actually now I think about it more carefully, your forward model should be fine. I believe at no point is MNE using the volume geometry explicitly in computations, so the coordinates should have been kept intact. 

**Something you can try:** 
`mne.viz.plot_alignment(raw.info, trans=trans, subject=subject,
                       src=src, subjects_dir=subjects_dir, meg=[], dig=True,
                       surfaces=['head', 'white'], coord_frame='head', interaction='terrain')`
and see if your source space looks upside-down from your electrodes. 

------------------------------------------------------------------------------------------------
Ok just checked version history for you. This bug was fine in version 0.19 due to the masking bug on line 1154
            `write_surface(s, rr, tris, volume_info=volume_info)` 
So this is a new issue that only affects people using v0.20 to generate forward models with the watershed algorithm since the release of v0.20 on March 27th, I think. 

> Actually now I think about it more carefully, your forward model should be fine. I believe at no point is MNE using the volume geometry explicitly in computations, so the coordinates should have been kept intact.

That's what I expect. @bloyl as long as `plot_alignment` looks correct, then things should be correct. We try to keep the transformations there and in the forward code consistent, and indeed we never use the volume information.

> 2. Do the transpose fix of _extract_volume_info method

This is the fix I would do. It's the least invasive (better than option 3 in this sense) yet keeps things the most correct / complete (better than option 1 in this sense).
> This is the fix I would do. It's the least invasive (better than option 3 in this sense) yet keeps things the most correct / complete (better than option 1 in this sense).

Sounds good. Will push later this week. 
> Oh, I see. I read the "Even something scripted that requires Freeview is fine" as the surface looks fine. Sure... I can do that, but which fix do you want?
> 
>     1. Remove volume geometry update all together from mne.bem.make_watershed_bem()
> 
>     2. Do the transpose fix of _extract_volume_info method
> 
>     3. Do a subprocess call of mris_convert --vol-geom to update volume geometry
>        Any of the three will fix this issue.

Hi @mh105 , I noticed that `list(zip(*header['Mdc'].T))` is the same as `header['Mdc']`:
```
>> list(zip(*header['Mdc'].T))
[(-1.0, 8.56817e-08, 1.4901168e-08),
 (1.15484006e-07, 3.1432176e-08, -1.0),
 (-1.9185245e-07, 1.0, 1.2805678e-08)]

>> header['Mdc']
array([[-1.00000000e+00,  8.56816982e-08,  1.49011683e-08],
       [ 1.15484006e-07,  3.14321760e-08, -1.00000000e+00],
       [-1.91852450e-07,  1.00000000e+00,  1.28056783e-08]], dtype=float32)
```

So `vol_info['xras'], vol_info['yras'], vol_info['zras'] = header['Mdc']` seems to fix the problem?
![Screenshot from 2020-04-15 15-08-57](https://user-images.githubusercontent.com/29604854/79308574-6602c400-7f2b-11ea-89c2-5f18fb926b40.png)


> Hi @mh105 , I noticed that `list(zip(*header['Mdc'].T))` is the same as `header['Mdc']`:

Very true. Let me just do that instead of transposing back. 
> > Hi @mh105 , I noticed that `list(zip(*header['Mdc'].T))` is the same as `header['Mdc']`:
> 
> Very true. Let me just do that instead of transposing back.

After digging in the file history, I think the bug was likely caused by misunderstanding of `header_dtd['Mdc']` in [`nibabel/freesurfer/mghformat.py`](https://github.com/nipy/nibabel/blob/e2f50b4cf11482e6f018195aa6b89ccf12a3141e/nibabel/freesurfer/mghformat.py).

### Details
`header_dtd['Mdc']` was originally considered to be the cosines transform according to [FreeSurfer WiKi](https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/MghFormat).
That might be the reason why `_extract_volume_info` acquires the RAS coordinates by transposing `header_dtd['Mdc']`. Because of the bug described in #7139, the output surfaces were not updated by `_extract_volume_info`. So they were correct and the bug went unnoticed. 
After several commits, the annotation of `header_dtd` then indicated that `Mdc` is actually the _TRANSPOSE_ of direction cosine matrix, which are the RAS coordinates (hopefully not wrong, I'm not familiar with the coordinate systems). Because #7139 was fixed in v20, the bug in `_extract_volume_info` then results in the coordinate issue.

Technically, I caused this error by #7144... I made some [changes](https://github.com/mne-tools/mne-python/compare/master...yh-luo:fix_volume_info) to deal with the it due to guilty. If you could not make a PR please let me know. Would fix it happily :sweat_smile: 
Oh I see. Yeah go ahead with PR. I was able to create the PR but I realized you already fixed it with the changes. You have better documentations on this as well =) Cheers

Resolving patch:
diff --git a/mne/bem.py b/mne/bem.py
--- a/mne/bem.py
+++ b/mne/bem.py
@@ -32,7 +32,7 @@
 from .transforms import _ensure_trans, apply_trans, Transform
 from .utils import (verbose, logger, run_subprocess, get_subjects_dir, warn,
                     _pl, _validate_type, _TempDir, _check_freesurfer_home,
-                    _check_fname)
+                    _check_fname, has_nibabel)
 from .fixes import einsum
 
 
@@ -1154,11 +1154,10 @@ def make_watershed_bem(subject, subjects_dir=None, overwrite=False,
     run_subprocess_env(cmd)
     del tempdir  # clean up directory
     if op.isfile(T1_mgz):
-        new_info = _extract_volume_info(T1_mgz)
-        if new_info is None:
-            warn('nibabel is required to replace the volume info. Volume info'
-                 'not updated in the written surface.')
-            new_info = dict()
+        new_info = _extract_volume_info(T1_mgz) if has_nibabel() else dict()
+        if not new_info:
+            warn('nibabel is not available or the volumn info is invalid.'
+                 'Volume info not updated in the written surface.')
         surfs = ['brain', 'inner_skull', 'outer_skull', 'outer_skin']
         for s in surfs:
             surf_ws_out = op.join(ws_dir, '%s_%s_surface' % (subject, s))
@@ -1209,25 +1208,21 @@ def make_watershed_bem(subject, subjects_dir=None, overwrite=False,
     logger.info('Created %s\n\nComplete.' % (fname_head,))
 
 
-def _extract_volume_info(mgz, raise_error=True):
+def _extract_volume_info(mgz):
     """Extract volume info from a mgz file."""
-    try:
-        import nibabel as nib
-    except ImportError:
-        return  # warning raised elsewhere
-    header = nib.load(mgz).header
-    vol_info = dict()
+    import nibabel
+    header = nibabel.load(mgz).header
     version = header['version']
+    vol_info = dict()
     if version == 1:
         version = '%s  # volume info valid' % version
-    else:
-        raise ValueError('Volume info invalid.')
-    vol_info['valid'] = version
-    vol_info['filename'] = mgz
-    vol_info['volume'] = header['dims'][:3]
-    vol_info['voxelsize'] = header['delta']
-    vol_info['xras'], vol_info['yras'], vol_info['zras'] = header['Mdc'].T
-    vol_info['cras'] = header['Pxyz_c']
+        vol_info['valid'] = version
+        vol_info['filename'] = mgz
+        vol_info['volume'] = header['dims'][:3]
+        vol_info['voxelsize'] = header['delta']
+        vol_info['xras'], vol_info['yras'], vol_info['zras'] = header['Mdc']
+        vol_info['cras'] = header['Pxyz_c']
+
     return vol_info
 
 


Test patch:
diff --git a/mne/commands/tests/test_commands.py b/mne/commands/tests/test_commands.py
--- a/mne/commands/tests/test_commands.py
+++ b/mne/commands/tests/test_commands.py
@@ -4,6 +4,7 @@
 import shutil
 import glob
 
+import numpy as np
 import pytest
 from numpy.testing import assert_equal, assert_allclose
 
@@ -23,7 +24,8 @@
 from mne.io import read_raw_fif, read_info
 from mne.utils import (run_tests_if_main, requires_mne,
                        requires_mayavi, requires_vtk, requires_freesurfer,
-                       traits_test, ArgvSetter, modified_env, _stamp_to_dt)
+                       requires_nibabel, traits_test, ArgvSetter, modified_env,
+                       _stamp_to_dt)
 
 base_dir = op.join(op.dirname(__file__), '..', '..', 'io', 'tests', 'data')
 raw_fname = op.join(base_dir, 'test_raw.fif')
@@ -209,11 +211,15 @@ def test_surf2bem():
 @pytest.mark.timeout(600)  # took ~400 sec on a local test
 @pytest.mark.slowtest
 @pytest.mark.ultraslowtest
+@requires_nibabel()
 @requires_freesurfer
 @testing.requires_testing_data
 def test_watershed_bem(tmpdir):
     """Test mne watershed bem."""
     check_usage(mne_watershed_bem)
+    # from T1.mgz
+    Mdc = np.array([[-1, 0, 0], [0, 0, -1], [0, 1, 0]])
+    Pxyz_c = np.array([-5.273613, 9.039085, -27.287964])
     # Copy necessary files to tempdir
     tempdir = str(tmpdir)
     mridata_path = op.join(subjects_dir, 'sample', 'mri')
@@ -229,15 +235,24 @@ def test_watershed_bem(tmpdir):
         with ArgvSetter(args):
             mne_watershed_bem.run()
     os.chmod(new_fname, old_mode)
-    out_fnames = list()
-    for kind in ('outer_skin', 'outer_skull', 'inner_skull'):
-        out_fnames.append(op.join(subject_path_new, 'bem', 'inner_skull.surf'))
-    assert not any(op.isfile(out_fname) for out_fname in out_fnames)
+    for s in ('outer_skin', 'outer_skull', 'inner_skull'):
+        assert not op.isfile(op.join(subject_path_new, 'bem', '%s.surf' % s))
     with ArgvSetter(args):
         mne_watershed_bem.run()
-    for out_fname in out_fnames:
-        _, tris = read_surface(out_fname)
-        assert len(tris) == 20480
+
+    kwargs = dict(rtol=1e-5, atol=1e-5)
+    for s in ('outer_skin', 'outer_skull', 'inner_skull'):
+        rr, tris, vol_info = read_surface(op.join(subject_path_new, 'bem',
+                                                  '%s.surf' % s),
+                                          read_metadata=True)
+        assert_equal(len(tris), 20480)
+        assert_equal(tris.min(), 0)
+        assert_equal(rr.shape[0], tris.max() + 1)
+        # compare the volume info to the mgz header
+        assert_allclose(vol_info['xras'], Mdc[0], **kwargs)
+        assert_allclose(vol_info['yras'], Mdc[1], **kwargs)
+        assert_allclose(vol_info['zras'], Mdc[2], **kwargs)
+        assert_allclose(vol_info['cras'], Pxyz_c, **kwargs)
 
 
 @pytest.mark.timeout(300)  # took 200 sec locally