Repository: mne-tools/mne-python

Issue:
ENH: Add GIfTI output for STC
Wanted to get thoughts from others who may find this feature useful, and insights into possible straightforward ways to implement.  

#### Describe the new feature or enhancement

In an effort to compare/overlay results in fMRI and EEG/MEG, several of us are seeking an easy way to export MNE source results to GIFTI (and potentially NIFTI in the future).  GIFTI would allow other programs (e.g. AFNI, Freesurfer) to then work with source-space data in useful ways (e.g. constrain solutions based on fMRI, make pretty pictures).  


#### Describe your proposed implementation

Add options to mne.SourceSpaces.save( ) or better - a new method .export( ) to allow the export of GIFTI images.  For added convenience, with option for hemisphere, time-window of interest, and overwrite.  Wrapper for nibabel's GIFTI implementation with nuances for writing the relevant data in. 

https://nipy.org/nibabel/reference/nibabel.gifti.html

#### Additional comments
This would be decently useful for folks trying to do EEG-fMRI simultaneous, MEG & fMRI joint analysis, or anyone interested in taking the source estimates from MNE into other softwares (FieldTrip and BrainStorm both have GIFTI imports).  



Discussion:
Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

I think it's worth trying. Now we need to see how to handle the fact stc
subsample the high resolution
surfaces of freesurfer. Not sure how to handle this with gifti unless we
upsample creating very
large files.

This would be great.  Currently using an amalgamation of MNE-C, Freesurfer, and AFNI to get surface estimates into .nii files.
Hello again. @jbteves and I finally have some bandwidth to work more on this.  

@agramfort & @larsoner, can you direct us to where the subsampling of the surfaces happens?  

Also @jhouck - can you point us to your code that does work-arounds? 
@pmolfese I think we're planning on working this into `nibabel` first, see discussions and hopefully join them (next meeting is a week from tomorrow IIRC!) -- it would be nice not to duplicate efforts here:

https://nipy.discourse.group/t/collecting-use-cases/58
https://nipy.discourse.group/t/surface-api-follow-up-meetings/79/9
https://github.com/nipy/nibabel/pull/1056

But to answer your question, the subsampling is here (and associated helper functions), it's not the easiest code to read:

https://github.com/mne-tools/mne-python/blob/022f1a8f69c52062f956f015fba3c3b6f5ee7a1d/mne/surface.py#L942-L993


I'm interesting in joining these discussions, how might I join? I see a standing zoom room in your second link, should I simply join at the appropriate time?
@jbteves it would be good to drop a comment in https://nipy.discourse.group/t/surface-api-follow-up-meetings/79/9 to mention your interest just so that other folks know you're coming
@pmolfese the code is not the best. It relies on the fact that a .stc file is basically a set of concatenated Freesurfer paint (.w) files. With the old MNE-C (2.7.4, ca. 2016), I was using mne_make_movie to extract the surface at the time of interest and export it to a .w file, then freesurfer's mri_surf2vol to convert the .w file to a nifti file.  For gifti I think you could use mri_surf2surf.  To avoid the old code you can set  ftype=w when you save the SourceEstimate in mne-python.

I will add my name to the linked discussion point too. For MNE-nirs I want to move to source reconstruction next and was planning to integrate tightly with nilearn.
@pmolfese why did you prefer to export as GIFTI? I am just curious as Im new to this topic. Some of the fNIRS software exports as NIFTI, and wondered if there was a specific reason for choosing between the two?
@rob-luke - GIFTI mostly comfort in handling source data from other software.  There are programs for converting between NIFTI and GIFTI for sure. I suspect ultimately both would be useful.  
@jbteves you pinged me with an email for a status update here -- so far there has been some progress at the `nibabel` end. I think our plan is basically to wait for `nibabel` to make `SurfaceImage` available, then allow something like `stc.as_gifti` or something to export a `SurfaceImage` class from our `SourceEstimate` class in MNE. (Or maybe someday `SourceEstimate` can be a wrapper for `SurfaceImage` -- but that would be more work.) In any case, I think any Gifti export would hopefully come from going STC->SurfaceImage (implemented at the MNE end) then SurfaceImage->Gifti (implemented at the nibabel end).

So I think at this point the bottleneck is at the nibabel end, maybe there is some way you could help their effort?

https://nipy.discourse.group/t/preliminary-notes-on-surface-representations/57
https://nipy.discourse.group/t/surface-api-follow-up-meetings/79
https://github.com/nipy/nibabel/pull/1056

Looks like the next meeting is scheduled for 9AM EST Jan 28th if you want to join and discuss!
Alright, thanks for letting me know!
Hi @larsoner, we're pretty eager to get this working and happy to help the `nibabel` efforts. From the `nibabel` meeting this morning and our own poking around in the codebase, we have some questions: 

for building the geometry we have found the necessary vertices on the decimated surfaces, but we cannot seem to find the triangles necessary to complete the geometry and thus get ready to build the matching GIFTIs for export. 

Any help or pointers you could provide on finding the decimated triangles would be much appreciated! 

-Pete (and @jbteves). 


> for building the geometry we have found the necessary vertices on the decimated surfaces, but we cannot seem to find the triangles necessary to complete the geometry and thus get ready to build the matching GIFTIs for export.

Where did you find the decimated vertices? From `stc.vertices`?

Typically that `stc.vertices` will correspond to some `src[0]['vertno']` from a SourceSpaces instance. If you look at that instance, it should have what you want, e.g.:
```
>>> import numpy as np
>>> import mne
>>> src = mne.read_source_spaces(mne.datasets.sample.data_path() + '/subjects/sample/bem/sample-oct-6-src.fif')
>>> src[0]['tris'].shape  # high-density
(310810, 3)
>>> src[0]['use_tris'].shape  # subsampled
(8192, 3)
>>> src[0]['vertno'].shape  # vertex numbers
(4098,)
>>> np.array_equal(np.unique(src[0]['use_tris']), np.sort(src[0]['vertno']))
True
```
Thanks for the response! Sorry, I may be misunderstanding something. We use something like this:
```
>>> se_rh = mne.read_source_estimate("sub-stc-rh.stc")
>>> right_vert = se_rh.right_verno
```
To index into the source vertices. However, there are the triangles that represent the faces to create a complete mesh/surface in the high-resolution sample. How might we get those triangles for the low-resolution surface?
> We use something like this:

Your `stc.rh_vertices` should correspond to `src[1]['vertno']` in my code above, as long as no vertices were removed during forward computation. Each STC corresponds to some SourceSpaces instance (the one that is preserved in `fwd['src']` when it's computed, and `inv['src']` when it's computed), and that SourceSpaces instance will (usually) contain *both* the full-resolution triangulation (`src[0]['tris']` for left and `src[1]['tris']` for right) and the decimated, low-resolution triangulation (`src[0]['use_tris']` and `src[1]['use_tris']`).

In other words, what I was trying to express with the commands above is "tris is the dense triangulation, use_tris is the decimated one, and the set of vertex numbers in the decimated one match exactly the vertno attribute", and the implied part that is probably the missing link for you is that "the vertno key of the two source space elements should match the stc.vertices", which in code would be something like (for a STC that has no "holes" / missing vertices):
```
assert np.array_equal(src[0]['vertno'], stc.vertices[0])
assert np.array_equal(src[1]['vertno'], stc.vertices[[1])
```
@larsoner ah, thanks so much! That makes things much more clear for me. I'll hopefully get some toy code together for this soon.
@larsoner after the surface API meeting prior to last week it seemed like the ball was moving back towards "MNE should implement the solution using the new API." While we're waiting for the API in nibabel to be fully implemented, could we merge in a more formalized version of the below gist?

https://gist.github.com/jbteves/384bfdd3777190f59620c3d988a314fd
I imagine some version of `src2subsurf` (everything before [line 50](https://gist.github.com/jbteves/384bfdd3777190f59620c3d988a314fd#file-stc2gii-py-L50)) with a suitable API and naming scheme could live in `mne/source_space.py`. Are you interested in doing this @jbteves ?

For an API/naming, perhaps `mne.source_space.get_decimated_surfaces(src)` that returned `len(surf_source_spaces)` (usually == 2) dicts, each with `'rr'` and `'tris'` keys would be general enough? The `list of dict` rather than `list of list` allows us to later add other entries (e.g., `nn`) for each surface source space if we realize they're helpful.
Sure, I can take a crack at this @larsoner. Not sure on what precise timeline though.

Resolving patch:
diff --git a/mne/source_space.py b/mne/source_space.py
--- a/mne/source_space.py
+++ b/mne/source_space.py
@@ -2821,3 +2821,40 @@ def compute_distance_to_sensors(src, info, picks=None, trans=None,
     depths = cdist(src_pos, sensor_pos)
 
     return depths
+
+
+def get_decimated_surfaces(src):
+    """Get the decimated surfaces from a source space.
+
+    Parameters
+    ----------
+    src : instance of SourceSpaces | path-like
+        The source space with decimated surfaces.
+
+    Returns
+    -------
+    surfaces : list of dict
+        The decimated surfaces present in the source space. Each dict
+        which contains 'rr' and 'tris' keys for vertices positions and
+        triangle indices.
+
+    Notes
+    -----
+    .. versionadded:: 1.0
+    """
+    src = _ensure_src(src)
+    surfaces = []
+    for s in src:
+        if s['type'] != 'surf':
+            continue
+        rr = s['rr']
+        use_tris = s['use_tris']
+        vertno = s['vertno']
+        ss = {}
+        ss['rr'] = rr[vertno]
+        reindex = np.full(len(rr), -1, int)
+        reindex[vertno] = np.arange(len(vertno))
+        ss['tris'] = reindex[use_tris]
+        assert (ss['tris'] >= 0).all()
+        surfaces.append(ss)
+    return surfaces


Test patch:
diff --git a/mne/tests/test_source_space.py b/mne/tests/test_source_space.py
--- a/mne/tests/test_source_space.py
+++ b/mne/tests/test_source_space.py
@@ -27,7 +27,7 @@
 from mne.surface import _accumulate_normals, _triangle_neighbors
 from mne.source_estimate import _get_src_type
 from mne.source_space import (get_volume_labels_from_src,
-                              _compare_source_spaces,
+                              _compare_source_spaces, get_decimated_surfaces,
                               compute_distance_to_sensors)
 from mne.io.pick import _picks_to_idx
 from mne.io.constants import FIFF
@@ -886,6 +886,22 @@ def test_morphed_source_space_return():
                   src, subjects_dir=subjects_dir)
 
 
+@testing.requires_testing_data
+@pytest.mark.parametrize('src, n, nv', [
+    (fname_fs, 2, 10242), (fname_src, 2, 258), (fname_vol, 0, 0),
+])
+def test_get_decimated_surfaces(src, n, nv):
+    """Test get_decimated_surfaces."""
+    surfaces = get_decimated_surfaces(src)
+    assert len(surfaces) == n
+    if n == 0:
+        return
+    for s in surfaces:
+        assert set(s) == {'rr', 'tris'}
+        assert len(s['rr']) == nv
+        assert_array_equal(np.unique(s['tris']), np.arange(nv))
+
+
 # The following code was used to generate small-src.fif.gz.
 # Unfortunately the C code bombs when trying to add source space distances,
 # possibly due to incomplete "faking" of a smaller surface on our part here.