Repository: mne-tools/mne-python

Issue:
Question about our simulation module
Hi all,

I just started playing with our simulation module. Taking a look at our evoked data example I wondered how to localize the generated evokeds, e.g. to compare dSPM against ground truth, etc. This seems straight forward, but the resulting source estimates look strange --- e.g. are signed where they should not be signed. This is probably because the covariance from that example is not appropriate for the epochs generated. Any pointers?

Cheers,
Denis

/cc @joewalter @agramfort



Discussion:
Hi Denis,

do you apply an IIR filter to the data in order to simulate temporal correlation? Have you tried with iir_filter=None?

Bye Daniel

Regarding signed/unsiged, how does the simulated Evoked response look? Did you by any chance use a free-orientation SourceEstimate to generate the simulated evoked response? You have to use `force_fixed=True` or `pick_normal=True`, otherwise you will get unsigned source amplitudes and projecting them back to sensor space doesn't make sense (the simulation module actually gives a warning if you do this..).

@joewalter thanks, it seems the filtering was not the problem. @mluessi the evoked response looks fine. But I ended up loading a second a free-oriented fwd to compute an inverse solution, already assuming this is not the right way ...

But given this fwd,

`fwd = mne.read_forward_solution(fwd_fname, force_fixed=True, surf_ori=True)`

when trying to compute a new inverse operator from the generated evoked, None of the options seems to valid .. (depth, loose, fixed, ..)
Is this a feature?

@dengemann I'm not sure I understand the first problem, namely the source estimates being signed where they should not be signed. Why shouldn't they be signed? Unless you explicitly take the magnitude, or you use an unconstrained, free-orientation inverse, I wouldn't expect the estimates to all be positive...

Regarding the second problem, you'll proabably need to re-load the forward solution with a different combination of options (e.g., `force_fixed=True`, `surf_ori=False`). It's annoying to have to do this, but it might be necessary depending on how you make the inverse (combinations of depth weighting versus not, fixed, free, etc.). There are only certain types of combinations that work. A while back I was tinkering with the inverse code and @agramfort and I settled on this (slightly annoying) behavior in order to avoid users making bad choices in constructing inverses.

> @dengemann I'm not sure I understand the first problem, namely the source estimates being signed where they should not be signed. Why shouldn't they be signed? 

Because I don't want to pick normals but want to generate regular dSPM values from artificial evokeds. Makes sense?

> Regarding the second problem, you'll probably need to re-load the forward solution with a different combination of options (e.g., force_fixed=True, surf_ori=False). 

This is what I then did, unsuccessfully ... that's why you read me here.

> A while back I was tinkering with the inverse code and @agramfort and I settled on this (slightly annoying) behavior in order to avoid users making bad choices in constructing inverses.

I remember the conversation. You know what I was thinking yesterday? It would be cool to have tutorial on MNE methods (in the strict sense) where a table of all 'cells' resulting from parameter combinations are discussed. Would certainly be cool for newcomers.

I'll take a stab at making such a table today if I have time.

I understand that you want to generate regular dSPM values, but based on the mathematical operation used to calculate them, they will be signed (at least initially). They are typically _displayed_ as unsigned, but dSPM is just a multiplication in source space (of the MNE values) by a diagonal matrix consisting of all non-negative entries along the diagonal... thus the MNE values being signed (which they are) will lead to initially signed dSPM scores.

> I'll take a stab at making such a table today if I have time.

cool :-)

> I understand that you want to generate regular dSPM values, but based on the mathematical operation used to calculate them, they will be signed (at least initially).

Sure, I'm really talking about the final result. Typically, a dSPM non-normal-picked stc object does not include signed values (== negative F-values ?), or did I miss something fundamental here? 

Unless you use the normal orientation (either using `pick_normal` or `force_fixed`) the values should be unsigned. Can you post a code snipped on how you get a "a dSPM non-normal-picked stc object"?

@mluessi I think he's using a fixed-orientation inverse, thus why the values come out signed.

With loose orientation, the values along the three axes get combined for each location, which is done via a vector norm (yielding strictly nonnegative values) unless you use `pick_normal`. Now that you're working with a fixed orientation solution (which I think is the case?), in the mne-python code it is not necessary to do any combining to yield one component for each source space location, so the values by virtue of the mathematical operations carried out (see above) will still be signed.

There is a good reason for this, namely that it preserves the linearity of the operator in case you want to take advantage of it while still having meaningful noise normalization. It has the disadvantage of being less easily interpreted as an F-value -- but you can just take the magnitude manually to get the more familiar non-negative results.

> "a dSPM non-normal-picked stc object"

This is what our minimum norm examples produce, e.g. examples/inverse/plot_compute_mne_inverse.py

@Eric89GXL thanks for the hint --- I just realized that one part of my code did not update while trying differnt orientation options when creating my second fwd. It now looks like I expected.

Oh.. I was confused.. the question was about the negative dSPM values. As @Eric89GXL pointed out, this is not really a dSPM value, more like a "noise normalized current". Applying `abs` makes it into a dSPM. The signed "dSPM" values are often useful for e.g. connectivity estimation (where you need signed values) as they have less depth bias than MNE estimates and may also have less variability across subjects.

Oh.. also, you shouldn't use a dSPM source estimate to generate a simulated evoked response, use MNE instead (so the unit of the source estimate is Am and they can be projected to uV, fT, fT/cm in sensor space using the forward operator). 

> also, you shouldn't use a dSPM source estimate to generate a simulated evoked response,

Which I did not do (and would not do -- would be a bit nonsensical to have F-Values as true source ...)

And as a safeguard, I'm pretty sure the simulation code has a built-in warning if it sees currents that are too high (as a dSPM estimate would be).

> And as a safeguard, I'm pretty sure the simulation code has a built-in warning if it sees currents that are too high (as a dSPM estimate would be).

btw. I'm pretty amazed by the simulation code. Is there actually a reason why generate_stc is hidden inside the simulation.source name space?

I think the percentage of people interested in simulating data (as opposed to analyzing real data) is smaller than the biased sampling you get from talking to those of us on github would lend you to believe :)

> I think the percentage of people interested in simulating data (as opposed to analyzing real data) is smaller than the biased sampling you get from talking to those of us on github would lend you to believe :)

That's a good answer. It might have indicated that the function is less 'ready' / recommend then generate_sparse_stc. So I'd suggest to expose it in the simulation name space. Opinions?

If that's where the other simulation functions are, makes sense to me.

>  The signed "dSPM" values are often useful for e.g. connectivity estimation (where you need signed values) as they have less depth bias than MNE estimates and may also have less variability across subjects.

This is what I assumed. But good that you confirm it.

@Eric89GXL so we almost discussed each of our cells here, defining them from how the results will look like -- 1) value range: signed VS unsigned 2) unit: physiological VS dimensionless-score, 3) shape of the estimates -- related to the cross talk function : complex VS peaky, 2) depth-bias: superficial VS deep. 4) point-spread bias: high vs lower ...  

Sorry I missed it -- are you saying these are things you want to test via simulation?

> Sorry I missed it

No the 'cells' for the tutorial on different MNE options.

Ahh -- I think that's certainly a good start. By the way, the "matrix" I talked about making was separate from this tutial. I'll probably just make a list of which options combinations work (and which don't) in terms of fwd/inv construction with rationales, and add it to the inverse docstring.

> Ahh -- I think that's certainly a good start. By the way, the "matrix" I talked about making was separate from this tutial. I'll probably just make a list of which options combinations work (and which don't) in terms of fwd/inv construction with rationales, and add it to the inverse docstring.

This, together with a more extensive tutorial would make up a nice show ;-)

> That's a good answer. It might have indicated that the function is less
> 'ready' / recommend then generate_sparse_stc. So I'd suggest to expose it in
> the simulation name space. Opinions?

fine with me.


Resolving patch:
diff --git a/mne/simulation/__init__.py b/mne/simulation/__init__.py
--- a/mne/simulation/__init__.py
+++ b/mne/simulation/__init__.py
@@ -1,6 +1,6 @@
 """Data simulation code
 """
 
-from .evoked import generate_evoked
+from .evoked import generate_evoked, generate_noise_evoked, add_noise_evoked
 
-from .source import select_source_in_label, generate_sparse_stc
+from .source import select_source_in_label, generate_sparse_stc, generate_stc


Test patch: