mne-tools/mne-python — pull request #481 by mluessi (2013-02-20 21:08:17+00:00)
Title: WIP: Smarter SourceEstimate with delayed computation

We currently have the problem that e.g. `spectral_connectivity` is slow for connectivity computation in source space, as the time-frequency transforms are applied in source space instead of applying them in sensor space and then multiplying the result with the inverse kernel. This PR alleviates this problem by making `SourceEstimate` smarter such that the data can be represented by a kernel and sensor space data. It also adds a method for applying linear transforms to the data, which will automatically apply the transform in sensor space if possible.

To save memory, `SourceEstimate.data` is now an attribute which only computes the source space data when it is accessed the first time. Naturally, this leads to the problem that a user can have an stc which has the computed source space data, the kernel and the sensor space data. The user could then modify the data, e.g., `stc.data[0] = 1` after which the result of `stc.transformed_data(..)` would be wrong (since the kernel and sensor data are used for the computation). To avoid this problem, a special `_NotifyArray` class is used for `stc._data`, which will notify `stc` when its contents are modified and the kernel and sensor data are removed. This is somewhat of a hack, I'm all ears if someone has a better idea for how to do this.

I did some test, one example is here https://gist.github.com/mluessi/4999358 when the spectra are estimated using FFT, the speedup is about 30% but when mulitaper estimation is used, the speedup is 400%. The results are similar for `spectral_connectivity`. Also, for the sample data we use `oct-6` resolution, which is quite low, for higher resolution source spaces the difference is even larger.

There are different ways to achieve the same result. One would be to apply the transforms to epochs and then apply the inverse method. However, there are several problems: 1) the users can no longer write a straightforward linear script, i.e., they have to know that applying the transform in sensor space is more efficient 2) all inverse methods have to be able to accept transformed data as input 3) the time-frequency (or just frequency) epochs and source estimates can no longer be stored as `.fiff` or `.stc` files (unless we introduce more fiff tags and extend the stc format somehow, but this is a mess).

Right now, the behavior can be controlled using the `delayed` keyword argument of `apply_inverse_epochs` but I feel if this is accepted it would make sense to always use delayed computation (if possible) for linear inverse methods that accept Epochs as input. Like that it will be transparent to the user.

This is WIP and experimental.. I thought I would get some feedback before I invest more time in it.