nilearn/nilearn — pull request #196 by VirgileFritsch (2014-05-22 12:10:59+00:00)
Title: NF: Randomized Parcellation Based Inference

Tadaaaaam... Implementation of Randomized Parcellation Based Inference [1].

Main accomplished tasks:
* Refactoring: move some functions that were common to permuted OLS and RPBI into a utils.py file.
orthonormalize_design/matrix, f_score computation, ...
Tests have been changed accordingly.
* Re-introduce GrowableSparseArrays (and corresponding tests).
They have already been reviewed in a former PR about permuted OLS before they were finally removed.
Now, we needed to re-introduce them because we need to address the data efficiently, in different
manners and for different purposes.
* Introduce some functions to generate/deal with randomized parcellations.
They could be moved to a more generic module that serves this purpose (creating multiple
random parcellations, transforming a dataset with all of the built parcellations)
* Examples: the three examples on massively univariate analysis have been duplicated and RPBI is now
applied. This is a lot of examples, but they will be structured at some point (see issue #195).
Issue: examples can take up to 30 minutes on a laptop, using one single CPU (see discussion below).

The code for RPBI is optimized but a run of the algorithm remains costly. This is because RPBI requires the computation of many parcellations (97% of the total execution time). Only 3% of the computation time is spent on the permutations, when 10,000 are performed (which is already kind of a lot, but recommended).
Fortunately, both the computation of the parcellation and the statistical inference part can be parallelized.
Therefore, on a 10-core station, the computation time boils down to less than three minutes, which is acceptable. Still, the examples policy requires that n_jobs=1 by default.

[1] Benoit Da Mota et al. Randomized parcellation based inference, NeuroImage, Volume 89, 1 April 2014.