nipy/dipy — pull request #235 by arokem (2013-08-24 05:47:03+00:00)
Title: Cross-validation

Another bit of WIP. The idea here is to use the nascent prediction API on the tensor model (and hopefully also on other models soon enough...) to perform k-fold cross validation.

The scheme is that in each iteration, a certain number of random diffusion-weighted directions is dropped. A model is fit to the rest of the directions and then a prediction is made for the left out directions. This is performed so that every direction is dropped in one iteration, so we get a complete prediction of out-of-sample measurements, which in the end comprise all the measurements in the original sample.

Questions at this point: this function relies on a rather uniform API: it requires that the Fit object derived from the model have a `predict` method that takes a `GradientTable` as input and returns a signal for each of the directions/b-values in the that input. The function currently also assumes that the model itself has `bval` and `bvec` attributes. This is true for the `TensorModel` class, with which I have been testing this (and that is currently the only model for which prediction is implemented).

Maybe we should implement a base-class from which all the models inherit that enforces these assumptions? Are there any models that are not initialized with a `GradientTable` in this way?

Also - I created a separate module for this under `stats`, but does this belong in some other pre-existing module?

Comments and ideas are most welcome at this point. I am still trying to hammer this out in the best possible way.