ENH Add cross_val_predict function

This function returns the cross-validated prediction for each element in
the input data (i.e., the prediction that is made when that object is in
the test set).

This is a loop that I keep writing over and over again, so I feel it should
be part of scikit-learn. Additionally, this uses joblib to take advantage of
multiple CPUs
I think Luis' concern here is that the user also needs to reorder `y` to
match, which means either the indices need to be returned or the user needs
to run `check_cv` in addition. So the API is much more complicated once you
consider non-partitions.

On 29 September 2014 01:52, Alexandre Gramfort <notifications@github.com>
wrote:

>  In sklearn/cross_validation.py:
>
> > +    estimator : estimator object implementing 'fit' and 'predict'
> > +        The object to use to fit the data.
> > +
> > +    X : array-like
> > +        The data to fit. Can be, for example a list, or an array at least 2d.
> > +
> > +    y : array-like, optional, default: None
> > +        The target variable to try to predict in the case of
> > +        supervised learning.
> > +
> > +    cv : cross-validation generator or int, optional, default: None
> > +        A cross-validation generator to use. If int, determines
> > +        the number of folds in StratifiedKFold if y is binary
> > +        or multiclass and estimator is a classifier, or the number
> > +        of folds in KFold otherwise. If None, it is equivalent to cv=3.
> > +        Note that this function does not make sense when elements are included
>
> Secondly that assumes that elements are included in exactly one test set
> (i.e. they constitute a partition of the dataset), not at most one.
> I am also concerned by this. How returning a list of arrays ie. one y_pred
> per fold. Then if you use k-fold use np.concatenate to do the plot you
> suggest.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/scikit-learn/scikit-learn/pull/3703/files#r18130402>.
>