rmcgibbo/osprey — pull request #50 by rmcgibbo (2014-11-13 22:35:51+00:00)
Title: Mixtape CV fold averaging

This PR attempts to fix a bug with mixtape models related to averaging the train and test scores over the CV folds with trajectories of unequal length. The average test score should, I think, be weighted by the amount of data in each of the test sets. The default sklearn cross validation does this, but because of the difference between the sklearn and mixtape APIs, I don't think it does the right thing for mixtape. In the mixtape models, `X` is a list of trajectories, each of which may be of unequal length. So the appropriate weighting factor, IMO, is not `len(X)`, but `sum(len(xx) for xx in x)`.

Doing this right requires checking which API the estimator supports, which is the impetus for https://github.com/rmcgibbo/mixtape/issues/281.