Patent Document ID: 9336494
Application ID: 13969193
Patent Flag: 1

Claim One:
1. A method performed by one or more computers, the method comprising: receiving an ordered sequence of feature vectors; for each feature vector of a plurality of feature vectors in the ordered sequence: using a predictive model having a plurality of parameters to generate a predicted output for the feature vector, wherein the predictive model has been trained on a plurality of old feature vectors using a model training process that generates respective first parameter values for each of the plurality of parameters of the predictive model, identifying recent feature vectors in the ordered sequence, wherein each recent feature vector is within a window of predetermined size preceding the feature vector in the ordered sequence, and computing a measure of the quality of the output of the predictive model on the recent feature vectors; determining, for a first feature vector, that the quality of the output of the predictive model on first recent feature vectors within a first window of the predetermined size preceding the first feature vector in the ordered sequence has become unacceptable as of the first feature vector, and in response: selecting retraining data for retraining the predictive model from a collection of feature vectors consisting of the first recent feature vectors and the plurality of old feature vectors, wherein the ratio of first recent feature vectors to old feature vectors in the retraining data is greater than the corresponding ratio in the collection by an amount based on how unacceptable the quality of the output has become as of the first feature vector, whereby a more unacceptable quality of the output results in the retraining data having a greater ratio of first recent feature vectors to old feature vectors; and retraining the predictive model on the retraining data.