Patent ID: 8768863

Claim:
A method comprising: determining a plurality of sets of users of a social networking system, each set characterized by values of attributes describing the users; for each set of users: training a model configured to rank news feed stories for presentation to users of the set, the training utilizing training data sets obtained from the set of users, wherein features used for the model comprise interactions of users of the set with news feed stories presented to the users; ranking stories identified for presentation to a user using the model and presenting the ranked stories to the user; receiving a first rate of interactions with news feed stories from users of the set, the first rate measured over a first time interval; receiving a second rate of interactions with news feed stories from users of the set, the second rate measured over a second time interval occurring after the first time interval; determining whether to retrain the model based on comparison of the first rate of interactions and the second rate of interactions; and retraining the model based on the determination.