Patent ID: 8504558

Claim:
A computer-implemented method comprising: collecting data about user activities; wherein, the data include behavior information that indicates how each user, of a set of users, interacted with content items on a web portal during a particular period of time; generating, based on at least some of the collected data, a plurality of user profile models, wherein each user profile model, of the plurality of user profile models, at least indicates a probability that users that match the user profile model will interact with a content item, on the web portal, having certain characteristics; evaluating a plurality of content-selection algorithms, wherein each particular content-selection algorithm, of the plurality of content-selection algorithms, implements a different approach for selecting content items to display on the web portal, wherein each particular content-selection algorithm is not a user profile model; wherein evaluating the plurality of content-selection algorithms comprises, for each particular content-selection algorithm of the plurality of content-selection algorithms: using the particular content-selection algorithm to identify one or more sets of content items for display on a simulated web portal; based on the plurality of user profile models, and at least some of the data about the user activities, simulating visits to the simulated web portal, in which simulated users select one or more content items from the identified one or more sets of content items; and generating simulation data indicating content-item selections during the simulated visits; wherein evaluating the plurality of content-selection algorithms further comprises, based on the simulation data from the simulated visits, computing performance metrics that measure a predicted effectiveness, of each particular content-selection algorithm of the plurality of content-selection algorithms, in choosing content items with which users will interact on the web portal; wherein the method is performed by one or more computing devices.