Patent ID: 7783512
Filing Date: 2010-08-24
Classification: G06Q

Abstract:
1. A method of evaluating a recommender computing system, which recommender is used for recommending a set of items of interest to online users of an online content service provider, the method being implemented using a computing system executing computer software instructions embodied in a computer usable medium and comprising the steps of: (a) identifying a first set of reference items to be used in evaluating the recommender computing system, said first of reference items constituting a first subset of the set of items; (b) receiving a request with the computing system from a first online user for a list of recommendations for one or more of the set of items; (c) automatically generating said list containing a plurality of separate recommendations with the recommender system for the first online user in response to said request, including a second subset of items taken from the set of items, said second subset of items including one or more of said first set of reference items as well as additional items not part of said first set of reference items; (d) automatically correlating said first set of reference items with said ordered list of second subset of items using the computing system to identify an awareness level exhibited by the recommender system for said first set of reference items; (e) automatically generating a rating for the recommender system for said first set of reference items with the computing system based at least in part on an evaluation of said awareness level for each of said first set of reference items; (f) setting up a plurality of proxy accounts and respective demographic profiles prior to step (b) adapted to interact with the recommender system, and wherein said proxy accounts include at least some data for said first online user that is an estimate and/or a prediction of a rating of a real person.