Patent ID: 8442984
Filing Date: 2013-05-14
Classification: G06Q

Abstract:
1. A method, comprising: computing, for each of a plurality of websites, an initial quality score, the initial quality score for each website being computed based, at least in part, on an attribute of the website; selecting websites to be rated by raters, the websites being selected based on the initial quality scores of the websites and a specified quality distribution, the websites being selected to include: a first quantity of websites having initial quality scores that are between a low quality threshold score and a high quality threshold score, the low quality threshold score being a maximum initial quality score for a low quality website, the high quality threshold score being a minimum initial quality score for a high quality website, the high quality threshold score being higher than the low quality threshold score; a second quantity of websites having initial quality scores that are below the low quality, threshold score, the second quantity being lower than the first quantity; and a third quantity of websites having initial quality scores that are above the high quality threshold score, the third quantity being lower than the first quantity; selecting a group of the websites for each of the raters, each group of websites including: at least one website having an initial quality score that is below the low quality threshold; at least one website having an initial quality score that is above the high quality threshold; and at least one website having an initial quality score that is between the low quality threshold and the high quality threshold; providing each of the raters with one group of websites; receiving, from the raters, website quality ratings specifying rater selected measures of quality for the websites, the website quality rating for each website being based on an aggregate quality of a plurality of web pages in the website; for each of the websites in the group, associating a website quality rating with website signals that represent attributes of the website; creating a machine learned model based on the website quality ratings and the website signals, wherein the model characterizes relationships between the website quality ratings and the website signals; and applying the model to the website signals of unrated websites to generate calculated quality ratings for the unrated websites.