Patent ID: 7603343

Claim:
A method having computer instructions that are executable on a computing device, the method comprising: collecting a rating for each of multiple web pages from a group of recipients of web pages; presenting a rating game to a recipient where the recipient receives a random web page from the multiple web pages to rate; rewarding or demeriting the recipient based on a rating submitted by the recipient, wherein the random web page has a combined rating of the group of recipients when the recipient receives the random web page; collecting a rating estimate from the recipient on how the group of recipients rate the random web page of the multiple web pages, the rating estimate comprising an estimate of the combined rating of the group of recipients, wherein the estimate of the combined rating comprises an estimate of a subjective impression the web page makes on the other recipients; assigning a weighting factor to the recipient, wherein a recipient with a history of accurately estimating the ratings of how the group of recipients rate each of the multiple web pages has a greater effect on the ratings of the web pages used in prioritizing web search results; ranking at least some of the results of a web search that returns at least some of the multiple web pages according to the ratings, wherein the ranking results of the web search is based on the combined, weighted ratings of the multiple web pages from the group of recipients and the recipient; updating the combined, weighted ratings of the multiple web pages by including a new rating divided by a new total number of recipients who have rated the web page or including filters to discard guesses from recipients that tend to select a same rating, without performing a function on the web page; and presenting a next web page which includes another random web page or a consequential web page to the recipient for rating, the consequential web page corresponds to the recipient being rewarded or demerited based on the rating submitted by the recipient, and the another random web page is presented to prevent a compromising feedback to the recipient that affects the rating estimate following a rewarded or a demerited rating by the recipient.