Patent Document ID: 9223897
Application ID: 13117006

Base Claim:
1. A method performed by one or more server devices, the method comprising: obtaining, by the one or more server devices, a rank position map, the rank position map specifying, for a plurality of rank positions, an expected utility rate for a document presented at a particular one of the plurality of rank positions as a search result, the expected utility rate being derived from an aggregate quantity of selections of the document presented as a search result at the particular one of the plurality of rank positions; determining, by the one or more server devices, an expected utility rate, for a particular document, based on the obtained rank position map and based on a quantity of times that the particular document was presented as a search result at particular ones of the plurality of rank positions; determining, by the one or more server devices, an actual utility rate for the particular document based on an actual quantity of selections of the particular document and based on a total quantity of times that the particular document was presented as a search result; calculating, by the one or more server devices, a correction factor, for the particular document, based on the determined expected utility rate and the determined actual utility rate; determining, by the one or more server devices, a confidence level for the correction factor, the confidence level representing a measure of confidence associated with the correction factor; adjusting, by the one or more server devices, the correction factor based on the determined confidence level; selecting, by the one or more server devices and based on determining that the confidence level is less than an aggregation threshold, a set of documents, the particular document being included in the set of documents; calculating, by the one or more server devices, an aggregated correction factor for the set of documents; using, by the one or more server devices, the aggregated correction factor as the correction factor for the particular document; adjusting, by the one or more server devices, a score of the particular document based on the correction factor; and providing, for presentation, a ranked list of search results, one of the search results corresponding to the particular document, and a ranking of the search result corresponding to the particular document being based on the adjusted score.

---

Claim 14:
14. The method of claim 1 , where determining the expected utility rate includes: for each of the plurality of rank positions, multiplying a quantity of times that the particular document was presented at a rank position by the expected utility rate at the rank position as indicated in the obtained rank position map to generate a plurality of products; generating a sum of the plurality of products; and dividing the generated sum by a quantity of the plurality of rank positions to obtain the expected utility rate.