Patent Document ID: 7836058
Application ID: 12056302

Base Claim:
1. A method for improving relevance of web searches for a query, comprising: providing a pre-existing relevance ranking of a plurality of query/uniform resource locator (URL) pairs, respective labels being associated with the plurality of query/URL pairs; computing a click relevance ranking of the plurality of query/URL pairs based upon user log data comprising user click data; and identifying and correcting mislabeled query/URL pairs in the pre-existing relevance ranking based upon the click relevance ranking, the user log data comprising: respective rank associated with a query/URL pair within the plurality of query/URL pairs at one or more times; respective total numbers of impressions (I) for the rank associated with the query/URL pair; and respective total numbers of clicks received (C) for the rank associated with the query/URL pair, wherein the click rate (CR) is the sum of the total number of clicks for all ranks associated with the query/URL pair divided by the sum of the total number of impressions for all ranks associated with the query/URL pair, and computing the click relevance ranking comprises: aggregating the user log data by query and URL; reducing click bias; creating a click relevance ordering of the plurality of query/URL pairs; creating a directed acyclic graph of a relevance relationship between the plurality of query/URL pairs; and mapping the directed acyclic graph into a linear ordering, at least some of one or more of the providing, computing and identifying performed at least in part with a microprocessor.

---

Claim 10:
10. The method of claim 1 , comprising: computing a distribution of labels in the pre-existing relevance ranking; and relabeling a label associated with the query/URL pairs in the click relevance ranking according to the distribution of labels in the pre-existing relevance ranking, the labels decreasing with respect to click relevance.