Patent Document ID: 7725463
Application ID: 10879002
Patent Status: 1

Claim One:
1. A system comprising one or more computer storage media embodying one or more executable software components for generating a normalized relevance output for use in search result analysis, the one or more software components comprising: a normalization engine to receive a set of independently scaled relevance metrics and a user's empirical result selections, wherein at least a portion of the set of independently scaled relevance metrics comprises independently scaled relevance metrics originating from a plurality of disparate sources and being incapable of direct comparison, wherein the independently scaled relevance metrics include human input feeds that are unanalyzed by a search engine, ordinal search results generated by one or more disparate relevance sources, and server-captured user behaviors, to convert each of the independently scaled relevance metrics to one or more normalized metrics, wherein converting comprises: (a) accessing the set of ordinal search results from the one or more disparate relevance sources; (b) determining a degree of deviation upon comparing ratings of individual results within the set of ordinal search results against the user's empirical result selections derived from query logs and event data that capture search activities of the user; and (c) assigning each of the individual results of the set of ordinal search results a hierarchical rating level based on the degree of deviation, wherein the greater the degree of deviation, the greater the hierarchical rating level, and to rank the one or more normalized metrics based on the hierarchical rating level, such that the normalized relevance output is generated.