Patent Document ID: 9449078
Application ID: 12243937
Patent Flag: 1

Claim One:
1. One or more processor-accessible storage media comprising processor-executable instructions for evaluating a ranking quality of a ranked list, wherein the processor-executable instructions, when executed, direct a system to perform acts comprising: accessing log data that reflects user interactions by a user with the ranked list, the ranked list having multiple entries; determining, from the log data, that the multiple entries of the ranked list include viewed entries that have been viewed by the user and unviewed entries that have not been viewed by the user; ascertaining, from the log data, that the viewed entries include skipped viewed entries that the user has skipped and selected viewed entries that the user has selected, wherein each of the skipped viewed entries and each of the selected viewed entries are ranked relatively higher in the ranked list than the unviewed entries and the skipped viewed entries are interspersed with the selected viewed entries in the ranked list; determining a first number of the skipped viewed entries and a second number of the viewed entries including both the skipped viewed entries and the selected viewed entries; determining a ranking quality metric for the ranked list using the first number of the skipped viewed entries and the second number of the viewed entries; and using the ranking quality metric to tune a search application, wherein the skipped viewed entries include partially skipped viewed entries and fully skipped viewed entries, and the ascertaining comprises using dwell times to identify the partially skipped viewed entries.