Patent ID: 8010535

Claim:
A computer-implemented method comprising: selecting a first search object from a set of search objects; performing a pairwise comparison between the first search object and each other search object from the set of search objects to generate a rank distribution for the first search object, wherein the pairwise comparison is based on a score distribution for each of the set of search objects, wherein the rank distribution comprises a set of probabilities, each probability corresponding to the probability that the first search object has a particular rank; repeating the selection and pairwise comparison for each search object in the set of search objects to create a set of rank distributions, each corresponding to one of the set of search objects; generating a matrix comprising the set of rank distributions; converting the matrix into a doubly-Stochastic matrix; substituting the set of rank distributions into a Normalized Discounted Cumulative Gain to generate a smoothed information retrieval metric, wherein the smoothed information retrieval metric exhibits a substantially continuous change in value in response to changes in input parameters, the substituting comprising: rewriting the Normalized Discounted Cumulative Gain as a sum over search object indices; replacing a deterministic discount term with an expected discount based on the rank distributions, wherein the expected discount is obtained by mapping the rank distributions through the deterministic discount function; and training a machine learning model using the smoothed information retrieval metric as an objective function.