Patent ID: 7805438

Claim:
A method in a computing device for determining loss between a target probability and a model probability for documents when training a ranking function based on training data, the training data including documents and the target probability of relative relevance of pairs of documents to queries, the model probability being generated by a ranking function that ranks documents, the method comprising: training the ranking function by repeating the following until a calculated loss is below a threshold loss: selecting a new ranking function by modifying a previous ranking function to reduce the calculated loss; applying the new ranking function to the pairs of documents of the training data to provide new rankings of the documents based on the queries; calculating by the computing device a model probability from the new rankings of the documents; and calculating by the computing device a loss between the calculated model probability and the target probability to indicate a difference between the new ranking of a pair of documents represented by the calculated model probability and a ranking of the pair of documents represented by the target probability, the loss varying between 0 and 1 and the loss being 0 when the calculated model probability is the same as the target probability wherein the calculated loss is a fidelity loss and wherein the fidelity-based loss is represented by the following equation: F ij = 1 - ( P ij * · P ij + ( 1 - P ij * ) · ( 1 - P ij ) ) where F ij represents the fidelity loss, P ij * represents the target probability for documents i and j, and P ij represents the calculated model probability for documents i and j.