Patent ID: 8112421

Claim:
A computer-implemented method for refining a ranking function model the method comprising: storing an unlabeled training set comprising a plurality of unlabeled queries and a plurality of unlabeled documents associated with the plurality of unlabeled queries; storing a labeled validation set comprising a plurality of labeled queries and a plurality of labeled documents associated with the labeled queries, the labeled documents being labeled to reflect relevance to the associated labeled queries; iteratively performing learning steps including: determining similarity values between the unlabeled documents associated with the unlabeled queries and the labeled documents associated with the labeled queries; determining accuracy values of the model for ranking individual associated labeled documents of associated labeled neighbor queries in the labeled validation set; and determining weights of the unlabeled queries based on the similarity values and the accuracy values; selecting a first query, from the unlabeled training set of unlabeled queries to present to a user for labeling, wherein the first query is selected based on the similarity values of the unlabeled documents associated with the first query to the labeled documents associated with the labeled queries, and wherein the first query is selected based on the determined weights of the unlabeled queries; presenting the selected first query to the user for labeling of individual documents associated with the selected first query to provide newly labeled documents, wherein the set of documents associated with the first query are presented for the labeling; accumulating the newly labeled documents into a labeled training set of labeled documents; learning a refined model based on the labeled training set; and determining whether the refined model is adequate; and storing the refined model.