Patent ID: 8589371

Claim:
A method for selecting a relevance function to determine a relevance of a query-content item pair, the method comprising: generating, using a processing device, a training set comprising one or more content item-query pairs, the content items-query pairs having a training relevance score assigned by a user; for each query-content item pair, determining, using the processing device, a relevance function; modifying, using the processing device, the relevance function using a loss function according to a relevance score adjustment function that accounts for a query differentiation, wherein the relevance score adjustment function receives the determined relevance function as input, and wherein the query differentiation is a difference between the training relevance score and an output of the relevance score adjustment function; and selecting, using the processing device, a relevance function trained with the query differentiation that produces a smallest loss, when the modified relevance function and the training relevance value converge.