Patent ID: 8903166

Claim:
A computer-implemented method comprising: accessing training data including: at least one query; a set of images associated with the at least one query; and for an individual image of the set of images, one or more user-assigned labels indicating a degree of relevance of the individual image to the at least one query, the degree of relevance selected from a plurality of degrees of relevance; ranking the set of images associated with the at least one query based at least in part on the respective one or more user-assigned labels thereby providing a first relevance ranking of the set of images; extracting visual features from the set of images; extracting textual features associated with the set of images; ranking the set of images using a ranking model that jointly uses the visual features and the textual features thereby providing a second relevance ranking of the set of images; comparing the first relevance ranking of the set of images to the second relevance ranking of the set of images; and updating the ranking model based in part on ranking differences produced by the comparing.