Patent ID: 8761512

Claim:
A method performed by data processing apparatus, the method comprising: receiving a group of images, each image having associated metadata; generating one or more local descriptor vectors for each image of the group of images; building an initial index for the group of images using the local descriptor vectors, wherein the initial index is configured to return images of the group of images that are visually similar to a query image; generating a set of matching images for each image using the initial index, wherein a matching image is determined based on a match score that is correlated to a visual similarity between images using the local descriptor vectors; using the group of images and their respective matches to determine a final image selection, including: generating an undirected graph using the group of images wherein each vertex is an image and each edge is a match between images and wherein each edge is weighted according to the respective match scores, using the undirected graph to determine pairwise image affinity based on the edge weights along a path between images in the undirected graph, and selecting a final image selection from the group of images using the pairwise image affinity; calculating a set of n-grams and corresponding n-gram scores for each image using image affinity and image metadata; and generating a new index using the images of the final image selection and including corresponding n-grams for each image.