For some images, descriptions written by multiple peo ple are consistent with each other. But for other images, de scriptions across people vary considerably. In other words, some images are specific -they elicit consistent descriptions from dif f erent people -while other images are ambiguous. Applications involving images and text can benefit from an understanding of which images are specific and which ones are ambiguous. For instance, consider text-based image re trieval. If a query description is moderately similar to the caption (or reference description) of an ambiguous image, that query may be considered a decent match to the image. But if the image is very specific, a moderate similarity be tween the query and the reference description may not be sufficient to retrieve the image. In this paper, we introduce the notion of image speci ficity. We present two mechanisms to measure specificity given multiple descriptions of an image: an automated mea sure and a measure that relies on human judgement. We analyze image specificity with respect to image content and properties to better understand what makes an image spe cific. We then train models to automatically predict the specificity of an image from image features alone without requiring textual descriptions of the image. Finally, we show that modeling image specificity leads to improvements in a text-based image retrieval application.
In text-based image retrieval, image specificity REF based on image content and properties has been used to discriminate easily describable images.
Image specificity
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Computer Science