AbstractÐWe address the problem of finding the subject of a photographic image intended to illustrate some physical object or objects (ªdepictiveº) and taken by usual optical means without magnification (ªnaturalº). This could help in developing digital image libraries since important image properties like subject size and color of a photograph are not usually mentioned in accompanying captions and can help rank the photograph retrievals for a user. We explore an approach that identifies the ªvisual focusº of the image and the ªdepicted conceptsº in a caption and connects them. The visual focus is determined by using eight domainindependent characteristics of regions in the segmented image, and the caption depiction is identified by a set a rules applied to the parsed and interpreted caption. The visual-focus determination also does combinatorial optimization on sets of regions to find the set that best satisfies focus criteria. Experiments on 100 randomly selected image-caption pairs show significant improvement in precision of retrieval over simpler methods, and, particularly, emphasizes the value of segmentation of the image.
With respect to the automatic segmentation and labeling of images, Rowe REF presents an approach to find the visual focus of an image by applying image processing in terms of segmentation and low-level features.
Finding and labeling the subject of a captioned depictive natural photograph
IEEE Trans. Knowl. Data Eng.
IEEE Trans. Knowl. Data Eng.
Computer Science