icture yourself as a fashion designer needing images of fabrics with a particular mixture of colors, a museum cataloger looking P for artifacts of a particular shape and textured pattern, or a movie producer needing a video clip of a red car-like object moving from right to left with the camera zooming. How do you find these images? Even though today's technology enables us to acquire, manipulate, transmit, and store vast on-line image and video collections, the search methodologies used to find pictorial information are still limited due to difficult research problems (see "Semantic versus nonsemantic" sidebar). Typically, these methodologies depend on file IDS, keywords, or text associated with the images. And, although powerful, they don't allow queries based directly on the visual properties of the images, are dependent on the particular vocabulary used, and don't provide queries for images similar to a given image. Research on ways to extend and improve query methods for image databases is widespread, and results have been presented in workshops, conferences,'.* and surveys. We have developed the QBIC (Query by Image Content) system to explore content-based retrieval methods. QBIC allows queries on large image and video databases based on example images, user-constructed sketches and drawings, selected color and texture patterns, At first glance, content-based querying appears deceptively simple because we humans seem to be so good at it. If a program can be written to extract semantically relevant text phrases from images, the problem may be solved by using currently available text-search technology. Unfortunately, in an unconstrained environment, the task of writing this program is beyond the reach of current technology in image understanding. At an artificial intelligence conference several years ago, a challenge was issued to the audience to write a program that would identify all the dogs pictured in a children's book, a task most 3-year-olds can easily accomplish. Nobody in the audience accepted the challenge, and this remains an open problem. Perceptual organization-the process of grouping image features into meaningful objects and attaching semantic descriptions t o scenes through model matching-is an unsolved problem in image understanding. Humans are much better than computers at extracting semantic descriptions from pictures. Computers, however, are better than humans at measuring properties and retaining these in long-term memory. One of the guiding principles used by QBIC is t o let computers do what they do best-quantifiable measurementand let humans do what they do best-attaching semantic meaning. QBIC can find "fish-shaped objects," since shape is a measurable property that can be extracted. However, since fish occur in many shapes, the only fish that will be found will have a shape close t o the drawn shape. This is not the same as the much harder semantical query of finding all the pictures of fish in a pictorial database. 0018-9162/95/$4 00 1995 IEEE
In the commercial domain, IBM QBIC REF is one of the earliest systems.
Query by image and video content: the QBIC system
IEEE Computer
IEEE Computer
Computer Science