If storage space is limited, it may not be feasible to construct a database which contains digital representations of all videos in their entirety. We have developed a content-based approach to selecting such images which has been very successful in the test cases we have run to date. We have also been developing a repertoire of feature analysis algorithms which may be applied to indexing of these images, once they have been selected. Topical indices resemble the sorts of indices we normal associate with books. Given the multimedia nature of video source material, we believe it is important that the indexing and retrieval of those data should be multimedia-based.
Text data are necessary in all types of indices, especially in topical indices.
Indexing based on image content is currently built on visual features such as color, texture, and shape of objects. Our work on developing an audio index structure is still very preliminary, but we plan to proceed by analogy to the structure of our image index. A database schema is being implemented in our prototype system to accommodate these three types of indices. A frame-based knowledge base is used to represent the semantic objects, their attributes, and relations to their respective video sources. These objects are assigned as instances in a hierarchical organization of categories.