This project seeks to apply visual and aural processing by computer to automated annotation of video. For this application we do not rely directly on any single low level processing, but rather use multiple modes of low level data processing to form a hypothesis about clips within the video. This allows further, better directed processing to be applied to refine the annotation. Structure automatically detected in a news video. Structure Detection Of prime importance in this work is the detection of structure.
Simple content based methods are of limited use as a direct query mechanism in video.
Instead they can be used to detect a basis for inference of more useful, higher level structure. As an example, similarity of spatial colour information can be used to detect shots which are likely to be anchor people or reporters. Face detection can be applied to the selected shots to further refine the detection, and voice separation may refine the selections even further. Given classification of some of the shots within a video, the shot syntax for a production type, be it interview, sport, or magazine style news builettin, may be used to hypothesise about other shots. Given a suitable hypothesis, higher level processing such as face detection and text recognition may be applied to only those shots where is is most useful.