making sense of video content
Our aim in this research is to make sense of scenes in video. We expect this will enable us to identify different scenes sharing the same semantic, even if they do not share any multimedia cues. Our approach is based on emergence, and involves classification and reasoning. We use patterns of cues to synthesize semantic classifications. These classifications need to be consistent with the observed cues, and suggest other elements that might be present. The suggestions might be based on sets of related patterns, ontology, or other knowledge bases. With an idea of what we are looking for, we can re-examine the scenes for multimedia elements, which support our hypothesis, or at least are not inconsistent. We expect that sufficiently detailed semantic descriptions can be generated, by cycling through these steps.
scene analysis semantic modeling video semantics semantic gap synthesis