The work presents the tirst effort to automatically annotate the semantic meanings of temporal video pattems obtained through unsupervised discovery proccsses. This problem is interesting in doindins where neither perceptual patterns nor semantic concepts have simple structures. The patterns in video are modeled with hierarchical hidden Markov models (HHMMj, with efficient algorithms to learn the parameters. the model complexity, and the relevant features; the meanings are contained in words of the speech transcript of the video. The pattern-word association is obtained via co-occurrence analysis and statistical machine translation models. Promising results are obtained through extensive experiments on 20+ hours 01' TRECVID news videos: video patterns that associate with distinct topics such as el-nitlo and politics are identitied the HHMM temporal structure model compares favorably to a non-temporal clustering algorithm.
REF present an approach for discovering meaningful structures in video through unsupervised learning of temporal clusters and associating these with metadata using co-occurrence analysis and models similar to machine translation.
Discovering meaningful multimedia patterns with audio-visual concepts and associated text
2004 International Conference on Image Processing, 2004. ICIP '04.
2004 International Conference on Image Processing, 2004. ICIP '04.
Computer Science