Patent Document ID: 20130089304
Application ID: 13269753
Patent Flag: 0

Claim One:
1. A method for determining a semantic concept classification for a digital video clip including a temporal sequence of video frames and a corresponding audio soundtrack, comprising: a) receiving an audio-visual dictionary including a plurality of audio-visual grouplets, the audio-visual grouplets including visual background codewords representing visual background content, visual foreground codewords representing visual foreground content, audio background codewords representing audio background content and audio foreground codewords representing audio foreground content, wherein the visual background codewords, the visual foreground codewords, the audio background codewords and audio foreground codewords in a particular audio-visual grouplet were determined to be correlated with each other during a training process based on an analysis of a plurality of training videos; b) determining reference video codeword similarity scores for each reference video clip in a set of reference video clips by: i) analyzing the temporal sequence of video frames for a particular reference video clip to determine a set of reference video visual features; ii) analyzing the audio soundtrack for the particular reference video clip to determine a set of reference video audio features; and iii) determining the reference video codeword similarity scores for the particular reference video clip by comparing the set of reference video visual features to the visual background codewords and the visual foreground codewords and by comparing the set of reference video audio features to the audio background codewords and audio foreground codewords; c) using a data processor to determine codeword similarity scores for the digital video clip by: i) analyzing the temporal sequence of video frames in the digital video clip to determine a set of visual features; ii) analyzing the audio soundtrack in the digital video clip to determine a set of audio features; iii) determining the codeword similarity scores for the digital video clip by comparing the set of visual features to the visual background codewords and the visual foreground codewords and by comparing the set of audio features to the audio background codewords and audio foreground codewords; d) determining a reference video similarity score for each reference video clip representing a similarity between the digital video clip and the reference video clip responsive to the audio-visual grouplets, the codeword similarity scores and the reference video codeword similarity scores; e) determining one or more semantic concept classifications using trained semantic classifiers responsive to the determined reference video similarity scores; and f) storing indications of the one or more semantic concept classifications in a processor-accessible memory.