Patent ID: 8867891

Claim:
A method for determining a semantic concept classification for a digital video clip including a temporal sequence of video frames and a corresponding audio soundtrack, the method comprising: analyzing, by a processing device, the temporal sequence of video frames to determine a set of visual features; analyzing, by the processing device, the audio soundtrack to determine a set of audio features; determining, by the processing device, similarity scores between the digital video clip and each of a plurality of audio-visual grouplets from an audio-visual dictionary, wherein the plurality of audio-visual grouplets includes distinct visual background codewords representing visual background content, distinct visual foreground codewords representing visual foreground content, distinct audio background codewords representing audio background content, and distinct audio foreground codewords representing audio foreground content, wherein the distinct visual background codewords and the distinct visual foreground codewords are separate and distinct from each other, wherein the distinct audio background codewords and the distinct audio foreground codewords are separate and distinct from each other, and wherein the distinct visual background codewords and the distinct visual foreground codewords are separate and distinct from the distinct audio background codewords and the distinct audio foreground codewords, and wherein the determining similarity scores comprises: comparing the set of visual features to distinct visual background codewords and distinct visual foreground codewords associated with a particular audio-visual grouplet; and comparing the set of audio features to distinct audio background codewords and distinct audio foreground codewords associated with the particular audio-visual grouplet; determining, by the processing device, one or more semantic concept classifications using trained semantic classifiers responsive to the determined similarity scores; and storing, by the processing device, indications of the one or more semantic concept classifications in a processor-accessible memory.