Patent ID: 8196164

Claim:
A computer implemented method, comprising: receiving television broadcast data for a plurality of television channels, the television broadcast data having programming portions during which television programming is aired, and advertising portions during which advertisements are aired, and including caption data that includes text that corresponds to audio content of the programming portions and advertising portions and frame data defining video frames; storing the caption data in a data store and indexing the caption data by time and channel, the time corresponding to a time the caption data was received in the broadcast data and the channel specifying a channel on which the stored caption data was provided in the broadcast data; storing frame data of video frames in the data store and indexing each video frame by time and channel, the time corresponding to a time the video frame was received in the broadcast data and the channel specifying a channel on which the video frame was provided in the broadcast data; identifying, from the caption data in the data store, caption data that has been received at least a threshold number of times and received over at least a threshold number of channels as repeated caption data; identifying, from the index times and channels of the repeated caption data, the advertisement portions of the broadcast data; selecting video frames indexed at times and channels that that correspond to the times and channels of the advertisement portions; providing the selected video frames to a video processing system that identifies advertising objects in the video frame data and receiving from the video processing system the advertising objects; and for each advertising object, associating the advertising object with the repeated caption data having times and channel indices that match the time and channel index of the video frame from which the advertising object was identified.