Patent Publication Number: US-10785547-B2

Title: System and method for synchronizing metadata with audiovisual content

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     The present application is a continuation of U.S. patent application Ser. No. 15/900,665, filed Feb. 20, 2018 and entitled “System and Method for Synchronizing Metadata with Audiovisual Content” and claims priority to U.S. Provisional Patent Application Ser. No. 62/568,975, filed Oct. 6, 2017 and entitled “System and Method for Synchronizing Metadata with Audiovisual Content,” and U.S. Provisional Patent Application Ser. No. 62/605,526, filed Aug. 17, 2017 and entitled “System and Method for Synchronizing Metadata with Audiovisual Content,” both of which are incorporated by reference herein. 
    
    
     TECHNICAL FIELD 
     Embodiments of the present invention related generally to the synchronization of video and audio streams with metadata using out-band signaling. 
     BACKGROUND 
     Historically, in analog television broadcasting, the locations where commercials could be inserted was marked using in-band signaling such as dual-tone multi-frequency (DTMF) tones. DTMF tones can be used to indicate the start and stop times of commercials. These tones may be inserted by the broadcast or cable network and may be sent to local networks as part of a network feed. Subsequently, local television stations may insert or replace part of the network feed based on the DTMF tones with local content or commercials. 
     In digital television broadcasting, instead of using DTMF tones, in-band signaling (e.g., the Society of Cable Telecommunications Engineers (SCTE) 35 standard) is often used to signal the insertion of a commercial or other content. SCTE 35 metadata messages have long been used in the cable and broadcast industry to signal the start of local advertising breaks. The SCTE 35 Standard has evolved over the years through multiple revisions to also enable signaling and metadata for dynamic ad insertion, content replacement, blackout, start over, look back, DVR and live-to-VOD applications. 
     Metadata may include, for example, broadcaster-specific advertisements, location-specific advertisements, graphical overlays, tickers, timing of events within the video (e.g., a goal scored in a sports match), asset identification (e.g., Entertainment Identifier Registry (EIDR), Ad-ID) and/or the like. However, the SCTE 35 standard is ambiguous in many places with many sections completely optional and as a result there are multiple ways for the standard to be interpreted. This has resulted in a situation where many content providers that currently use SCTE 35 have different metadata configurations (still technically compliant with the SCTE 35 standard), even though the same temporal point or type of content is being signaled. For example, one content provider may signal the start of an advertising break using a particular type identifier, whereas another may choose to use a different type identifier where both type identifiers are perfectly valid according to SCTE 35. This causes confusion for distributors, especially those that are receiving content from multiple different content providers (e.g., multichannel video programming distributors (“MVPDs”)). 
     Synchronization of this metadata with the video timeline is quite often lost as the audiovisual content is transmitted, processed and/or stored and as such the usefulness of the metadata degrades significantly. This causes events to be triggered at the wrong point in the content resulting in a poor on-screen experience for viewers. For example, if the SCTE 35 message that is signaling the start of a local advertising break at an MVPD arrives too early, the advertising insertion equipment will cut off the end of the previous program segment early. Similarly, if the SCTE 35 message arrives too late, there will be “peek through” of the programming that should have been replaced by a local commercial. These problems are hindering the ability for content providers and MVPDs alike to monetize content, especially as it relates to distribution of linear/live television via the internet. 
     Further, conversion between different viewing formats and/or transmission over different distribution channels may result in SCTE signaling and/or DTMF tones associated with the insertion of commercials and other visual graphics or audio components being lost. Metadata may be lost during the processing of audiovisual content, converting the audiovisual content from analog to digital (or vice versa), video encoding, video decoding, or changing the resolution of the video. In another example, SCTE signaling and/or metadata may be lost changing the frame rating to conform to a particular standard (e.g., high definition, ultra-high definition) or a particular regional requirement (e.g., PAL, NTSC, or the like). 
     SUMMARY 
     An example method comprises receiving, at a first digital device, video data, scanning, at the first digital device, video content of the video data for visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, retrieving, at the first digital device, a different time for each visual transition to timestamp each visual transition and create a first set of temporal video fingerprints, each of the first set of temporal video fingerprints being relative to a different visual transition and the retrieved time, identifying, at the first digital device, items of metadata to be associated with the video data, identifying, at the first digital device, a location within the video data using the temporal video fingerprints for the identified items of metadata, generating, at the first digital device, a metadata index identifying each item of metadata and a location for each item of metadata relative to the video data using at least one of the temporal video fingerprints, and transmitting, at the first digital device, the video data, the first set of temporal video fingerprints, and the metadata index to a different digital device. 
     Each of the temporal fingerprints may further identify a different frame associated with a visual transition. 
     In various embodiments, the method further comprises receiving, at a second digital device, the video data and the first set of temporal video fingerprints, scanning, at the second digital device, the video content of the video data for the visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, retrieving, at the second digital device, a different time for each visual transition to timestamp each visual transition and create a second set of temporal video fingerprints, each of the second set of temporal video fingerprints being relative to a different visual transition and the retrieved time, comparing, at the second digital device, the first set of temporal video fingerprints and the second set of temporal video fingerprints to determine a time offset and determine a timing difference, and inserting, at the second digital device, at least one of the items of metadata into the video data using the metadata index and the time offset to create modified video data. 
     The method may further comprise transmitting, at the second digital device, the modified video data to a user device for playback of the modified video data. In some embodiments, the method may further comprise the second digital device playing the modified video data. 
     In some embodiments, scanning, at the first digital device, the video content of the video data for visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, comprises calculating histograms H=(h0, h1 . . . hn) for every chroma/luma/color channel where n is the number of possible pixel brightness levels for the particular channel and hi is the count of pixels with value i in each frame of video, calculating a sum of absolute differences (SAD) between histogram vectors of every pair of adjacent frames to generate resulting values, and comparing the resulting values to a threshold to identify at least one visual transition within the video content. The threshold may be a moving average of the SAD of histograms from previous frames with a sample window of double the video framerate. 
     In various embodiments, scanning, at the first digital device, the video content of the video data for visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, comprises decimating the histogram vectors H(A) and H(B) to the size of 16(H′(A)&amp;H′(B)) as follows: where 
                 H   ⁡     (   A   )       =     (       a   0     ,       a   1     ⁢           ⁢   …   ⁢           ⁢     a   n         )       ;                   H   ′     ⁡     (   A   )       =         (       h   0     ,       h   1     ⁢           ⁢   …   ⁢           ⁢     h   15         )     ⁢           ⁢     h   i       =       ∑     j   =       i   ⁡     (     n   +   1     )       16               (     i   +   1     )     ⁢     (     n   +   1     )       16     -   1       ⁢     a   j               
the result of this subtraction is used as a temporal video fingerprint.
 
     Scanning, at the second digital device, the video content of the video data for visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, may comprise calculating histograms H=(h0, h1 . . . hn) for every chroma/luma/color channel where n is the number of possible pixel brightness levels for the particular channel and hi is the count of pixels with value i in each frame of video, calculating a sum of absolute differences (SAD) between histogram vectors of every pair of adjacent frames to generate resulting values, and comparing the resulting values to a threshold to identify at least one visual transition within the video content. 
     In some embodiments, inserting, at the second digital device, at least one of the items of metadata into the video data using the metadata index and the time offset to create modified video data comprises adjusting audio of the video data. 
     An example system may include a first digital device including one or more first processors and memory, the memory containing instructions executable by at least one of the one or more first processors to: receive video data, scan video content of the video data for visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, retrieve a different time for each visual transition to timestamp each visual transition and create a first set of temporal video fingerprints, each of the first set of temporal video fingerprints being relative to a different visual transition and the retrieved time, identify items of metadata to be associated with the video data, identify a location within the video data using the temporal video fingerprints for the identified items of metadata, generate a metadata index identifying each item of metadata and a location for each item of the metadata relative to the video data using at least one of the temporal video fingerprints, and transmit the video data, the first set of temporal video fingerprints, and the metadata index to a different digital device. 
     In some embodiments, the system further comprises a second digital device including one or more second processors and memory, the memory containing instructions executable by at least one of the one or more second processors to: receive the video data and the first set of temporal video fingerprints, scan the video content of the video data for the visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, retrieve a different time for each visual transition to timestamp each visual transition and create a second set of temporal video fingerprints, each of the second set of temporal video fingerprints being relative to a different visual transition and the retrieved time, compare the first set of temporal video fingerprints and the second set of temporal video fingerprints to determine a time offset and determine a timing difference, and insert at least one of the items of metadata into the video data using the metadata index and the time offset to create modified video data. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  depicts a block diagram of an example audiovisual broadcast system capable of providing synchronizing of audio and/or video stream as well as metadata using out-band signaling according to some embodiments. 
         FIG. 2  depicts a block diagram of an example first broadcast system according to some embodiments. 
         FIG. 3  depicts a block diagram of an example second broadcast system according to some embodiments. 
         FIG. 4  depicts a flowchart of a video broadcasting method of a first broadcast server according to some embodiments. 
         FIG. 5  depicts a flowchart of an audio broadcasting method of the first broadcast server according to some embodiments. 
         FIG. 6  depicts a flowchart of a video broadcasting method of a second broadcast server local stock image upload method according to some embodiments. 
         FIG. 7  depicts a flowchart of an audio broadcasting method of the second broadcast server according to some embodiments. 
         FIG. 8  depicts a flowchart of re-synchronization of video method of the second broadcast server according to some embodiments. 
         FIG. 9  depicts a flowchart of correcting lip sync method of the second broadcast server according to some embodiments. 
         FIG. 10  depicts a block diagram of an example a digital device according to some embodiments. 
     
    
    
     DETAILED DESCRIPTION 
     Some embodiments described herein provide for re-synchronizing metadata at any point downstream of an original transmission. There are many other benefits including but not limited to: lip-sync error correction (where the audio and video are out of sync), application of custom graphic overlays at any point prior to presentation, continuous measurement of transmission delay, synchronization of the audiovisual content to a stream of data sent via another path (e.g., the internet), auditing delivery of the audiovisual content to a particular point, synchronization of a disaster recovery broadcast origination with the main broadcast origination, application of synchronized timecode at multiple independent receive locations, and/or custom formatting of metadata to suit the needs of a distributor or viewer or other entity processing or viewing the content. 
     Embodiments of systems and methods are described for synchronizing metadata with audio and/or visual content at any number of points in a content delivery path. Temporal fingerprints may be generated at certain points of the audio and video, such as when an abrupt transition (or when a full cut) between successive frames of video takes place. The temporal fingerprints may be identified and sent out of band (e.g., via the internet), and/or may be stored in a network-based storage, such as the cloud. Temporal fingerprints may refer to temporal video fingerprints, temporal audio fingerprints, or temporal multimedia fingerprints. As discussed herein, temporal video fingerprints include timestamps associated with transitions or changes of content within video data. Video data may include video content. Video data may optionally include audio content (e.g., a YouTube video may be video data). Temporal audio fingerprints include timestamps associated with transitions or change of content within audio data (e.g., an audio file). 
     The temporal fingerprints may be utilized to signal positions within audio and/or video for insertion of metadata. In some embodiments, metadata may be used to signal precise times of events within the audiovisual content, for example, the start of an advertisement or a program. Metadata may also be used to signal the type of content (e.g. an advertisement, a chapter or an advertising break) as well as more specific information such as an event ID or characteristics of a particular scene. 
     In some embodiments, the temporal fingerprints may be generated by a first broadcast server and then sent from the first broadcast server to a second broadcast server, independent of the audio and/or video data itself (e.g., independent of the audio and/or video file). The audio and/or video data may be sent from the first broadcast server to the second broadcast server via a satellite, antenna, or any other means. The second broadcast server may receive the audio and/or video data and generate its own temporal fingerprints from the audio and/or video data. It will be appreciated that the temporal fingerprints may be generated by both the first and second broadcast servers, independently. 
     Although the timestamps for the temporal fingerprints generated by the first and second broadcast servers may be different, the positions of the temporal fingerprints relative to the audio and/or video data may be the same (or substantially the same). For example, if the first broadcast server generated timestamps associated with transitions of a video using a GPS system and a second broadcast server generated its own timestamps associated with the same transitions within the same video received from the first broadcast server eight hours later, then the timestamps of the second broadcast server will be eight hours after the timestamps of the first broadcast server. As such, time between timestamps of the two sets of temporal fingerprints may be consistent. 
     By matching the temporal fingerprints generated by the different broadcast servers and determining time differences between the temporal fingerprints, elapsed time can be determined. The elapsed time may represent time for transmission and processing of the audio and/or video signal by different recipients. The elapsed time may be utilized, in conjunction with the temporal fingerprints, to synchronize audio and/or video data (e.g., for consistent and precise insertion of metadata). 
     In addition to the ability to synchronize metadata, temporal fingerprints may be used to synchronize a broadcast with different end user devices. For example, a user may bring a smart phone displaying a particular football game, within listening range of a television broadcasting the same football game. In this example, the smart phone may generate audio temporal fingerprints from the sound of the television broadcast, and determine the elapsed time (e.g., delay) between the broadcast of the audio on the television to the receipt and processing by the smart phone. Using this information, the smart phone may be able to successfully synchronize the broadcast of the football game to the smart phone. 
     Some embodiments may also be used to synchronize two (or more) independent broadcast feeds, which may be useful for disaster recovery or purposes of redundancy. In case of natural disaster, for example, it may be necessary to have two independent broadcast feeds become synchronized if one fails to broadcast. To avoid interruption to the broadcast feed, it may be necessary to switch from one broadcast feed to another. By having each of the independent broadcast servers generate temporal fingerprints, the delay between the two broadcast feeds can be determined and the broadcasts can be locked together. 
       FIG. 1  depicts a block diagram of an example audiovisual broadcast system capable of providing synchronization of audio and/or video data as well as metadata using out-band signaling according to some embodiments. The audiovisual broadcast system  100  includes a first broadcast system  102 , a second broadcast system  104 , a satellite system  106 , a network-based storage  108 , and a user system  110 . 
     A broadcast system includes any number of digital devices that broadcast (e.g., transmit) audio and/or video to any number of receivers (e.g., any number of user devices  110  and/or any number of other broadcast systems). In some embodiments, a broadcast system may transmit audio and/or video data to other broadcaster systems that may, subsequently, further broadcast the audio and/or video data. 
     A broadcast system may, in some embodiments, receive and capture audio and/or video content for transmission. For example, a broadcast system may record a football game or other live event for broadcasting. The broadcast system may transmit the programming or any other audio and/or video data to others. Although broadcast systems are depicted in  FIG. 1 , there may be multicast systems, narrowcast systems, and/or any other digital devices that transmit audio and/or video data that are in addition to or in place of one or more broadcast systems. 
     In some embodiments, first broadcast system  102  may capture audio and/or video data for viewing by the users on user system  110 . Subsequent to the capture of the audio and/or video content (e.g., to generate audio and/or video data based on the content), the first broadcast system  102  may determine video transitions from the video component of the audiovisual broadcast and generate temporal video fingerprints by time stamping each video transition. Video transition indicates changes in the video (e.g., including color changes, movement, and/or the like in the video content). 
     The first broadcast system may generate timestamps using any clock (e.g., a GPS system). In some embodiments, the first broadcast system may utilize a clock that is available to one or more other digital devices (e.g., the second broadcast system  104  and/or the user device(s)  110 ). 
     The first broadcast system  102  may determine metadata to be inserted into the video content and generate a metadata index identifying metadata and a position for the metadata within the video content of the video data. For example, the first broadcast system  102  may determine that a particular advertisement should be inserted fifteen frames after a particular temporal fingerprint (e.g., after a particular timestamp) or should be inserted half a second after the particular temporal fingerprint. The first broadcast system  102  may include a metadata identifier identifying the particular advertisement and the insertion point in the temporal index. 
     The first broadcast system  102  may transmit the temporal video fingerprints, metadata, and/or metadata index to network-based storage  108  at any time (e.g., after the temporal video fingerprints are generated), any number of user devices  110 , and/or any number of second broadcast systems  104 . The audiovisual broadcast (e.g., the audio and/or video data) may be transmitted to second broadcast system  104  over a first network at any time (e.g., before or after first broadcast system  102  identifies audio and/or video transitions). 
     The first broadcast system  102  may determine video metadata timestamps relative to temporal video fingerprints. For example, the first broadcast system  102  may determine commercial insertion points in the video content relative to any number of temporal video fingerprints (e.g., relative to timestamps associated with video transitions in the content). 
     Packaged video metadata may be determined using the video metadata fingerprints. In some embodiments, first broadcast system  102  may transmit the packaged video metadata after the metadata is packaged. In some embodiments, first broadcast system  102  may generate a video metadata index using packaged video metadata. 
     The first broadcast system  102  may transmit the audio and/or video data to the second broadcast system  104  over a first network and/or any number of networks. In some embodiments, first broadcast system  102  may transmit the video metadata to second broadcast system  104  over a second network to network-based storage  108 . 
     The second broadcast system  104  may receive the audio and/or video data to the second broadcast system  104 . The second broadcast system  104  may receive the metadata index, temporal fingerprints generated by the first broadcast system  102 , and metadata from the first broadcast system  104 . It will be appreciated that the second broadcast system  104  may receive the audio and/or video data over a first network and may receive the metadata index, temporal fingerprints, and metadata over a different network (or any number of networks). As such, the second broadcast system  104  may receive the metadata index, temporal fingerprints, and metadata over a different network or connection. 
     The second broadcast system may determine temporal fingerprints of the video content of the audio and/or video data using methods similar to first broadcast system  102 . For example, the second broadcast system may determine video transitions from the video component of the audiovisual broadcast and generate temporal video fingerprints by time stamping each video transition. Video transition indicates changes in the video (e.g., including color changes, movement, and/or the like in the video content). 
     The second broadcast system  104  may generate timestamps using any clock (e.g., a GPS system). In some embodiments, the second broadcast system  104  may utilize a clock that is available to one or more other digital devices (e.g., the first broadcast system  102  and/or the user device(s)  110 ). 
     It will be appreciated that video content may not change or may not significantly change regardless of formatting, compression, conversion, and/or any other transmission process. In order for viewers to enjoy the same programming, the video content of the program is the same (or substantially the same) regardless of medium, device, or channel. For example, size, speed, or enhancements to a presentation of a program may change, but the video content itself may not change unless it is a different program (otherwise the viewer is no longer watching the same program). As such, video transitions within the video content are consistent through the broadcast system regardless of which viewer is watching the video. 
     The second broadcast system  104  may match temporal fingerprints generated by the second broadcast system  104  with the temporal fingerprints generated by the first broadcast system  102  to synchronize the audio and/or video data. By matching the temporal fingerprints generated by first broadcast system  102  and second broadcast system  104 , an elapsed time can be determined using the timestamps of the two sets of temporal fingerprints. 
     The second broadcast system  104  may use timing of the timestamps to determine the correct location within the audio and/or video data for metadata (e.g., based on the metadata index). The second broadcast system  104  may insert video associated with the video metadata into the location of the audio and/or video data and display and/or transmit the modified audio and/or video data to the user system(s)  110  and/or other broadcast systems over any number of networks. 
     First broadcast system  102  may transmit the video content, along with the metadata over the first network. The first broadcast system  102  may, in some embodiments, provide the temporal fingerprints generated by the first broadcast system  102  to the second broadcast system  104  in any number of ways (e.g., via the first network, a different network, or a different path on the same network). In various embodiments, the first network may include satellite communications provided by the satellite system  106 . The satellite system may include any number of satellites configured to receive and transmit data (e.g., audio and/or video data, metadata, temporal fingerprints, and/or the like) from a broadcast system to any number of digital devices or systems (e.g., to a second broadcast system  104 ). Second broadcast system  104  may transmit the video content to user system  110  over a network (e.g., internet, satellite system  106 , and/or any other networks). 
     The satellite system  106  includes one or more satellites. In some embodiments, the satellite system  106  may include other forms of electronic communication including wired or wireless network. It will be appreciated that the satellite system  106  may be any medium that can convey video and audio data using electromagnetic waves, whether guided (using wires) or unguided (without wires) or any storage medium whether magnetic, optical or in silicon. 
     The optional network-based storage  108  may store metadata (e.g., advertisements, graphics, video, audio, and/or the like) from first broadcast  102  and/or from any other digital devices. In some embodiments, network-based storage  108  may store temporal video fingerprints from the first broadcast system  102 . In some embodiments, network-based storage  108  may store audio metadata and/or temporal audio fingerprints from first broadcast system  102 . The network-based storage  108  may include any number of logical and/or hardware storage devices. Although only a single network-based storage  108  is depicted in  FIG. 1 , it will be appreciated that there may be any number of network-based storage  108 . 
     In various embodiments, the network-based storage  108  may generate a first video fingerprint index from video fingerprints from first broadcast system  102  and store the first video fingerprint index in network-based storage  108 . In some embodiments, network-based storage  108  may be omitted, and first video fingerprint index may be stored in the first broadcast system  102 . 
     In some embodiments, the user system  110  may be or include one or more mobile devices (e.g., smartphones, cell phones, smartwatches, tablet computers, or the like), desktop computers, laptop computers, televisions and/or the like. In the illustrated embodiment, second broadcast system  104  provides the modified audio and/or video content (e.g., including metadata inserted by the second broadcast system  104 ) to user system  110 . In various embodiments, the user system  110  may receive the audio and/or video content for display to a user. 
     In various embodiments, the user system  110  may determine its own temporal fingerprints using methods similar to the first broadcast system  102  and the second broadcast system  104 . The user system  110  may receive previously generated temporal fingerprints from the first and/or second broadcast systems in any number of ways (e.g., over a network or a network communication path that is different than a network or a network communication path that was utilized to provide the audio and/or video content to the user system  110 ). 
     By matching the temporal fingerprints generated by second broadcast system  104  and/or user system  110  with temporal fingerprints generated by the user system  110 , the user system may synchronize the modified audio and/or video data for further precision with metadata (e.g., for lip synchronization, audio synchronization, metadata synchronization, and/or the like). In various embodiments, the user system  110  may receive all or some of the metadata identified by the first broadcast system  102  and/or all or some of the metadata index. The user system  110  may synchronize or reposition metadata (e.g., audio, advertisements, and/or the like) using the identified metadata and/or the metadata index. 
     In the illustrated embodiment, two broadcast systems are shown, in other embodiments, audiovisual broadcast system  100  may comprise three or more broadcast system which may function similarly to first broadcast system  102  or second broadcast system  104 . 
     Although some embodiments described herein utilize the temporal fingerprints to assist in determining or identifying locations for metadata, it will be appreciated that synchronization of video based on video content has a myriad of uses. 
       FIG. 2  depicts a block diagram of the first broadcast system  102  according to some embodiments. The first broadcast system  102  includes a controller engine  202 , a communication engine  204 , a transition detection engine  206 , a timing engine  208 , a packet engine  210 , a media fingerprint datastore  212 , a metadata store  214  and a metadata engine  216 . An engine may be hardware (e.g., an ASIC), software (e.g., including instructions executable by a processor), or a combination of both. 
     The controller engine  202  may manage one or more of the various engines shown in  FIG. 2 . For example, the controller engine  202  may direct the transition detection engine  206  to detect abrupt transitions between successive frames of video or abrupt transitions in the audio between successive moments in time. The controller engine  202  may direct the timing engine  208  to timestamp video transitions to generate temporal video fingerprints utilizing a Global Positioning System (GPS) clock to timestamp the audio and/or video transitions detected by transition detection engine  206 . In another example, the controller engine  202  may direct the packet engine  210  to package temporal fingerprints with a network identifier. A network identifier may identify the broadcast network associated with the audio and/or video content. The controller engine  202  may be configured to facilitate communication engine  204  to transmit the video fingerprint or video metadata to network-based storage  108 . 
     The communication engine  204  may transmit and/or receive data. For example, the communication engine  204  may transmit data between first broadcast system  102  and satellite system  106 , and between first broadcast system  102  and network-based storage  108 . 
     The transition detection engine  206  may scan audio and/or video data to detect transitions between successive frames of video or transitions in the audio between successive moments in time. In various embodiments, the transition detection engine  206  may scan through a video or audio component to identify video or audio transitions. In one example, the transition detection engine  206  detects one or more transitions within video content (e.g., abrupt changes in color, brightness, and/or the like). In some embodiments, the transition detection engine  206  may detect one or more transitions within audio content (e.g., abrupt changes in volume or sound(s) after a period of silence). A video or audio component of the audiovisual broadcast may be transmitted to second broadcast system  104  over the first network at any time before or after transition detection engine  206  identifies video or audio transitions. 
     Once any number of audio or video transitions have been detected, the transition detection engine  206  may send a request to the timing engine  208  for clock values (e.g., based on a GPS clock) to timestamp the audio or video transition. Timing engine  208  may receive the request from the transition detection engine  206  to create one or more temporal audio fingerprints or one or more temporal video fingerprints. 
     Video transitions may be detected and/or determined in any number of ways. In one example, the transition detection engine  206  may determine video transitions between successive frames of video with the following steps: 1) for each frame of video, the transition detection engine  206  may calculate histograms H=(h 0 , h 1  . . . h n ) for every chroma/luma/color channel where n is the number of possible pixel brightness levels for the particular channel and h i  is the count of pixels with value i in each frame of video. For example, if YUV color space is used with 8 bits of depth there would be three histograms created for each video frame:
 
 H   Y   =H   Y ( h   0   ,h   1   , . . . h   255 )
 
 H   U   =H   U ( h   0   ,h   7   , . . . h   255 )
 
 H   B   =H   V ( h   0   ,h   2   , . . . h   255 )
 
     2) the transition detection engine  206  may calculate a sum of absolute differences (SAD) is calculated between the histogram vectors of every pair of adjacent frames. For two histogram vectors A=(a 0 , a 1  . . . a n ) and B=(b 0 , b 1  . . . b n ) the sum of absolute differences is: 
     
       
         
           
             
               SAD 
               ⁡ 
               
                 ( 
                 
                   A 
                   , 
                   B 
                 
                 ) 
               
             
             = 
             
               
                 ∑ 
                 
                   i 
                   = 
                   0 
                 
                 n 
               
               ⁢ 
               
                  
                 
                   
                     a 
                     i 
                   
                   - 
                   
                     b 
                     i 
                   
                 
                  
               
             
           
         
       
     
     The resulting value quantifies a similarity of two adjacent frames; the more similar the frames are, the closer the value is to zero. The transition detection engine  206  may compare this value to a threshold or moving average of the SAD of histograms from previous frames with a sample window of double the video framerate. In one example, when the value is 20-30 times higher than the moving average, a suitable pair of frames may be found. It will be appreciated that transitions may be determined based on any comparison of the value with a reference (e.g., moving average). For example, if the value is significantly higher (e.g., two more times higher) than the value, a transition may be determined. 
     3) “Abrupt transitions” between frames A and B may be the basis for temporal fingerprints, calculated in this example by the transition detection engine  206  as follows: a) the histogram vectors H(A) and H(B) may be decimated to the size of 16(H′(A)&amp;H′(B)) as follows: where H(A)=(a 0 , a 1  . . . a n ); H′(A)=(h 0 , h 1  . . . h 15 ) 
     
       
         
           
             
               h 
               i 
             
             = 
             
               
                 ∑ 
                 
                   j 
                   = 
                   
                     
                       i 
                       ⁡ 
                       
                         ( 
                         
                           n 
                           + 
                           1 
                         
                         ) 
                       
                     
                     16 
                   
                 
                 
                   
                     
                       
                         ( 
                         
                           i 
                           + 
                           1 
                         
                         ) 
                       
                       ⁢ 
                       
                         ( 
                         
                           n 
                           + 
                           1 
                         
                         ) 
                       
                     
                     16 
                   
                   - 
                   1 
                 
               
               ⁢ 
               
                 a 
                 j 
               
             
           
         
       
     
     b) the resulting 16-dimensional vectors are subtracted as matrices, and the result of this subtraction is used as a temporal video fingerprint. 
     The timing engine  208  may be configured to generate a timestamp associated with an audio or video transition detected by the transition detection engine  206  to generate temporal fingerprint. When the timing engine  208  receives a request from the transition detection engine  206  for the current time, the timing engine  208  may retrieve a clock signal or time (e.g., according to the GPS clock) to timestamp the audio or video transition. In various embodiments, timing engine  208  may provide the current time according to a UNIX clock or any other clock. It will be appreciated that the timing engine  308  may provide the current time in any number of ways (e.g., according to Coordinated Universal Time (UTC)). 
     The packet engine  210  may package the timestamp and/or network identifier together. In various embodiments, the packet engine  210  may package an audio and/or video data identifier (i.e., an identifier that identifies audio data, video data, or the like). In some embodiments, the packet engine  210  may package metadata, timestamp, and/or network identifier together. The network identifier may identify the broadcast network associated with the audio and/or video data (e.g., the broadcast system or network associated with generating the audio and/or video data, generating the initial temporal fingerprints, and/or the like). 
     In some embodiments, the network identifier may be used to information used to identify the show, such as production code. Network identifiers may be used by the first broadcast system  102  and second broadcast system  104  during the comparison of temporal video fingerprints. For example, the network identifier received from first broadcast system  102  may match the network identifier generated by second broadcast system  104 . Once a video temporal fingerprint has been packaged, it may be sent by the first network to network-based storage  108 , the second broadcast system  104 , and/or any number of user systems  110 . In some embodiments, the packet engine  210  may package video transitions, audio transitions, timestamps, and/or the network identifier. 
     The media fingerprint datastore  212  may be or may include any structure and/or structures suitable for storing data entries or records (e.g., an active database, a relational database, a self-referential database, a table, a matrix, an array, a flat file, a documented-orientated storage system, a non-relational No-SQL system, an FTS-management system such as Lucene/Solar, and the like). Media fingerprints may store packaged audio and/or video fingerprints. In some embodiments, media fingerprint datastore  212  may store audio fingerprint index and/or video fingerprint index. 
     The metadata engine  216  may be configured to determine locations within the audio and/or video data associated or for metadata. In one example, the metadata engine  216  may utilize a video metadata timestamp relative to a temporal video fingerprint. For example, metadata engine  216  may determine that video metadata (e.g., a title, length, start time, and/or the like of a car commercial) may be inserted at, for example, 1 frame or at specific times (e.g., 1/30 th  of a second), after a particular temporal video fingerprint. The metadata engine  216  may associated the video metadata with a video file of the car commercial. The video file of the car commercial in this example may be stored in metadata datastore  214  and/or another digital device. In other embodiments, the video metadata may be stored in network-based storage  108 . Similarly, metadata engine  216  may be configured to determine an audio metadata timestamp relative to a temporal audio fingerprint. 
     The metadata engine  216  may generate an audio or video metadata index using the audio or video metadata respectively. The metadata index may include metadata identifiers associated with temporal fingerprints (e.g., timestamps) of audio and/or video data (e.g., of a particular audio and/or video file). In some embodiments, the metadata index may include an audio and/or video data identifier that identifiers the audio and/or video data. The metadata index may include metadata identifiers and temporal fingerprints associated with any number of different audio and/or video files. Although the metadata index is identified as an index, it will be appreciated that any data structure (e.g., a table) may be utilized to store the information. 
     The metadata datastore  214  may be or may include any structure and/or structures suitable for storing data entries or records (e.g., an active database, a relational database, a self-referential database, a table, a matrix, an array, a flat file, a documented-orientated storage system, a non-relational No-SQL system, an FTS-management system such as Lucene/Solar, and the like). The metadata datastore  214  may store audio and/or video metadata associated with respective temporal audio and/or video temporal fingerprint. In some embodiments, metadata datastore  214  may store a hyperlink to a video and/or audio file stored in another location, or on the world wide web. 
       FIG. 3  depicts a block diagram of second broadcast system  104  according to some embodiments. The second broadcast system  104  may receive audio and/or video data from the first broadcast system  102  and generate temporal fingerprints in a manner similar to that of the first broadcast system  102 . The second broadcast system  104  may also receive the temporal fingerprints from the first broadcast system, compare the two sets of temporal fingerprints to synchronize the audio and/or video data and determine locations associated with or to insert metadata. The second broadcast system  104  may retrieve the metadata from any number of storage devices or receive the metadata (e.g., from the first broadcast system  102  or network-based storage  108 ). The second broadcast system  104  may subsequently broadcast the modified audio and/or video data (e.g., the audio and/or video data with the inserted metadata) to other broadcast systems and/or digital devices (e.g., user devices  110 ). 
     The second broadcast system  104  may include a controller engine  302 , a communication engine  304 , a transition detection engine  305 , a timing engine  308 , a packet engine  310 , a media fingerprint datastore  312 , a comparison engine  314 , a media engine  316 , and a metadata datastore  316 . Although  FIG. 3  depicts the second broadcast system  104 , any number of the engines and/or datastores, as well as their functions, may be performed by a user device  110  (e.g., for synchronization of audio and/or video data, insertion of metadata, correcting locations of metadata, lip synchronization, audio synchronization, metadata synchronization, and/or the like). 
     The controller engine  302  may manage one or more of the various engines shown in  FIG. 3 . For example, the controller engine  302  may direct the transition detection engine  306  to detect abrupt transitions between successive frames of video or abrupt transitions in the audio between successive moments in time. The controller engine  302  may direct the timing engine  308  to timestamp video transitions to generate temporal video fingerprints utilizing a Global Positioning System (GPS) clock to timestamp the audio and/or video transitions detected by transition detection engine  306 . In another example, the controller engine  202  may direct the comparison engine  314  to compare temporal fingerprints received from first broadcast system  102  with temporal fingerprints detected by transition detection engine  306 . 
     The communication engine  304  may transmit data between second broadcast system  104  and network-based storage  108  and/or between second broadcast system  104  and user systems  110 . 
     The transition detection engine  306  of the second broadcast system  104  may be similar to the transition detection engine  206  of the first broadcast system  102 . The transition detection engine  306  may scan audio and/or video data to detect transitions between successive frames of video or transitions in the audio between successive moments in time. In various embodiments, the transition detection engine  306  may scan through a video or audio component to identify video or audio transitions. In one example, the transition detection engine  306  detects one or more transitions within video content (e.g., abrupt changes in color, brightness, and/or the like). In some embodiments, the transition detection engine  306  may detect one or more transitions within audio content (e.g., abrupt changes in volume or sound(s) after a period of silence). 
     Once any number of audio or video transitions have been detected, the transition detection engine  306  may send a request to the timing engine  308  for clock values (e.g., based on a GPS clock) to timestamp the audio or video transition. The timing engine  308  may receive the request from transition detection engine  306  to create one or more temporal audio fingerprints or one or more temporal video fingerprints. 
     Video transitions may be detected and/or determined in any number of ways. in some embodiments, the transition detection engine  306  detects and/or determines video transitions within video data using the method described regarding the transition detection engine  206  discussed herein. 
     The timing engine  308  may be configured to generate a timestamp associated with an audio or video transition detected by the transition detection engine  306  to generate temporal fingerprint. When the timing engine  308  receives a request from the transition detection engine  306  for the current time, the timing engine  308  may retrieve a clock signal or time (e.g., according to the GPS clock) to timestamp the audio or video transition. In various embodiments, timing engine  308  may provide the current time according to a UNIX clock or any other clock. It will be appreciated that the timing engine  308  may provide the current time in any number of ways (e.g., according to Coordinated Universal Time (UTC)). 
     The packet engine  310  may package the timestamp and/or network identifier together. In various embodiments, the packet engine  310  may package an audio and/or video data identifier (i.e., an identifier that identifies audio data, video data, or the like). In some embodiments, the packet engine  310  may package metadata, timestamp, and/or network identifier together. The network identifier may identify the broadcast network associated with the audio and/or video data (e.g., the broadcast system or network associated with generating the audio and/or video data, generating the initial temporal fingerprints, and/or the like). 
     Once a video temporal fingerprint has been packaged, it may be sent to any number of digital devices (e.g., other broadcast systems and/or other user systems  110 ). In some embodiments, the packet engine  310  may package video transitions, audio transitions, timestamps, and/or the network identifier. 
     The media fingerprint datastore  312  may be or include any structure and/or structures suitable for storing data entries or records (e.g., an active database, a relational database, a self-referential database, a table, a matrix, an array, a flat file, a documented-orientated storage system, a non-relational No-SQL system, an FTS-management system such as Lucene/Solar, and the like). The media fingerprint datastore  312  may store temporal fingerprints (e.g., audio fingerprints, video fingerprints, and/or multimedia fingerprints. 
     The comparison engine  314  may be configured to compare the temporal fingerprints received from first broadcast system  102  with the temporal fingerprints generated by the transition detection engine  306 . When the same transitions are detected by transition detection engine  306 , the difference between the timestamp of the two temporal fingerprints may be determined. This difference, also referred to as the time offset, may be used to synchronize audio and/or video for consistent and precise insertion of metadata, correct lip sync issues and two independent broadcast feeds which may be useful for disaster recovery. 
     In other embodiments, comparison engine  314  may be configured to detect audio transitions received from first broadcast system  102  with audio transitions detected by transition detection  306 . 
     The metadata engine  316  may be configured to identify metadata using the metadata index (e.g., received from the first broadcast system  102 ) as well as identify one or more locations within the audio and/or video data for the identified metadata (e.g., to insert or adjust the identified metadata relative to the audio and/or video data). For example, the metadata engine  316  may identify one or more locations within the audio and/or video data based on the temporal fingerprints generated by the second broadcast system  104 , the comparison of the temporal fingerprints (e.g., based on the time offset), and/or number of frames before or after any number of abrupt transitions within the video content of the audio and/or video data. 
     In various embodiments, the first broadcast system  102  may be configured to inform the metadata engine  316  of the video to associate with a particular video metadata timestamp. The second broadcast system  104  may replace video (e.g., a commercial) associated with the particular video metadata timestamp with a different video (e.g., a different commercial). In some embodiments, the metadata engine  316  may generate an audio or video metadata index using the audio or video metadata fingerprints respectively. Similarly, the metadata engine  316  may be configured to determine an audio metadata timestamp relative to a temporal audio fingerprint, and associate an audio file with the audio metadata timestamp. 
     The metadata datastore  318  may be or include any structure and/or structures suitable for storing data entries or records (e.g., an active database, a relational database, a self-referential database, a table, a matrix, an array, a flat file, a documented-orientated storage system, a non-relational No-SQL system, an FTS-management system such as Lucene/Solar, and the like). The metadata datastore  316  may store audio and/or video data associated with their respective temporal audio and/or video metadata fingerprint. In some embodiments, metadata datastore  318  may store a hyperlink to a video and/or audio file stored in another location, or on the world wide web. 
       FIG. 4  depicts a flowchart of a first video broadcasting method  400  utilizing the first broadcast system  102  according to some embodiments. In step  402 , the first broadcast system  102  may receive the video component of the audiovisual broadcast (e.g., video data that includes audio). The first broadcast system  102  may receive audio and/or video content as part of a live broadcast and/or generate audio and/or video data. 
     In step  404 , the transition detection engine  206  may scan through the video component of the audiovisual broadcast to identify video transitions. The transition detection engine  206  may be configured to identify video transitions by scanning through the video component of the audiovisual broadcast until the end of the program or show or over any portion of the program or show. 
     It will be appreciated that the video component of the audiovisual broadcast may be transmitted to the second broadcast system  104  and/or user devices  110  over the first network via the satellite system  106  at any time before or after the transition detection engine  206  identifies video transitions. In one example, such as a live broadcast, the first broadcast system  102  may send the video component of the audiovisual broadcast before the transition detection engine  206  identifies video transitions. In another example, the first broadcast system  102  may send a segment of the video component of the audiovisual broadcast after it has been scanned by transition detection engine  206 . The transition detection engine  206  may determine the video histogram for every chroma/luma/color channel and calculate a sum of absolute differences between the histogram vectors of every pair of adjacent frames. For example, the transition detection engine  206  may detect video transition ‘A’ at the beginning of frame  150 , video transition ‘B’ at the beginning of frame  234  and video transition ‘C’ at the beginning of frame  357 . 
     In step  406 , for each detected video transition, the timing engine  208  may generate a timestamp based on the current time according to the GPS clock to timestamp the video transition. For example, the timing engine  208  may generate a timestamp of a current time X1 for video transition ‘A.’ In one example, the timing engine  208  may provide the current time according to an UNIX clock. It will be appreciated that the timing engine  208  may retrieve a time for the timestamp(s) from any source (e.g., according to Coordinated Universal Time (UTC)). 
     In step  408 , the metadata engine  216  determines specific metadata such as a commercial or other metadata relative to a temporal video fingerprint. For example, the video metadata associated with, for example, a car commercial may be placed 100 frames after video transition ‘A’ or 16 frames after video transition ‘B.’ In another example, the video metadata may be placed 32 milliseconds after video transition ‘A’ or 28 milliseconds before video transition ‘B.’ 
     In some embodiments, upon determination of the relative frame position of the video metadata relative to one or more temporal video fingerprints, a video metadata timestamp may be determined. There may not be a video metadata timestamp associated with every temporal fingerprint, but each video metadata maybe associated with more than one temporal video fingerprints. 
     In various embodiments, the metadata engine  216  creates a metadata index identifying metadata and associating different identified metadata with positions in the audio and/or video data using the temporal timestamps. 
     In step  410 , the packet engine  210  may package the temporal timestamps and network identifier together. The network identifier may be used by second broadcast system  104  during the comparison of temporal fingerprints (e.g., by the second broadcast system  104  and/or user device(s)  110 ). The network identifier of the temporal video fingerprint from the first broadcast  102  and the temporal video fingerprint from the second broadcast  104  may be compared to ensure that the temporal fingerprints are from the same audiovisual broadcast and for synchronization. 
     In step  412 , the packet engine  210  may package metadata and the network identifier together. Similar to the network identifier associated with packaging temporal fingerprints, the network identifier may be used to ensure that the metadata is being inserted into the correct audiovisual broadcast. In various embodiments, packaging the metadata, temporal timestamps, metadata index, or the like is optional. It will be appreciated that the metadata, temporal timestamps, metadata index, or the like may be sent individually, together, or in any combination to another digital device. 
     In step  414 , the communication engine  204  may transmit the audio and/or video data over the first network to the satellite system  106 . In other embodiments, the communication engine  204  may transmit the audio and/or video data over the first network to user system  110  via satellite system  106 . In various embodiments, the first network may include satellite communications provided by satellite system  106 . In other embodiments, the first network may include wired or wireless computer network. 
     In step  416 , the communication engine  204  may transmit the temporal fingerprints, metadata index, and/or metadata over the second network to network-based storage  108 . In some embodiments, the communication engine  204  may be transmit the temporal fingerprints, metadata index, and/or metadata over the second network to user system  110  and/or the second broadcast system  104 . The second network may represent a different band or connection to the receiving device(s). 
     It will be appreciated that the first broadcast system  102  may provide any portion of the audio and/or video data as well as portions temporal fingerprints, a portion of the metadata index, and/or any metadata at any time including during scanning and before the determination of all temporal fingerprints of the same audio and/or video file is complete. The first broadcast system  102  may complete scanning all or a portion of the video component of the audiovisual broadcast, identify additional video transitions, and package the additional temporal fingerprints and metadata. 
       FIG. 5  depicts a flowchart of a first audio broadcasting method  500  of first broadcast system  102  according to some embodiments. In step  502 , the first broadcast system  102  may receive audio data (e.g., an audio file). 
     In step  504 , the transition detection engine  206  may scan through any portion of an audio component of the audio broadcast (or an audio/video broadcast) to identify audio transitions. For example, the transition detection engine  206  may detect audio transition ‘A’, audio transition ‘B’ and audio transition ‘C’. In various embodiments, the audio component of the audiovisual broadcast may be transmitted to the second broadcast system  104  over the first network via satellite system  106  at any time before or after transition detection engine  206  identifies audio transitions. 
     The transition detection engine  206  may determine audio transitions between successive moments in time. In one example:
         1) The transition detection engine  206  may downmix audio to a monaural signal by adding corresponding samples. In the downmix process, a common downmix formula may be applied where the center channel(s) may be taken at −3 dB, LS, RS at −6 dB, LFE channel discarded. The transition detection engine  206  may choose a time interval for binning in such manner as to: a) produce an integer number that is a multiple of 16 of digital audio signal samples for this interval given the original sampling rate of the signal; b) have the time interval that is several times lower than the duration of one video frame of the content. For example, in a scenario with 48 kHz sampled PCM audio with 30 frame per second video, the transition detection engine  206  may choose 10 ms, which may result in 480 samples per interval (16×30), and approximately 3 intervals per frame.   2) The transition detection engine  206  may split continuous digital audio amplitude samples into sections representing the selected time interval. Each section may be converted from time-domain to frequency domain using a Fast Fourier Transform algorithm, and the resulting vector may be decimated into 16 bins by summing adjacent results up. In the example with 480 sample segments, bin 0 may be sum of FFT result values 0 through 29, and so on.   3) The transition detection engine  206  may analyze a sequence of resulting 16-dimensional vectors, and abrupt audio transitions may be detected by evaluating the sum of absolute differences between the vectors calculated from adjacent intervals, in the same or similar manner as described in step 1b for video fingerprints.   4) Once the abrupt transitions are identified, they may become the basis for audio temporal fingerprints, that include both 16-dimensional vectors calculated based on adjacent time intervals.       

     The timing engine  208  may retrieve a clock signal (e.g., according to the GPS clock) to create a timestamp for each audio transition. In some embodiments, the transition detection engine  206  creates the timestamp using a clock signal from the timing engine  208  to generate a temporal audio fingerprint. For example, the timing engine  208  may receive the request from transition detection engine  206  of audio transition ‘A’ and send the current time of X4. In some embodiments, the timing engine  208  may provide the current time according to an UNIX clock. In other embodiments, timing engine  208  may provide the current time according to UTC. 
     In step  508 , the metadata engine  216  determines location of metadata associated with a commercial or other metadata relative to one or more temporal audio fingerprints. For example, audio metadata associated with, for example, a sportscaster&#39;s commentary may be placed 2 seconds after audio transition ‘A’ or 16 frames after audio transition ‘B.’ 
     Upon determination of the relative position of the metadata relative to one or more temporal audio fingerprint, the audio metadata timestamp or an index identifier for the metadata can be determined. There may not be an audio metadata timestamp associated with every temporal fingerprint, but each audio metadata maybe associated with more than one temporal audio fingerprint. 
     In step  510 , the packet engine  210  may be optionally configured to package the temporal audio fingerprints and network identifier together. The network identifier may be used by the second broadcast system  104  during the comparison of temporal audio fingerprints. The network identifier of the temporal audio fingerprint from the first broadcast  102  and the temporal video fingerprint from the second broadcast  104  may be compared to ensure that the temporal fingerprints are from the same audiovisual broadcast. 
     In step  512 , the packet engine  210  may optionally be configured to package audio metadata, temporal fingerprints, and/or network identifier together. Similar to the network identifier associated with packaging temporal audio fingerprints, the network identifier may be used to ensure that the metadata is being inserted into the correct audiovisual broadcast. 
     In step  514 , the audio data may be transmitted over the first network to the second broadcast system  104  via the satellite system  106 . In other embodiments, the audio data may be transmitted over the first network to user system  110 . In some embodiments, after transition detection engine  206  detects an audio transition, such as in step  504 , the section of audio that has already been scanned may be transmitted. 
     In step  516 , the temporal audio fingerprints may be transmitted over the second network to network-based storage  108 . In other embodiments, the temporal audio fingerprints may be transmitted over the second network to the user system  110 . 
       FIG. 6  depicts a flowchart of a second video broadcasting method  600  of second broadcast server  104  according to some embodiments. In step  602 , the second broadcast system  104  may receive the audio and/or video data from the first broadcast system  102  (e.g., via the satellite system  106 ). 
     In step  604 , the transition detection engine  306  may scan through any portion of the video component of the video and/or audio data to identify video transitions. In various embodiments, the video and/or audio data may be transmitted to the user system  110  over the first network via satellite system  106  at any time before or after transition detection engine  306  identifies video transitions. Similar to step  404  of first video broadcasting method  400 , the transition detection engine  306  may detect the same video transitions in step  604 . For example, the transition detection engine  306  may detect video transition ‘A’ at the beginning of frame  150 , video transition ‘B’ at the beginning of frame  234  and video transition ‘C’ at the beginning of frame  357 . 
     Once a video transition has been detected, the timing engine  308  may retrieve a current time according to the GPS clock to timestamp the video transition in step  606 . The timing engine  308  may receive a request and transmit the current time to the transition detection engine  306 . The transition detection engine  306  may receive the current time and associate the received time to generate a temporal video fingerprint. For example, the timing engine  308  may receive the request from the transition detection engine  306  of video transition ‘A’ and send the current time of X1+Y1. Where Y1 represents the time elapsed between the first broadcast system  102  detect the video transition, generate the temporal video fingerprint, package the temporal video fingerprint with the network identifier, transmit the package to second broadcast system  104 , the second broadcast system  104  detect the video transition and generate the temporal video fingerprint. 
     For live audiovisual broadcast, the time elapsed may represent the transmission and processing time of the audio and/or video content. For on-demand broadcast, users of user system  110  may select audio and/or video content when they choose rather than at the specific broadcast time. Y1, in this case, may represent the time elapsed to process the audio and/or video content as well as the time between the original broadcast and when the user chooses to access the audio and/or video content. 
     In step  608 , the comparison engine  314  may compare the temporal video fingerprint received from first broadcast system  102  with temporal video fingerprints generated from transition detection engine  306 . It will be appreciated that the second broadcast system  104  may receive the temporal fingerprints generated by the first broadcast system  102  in any number of ways. In some embodiments, the comparison engine  314  may require some number of consecutive matches between temporal video fingerprints from first broadcast system  102  and second broadcast system  104  with matching timestamp deltas to determine the time offset of the video component of the audiovisual broadcast. A timestamp delta may be the time difference between consecutive temporal video fingerprints. 
     In step  610 , the comparison engine  314  determines the time offset between the temporal video fingerprint(s) from first broadcast system  102  and the temporal video fingerprint(s) of the second broadcast system  104 . 
     Once the time offset has been determined, metadata may be inserted into the video component of the video and/or audio data in step  612 . The second broadcast system  104  may insert a video according to the metadata sent by first broadcast system  102  or retrievable from network storage. In some embodiments, second broadcast system  104  may replace previously existing metadata with other metadata chosen by the second broadcast system  104 . In some embodiments, video metadata may point to a video file stored in network-based storage  108 . In other embodiments, video metadata may include a hyperlink, a video graphic, a graphic interchange format (GIF), or the like. Once metadata is added to the video and/or audio data, the resulting combination may be referred to as modified video and/or audio data. 
     In step  614 , the modified video and/or audio data may be transmitted over the first network to user system  110  (e.g., via satellite system  106 ). In some embodiments, after the transition detection engine  306  detects a video transition, such as in step  604 , the section of video that has already been scanned may be transmitted over the first network. In various embodiments, the first network may include satellite communications provided by satellite system  106 . In other embodiments, the first network may include wired or wireless computer network. 
     In step  616 , the temporal video fingerprint and video metadata may be optionally transmitted over the second network to network-based storage  108 . 
     The second broadcast system  104  may finish scanning the video component of the audiovisual broadcast and identify the video transitions and package the video fingerprints and video metadata. 
       FIG. 7  depicts a flowchart of a second audio broadcasting method  700  of second broadcast server  104  according to some embodiments. In step  702 , the second broadcast system  104  may receive the audio data from the first broadcast system  102 . 
     In step  704 , the transition detection engine  306  may scan through the audio data (or an audio component of audio/video data) to identify audio transitions. The transition detection engine  306  may identify audio transitions by scanning through any portion of the audio component of the audiovisual broadcast. In various embodiments, the audio component of the audiovisual broadcast may be transmitted to user system  110  over the first network at any time before or after transition detection engine  306  identifies audio transitions. Similar to step  504  of first audio broadcasting method  500 , the transition detection engine  306  may detect the same audio transitions in step  704 . For example, the transition detection engine  306  may detect audio transition ‘A’ 
     Once an audio transition has been detected, the timing engine  308  may retrieve the current time according to the GPS clock to timestamp the audio transition. The timing engine  308  or the transition detection engine  306  may receive the current time and associated the received time to generate a temporal audio fingerprint. For example, the timing engine  308  may receive the request from transition detection engine  306  of audio transition ‘A’ and send the current time of X4+Y2. Where Y2 represents the time elapsed between when the first broadcast system  102  detected the audio transition, the first broadcast system  102  generated the temporal audio fingerprint, the first broadcast system  102  packaged the temporal audio fingerprint with the network identifier, the first broadcast system  102  transmitted the package to second broadcast system  104 , and the second broadcast system  104  detected the audio transition and generate the temporal audio fingerprint. 
     For live audiovisual broadcast, the time elapsed may represent the transmission and processing time of the audio and/or video content. For on-demand broadcast, the user system  110  may select audio and/or video content when they choose rather than at the specific broadcast time. Y2, in this case, represents the time elapsed to process the audio and/or video content as well as the time between the original broadcast and when the user chooses to access the audio and/or video content. 
     In step  708 , the comparison engine  314  compares the temporal audio fingerprint received from first broadcast system  102  with temporal audio fingerprints generated from the transition detection engine  306 . In some embodiments, the comparison engine  314  may require some number of consecutive matches between temporal audio fingerprints from the first broadcast system  102  and the temporal audio fingerprints from the second broadcast system  104 . As discussed herein, matching timestamp deltas may be used to determine the time offset of the audio component of the audiovisual broadcast. A timestamp delta may be the time difference between consecutive temporal audio fingerprints. 
     In step  710 , the comparison engine  314  may determine the time offset between the temporal audio fingerprint from the first broadcast system  102  and the second broadcast system  104 . 
     Once the time offset has been determined, audio metadata may be inserted into the audio component of the audiovisual broadcast in step  712 . The second broadcast system  104  may insert an audio according to the audio metadata sent by the first broadcast system  102 . In other embodiments, the second broadcast system  104  may replace an entry from the audio metadata with audio chosen by the second broadcast system  104 . In some embodiments, audio metadata may point to an audio file stored in network-based storage  108 . In other embodiments, audio metadata may include a hyperlink or the like. 
     In step  714 , the audio component of the audiovisual broadcast may be transmitted over the first network to user system  110 . In some embodiments, after transition detection engine  306  detects an audio transition, such as in step  704 , the section of audio that has already been scanned may be transmitted over the first network. In various embodiments, the first network may include satellite communications provided by satellite system  106 . In other embodiments, the first network may include wired or wireless computer network. 
     In step  716 , the temporal audio fingerprint and audio metadata may be optionally transmitted over the second network to network-based storage  108 . 
     The second broadcast system  104  may complete scanning of all or a portion of the audio component of the audiovisual broadcast, may identify the audio transitions, and may package the audio fingerprints and audio metadata. 
       FIG. 8  depicts a flowchart of re-synchronization of video method  800  of the second broadcast server  104  according to some embodiments. In step  802 , the first broadcast system  102  generates temporal video fingerprints. This is similar to steps  404 ,  406 ,  408  and  412  of first video broadcasting method  400  of  FIG. 4 . The transition detection engine  206  of the first broadcast system  102  may scan through any portion of the video component of the audiovisual broadcast to identify video transitions. When a video transition is detected, the transition detection engine  206  may send a request to the timing engine  208  for the current time according to the GPS clock to timestamp the video transition. The timing engine  208  may receive this request and transmit the current time to the transition detection engine  206 . The transition detection engine  206  may receive the current time and associate the received time to generate the temporal video fingerprint. The packet engine  210  may package the video transition, timestamp and network identifier together. 
     In step  804 , the communication engine  204  may transmit the temporal video fingerprint to network-based storage  108 . 
     In step  806 , similar to step  802 , the second broadcast system  104  generates temporal video fingerprints. This is similar to steps  604 ,  606 ,  608  and  612  of second video broadcasting method  600  of  FIG. 6 . The transition detection engine  306  of the second broadcast system  104  may scan through any portion of the video component of the audiovisual broadcast to identify video transitions. When a video transition is detected, the transition detection engine  306  may send a request to timing engine  308  for the current time according to the GPS clock to timestamp the video transition. The timing engine  308  may receive this request and transmit the current time to the transition detection engine  306 . The transition detection engine  306  may receive the current time and associate the received time to generate the temporal video fingerprint. The packet engine  310  may be configured to package the video transition, timestamp and network identifier together. 
     As the second broadcast system  104  generates video temporal fingerprints, the second broadcast system  104  may retrieve temporal video fingerprints from network-based storage  108  over the second network in step  808 . 
     In step  810 , the comparison engine  314  compares the temporal video fingerprint received from first broadcast system  102  with temporal video fingerprints generated from transition detection engine  306  to find video fingerprints that match for synchronization. 
     In step  812 , the comparison engine  314  may match temporal video fingerprints. In some embodiments, the comparison engine  314  may require some number of consecutive matches between temporal video fingerprints from first broadcast system  102  and second broadcast system  104  with matching timestamp deltas to determine the time offset of the video component of the audiovisual broadcast. 
     In step  814 , the comparison engine  314  determines the time offset between the temporal video fingerprint(s) from first broadcast system  102  and the temporal video fingerprint(s) generated by second broadcast system  104 . 
     In step  816 , the media engine  316  utilizes the time offset to re-synchronize the video metadata and video component on the second broadcast system  104 . 
       FIG. 9  depicts a flowchart of correcting lip sync method  900  of the second broadcast server  104  according to some embodiments. Steps  902  thru  914  are similar to those of steps  802  thru  814  of  FIG. 8 . For example, in step  902 , the first broadcast system  102  receives audiovisual broadcast (e.g., video data with audio) and generates temporal video fingerprints. The transition detection engine  206  of the first broadcast system  102  may scan through any portion of the audiovisual broadcast to identify video transitions. When a video transition is detected, the timing engine  208  may retrieve a time according to the GPS clock to timestamp the video transition. The timing engine  208  or the transition detection engine  206  may generate the temporal video fingerprint. 
     In step  904 , the communication engine  204  may transmit the temporal video fingerprint to network-based storage  108 . 
     In step  906 , the second broadcast system  104  generates temporal video fingerprints. This is similar to steps  604 ,  606 ,  608  and  612  of second video broadcasting method  600  of  FIG. 6 . The transition detection engine  306  of the second broadcast system  104  may scan through any portion of the video data to identify video transitions. When a video transition is detected, the timing engine  308  may retrieve the current time according to the GPS clock to timestamp the video transition. The timing engine  308  or the transition detection engine  306  may utilize the time retrieved by the timing engine  308  to generate the temporal video fingerprint. 
     As the second broadcast system  104  generates video temporal fingerprints, the second broadcast system  104  may retrieve temporal video fingerprints from network-based storage  108  over the second network in step  808 . 
     In step  910 , the comparison engine  314  compares the temporal video fingerprint received from first broadcast system  102  with temporal video fingerprints generated from transition detection engine  306  to find video fingerprints that match for synchronization. 
     In step  912 , the comparison engine  314  may match temporal video fingerprints. In some embodiments, the comparison engine  314  may require some number of consecutive matches between temporal video fingerprints from first broadcast system  102  and second broadcast system  104  with matching timestamp deltas to determine the time offset of the video component of the audiovisual broadcast. 
     In step  914 , the comparison engine  314  determines the time offset between the temporal video fingerprint(s) from first broadcast system  102  and the temporal video fingerprint(s) generated by second broadcast system  104 . 
     In step  916 , the transition detection engine  206  may scan through the audio component of the audiovisual broadcast to identify audio transitions. The timing engine  208  may retrieve the time according to the GPS clock to timestamp the audio transition. The timing engine  208  and/or the transition detection engine  206  may utilize the retrieved time to generate a temporal audio fingerprint. 
     In step  918 , the communication engine  204  may be configured to transmit the temporal audio fingerprint to network-based storage  108 . 
     In step  920 , similar to step  916 , the second broadcast system  104  generates temporal audio fingerprints. This may be similar to steps  704 ,  706 ,  708  and  712  of second audio broadcasting method  700  of  FIG. 7 . The transition detection engine  306  of second broadcast system  104  may scan through any portion or all of the audio component of the audiovisual broadcast to identify audio transitions. When an audio transition is detected, the timing engine  308  for the current time according to the GPS clock to timestamp the audio transition. The timing engine  308  or the transition detection engine  306  may utilize the retrieved time to generate the temporal audio fingerprint. 
     As the second broadcast system  104  generates audio temporal fingerprints, second broadcast system  104  may retrieve temporal audio fingerprints from network-based storage  108  over the second network, as seen in step  922 . 
     In step  924 , the comparison engine  314  compares the temporal audio fingerprint received from first broadcast system  102  via network-based storage  108  with temporal audio fingerprints generated from transition detection engine  306  to find audio fingerprints which match. 
     In step  926 , the comparison engine  314  may match temporal audio fingerprints. In some embodiments, the comparison engine  314  may require some number of consecutive matches between temporal audio fingerprints from the first broadcast system  102  and the second broadcast system  104  with matching timestamp deltas to determine the time offset of the audio component of the audiovisual broadcast. 
     In step  928 , the comparison engine  314  determines the time offset between the temporal audio fingerprint from the first broadcast system  102  and the second broadcast system  104 . 
     In step  930 , the comparison engine  314  takes the video timing offset determined in step  914  and audio timing offset determined in step  928  to calculate the lip synchronization offset. 
     In step  932 , the second broadcast system  104  corrects the lip synchronization utilizing the lip synchronization offset. 
     In the presented embodiments, the audiovisual broadcast system  100  includes two broadcast systems. In other embodiments, the audiovisual broadcast system  100  may include three or more broadcast systems which may function similarly to first broadcast system  102  or second broadcast system  104 . 
       FIG. 10  depicts a block diagram of an example a digital device  1002  according to some embodiments. Any user system  102  may comprise an instance of a digital device  1002  (e.g., a computing device). The digital device  1002  comprises a processor  1004 , memory  1006 , storage  1008 , an input device  1010 , a communication network interface  1012 , and an output device  1014 . 
     The processor  1004  may be configured to execute executable instructions (e.g., programs). In some embodiments, the processor  1004  comprises circuitry or any processor capable of processing the executable instructions. 
     The memory  1006  stores data. Some examples of memory  1006  include storage devices, such as RAM, ROM, RAM cache, virtual memory, etc. In various embodiments, working data is stored within memory  1006 . The data within memory  1006  may be cleared or ultimately transferred to storage  1008 . 
     The storage  1008  includes any storage configured to retrieve and store data. Some examples of the storage  1008  includes flash drives, hard drives, optical drives, and/or magnetic tape. Each of memory system  1006  and the storage system  1008  may comprise a computer-readable medium, which stores instructions or programs executable by processor  1004 . 
     The input device  1010  is any device that inputs data (e.g., mouse, keyboard, stylus). 
     The output device  1014  outputs data (e.g., speaker, display, virtual reality headset). It will be appreciated that the storage  1008 , input device  1010  and output device  1014  may be optional. For example, routers/switchers may comprise processor  1004  and memory  1006  as well as a device to receive and output data (e.g., communication network interface  1012  and/or output device  1014 ). 
     The communication network interface  1012  may be coupled to a network (e.g. network-based storage  108  or satellite system  106 ) via a communication network interface  1012 . The communication network interface  1012  may support communication over an Ethernet connection, a serial connection, a parallel connection, and/or an ATA connection. Communication network interface  1012  may also support wireless communication (e.g., 1102.11 a/b/g/n, WiMax, LTE, WiFi). It will be apparent that communication network interface  1012  may support many wired and wireless standards. 
     An engine may be hardware or software. In some embodiments, the engine may configure one or more processors to perform functions associated with the engine. Although different engines are discussed herein, it will be appreciated that the server system  106  may include any number of engine performing any or all functionality discussed herein.