Abstract:
A method and apparatus is provided for encoding a video signal stream. The method includes receiving a video signal stream, identifying a plurality of individual segments within the video signal stream and encoding, off-line, recurring ones of the individual segments that recur with at least a first frequency to produce encoded recurring segments. The video signal stream is also encoded in real-time to generate a real-time encoded video stream. Segments in the real-time encoded video stream are replaced with their corresponding encoded recurring segments to thereby produce an encoded video output stream that includes some segments encoded in real-time and other segments encoded off-line.

Description:
FIELD OF THE INVENTION 
     The present invention relates generally to video encoders and more particularly to video encoders that employs a combination of real-time and off-line encoding. 
     BACKGROUND 
     Digital video compression is an important technology in video communications, broadcasting, and storage. MPEG video coding standards have been successfully used to reduce the transmission bandwidth and storage space requirements in many applications, such as digital TV broadcast through satellite and cable, Digital Video Disk (DVD), Video on-Demand and video streaming over the Internet, etc. However, emerging applications and new services place increasing demands on compression techniques. For example, live news and sports events are transmitted in real-time to subscribers using MPEG-2 video coding standard (ISO/IEC 13818-2) at a constant bit rate (CBR) in the range of 0.6 to 2 Mbits/second for standard definition content. It is very challenging for conventional MPEG encoders available on the commercial market to produce acceptable picture quality at such bit rates. 
     Conventional real-time video encoders often employ a coding strategy that uses information retained in coding only the previously received video frames to encode a current frame. However, prediction and estimation based on past frames generally will not correctly describe the current frame. Such encoders are not able to determine and apply the best coding strategy to encode incoming video frames because they lack information about future frames. As a result real-time encoders generally do not achieve compressed video with as high quality encoding characteristics (e.g., less distortion in an encoded image while using fewer bits of information) as off-line encoders. 
     In comparison to real-time encoders, off-line (i.e., non-real-time) video encoders can provide compressed video with higher quality encoding characteristics by using complex algorithms that require higher computational overhead. Such algorithms can perform non-casual rate control and may exhaustively perform rate/distortion optimization to determine the number of bits required to reduce artifacts that arise from the compression process. For example, off-line encoders can employ techniques such as multipass encoding. With multipass encoding, a video sequence is encoded several times and each encoding pass uses the results of the preceding pass to adjust coding parameters to optimize, for example, average bit rate and/or decoder buffer fullness. Overall, the multipass encoding process is a trial and error process: select initial coding parameters, code the video sequence, examine the results to determine if performance requirements are met and recode as necessary using adjusted coding parameters for each subsequent iteration. For long sequences of digital video, however, coding the entire video sequence several times is inefficient and greatly increases the time required to generate an efficiently compressed sequence and thus multipass encoding, like other complex algorithms that achieve higher quality, are often reserved for off-line encoding. 
     SUMMARY 
     In accordance with the present invention, a method and apparatus is provided for encoding a video signal stream. The method includes receiving a video signal stream, identifying a plurality of individual segments within the video signal stream and encoding, off-line, recurring ones of the individual segments that recur with at least a first frequency to produce encoded recurring segments. The video signal stream is also encoded in real-time to generate a real-time encoded video stream. Segments in the real-time encoded video stream are replaced with their corresponding encoded recurring segments to thereby produce an encoded video output stream that includes some segments encoded in real-time and other segments encoded off-line. 
     In accordance with another aspect of the invention, a video encoder arrangement is provided that includes a video segmentation module for segmenting an input video stream into a plurality of video segments. The arrangement also includes a real-time video encoder for encoding the input video stream and a processor configured to identify recurring video segments in the input video stream that recur with at least a first frequency. The arrangement also includes an off-line video encoder for encoding the recurring video segments. The processor is further configured to replace segments in the input video stream encoded by the real-time encoder with corresponding ones of the encoded recurring segments. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a functional block diagram of one example of a hybrid encoder that may implement the techniques and processes described herein. 
         FIG. 2  is a flow diagram illustrating one example of the operation of the hybrid encoder depicted in  FIG. 1 . 
         FIG. 3  shows one example of a video encoder that may be employed in the hybrid encoder shown in  FIG. 1 . 
         FIG. 4  is a flowchart showing one example of a method for encoding a video signal stream. 
     
    
    
     DETAILED DESCRIPTION 
     As detailed below, a live video stream that is being delivered to a viewer is encoded using a combination of both real time and off-line encoding. In particular, off-line encoding is used to encode recurring material that is repeated multiple times, within the same program and possibly among different programs as well. Such material may include, without limitation, certain commercials, opening shots of a sporting event or other program, graphics such as those that are presented as an introduction to a featured segment in a program (e.g., graphics that are presented as an instruction to a slow-motion replay in a football game) and graphics and material that are presented as interstitials (e.g., ESPN Sports Center interstitials). Recurring material that is repeated more than a threshold number of times during one or more programs is encoded at a relatively high quality using an off-line encoder. The other material (i.e., nonrecurring material and recurring material that is repeated less than a threshold number of times) in the live video is encoded using an on-line or real-time encoder, which generally will encode video at a lower quality than the off-line encoder. If incoming material in the live video stream is recognized as material that has been previously encoded and stored off-line, the encoded material is retrieved from storage and substituted for the real-time encoded material which would otherwise be included in the compressed video output stream. 
       FIG. 1  is a functional block diagram of one example of a hybrid encoder  100  that may implement the techniques and processes described herein. The hybrid encoder  100  includes real-time encoder  110 , off-line encoder  120 , segmentation module  130 , digital word comparator  135 , segment or scene comparator engine  140 , storage medium  150  and video decoder  160 . Additionally, the various elements shown in  FIG. 1  operate under the control of a processor  170 . The operation of the hybrid encoder  100  will be described with reference to  FIG. 2 . 
       FIG. 2  is a flow diagram illustrating one example of the operation of the hybrid encoder  100  depicted in  FIG. 1 . As shown, a baseband video input signal is received at block  205  and directed to blocks  210 ,  215 ,  217  and  219 . More specifically, copies of the video signal are respectively encoded in real-time by real-time encoder  110  at block  219 , delayed at block  217 , temporarily stored in a buffer at block  210  and directed to the segmentation module  130  at block  215 . 
     The segmentation module  130  extracts individual segments such as scenes at block  215 . In one implementation in which the video is segmented into scenes, segmentation is accomplished by identifying scene changes between temporally adjacent scenes in accordance with well known techniques. Each segment is assigned an identifier such as a digital word. The identifier is sufficiently descriptive to correctly recognize a segment with a certain degree of probability, which may vary from application to application. The digital word may represent various features in the segment such as the first or last frame, the number of frames in the segment, and so on. The segment descriptor word generated at block  215  is recorded on a histogram at block  220 . The histogram tabulates the frequency with which each word, and hence each segment, appears in the video input signal. The histogram may be maintained in a database located, for example, in the segmentation module  130  itself or in storage medium  150 . Next, at block  225  those segments that are repeated with a certain frequency and which were buffered at  210  are transferred to off-line encoder  120  and encoded at block  230 . The encoded segments are then stored at block  235  in, e.g., storage medium  150 . 
     Continuing at block  240 , the segment descriptor words generated from the baseband video input signal at block  215  are compared by word comparator  135  to the segment descriptor words identifying the pre-encrypted segments that have been stored at block  235 . If two words are found to match, there is a certain probability that the segments are the same. If such a match is found, additional steps may be performed to confirm that the segments are indeed the same. For instance, in this example, the pre-encrypted segment corresponding to the matched word is retrieved from storage and decoded at block  245  by the video decoder  160 . At block  250 , the decoded segment is compared by segment comparator  140  to the rendition of the segment that underwent a delay at block  217 . The comparison may be performed on a frame-by-frame and pixel-by-pixel basis in accordance with well-known techniques, such as by measuring the correlation between frames using a mean squared error metric, for example. If the segment descriptor word match identified at block  240  is confirmed by a segment match at block  250 , then, at block  255 , the pre-encrypted segment is substituted for the corresponding segment that has been encrypted in real-time at block  219 . 
     In some implementations the segment comparison performed at block  250  by segment comparator  140  may be eliminated if the segment descriptor word matching performed at block  240  has an adequately high rate of accuracy so that additional confirmation that the two segments being compared are the same is deemed unnecessary. In this case both the segment comparator  140  and the video decoder  160  shown in  FIG. 1  may be eliminated. 
     The recurring material that is encoded off-line in the previously discussed implementations has been described as a segment or scene, which generally consists of one or more consecutive video frames or pictures. In some implementations, however, the material that is encoded off-line may be a sub-set of one or more frames or pictures. For example, it is often the case that only portions of a frame are frequently repeated in a video program. As another example, the background in a newscast is not only frequently repeated, it may appear throughout virtually the entire newscast. Likewise, the background in a sporting event that is recorded by a fixed camera is often repeated multiple times. Similarly, foreground objects or graphics (e.g., a broadcaster&#39;s logo) may also be repeated. 
     The frequently recurring portion of the frame or picture may be separately encoded off-line and then combined with the remainder of the frame or picture, which is encoded in real-time. This may be accomplished, for instance, by treating the frequently recurring portion of the frame or picture as a horizontal slice, which, in the context of video encoding and compression, is a spatially distinct region of a frame or picture that is encoded separately from any other region in the same frame or picture. In such an implementation the recurring segments of the video that are identified and processed off-line are slices. The slices are otherwise treated as a segment of the video in the manner described above. 
       FIG. 3  shows one example of a video encoder  300 . The encoder  300  can implement digital video encoding protocols such as, for example, any one of the Moving Picture Experts Group (MPEG) standards (e.g., MPEG-1, MPEG-2, or MPEG-4) and/or the International Telecommunication Union (ITU) H.264 standard. Additionally, the constituent components of the encoder  300  can be implemented in hardware, software or any combination thereof. The video encoder  300  may be employed as a real-time encoder or an off-line encoder, depending, for example, on the complexity of the algorithms that are employed. For example, when used as a real-time encoder, the encoder  300  may employ single or double pass encoding. On the other hand, when used as an off-line encoder, the encoder  300  may employ more complex multipass encoding techniques that employ, for example, non-causal analysis to optimize quantization decisions. As shown, the input to the encoder  300  on which the video signal is received is connected to a non-inverting input of a summing junction  310 . The output of the summing junction  310  is connected to a transform function block  320  and the transformer  320  is connected to a quantizer  330 . The output of the quantizer  330  is connected to a variable length coder (“VLC”)  340 , where the output of the VLC  340  is an externally available output of the encoder  300 . The output of the quantizer  330  is further connected to an inverse quantizer  350 . The inverse quantizer  350  is connected to an inverse block transform function  360 , which, in turn, is connected to a reference picture store  370 . A first output of the reference picture store  370  is connected to a first input of a motion estimator  380 . The input to the encoder  300  is further connected to a second input of the motion estimator  380 . The output of the motion estimator  380  is connected to a first input of a motion compensator  390 . A second output of the reference picture store  370  is connected to a second input of the motion compensator  390 . The output of the motion compensator  390  is connected to an inverting input of the summing junction  310 . 
       FIG. 4  is a flowchart showing one example of a method for encoding a video signal stream. The method may be implemented by any of a variety of different hybrid encoders, including but not limited to the hybrid encoder shown in  FIG. 1 . The method begins in step  410  by receiving a video signal stream and continues in step  420  by dividing the video signal stream into a plurality of video segments. The video signal stream is encoded in real-time in step  430 . A digital word is assigned to each of the video segments in step  440 . Next, in step  450 , frequently recurring video segments are identified by tabulating the digital words assigned to each of the video segments. The frequently recurring video segments are encoded off-line and stored in step  460 . The video segments encoded off-line which match or correspond to video segments in the video signal stream are identified in step  470  by comparing the digital words assigned to the video segments in the video signal stream with the digital words assigned to the video segments encoded off-line. In step  480 , the corresponding video segments that have been encoded off-line and which have identified in step  470  are substituted for the frequently recurring video segments that have been encoded using real-time encoding. 
     The functions of the various elements shown in the figures above may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage. Other hardware, conventional and/or custom, may also be included in the processor. Such a processor will execute instructions, either at the assembly, compiled or machine-level, to perform the processes described above. Those instructions can be written by one of ordinary skill in the art following the description presented above and stored or transmitted on a computer readable medium. The instructions may also be created using source code or any other known computer-aided design tool. A computer readable medium may be any medium capable of carrying those instructions and include a CD-ROM, DVD, magnetic or other optical disc, tape, or silicon memory (e.g., removable, non-removable, volatile or non-volatile. In addition, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.