Abstract:
A method and system are disclosed. The method includes receiving a video clip including a plurality of frames. The method further includes detecting a telecine cadence of the video clip by comparing a top field and a bottom field of each of the plurality of frames. Further, based on the detected telecine cadence, the method reconstructs the video clip to an original frames-per-second (fps) value.

Description:
FIELD OF THE INVENTION 
     The embodiments of the invention relate generally to the field of video graphics and, more specifically, relate to video processing and rendering. 
     BACKGROUND 
     Digital video is often stored in a different frame rate from an original frame rate. For example, it is common to see video that was originally filmed at a 24 frames-per-second (fps) frame rate, and then before it is written to a digital video disc (DVD) the video is stored at 30 fps (often using the 3:2 cadence pulldown method) to match television (TV) requirements. This process is referred to as telecine cadence. A problem exists in the telecine process. When viewing telecined video on a progressive display monitor, interlacing artifacts repeatedly appear and disappear. This is commonly referred to as the jitter effect. In addition, due to the variety of different telecine cadences and patterns, a problem exists of accurately determining what cadence and pattern was used on a video clip when telecine was preformed. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The invention will be understood more fully from the detailed description given below and from the accompanying drawings of various embodiments of the invention. The drawings, however, should not be taken to limit the invention to the specific embodiments, but are for explanation and understanding only. 
         FIG. 1  illustrates a block diagram of one embodiment of a computer system; 
         FIG. 2  illustrates common telecine film cadences; 
         FIG. 3  illustrates an example of telecine being preformed on a video clip; 
         FIG. 4  illustrates one embodiment of a flow diagram illustrating telecine cadence detection, confirmation and reconstruction; and 
         FIG. 5  illustrates one embodiment of a flow diagram illustrating inverse telecine conversion. 
     
    
    
     DETAILED DESCRIPTION 
     A method and apparatus for telecine cadence detection and restoration is disclosed. In the following detailed description of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention. 
     Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment. 
       FIG. 1  is a block diagram of one embodiment of a computer system  100 . Computer system  100  includes a central processing unit (CPU)  102  coupled to interconnect  105 . In one embodiment, CPU  102  is a processor in the Pentium® family of processors Pentium® IV processors available from Intel Corporation of Santa Clara, Calif. Alternatively, other CPUs may be used. For instance, CPU  102  may be implemented multiple processors, or multiple processor cores. 
     In a further embodiment, a chipset  107  is also coupled to interconnect  105 . Chipset  107  may include a memory control component (MC)  110 . MC  110  may include a memory controller  112  that is coupled to a main system memory  115 . Main system memory  115  stores data and sequences of instructions that are executed by CPU  102  or any other device included in system  100 . 
     In one embodiment, main system memory  115  includes one or more DIMMs incorporating dynamic random access memory (DRAM) devices; however, main system memory  115  may be implemented using other memory types. Additional devices may also be coupled to interconnect  105 , such as multiple CPUs and/or multiple system memories. 
     MC  110  may be coupled to an input/output control component (IC)  140  via a hub interface. IC  140  provides an interface to input/output (I/O) devices within computer system  100 . IC  140  may support standard I/O operations on I/O interconnects such as peripheral component interconnect (PCI), accelerated graphics port (AGP), universal serial interconnect (USB), low pin count (LPC) interconnect, or any other kind of I/O interconnect (not shown). In one embodiment, IC  140  is coupled to a graphics interface card  150 . Graphics interface card  150  includes a graphics processing unit (GPU)  155  and a graphics pixel sampler (GPS)  157 . 
     In one embodiment, graphics interface card  150  is implemented to perform telecine film cadences.  FIG. 2  illustrates common telecine film cadences.  FIG. 3  illustrates an example of telecine increasing the frame rate of a video clip to 24 fps using a 3:2:3:2:2 pulldown (see row  7  of  FIG. 2 ). Referring back to  FIG. 2 , the original progressive video frames are divided into pairs of fields. The even lines of a frame form the top field, and the odd lines form the bottom field. The telecine frame rate conversion is done by creating new frames via replicating and interleaving the original frame fields. 
     The replacing and interleaving is done according to various patterns. 
     Table 1 Illustrates Three Types of Patterns to be Considered: 
     
       
         
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
             
             
               
                   
                 FRP (Field repetition pattern): (2 bits per frame), 1 bit indicates the 
               
               
                   
                 top field of the current frame is a repetition of the previous frame, 
               
               
                   
                 and 1 bit indicates the bottom field. 
               
               
                   
                 ITL (Interlaced or progressive): (1 bit per frame), 1 bit indicates the 
               
               
                   
                 current frame is considered interlaced. 
               
               
                   
                 PFF (Possible TFF (top-field-first) or BFF (bottom-field-first)): (2 bits 
               
               
                   
                 per frame), 1 bit indicates that the current top field and the previous 
               
               
                   
                 bottom field fit into a progressive frame, and 1 bit indicates that the 
               
               
                   
                 previous top field and the current bottom fit into a progressive frame. 
               
               
                   
               
             
          
         
       
     
     As discussed above, current processes suffer from various problems (e.g. jitter effect). According to one embodiment computer system  100  implements a process that eradicates problems associated with the converse telecine process. The process may be implemented as instructions for a driver associated with graphics interface card  150 . In the alternative or in addition to the process may be implemented as an instruction set within graphics interface card  150 . 
       FIG. 4  illustrates one embodiment of a flow diagram  400  for telecine detection and restoration. At processing block  410  a video clip is received. At decision block  420  the video clip is analyzed to determine if telecine has been preformed on the video clip. If telecine has not been preformed on the video clip, the video clip is rendered without any alterations, processing block  490  and the process ends. 
     However, if telecine has been preformed on the video clip, the frames of the video clip are analyzed to determine the active cadence of the video clip, processing block  430 . In one embodiment, the active cadence of the video clip is determined by comparing the frames of the clip. This determination is made within 5 to 10 frames, however other thresholds may be implemented. Each frame may be divided into two fields, a top field and a bottom field. The following are comparisons that may be preformed in order to determine the cadence of the video clip. However, other comparisons may also be preformed. 
     The current top field of the frame may be compared with the current bottom field frame. Further, the current top field may be compared with the previous bottom field, or with the previous top field. In addition, the current bottom field may be compared with the previous bottom filed, or with the previous top field. These comparisons may be preformed in any order and with any level of frequency. Further the comparisons may be limited to only the luma component of the frames. 
     Table 2 Illustrates Examples of the Above Comparisons. 
     
       
         
               
               
             
           
               
                 TABLE 2 
               
               
                   
               
             
             
               
                   
                 DiffcTpT = Diff(current_frame_top_field, 
               
               
                   
                 previous_frame_top_field) 
               
               
                   
                 DiffcBpB = Diff(current_frame_bottom_field, 
               
               
                   
                 previous_frame_bottom_field) 
               
               
                   
                 DiffcTcB = Diff(current_frame_top_field, 
               
               
                   
                 current_frame_bottom_field) 
               
               
                   
                 DiffcTpB = Diff(current_frame_top_field, 
               
               
                   
                 previous_frame_bottom_field) 
               
               
                   
                 DiffcBpT = Diff(current_frame_bottom_field, 
               
               
                   
                 previous_frame_top_field) 
               
               
                   
               
             
          
         
       
     
     In a further embodiment, the comparisons in Table 2 are analyzed for cross-frame variances. By analyzing the comparisons the pattern of the cadence can be determined (e.g. FRP, ITL, or PFF). For example, for a progressive frame the average of the DiffcTcB comparison may be expected to be significantly smaller than the averages of the other crossing field differences. Similarly, the DiffcTpB comparison average may be expected to be small for the TFF (PFF=1) pattern, or the DiffcBpT comparison average may be expected to be small for BFF (PFF=2). In one embodiment, these cross frame variances may be used to determine the cadence used in the video clip. 
     In one embodiment, the comparisons are preformed using a frame difference function, such as mean-square-error, signal-to-noise ratio (PSNR), or sum-absolute-difference (SAD). Nonetheless, other frame difference functions may be used. 
     In one embodiment, in order to reduce computational complexity, an equally distributed sub-sampling pattern may be used as opposed to performing the difference function calculations on all pixels in the frame. The sub-sampling pattern may include portions of the frames that have the greatest amount of change, in order to have an increase in the difference between each frame. Thus, the likelihood of determining the correct active cadence is increased. 
     Referring back to  FIG. 4 , once the active cadence is determined, confirmation of the cadence is attempted, decision block  435 . Confirmation of the cadence includes checking that the active cadence and expected cadence to determine if they are consistent. In one embodiment, consistency makes sure the field repetition pattern (FRP) is preserved as all repeated fields having smaller differences comparing any non-repeated fields within the periodicity. If the confirmation of the cadence fails, the process returns to processing block  430 , and the frames are analyzed again to determine the correct cadence. 
     Once confirmation of the cadence occurs, the cadence is locked and/or maintained, processing block  440 . Once the cadence is locked, inverse-telecine frame restoration (IVTC) is preformed on the video clip, processing block  445 . The type of IVTC performed on the video clip depends on the determined cadence. In one embodiment, if the cadence is determined to be interlaced, then a de-interlacing algorithm may be performed. 
     In a further embodiment, the inverse telecine process is designed into a pipeline of three components: statistics gathering of variance calculations, logic deduction of cadence decision, and memory re-mapping of frame restoration. In one embodiment, the process includes corresponding computer processing of parallel operations, logic operations, and memory operations respectively. However, other restoration processes and algorithms may be performed. 
     At decision block  460 , the active cadence is re-analyzed to determine whether the cadence has changed. If no change in the cadence is detected, then the video clip is analyzed to determine whether there are still more frames to process, decision block  470 . If there are no more frames to process then the process ends. Otherwise, if there are still more frames to process, then the process returns to processing block  430 . 
     If a change in the cadence is detected (decision block  460 ), then the video clip is further analyzed to determine whether frame editing and/or frame shifting of the video clip has occurred, decision block  480 . This determination may include matching possible after-shifting patterns under the same active cadence. After-shifting patterns may include cuts in the video clip, or post filming edits to the video clip. If it is determined that no frame editing and/or shifting has occurred, then the active cadence itself has changed, and a new active cadence is determined, processing block  430 . 
     If editing and/or shifting of the frame has occurred, then the pattern shift is determined, processing block  485 . This determination enables the cadence determination process to be film-cutting-and-editing proof. This means that a false determination of a new cadence may not occur due to an edit or a shift. 
     Once the pattern shift is determined the video clip is analyzed to determine whether there are still more frames to process, decision block  470 . If there are no more frames to process the process ends. Otherwise, if there are still more frames to process, then the process returns to processing block  430 . 
       FIG. 5  illustrates one embodiment of a flow diagram illustrating inverse telecine conversion. At processing block  510 , variances of the current top and bottom fields and previously stored fields are calculated. In one embodiment, these calculations are preformed by GPU  155 . At processing block  520 , the telecine cadence is detected. In one embodiment, this detection is preformed by CPU  102 . At processing block  530 , GPS  157  pairs the correct fields of each frame with a frame buffer  540 . 
     The above-described process and mechanism detects telecined frame rate conversion on video clips and restores the video clips to the original frame rate in order to improve progressive display. 
     The various embodiments of the invention set forth above may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general-purpose or special-purpose processor or a machine or logic circuits programmed with the instructions to perform the various embodiments. Alternatively, the various embodiments may be performed by a combination of hardware and software. 
     Various embodiments of the invention may be provided as a computer program product, which may include a machine-readable medium having stored thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process according to various embodiments of the invention. The machine-readable medium may include, but is not limited to, floppy diskette, optical disk, compact disk-read-only memory (CD-ROM), magneto-optical disk, read-only memory (ROM) random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical card, flash memory, or another type of media/machine-readable medium suitable for storing electronic instructions. Moreover, various embodiments of the invention may also be downloaded as a computer program product, wherein the program may be transferred from a remote computer to a requesting computer by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection). 
     Whereas many alterations and modifications of the present invention will no doubt become apparent to a person of ordinary skill in the art after having read the foregoing description, it is to be understood that any particular embodiment shown and described by way of illustration is in no way intended to be considered limiting. Therefore, references to details of various embodiments are not intended to limit the scope of the claims, which in themselves recite only those features regarded as the invention.