Patent Publication Number: US-2010124276-A1

Title: Method and apparatus for detecting video field sequence

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application claims priority to Chinese Patent Application No. 200810217805.6 filed on Nov. 18, 2008, which is hereby incorporated by reference in its entirety. 
     TECHNICAL FIELD 
     The present disclosure relates to video technology, and in particular, to a method and apparatus for detecting a video field sequence and a video processing system. 
     BACKGROUND 
     Video streams exist in the form of frames after being restored to picture sequences by a decoder. If a video source is interlaced or is formed through field interleaving of a progressive source, for example, through frame rate conversion, two fields must be displayed in a same sequence as the video source when a frame is displayed on an interlaced device; otherwise, obvious high-frequency jitters may occur, which affects the subjective feeling. If a stream does not include field sequence information or the included field sequence information is unreliable, when the video is displayed on the interlaced device, the field displaying sequence can be determined only by other means, for example, by judging whether the top field is displayed first or the bottom field is displayed first. 
     A method for determining the field sequence in the prior art uses a general practice in the industry. For example, for Phase Alternating Line (PAL) streams, the top field is displayed first; for National Television System Committee (NTSC) streams, the bottom field is displayed first. Another method in the prior art uses the encoding and decoding information to reversely deduce the field sequence. For example, a picture order count (POC) reflects the time sequence of the picture over the H.264 protocol. If an interlaced source is encoded by field coding, the POCs of two fields of a frame also reflect the displaying sequence of the interlaced source. Thus, the POC may be used to deduce the field sequence. 
     SUMMARY 
     A method for detecting a video field sequence according to an embodiment of the present disclosure includes: 
     calculating a difference D 1  between a top field T n  of a current frame and a bottom field B n-1  of a previous frame and a difference D 2  between a bottom field B n  of the current frame and a top field T n-1  of the previous frame; and 
     determining a displaying sequence of the top field and bottom field of the current frame by comparing D 1  and D 2 . 
     An apparatus for detecting a video field sequence according to an embodiment of the present disclosure includes: 
     a difference calculating module, configured to calculate a difference D 1  between a top field T n  of a current frame and a bottom field B n-1  of a previous frame and a difference D 2  between a bottom field B n  of the current frame and a top field T n-1  of the previous frame; and 
     a field sequence detecting module, configured to determine a displaying sequence of the top field and bottom field of the current frame by comparing D 1  and D 2 . 
     A video processing system according to an embodiment of the present disclosure includes: 
     a decoding module, configured to decode a video stream into multiple frames; 
     a difference calculating module, configured to calculate a difference D 1  between a top field Tn of a current frame and a bottom field Bn−1 of a previous frame and a difference D 2  between a bottom field Bn of the current frame and a top field Tn−1 of the previous frame; 
     a field sequence detecting module, configured to determine a displaying sequence of the top field and bottom field of the current frame by comparing D 1  and D 2 ; and 
     a displaying and outputting module, configured to output the top field and bottom field data of the current frame for displaying according to the displaying sequence determined by the field sequence detecting module. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The accompanying drawings illustrate some embodiments of the present disclosure. Evidently, the accompanying drawings are exemplary, and those skilled in the art can derive other drawings from the accompanying drawings without creative work. 
         FIG. 1  is a flowchart of a method for detecting a video field sequence according to an embodiment of the present disclosure; 
         FIG. 2  shows a structure schematics of an apparatus for detecting a video field sequence according to an embodiment of the present disclosure; 
         FIG. 3  shows a structure schematics of an apparatus for detecting a video field sequence according to another embodiment of the present disclosure; and 
         FIG. 4  shows a structure schematics of a video processing system according to an embodiment of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments of the present disclosure are hereinafter described in detail with reference to the accompanying drawings. It is evident that the embodiments are only exemplary embodiments of the present disclosure and the present disclosure is not limited to such embodiments. Other embodiments that those skilled in the art obtain based on embodiments of the present disclosure may also fall in the scope of protection of the present disclosure. 
     By using general practice to determine the field sequence in the prior art, most streams may be displayed properly even if no field sequence information is available. This practice is of some value with respect to being a reference but is not a mandatory regulation. There are still a large quantity of streams does not comply with this practice, and jitters may occur when these streams are displayed. 
     The other method mentioned above in the prior art is based on the assumptions that the interlaced source use field coding and that the POC values of the fields increase in a same sequence as the displaying sequence. Considering the general practice and encoding efficiency, the preceding assumptions are reasonable. However, no regulation specifies that the encoder must comply with the assumptions. Thus, this method may be implemented only to some extent, but for a lot of streams a correct field sequence cannot be obtained through this method. 
     The two methods for determining the field sequence in prior art are not reliable enough and may often cause misjudgment. In case of misjudgment, the subjective quality of videos may be seriously affected. 
     A method for detecting a video field sequence provided in an embodiment of the present disclosure is not based on any assumption. In this method, a difference between the top field and the bottom field of two adjacent frames is calculated according to the video content, and then the displaying sequence of the video frames is determined according to the difference; for example, the top field or the bottom field is determined to be displayed first. Thus, this method ensures that a correct displaying sequence may be obtained for videos encoded in various forms. A frame of a video consists of two fields. The displaying sequence or output sequence of the two fields is called a field sequence. 
       FIG. 1  is a flowchart of a method for detecting a video field sequence according to an embodiment of the present disclosure. The method includes the following steps: 
     S 1 : A difference D 1  between the top field of a current frame and the bottom field of a previous frame and a difference D 2  between the bottom field of the current frame and the top field of the previous frame are calculated. 
     The difference is used to measure the inconsistency between the two fields due to motions. For an interlaced video, each frame consists of a top field and a bottom field, where the top field is formed by odd lines (for example, lines 1, 3, 5 . . . ) and the bottom field is formed by even lines (for example, lines 2, 4, 6 . . . ). A time difference generally exists between the top field and the bottom field of a same frame; that is, either the top field is displayed first or the bottom field is displayed first. Similarly, among fields that are displayed by time in two adjacent frames, the difference between two fields with a closer time distance is smaller, while the difference between two fields with a farther time distance is bigger. For example, if f 1 , f 2 , f 3  and f 4  are fields displayed by time, the difference between f 2  and f 3  is generally small, while the difference between f 1  and f 4  is large. 
     A lot of methods may be used to represent the difference. For example, in a picture frame formed by interleaving two fields, if a pixel and its adjacent upper and lower pixels belong to different objects, the pixel is called an interlacing point. The more the interlacing points are, the larger the difference between the two fields is, and vice versa. Suppose: a decoded current frame is represented by F n ; the previous frame is represented by F n-1 ; the top field of F n-1  is represented by T n-1 ; the bottom field of F n-1  is represented by B n-1 ; similarly, the top field and bottom field of F n  are represented by T n  and B n . 
     The process of calculating D 1  between T n  and B n-1  includes: calculating the number N 1  of interlacing points of picture frame formed by interleaving T n  and B n-1 . 
     The process of calculating D 2  between B n  and T n-1  includes: calculating the number N 2  of interlacing points of picture frame formed by interleaving B n  and T n-1 . 
     Another method for representing the difference is to calculate the Sum of Absolute Difference (SAD) between two fields. The method for representing the difference by using the SAD is described in detail as follows. 
     First, a top field representative point set and a bottom field representative point set of a picture frame are defined. A frame is formed by two fields (a top field and a bottom field) that are mutually interleaved. Suppose N points are selected in the top field to form the top field representative point set, as shown in formula (1): 
         S   t ={( x   tk   ,y   tk )|0 ≦x   tk   &lt;w, 0 ≦y   tk   &lt;h− 2 ,y   tk =an even number,  k= 0,1 , . . . N −1}  (1) 
     Similarly, N points corresponding to the points of S t  are selected in the bottom field to form the bottom field representative point set, as shown in formula (2): 
         S   b ={( x   bk   ,y   bk )| x   bk   =x   tk   ,y   bk   =y   tk +1 ,k= 0,1 , . . . N −1}  (2) 
     In formulas (1) and (2), (x bk , y bk ) indicates a bottom field pixel coordinate; “w” indicates the width of the picture frame; and “h” indicates the height of the picture frame. The origin of the coordinates is located at the upper left corner of the picture frame; the right direction indicates the x axis, and the top-down direction indicates the y axis. In the top field representative set, “y tk ” is an even number, representing picture lines that have the y-axis values such as y tk =0, 2, 4, 6 . . . , which correspond to the odd lines (i.e. lines 1, 3, 5 . . . , h−2) of the picture frame. Similarly, in the bottom field representative set, “y bk =y tk +1” represents picture lines that have the y-axis values such as y bk =1, 3, 5 . . . , which correspond to the even lines (that is, lines 2, 4, 6 . . . h−1) of the picture frame. Suppose that the height of the picture frame is eight pixels, namely, eight lines, and that the y-axis value is equal to 0, 1, 2, 3, 4, 5, 6, 7, and 8. The picture lines corresponding to the even values of y=0, 2, 4, 6, 8 form the top field, while the picture lines corresponding to the odd values of y=1, 3, 5, 7 form the bottom field. 
     The selection of the top field representative point set S t  and the bottom field representative point set S b  usually is subject to selecting pixels that can cover a Region Of Interest (ROI) of a picture, where the number N of pixels in the representative point set is an integer greater than 1. As for the judgment of each frame, the number and location of pixels in the top field representative set and the bottom field representative set may be changed as occasion requires or according to the picture content. The point set capacity N may be adjusted according to allowable quantity of calculations. If the value of N is large, a more accurate result may be obtained, but a large quantity of calculations is required. 
     Suppose: a decoded current frame is represented by F n , and the previous frame is represented by F n-1 ; the top field of F n-1  is represented by T n-1 , and the bottom field of F n-1  is represented by B n-1 ; the top and bottom fields of F n  are represented by T n  and B n ; the luminance values of N pixels of F n-1  in S t  are represented by c tk , and the luminance values of N pixels in S b  are represented by c bk ; similarly, the luminance values of pixels of F n  in S t  and S b  are represented by f tk  and f bk  respectively, where k=0, 1, 2, . . . N−1. 
     The calculations are defined as follows: 
     
       
         
           
             
               
                 
                   
                     D 
                      
                     
                         
                     
                      
                     1 
                   
                   = 
                   
                     
                       SAD 
                        
                       
                           
                       
                        
                       1 
                     
                     = 
                     
                       
                          
                         
                           
                             B 
                             
                               n 
                               - 
                               1 
                             
                           
                           - 
                           
                             T 
                             n 
                           
                         
                          
                       
                       = 
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             0 
                           
                           
                             N 
                             - 
                             1 
                           
                         
                          
                         
                            
                           
                             
                               c 
                               bk 
                             
                             - 
                             
                               f 
                               tk 
                             
                           
                            
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
             
               
                 
                   
                     D 
                      
                     
                         
                     
                      
                     2 
                   
                   = 
                   
                     
                       SAD 
                        
                       
                           
                       
                        
                       2 
                     
                     = 
                     
                       
                          
                         
                           
                             T 
                             
                               n 
                               - 
                               1 
                             
                           
                           - 
                           
                             B 
                             n 
                           
                         
                          
                       
                       = 
                       
                         
                           ∑ 
                           
                             k 
                             = 
                             0 
                           
                           
                             N 
                             - 
                             1 
                           
                         
                          
                         
                            
                           
                             
                               c 
                               tk 
                             
                             - 
                             
                               f 
                               bk 
                             
                           
                            
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
     Where |B n-1 -T n | indicates the difference between B n-1  and T n . Similarly, |T n-1 -B n | indicates the difference between T n-1  and B n . 
     S 2 : The displaying sequence of the top field and bottom field of the current frame is determined by comparing D 1  and D 2 . 
     Because fields of the video picture frame are samples of objective scenarios, the longer the time distance is, the bigger the content difference between the fields will be, and vice versa. If the field sequence is “top field first”, that is, if the top field is to be displayed before the bottom field in a same frame, the four fields of F n-1  and F n  are displayed in the following sequence: T n-1 , B n-1 , T n , and B n . Thus, the difference D 1  between B n-1  and T n , should be smaller than the difference D 2  between T n-1  and B n . In other words, the similarity between B n-1  and T n  should be greater than that between T n-1  and B n , as illustrated by the following formula: 
     If the top field is displayed first, then: 
         P (| T   n-1   −B   n   |−|B   n-1   −T   n |&gt;0)&gt;0.5  (5) 
     Formula (5) is equivalent to P (D 2 &gt;D 1 )&gt;0.5, where P(X) indicates the probability of X. 
     Similarly, if the field sequence is “bottom field first”, that is, if the bottom field is displayed before the top field in a same frame, the four fields of F n-1  and F n  are displayed in the following sequence: B n-1 , T n-1 , B n , and T n . Thus, D 1  between B n-1  and T n  should be greater than D 2  between T n-1  and B n . In other words, the similarity between B n-1  and T n  should be smaller than that between T n-1  and B n , as illustrated by the following formula:: 
     If the bottom field is displayed first, then: 
         P (| T   n-1   −B   n   |−|B   n-1   −T   n |&lt;0)&lt;0.5  (6) 
     Formula (6) is equivalent to P (D 2 &gt;D 1 )&gt;0.5, where P(X) indicates the probability of X. 
     In conclusion, the field sequence is related to the cross difference between the top field and the bottom field of two adjacent frames. Thus, the field sequence can be determined by measuring and comparing the cross difference. A method for determining the field sequence includes: 
     determining that the displaying sequence of the current frame is “top field first” (that is, top field of the current frame is to be displayed first), if the difference D 1  between T n  and B n-1  is smaller than the difference D 2  between B n  and T n-1 ; or 
     determining that the displaying sequence of the current frame is “bottom field to be displayed first” (that is, bottom field of the current frame is to be displayed first), if the difference D 1  between T n  and B n-1  is greater than the difference D 2  between B n , and T n-1 . 
     If D 1  between T n  and B n-1  is equal to D 2  between B n  and T n-1 , the field sequence of the previous frame may be used as the field sequence detection result of the current frame, or the field sequence may be determined according to the general practice in the industry. 
     If the difference is represented by SAD, D 1  and D 2  may be SAD 1  and SAD 2 . If the difference is represented by the number of interlacing points, D 1  and D 2  may be N 1  and N 2  mentioned before. 
     The difference mentioned in the embodiments of the present disclosure is calculated based on video contents. The field displaying sequence of the current frame is determined according to the difference between fields of two adjacent frames, without depending on any assumption or guess, thus greatly improving the accuracy and reliability of the field sequence detection. 
     The preceding formulas (5) and (6) are derived from the perspective of probabilities. In most cases, video frames comply with such probabilities described in formulas (5) and (6). However, in actual picture sequences, there may be some uncertainties for two frames due to influences of various factors. For example, a single frame may be affected by scenario changes and encoding noises, and thus does not comply with the probabilities. To further improve the accuracy and reliability of the field sequence detection, the difference among multiple frames may be calculated, and an overall rule may be obtained from the calculating result of each frame. 
     Specifically, the following calculation may be executed after D 1  and D 2  are calculated: SumDiff=SumDiff+(D 2 −D 1 ). That is, the value of D 2  minus D 1  is added to a predetermined variable “SumDiff”, the initial value of which may be set to 0 or other numerical values. After calculating results of multiple frames are accumulated, the calculating results are used as the basis for detecting the field sequence of latter frames. A threshold “Thr” may be set to determine the field sequence according to the following conditions: 
     if SumDiff&gt;Thr, it is determined that the displaying sequence of the current frame is “top field first”; or 
     if SumDiff&lt;(−1)*Thr, it is determined that the displaying sequence of the current frame is “bottom field first”; or 
     if “SumDiff” does not meet the preceding two conditions, it is determined that the displaying sequence of the current frame is the same as that of the previous frame, or the field sequence may be determined according to the general practice in the industry; 
     where “Thr” is a predetermined threshold. The preceding accumulated value may also be D 1  minus D 2  (D 1 −D 2 ), but the judgment condition needs to be changed accordingly. 
     The following provides an embodiment in which multiple frames are calculated successively; the difference is still represented by SAD; and an accumulator “SadDiffAcc” with the initial value 0 is defined. The following calculations are executed on each decoded frame: 
         SadDiffAcc=SadDiffAcc +(SAD2−SAD1), that is 
         SadDiffAcc=SadDiffAcc +(| T   n-1   −B   n   |−|B   n-1   −T   n |) 
     Then, the condition for determining that the video field sequence is “top field first” is changed as follows: 
       SadDiffAcc&gt;thr  (7) 
     Contrarily, the condition for determining that the video field sequence is “bottom field first” is changed as follows: 
         SadDiffAcc &lt;(−1)* thr   (8) 
     wherein “thr” is a predetermined threshold. 
     A specific example is given below. Suppose: the initial value of “SadDiffAcc” is 0; thr=80; and there are four frames, namely, F 1 , F 2 , F 3 , and F 4 ; the cross difference between F 1  and F 2  is 100; the cross difference between F 2  and F 3  is −150; and the cross difference between F 3  and F 4  is −10. 
     For F 2 , if SadDiffAcc=0+100=100&gt;80, the field sequence of F 2  is “top field first”. 
     For F 3 , if SadDiffAcc=100+(−150)=−50, which falls in the range of −80 to 80, the field sequence of F 3  is the same as that of F 2 , or the field sequence is determined according to the general practice in the industry. 
     For F 4 , if SadDiffAcc=−50+(−40)=−90&lt;−80, the field sequence of F 4  is “bottom field first”. 
     If the difference is represented by the number of interlacing points, D 1  and D 2  only need to be changed to the preceding N 1  and N 2 , while other steps are similar. 
     In this embodiment, the cross difference between the top field and the bottom field of two adjacent frames is accumulated, which can restrain the uncertainty of a single frame. Thus, the calculating results that reflect the overall rule may appear as more and more frames are calculated, which can be used to improve the accuracy and reliability of the field sequence detection. 
     In conclusion, in this embodiment, the field sequence information is extracted from the content of the video picture, which fundamentally solves the problem of inability to detect the field sequence reliably. In addition, the field sequence detection method in this embodiment does not destroy the video motion information, nor does it interrupt the playing continuity of the video. Furthermore, the calculation overheads may be controlled flexibly and the quantity of calculations may be minimized. 
     It is understandable to those skilled in the art that all or part of the processes of the method in the preceding embodiments of the present disclosure may be completed by hardware instructed by a computer program. The program may be stored in a computer readable storage medium, and may include processes of the preceding method during execution. The storage medium may be a magnetic disk, a compact disk (CD), a read-only memory (ROM) or a random access memory (RAM). One embodiment of the present disclosure provides a 
     As shown in  FIG. 2 , an apparatus for detecting a video field sequence according to an embodiment of the present disclosure includes: 
     a difference calculating module  21 , configured to calculate a difference D 1  between the top field Tn of the current frame and the bottom field Bn−1 of the previous frame and a difference D 2  between the bottom field Bn of the current frame and the top field Tn−1 of the previous frame; and 
     a field sequence detecting module  22 , configured to determine the displaying sequence of the top field and bottom field of the current frame by comparing D 1  and D 2  obtained by the difference calculating module  21 . 
     The field sequence detecting module  22  may include: 
     a first judging unit  221 , configured to judge the difference between D 1  and D 2 ; and 
     a first determining unit  222 , configured to determine that the displaying sequence of the current frame is “top field first” if the first judging unit determines that D 1  is smaller than D 2 , or determine that the displaying sequence of the current frame is “bottom field first” if the first judging unit  221  determines that D 1  is greater than D 2 . 
     The apparatus for detecting a video field sequence in this embodiment calculates a difference according to video contents, and determines the field displaying sequence of the current frame according to the difference between fields of two adjacent frames, without depending on any assumption or guess, thus greatly improving the accuracy and reliability of the field sequence detection. 
       FIG. 3  shows a structure of an apparatus for detecting a video field sequence according to another embodiment of the present disclosure. The apparatus includes: 
     a difference calculating module  31 , configured to calculate a difference D 1  between the top field T n  of the current frame and the bottom field Bn−1 of the previous frame and a difference D 2  between the bottom field Bn of the current frame and the top field Tn−1 of the previous frame; 
     a field sequence detecting module  32 , configured to determine the displaying sequence of the top field and bottom field of the current frame by comparing D 1  and D 2  obtained by the calculating module  31 ; and 
     a selecting module  33 , configured to: select N points in the top field and bottom fields of the current frame respectively to form a top field representative point set S t  and a bottom field representative point set S b  of the current frame, where N points of the S t  correspond to N points of the S b ; and select N points corresponding to the current frame in the top field and bottom field of the previous frame respectively to form a top field representative point set S t  and a bottom field representative point set S b  of the previous frame, where N is an integer greater than 1. 
     The difference calculating module  31  includes: 
     a first SAD calculating unit  311 , configured to calculate SAD 1  between the top field representative set of the current frame and the bottom field representative set of the previous frame; 
     a second SAD calculating unit  312 , configured to calculate SAD 2  between the bottom field representative set of the current frame and the top field representative set of the previous frame. 
     If the difference is represented by the number of interlacing points, each of which indicates a pixel point belonging to an object different from that of its adjacent upper and lower pixels in a picture frame obtained by interleaving two fields, the difference calculating module  31  is configured to: calculate the number N 1  of interlacing points of a picture frame formed by interleaving T n  and B n-1  as D 1 , and calculate the number N 2  of interlacing points of a picture frame formed by interleaving B n , and T n-1  as D 2 . 
     The field sequence detecting module  32  includes: 
     an accumulating module  320 , configured to add the difference value between D 2  obtained by the difference calculating module  31  and D 1  obtained by the difference calculating module  31  to the value of a predetermined variable “SumDiff”; 
     a second judging unit  321 , configured to judge the difference between “SumDiff” obtained by the accumulating module  320  and a predetermined threshold “Thr”; and 
     a second determining unit  322 , configured to: determine that the displaying sequence of the current frame is “top field first” when the second judging unit  321  determines that “SumDiff” is greater than “Thr”, or determine that the displaying sequence of the current frame is “bottom field first” when the second judging unit  321  determines that “SumDiff” is smaller than (−1)*Thr, or determine that the displaying sequence of the current frame is the same as that of the previous frame when the second judging unit  321  determines that “SumDiff” does not meet the preceding two conditions. 
     In this embodiment, the cross difference between the top field and the bottom field of two adjacent frames is accumulated, which can restrain the uncertainty of a single frame. Thus, the calculating results reflecting the overall rule may appear as more and more frames are calculated, which can be used to further improve the accuracy and reliability of the field sequence detection. 
     As shown in  FIG. 4 , a video processing system according to an embodiment of the present disclosure includes: 
     a decoding module  41 , configured to decode a video stream into multiple frames; 
     a difference calculating module  42 , configured to calculate a difference D 1  between the top field Tn of the current frame and the bottom field Bn−1 of the previous frame and a difference D 2  between the bottom field Bn of the current frame and the top field Tn−1 of the previous frame, where the current frame and previous frame are obtained by the decoding module  41 ; 
     a field sequence detecting module  43 , configured to determine the displaying sequence of the top field and bottom field of the current frame by comparing D 1  and D 2 ; and 
     a displaying and outputting module  44 , configured to output the top field and bottom field data for displaying according to the displaying sequence determined by the field sequence detecting module  43 . 
     The difference calculating module  42  and the field sequence detecting module  43  are similar to those in the preceding apparatus embodiment. 
     In the video processing system in this embodiment of the present disclosure, the decoding module  41  obtains multiple frames by encoding a video data; the difference calculating module  42  calculates field differences by using the contents of two adjacent frames obtained by encoding; the field sequence detecting module  43  determines the displaying sequence of the current frame according to the field differences; and the displaying and outputting module  44  displays the fields of the frames according to the determined field sequence. Thus, the problem of inability to detect the field sequence reliably is solved, and the subjective quality of pictures is improved. 
     The method for detecting field sequence in present disclosure may be implemented by the apparatus provided in present disclosure. The apparatus may be a decoder, and the video processing system may include the decoder. 
     The present disclosure has been described through some exemplary embodiments and accompanying drawings. However, the scope of the disclosure should not be limited to such embodiments. It is apparent that those skilled in the art can make various modifications and variations to the disclosure without departing from the spirit and scope of the disclosure.