Abstract:
An adaptive blending MCI method and device thereof are disclosed in the embodiments of the present invention. The adaptive blending MCI method uses adjacent four motion vectors to get the corresponding pixels, and uses linear interpolation equation to blend four pixels to reduce block artifacts. The method uses adaptive weighting coefficient to favor reliable motion vector to avoid bad motion vector degrade image quality.

Description:
BACKGROUND OF THE INVENTION 
     (a) Field of the Invention 
     The invention relates to a frame rate up-conversion (FRC) technique, particularly to a device and a method for adaptive blending motion compensation interpolation (MCI). 
     (b) Description of the Related Art 
     The frame rate up-conversion (FRC) technique is to increase the frame rate of a video source and has been applied in various fields, such as low-bit network video transmission for bandwidth-saving, converting a video source with a frame rate of 25 Hz into a higher frame rate for reducing frame juddering, and applying in hold-type liquid crystal displays (LCD) for avoiding frame blurring and achieving clear image quality. 
     Most of the frame rate up-conversion (FRC) techniques use motion estimation (ME) to calculate the motion vector of an object and perform motion compensation interpolation (MCI) to allocate the moving object in different frames. Most of the motion estimation techniques utilize blocks in calculation. One of main purposes of a motion estimation technique is to reduce block artifacts caused by block interpolation. The prior technique uses a method of blending adjacent blocks to reduce block artifacts but generally such a method cannot differentiate the correctness of the adjacent blocks. Although some methods select adjacent blocks based on the reciprocal of the sum of absolute difference (SAD), the calculation is too complicate to assure the correctness of the adjacent blocks. 
     BRIEF SUMMARY OF THE INVENTION 
     One embodiment of the invention provides a motion compensation interpolation device, comprising an adaptive weighting unit and a blending unit. The adaptive weighting unit receives at least a frame where the frame comprises a plurality of blocks and allocates an interpolation block in a predetermined number of blocks where the interpolation block comprises at least one target pixel to be interpolated. The adaptive weighting unit receives a plurality of vectors corresponding to a plurality of adjacent blocks of the interpolation block, checks these vectors, and calculates a plurality of weighting values based on at least one vector that passes the check. The blending unit receives the corresponding pixel values of the vectors and generates the pixel value of the target pixel to be interpolated according to the weighting values and the pixel values. 
     One embodiment of the invention provides motion compensation interpolation method, comprising the following steps. At first, an interpolation block is allocated in a frame where the interpolation block comprises at least one target pixel to be interpolated. The reliability of the vectors corresponding to the adjacent blocks of the interpolation block is checked. Then, at least one preferred vector having high reliability is selected and a plurality of weighting values are generated based on the at least one preferred vector. The corresponding pixel values of the vectors are received and the pixel value of the target pixel to be interpolated is determined according to the weighting values and the pixel values. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1A  shows a schematic diagram illustrating the imaging generating device according to one embodiment of the invention. 
         FIG. 1B  shows a schematic diagram illustrating the block interpolation in a frame according to one embodiment of the invention. 
         FIG. 1C  shows a schematic diagram illustrating the block interpolation in a frame according to another embodiment of the invention. 
         FIG. 1D  shows a schematic diagram illustrating the block interpolation in a frame according to another embodiment of the invention. 
         FIG. 2A  shows a schematic diagram illustrating the adaptive blending motion compensation interpolation device according to one embodiment of the invention. 
         FIG. 2B  shows a schematic diagram illustrating the adaptive weighting unit according to one embodiment of the invention. 
         FIG. 3  shows a schematic diagram illustrating the block interpolation process in a frame according to another embodiment of the invention. 
         FIG. 4A  shows a schematic diagram illustrating the vector weighting determination process according to one embodiment of the invention. 
         FIG. 4B  shows a schematic diagram illustrating the vector weighting determination process according to another embodiment of the invention. 
         FIG. 5  shows a flow chart illustrating the adaptive blending motion compensation interpolation method according to one embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     The following describes several preferred embodiments of the inventions, such as various electric circuits, elements, and related methods. Those who are skilled in the art should understand that the invention can be implemented by various methods, not limited by the following embodiments or characteristics in the embodiments. The well-known details will not be repeatedly described to avoid distracting the attention from the key point of the invention. 
     Furthermore, the image inputted into the image generating device, the adaptive blending MCI device and the method thereof according to the invention can be a frame or a field. The following examples are described by using a frame. 
       FIG. 1  shows a structural block diagram illustrating the image generating device according to one embodiment of the invention. Referring to  FIG. 1 , the image generating device  10  according to the invention comprises a motion vector estimation circuit  11  and a motion compensation interpolation circuit  12 . 
     The operating method of the motion vector estimation circuit  11  is to divide the original inputted frame into a plurality of blocks where each block comprises N*M points (for example, 8*8 or 8*16) and then perform motion estimation (ME) to search for a position having the highest similarity in a to-be-matched frame (for example, the preceding frame). Thus, the relative position is the motion vector corresponding to this block. Then, searching for each corresponding vector of each block can generate all the vectors between two adjacent frames. 
     In one embodiment, the motion vector estimation circuit  11  receives a current frame F 1  and a preceding frame F 0  and divides the current frame F 1  and the preceding frame F 0  into the same number of blocks. The motion vector estimation circuit  11  compares a target block with a predetermined search area of the preceding frame F 0  to calculate a plurality of motion estimation differences. By comparing the motion estimation differences, the motion vector estimation circuit  11  obtains the motion vector of the target block and the plurality of motion estimation differences corresponding to the target block. By repeating the above process, the whole frame is processed and then the motion vector estimation circuit  11  can generate all the vectors MV between the adjacent two frames. 
     It should be noted that the invention is not limited to this example. In another embodiment, a target block B i  is compared with a predetermined search area of the current frame F 1  to calculate a plurality of motion estimation differences and the motion vector MV of the target block. 
     The motion compensation interpolation circuit  12  performs compensation based on these motion vectors MV and interpolates the frame to generate at least one output frame. When the motion compensation interpolation circuit  12  according to one embodiment of the invention is in operation, a detecting block DB with predetermined size in the frame is used to perform interpolation. For example, as shown in  FIGS. 1B ,  1 C, and  1 D, the motion compensation interpolation circuit  12  in  FIG. 1B  uses four blocks, forming a criss-cross shape, as the detecting block DB in processing. After processing, the position is shifted right to the position shown in  FIG. 1C  to proceed the following process. Certainly, the invention is not limited to the case of using four blocks forming a criss-cross shape. A different number of blocks can also be used in processing. Furthermore, the sequence of moving directions of the detecting block can be adjusted when needed. 
     As shown in  FIG. 1D , in the detecting block DB, the four blocks forming a criss-cross shape are the above mentioned blocks and each block corresponds to one vector where four blocks in the figure correspond to the vectors MV 1 , MV 2 , MV 3 , and MV 4 . The center hatched block is the interpolation block IB to be interpolated. The interpolation block IB also comprises N*M points. The distance between the interpolation block IB and the upper-left block is half of a block and there are four adjacent vectors MV 1 , MV 2 , MV 3 , and MV 4  for every target pixel (x,y) to be interpolated in the interpolation block IB. According to the four vectors, the corresponding pixels P 1 , P 2 , P 3 , and P 4  in the preceding or subsequent frame can be found. 
     Assuming the distance between the target pixel (x,y) and the upper-left point of the interpolation block IB is (fx,fy), the output pixel value Pout is as follows: 
                       P   12     =         (     1   -     Δ   ⁢           ⁢   fx       )     ×     P   1       +     Δ   ⁢           ⁢   fx   ×     P   2           ⁢     
     ⁢       P   34     =         (     1   -     Δ   ⁢           ⁢   fx       )     ×     P   3       +     Δ   ⁢           ⁢   fx   ×     P   4           ⁢     
     ⁢             P   out     =       ⁢         (     1   -     Δ   ⁢           ⁢   fy       )     ×     P   12       +     Δ   ⁢           ⁢   fy   ×     P   34                     =       ⁢         (     1   -     Δ   ⁢           ⁢   fy       )     ⁢     (     1   -     Δ   ⁢           ⁢   fx       )     ⁢     P   1       +       (     1   -     Δ   ⁢           ⁢   fy       )     ⁢     (     Δ   ⁢           ⁢   fx     )     ⁢     P   2       +                     ⁢         (     Δ   ⁢           ⁢   fy     )     ⁢     (     1   -     Δ   ⁢           ⁢   fx       )     ⁢     P   3       +       (     Δ   ⁢           ⁢   fy     )     ⁢     (     Δ   ⁢           ⁢   fx     )     ⁢     P   4                     =       ⁢         W   1     ⁢     P   1       +       W   2     ⁢     P   2       +       W   3     ⁢     P   3       +       W   4     ⁢     P   4                         ⁢     (         W   1     +     W   2     +     W   3     +     W   4       =   1     )                     (   1.1   )               
where Pn is the pixel value corresponding to the vector MVn of the point and n=1˜4; Δfx is the value (fx) of the horizontal distance in the hatched interpolation block IB divided by the length of the interpolation block IB; and Δfy is the value (fy) of the vertical distance in the hatched interpolation block IB divided by the length of the interpolation block IB.
 
     For example, it is assumed that the adjacent four vectors are (0,0), (0,0), (6,0), and (6,0) and the pixel values corresponding to the target pixel (x,y) to be interpolated are P 1 =100, P 2 =200, P 3 =300, and P 4 =400. It is also assumed that fx=4 and fy=8 for the target pixel (x,y) to be interpolated and the length and width of the interpolation block IB are 16*16. Thus, the output pixel value Pout is that Pout=(1−8/16)*(1−4/16)*100+(1−8/16)*(4/16)*200+(8/16)*(1−4/16)*300+(8/16)*(4/16)*400=225. 
     By this way, repeating the operation can calculate the output pixel value Pout for every pixel in the interpolation block IB and thereby calculate all the pixel values for the whole interpolated frame. However, not all the vectors of the adjacent blocks are highly reliable. Thus, it is required to assign weighting values to each vector based on the reliability of the vector. 
       FIG. 2A  shows a schematic diagram illustrating the adaptive blending motion compensation interpolation device  12  according to one embodiment of the invention. The interpolation device  12  comprises an adaptive weighting unit  121  and a blending unit  122 . The adaptive weighting unit  121  receives at least one adjacent vector MV of an interpolation block IB where the interpolation block IB comprises one target pixel (x,y) to be interpolated and at least one weighting value W is generated after processing. The blending unit  122  receives at least one pixel value P corresponding to the at least one adjacent vector MV and processes at least one pixel value P based on the at least one weighting value W so as to generate the output pixel value Pout of the target pixel. 
     In the example shown in  FIG. 2A , the adaptive weighting unit  121  receives four adjacent vectors MV 1 , MV 2 , MV 3 , and MV 4  of an interpolation block IB where the interpolated position of the interpolation target pixel (x,y) is (fx,fy). The adaptive weighting unit  121  calculates the preferred vector MV in the four adjacent vectors and processes to generate the weighting values W 1 , W 2 , W 3 , and W 4  provided to the blending unit  122  for the subsequent process. 
     The blending unit  122  receives the corresponding pixel values P 1 , P 2 , P 3 , and P 4  of the four adjacent vectors and processes the pixel values P 1 , P 2 , P 3 , and P 4  based on the weighting values W 1 , W 2 , W 3 , and W 4  to generate the output pixel value OP of the interpolation target pixel. 
     It should be noted that the adaptive weighting unit  121  can output a proper weighting value W in order to have the result inclined to the vector MV having higher reliability. For example, if the reliability of the vector MV 1  is higher than that of the other vectors, the weighting value W 1  of the pixel value P 1  corresponding to the vector should be increased. 
     For example, the following device and method can be used to determine the reliability of a vector and calculate the appropriate converting value. In one embodiment, as shown in  FIG. 2B , the adaptive weighting unit  121  comprises a detail checking unit  121   a , a motion estimation difference checking unit  121   b , a vector coherence checking unit  121   c , and a look-up table  121   d . If the block that the vector corresponds to has one of the following property or combination thereof: high detail, low SAD, and high MV (moving vector) coherence, the vector has high reliability. 
     In one embodiment, the weighting value can be adjusted by adjusting the values of interpolation position fx, fy to thereby have the output pixel value OP inclined to the corresponding pixel value of the vector. 
     It should be noted that the device and method to determine the reliability of a vector are not limited to the above example and any other current device or any device to be developed in the future can be used in determination. 
     The checking method of the adaptive weighting unit  121  is as follows: 
     1. The detail checking unit  121   a  performs detail checking. The detail checking unit  121   a  checks details of each vector block. The vector block having higher detail is more sensitive to reflect the correct vector while searching the motion estimation difference. Thus, the vector block having higher detail should be chosen. The operation method of detail checking can use the following equation in calculation where the absolute difference between the vector block and the other block that is the lower-right block of the block is calculated. As the calculated value is larger, the value of detail is larger. The equation is shown in the following: 
                       ∑   block             ⁢           ⁢     (            F   ⁡     (     x   ,   y     )       -     F   ⁡     (       x   +   1     ,     y   +   1       )              )       &gt;     threshold   ⁢           ⁢   1             (   1.2   )               
Where “block” is the block, “F” is the frame, and “threshold 1 ” is a first threshold value.
 
     2. The motion estimation difference checking unit  121   b  performs low SAD checking. The motion estimation difference checking unit  121   b  calculates the motion estimation difference of the corresponding adjacent vectors of the interpolation block (such as MV 1 ˜MV 4  shown in  FIG. 1D ). A smaller motion estimation difference indicates the higher reliability of the vector. Motion estimation difference checking can use the following equation to calculate: 
                       ∑   block             ⁢           ⁢     (            F   ⁢           ⁢   0   ⁢     (     x   ,   y     )       -     F   ⁢           ⁢   1   ⁢     (       x   +   MVx     ,     y   +   MVy       )              )       &lt;     threshold   ⁢           ⁢   2             (   1.3   )               
where “block” is the interpolation block, “F 0 ” is the preceding frame, “F 1 ” is the current frame, “MVx” and “MVy” are vectors, and “threshold 2 ” is a second threshold value.
 
     3. The vector coherence checking unit  121   c  performs moving vector coherence checking. Each vector is checked with the other adjacent vectors, separately. For example, the vector MV 1  is checked with the vectors MV 2 ˜MV 4 , separately. If the vector MV 1  to be checked is similar to the adjacent vectors, the reliability of the vector is higher and this vector is selected to be inclined to. The following equation can be used to calculate vector coherence: 
                       ∑   neighborhood             ⁢           ⁢            neighbor   ⁢           ⁢   MV     -   MV            &lt;     threshold   ⁢           ⁢   3             (   1.4   )               
where “neighborhood” is the adjacent block, “MV” is a vector to be checked, “neighborMV” is an adjacent vector of the vector to be checked, and “threshold 3 ” is a third threshold value.
 
     It should be noted that the block which a vector corresponds to having higher detail, low SAD, and high vector coherence indicates that this vector has higher reliability. Certainly, the invention can adjust the vector inclination based on the need of a designer. For example, only any one or two of the above checking methods are used in determination; or some other checking method or other current checking method or any checking method to be developed in the future can be used. 
     For the vectors having higher reliability, the look-up table  121   d  can be used to adjust the values of the interpolation position fx, fy to fx′, fy′ and adjust the weighting values W 1 , W 2 , W 3 , and W 4  to the new weighting values W 1 ′, W 2 ′, W 3 ′, and W 4 ′ to achieve the effect of inclining to the corresponding pixel value, as show in  FIG. 3 . The equation is shown in the following: 
     
       
         
           
             
               
                 
                   
                     
                       
                         
                           P 
                           out 
                         
                         = 
                           
                         ⁢ 
                         
                           
                             
                               ( 
                               
                                 1 
                                 - 
                                 
                                   Δ 
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   
                                     fy 
                                     ′ 
                                   
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               ( 
                               
                                 1 
                                 - 
                                 
                                   Δ 
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   
                                     fx 
                                     ′ 
                                   
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               P 
                               1 
                             
                           
                           + 
                           
                             
                               ( 
                               
                                 1 
                                 - 
                                 
                                   Δ 
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   
                                     fy 
                                     ′ 
                                   
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               ( 
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   fx 
                                   ′ 
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               P 
                               2 
                             
                           
                           + 
                         
                       
                     
                   
                   
                     
                       
                           
                         ⁢ 
                         
                           
                             
                               ( 
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   fy 
                                   ′ 
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               ( 
                               
                                 1 
                                 - 
                                 
                                   Δ 
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   
                                     fx 
                                     ′ 
                                   
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               P 
                               3 
                             
                           
                           + 
                           
                             
                               ( 
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   fy 
                                   ′ 
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               ( 
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   fx 
                                   ′ 
                                 
                               
                               ) 
                             
                             ⁢ 
                             
                               P 
                               4 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                           
                         ⁢ 
                         
                           
                             
                               W 
                               1 
                               ′ 
                             
                             ⁢ 
                             
                               P 
                               1 
                             
                           
                           + 
                           
                             
                               W 
                               2 
                               ′ 
                             
                             ⁢ 
                             
                               P 
                               2 
                             
                           
                           + 
                           
                             
                               W 
                               3 
                               ′ 
                             
                             ⁢ 
                             
                               P 
                               3 
                             
                           
                           + 
                           
                             
                               W 
                               4 
                               ′ 
                             
                             ⁢ 
                             
                               P 
                               4 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                           
                         ⁢ 
                         
                           ( 
                           
                             
                               
                                 W 
                                 1 
                                 ′ 
                               
                               + 
                               
                                 W 
                                 2 
                                 ′ 
                               
                               + 
                               
                                 W 
                                 3 
                                 ′ 
                               
                               + 
                               
                                 W 
                                 4 
                                 ′ 
                               
                             
                             = 
                             1 
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   1.5 
                   ) 
                 
               
             
           
         
       
     
     For example, if the vector MV 4  passes “detail checking”, “motion estimation difference checking”, and “vector coherence checking”, the vector has reliability. Then, the look-up table  121   d  is used to convert the values of the interpolation position fx, fy to obtain a larger weighting value W 4 ′. The larger value W 4 ′ can reflect more P 4  that is the pixel value of the vector MV 4 . Thus, the result can be inclined to the vector MV 4  having the higher reliability by converting the values of the interpolation position fx, fy. 
     Converting the values of the interpolation position fx, fy can be done by utilizing the table built in the look-up table  121   d , as shown in  FIGS. 4A and 4B , illustrated as follows: 
     if it is the original linear interpolation, the case X 1 +case Y 1  are selected as shown in  FIGS. 4A and 4B ; 
     if it is inclined to MV 1  (upper left vector), the case X 2 +case Y 2  are selected as shown in  FIGS. 4A and 4B ; 
     if it is inclined to MV 2  (upper right vector), the case X 3 +case Y 2  are selected as shown in  FIGS. 4A and 4B ; 
     if it is inclined to MV 3  (lower left vector), the case X 2 +case Y 3  are selected as shown in  FIGS. 4A and 4B ; and 
     if it is inclined to MV 4  (lower right vector), the case X 3 +case Y 3  are selected as shown in  FIGS. 4A and 4B . 
     For example, it is assumed that the adjacent four vectors are (0,0), (0,0), (6,0), and (6,0) and the pixel values corresponding to the target pixel (x,y) to be interpolated are P 1 =100, P 2 =200, P 3 =300, and P 4 =400. It is also assumed that the values of the interpolation position for the target pixel (x,y) are fx=4 and fy=8 and the length and width of the interpolation block IB are 16*16. Assuming that the vector MV 4  is reliable after checked by the adaptive weighting unit  121 , the case X 3 +case Y 3  are selected and fx and fy are converted to be inclined to the vector MV 4 . Thus, fx′=8 and fy′=14 are obtained. These values are applied in the equation (1.5) and then the output pixel value Pout inclined to the vector MV 4  is
 
 P out=(1−14/16)*(1−8/16)*100+(1−14/16)*(8/16)*200+(14/16)*(1−8/16)*300+(14/16)*(8/16)*400=0.0625*100+0.0625*200+0.4375*300+0.4375*400=325.
 
The weighting values W 1 ′=0.0625, W 2 ′=0.0625, W 3 ′=0.4375, and W 4 ′=0.4375 are obtained where W 1 ′+W 2 ′+W 3 ′+W 4 ′=1.
 
     By this method, the way of inclining to the vector MV 4  is repeated to calculate the output pixel values Pout of all the target pixels in the interpolation block IB to thereby generate a whole interpolated frame. That is, the output frame OF of the imaging generating device shown in  FIG. 1A  is generated. Since the output frame OF is generated from the vector(s) having higher reliability, the block artifacts in the prior art can be removed. Compared to the prior art that uses the method of blending adjacent blocks to reduce block artifacts, the method according to the invention is simpler and the reliability of the method according to the invention is higher. Thus, correctly reflecting adjacent blocks can be achieved. 
     It should be noted that, in one embodiment for the imaging generating device and the method thereof, if all the adjacent four vectors cannot pass the reliability test, the original linear interpolation is selected in blending. If there are more than two vectors having high reliability, one of them can be selected in processing. 
     It should be noted that the above description uses four vectors in processing but the invention is not limited to the above examples and can use a different number of vectors. The number of vectors in use can be equal to N where N is a positive integer less than infinity. 
     Moreover,  FIG. 5  shows a flow chart illustrating the motion compensation interpolation method according to one embodiment of the invention. The method comprises the following steps: 
     Step S 502 : start; 
     Step S 504 : allocating an interpolation block in a frame where the interpolation block comprises at least one target pixel to be interpolated; 
     Step S 506 : checking the reliability of the vectors corresponding to the adjacent blocks of the interpolation block; 
     Step S 508 : selecting at least one preferred vector having high reliability to generate a plurality of weighting values based on the at least one preferred vector; 
     Step S 510 : receiving the corresponding pixel values of the vectors and determining the pixel value of the target pixel to be interpolated according to the weighting values and the pixel values; and 
     Step S 512 : end. 
     Although the present invention has been fully described by the above embodiments, the embodiments should not constitute the limitation of the scope of the invention. Various modifications or changes can be made by those who are skilled in the art without deviating from the spirit of the invention.