Abstract:
A method and apparatus for determining a region-based weighted prediction with improved global brightness detection. The method includes applying a global brightness change detection methods by computing the weighted prediction parameters, determining if the brightness change is different amount of change for the different regions, if the change is not different, calculate motion estimation and setting Refidx to 1 and setting Refidx is set to 0 when there is change, determining the best motion vector, motion vector cost and the best reference input, and determining a region-based weighted prediction with improved global brightness detection based on the motion vector data.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims benefit of U.S. provisional patent application Ser. No. 61/363,454, filed Jul. 12, 2010, which is herein incorporated by reference. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    Embodiments of the present invention generally relate to a method and apparatus for region-based weighted prediction; more specifically, to region-based weighted prediction with improved global brightness detection. 
         [0004]    2. Description of the Related Art 
         [0005]    Weighted prediction (WP) is introduced in H.264 video coding standard to improve coding efficiency for the video sequences that have brightness changes between frames. It is found to provide high coding efficiency for the video sequence with global brightness changes. However, since WP in H.264 is slice based method, it also leads to significant coding loss for the sequences that have lots of local brightness changes and high motions when it is applied without caution. 
         [0006]    Some embodiments of global brightness change detection (GBD) method to apply WP when there are global brightness change between the current and the reference frames. Such embodiments are effective in not introducing coding loss for normal sequences while improving coding efficiency for the sequences with global brightness changes. However, such embodiments could be worse than applying WP always especially when there are local brightness changes. 
         [0007]    Therefore, there is a need for a region-based weighted prediction with improved global brightness detection. 
       SUMMARY OF THE INVENTION 
       [0008]    Embodiments of the present invention relate to a method and apparatus for determining a region-based weighted prediction with improved global brightness detection. The method includes applying a global brightness change detection methods by computing the weighted prediction parameters, determining if the brightness change is different amount of change for the different regions, if the change is not different, calculate motion estimation and setting Refidx to 1 and setting Refidx is set to 0 when there is change, determining the best motion vector, motion vector cost and the best reference input. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]    So that the manner in which the above recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments. 
           [0010]      FIG. 1  is an embodiment of a local brightness changes and association to reference index; and 
           [0011]      FIG. 2  is an embodiment of a flow diagram depicting a method for region-based weighted prediction with improved global brightness detection. 
       
    
    
     DETAILED DESCRIPTION 
       [0012]    A region based WP method to handle local brightness change as well as improved GBD method is proposed. First, improved GBD is employed to determine if there is local brightness change and to identify such regions. Once local brightness changes are found, region based WP is applied using reference picture reordering to efficiently handle such regions. Then, motion estimation (ME) for each block is performed over one reference index (frame) depending on whether the region, in which the block is located, has brightness change or not. 
         [0013]    Such a solution is useful when there are strict limitations on resources, such as, cycle count, memory bandwidth, etc. In the current hardware implementation of H.264 encoder, a single reference is used for ME due to such resource constraints. Since one reference is loaded and one ME per block is needed, no additional computations and memory bandwidth for ME are needed. 
         [0014]    H.264 supports reference picture reordering by which the reference pictures in decoded picture buffer (DPB) could be reordered if necessary to improve coding efficiency. It is even possible that the same reference picture is placed in multiple reference indices. It is worth noting that each reference picture may have its own WP parameters in H.264. Using these two H.264 features, the effective region may be based on a WP method. One may assume that frame picture encoding with a single reference frame and each frame consist of a single slice. 
         [0015]    To improve global brightness change detection, one may divide the current frame and the reference frame into the same arbitrary N regions of any size and shapes. Then, apply the global brightness change detection by which we can also identify the regions that have brightness changes can be identified by: (1) calculating the mean pixel values and variances of the n-th region of current picture and reference picture, where they are denote by m cur (n), m ref (n), v cur (n) and v ref (n), respectively. Then, the absolute difference is calculated between the two mean values, m diff (n)=|m cur (n)−m ref (n)|; and then (2) comparing the absolute mean difference, m diff (n), to a predefined threshold, TH. If m diff (n) is larger than TH, then increase the number of regions that have brightness change, Rcount, by 1. These steps may be repeated for a specific number or type of regions or for all regions; and (3) checking if either v cur (n)−v ref (n)&gt;=0 or v cur (n)−v ref (n)&lt;=0 for all N regions. Based on the observation that, with fade-in and fade-out, the variances of all regions either increase (fade-in) or decrease (fade-out), if the mean pixel values changes are determined to be caused by fade-in/fade-out, then set Fade=1. Where if the Fade is set to 1 and Rcount is larger than or equal to N max  (N min &lt;N max &lt;=N), we compute WP parameters, associate them to refidx equal to 0 and encode all MBs with motion estimation (ME) using refidx equal to 0. Otherwise, if Fade=0 or Rcount is less than or equal to N min , we turn off WP and encode all MBs with ME using refidx equal to 0. 
         [0016]    In the region-based weighted prediction, if the fade is set to 1 and N min &lt;Rcount&lt;N max , the reference frame is reordered in order for the same reference frames to sit in two places, refidx=0 and refidx=1, respectively. In such a case, WP parameters are computed and associated to refidx equal to 0 and zero WP parameters are associated to refldx equal to 1. The encoder may send reference reordering syntaxes for decoders. However, when encoding each MB, if the MB is in the region with brightness change, ME is performed with refidx equal to 0. However, if the MB is in the regions without brightness change, ME is performed with refidx equal to 1. In one embodiment, if 2 reference frames are used without global brightness change, coding gains are lost due to the increased overhead bits to encode reference index. Such losses are very noticeable with normal sequences that have no fade-in, fade-out and brightness changes. 
         [0017]    In one embodiment, one may consider the percentage of regions with brightness change for it. For example, the number of regions with brightness change is smaller than the other, zero WP may be associated with refidx=0. This is important because it affects the number of skipped blocks, whose refidx should be 0. 
         [0018]      FIG. 1  is an embodiment of a local brightness changes and association to reference index. In  FIG. 1 , the frames are divided into 5 rectangular regions; two of frames include luminance change (gray regions) and the others do not (white regions). In such a case, the reference list is reordered to put the same reference frame (frame t−1) into refidx=1 as well. Then, the computed WP parameters (WP 0 ) is associated with refidx=0 and zero WP parameters (WP 1 ) with refidx=1. Then we perform ME using refidx=0 for MBs in the gray regions and refidx=1 for MBs in the white regions. 
         [0019]      FIG. 2  is an embodiment of a flow diagram depicting a method  200  for region-based weighted prediction with improved global brightness detection. The method starts at step  202  and continues to step  204 . At step  204 , the method  200  determines if a change in global brightness was detected. If a change was detected, the method  200  proceeds to step  206 , wherein weighted prediction parameters are calculated. Otherwise, the method  200  proceeds to step  208 , wherein the method  200  determines local brightness has changed. If there is no change, the method  200  proceeds to step  210 , wherein the weighted prediction parameters are set to zero (0). Otherwise, the method  200  proceeds to step  222 . 
         [0020]    From steps  206  and  210 , the method  200  proceeds to step  212 , wherein the method  200  sets micro block index to zero (0) and proceeds to step  214 . At step  214 , the method  200  sets motion estimation with refldx to zero (0). At step  216 , the method  200  determines if it is the last micro block. If it is not the last block, the method  200  proceeds to step  220  and sets micro block in increased by one (1). The method  200  proceeds from step  220  to step  214 . From step  208 , if the local brightness changed, the method  200  proceeds to step  222 . At step  222 , the method  200  calculates the weighted prediction parameters and reference re-ordering. At step  224 , the method  200  step the micro block index to zero (0). At step  226 , the method  200  determines if the micro block is in the region of luminance change. It is in the region, the method proceeds to step  230 . At step  230 , the method  200  calculates motion estimation with refidx set to zero (0). Otherwise, the method proceeds from step  226  to step  228 , wherein the method  200  calculates motion estimation with refidx set to one (1) and weighted prediction parameters set to zero (0). 
         [0021]    From steps  228  and  230 , the method  200  proceeds to step  232 , wherein the method  200  determines if it is the last micro block. If it is not the last micro block, the method  200  proceeds to step  234 , wherein the micro block index is incremented by one (1). From step  234 , the method  200  returns to step  226 . Otherwise, the method  200  proceeds from step  232  to step  236 . The method  200  ends at step  236 . 
         [0000]    To compute weighted prediction (WP) parameters, various method and different regions may be considered for weighted prediction parameter calculation, such as, regions where brightness have changed or all regions regardless of how brightness changes locally. In one embodiment, N reference indices may use reference picture reordering and associate N different WP parameters to each of N regions. Then, for each block, the corresponding reference index for motion estimation may be chosen according to its position. When a single block overlaps multiple regions, there are several criteria to decide which region the block is included in. In the current embodiment, we used top-left block position for the decision. Thus, in this case, the block is in the region where top-left pixel of the block is included. 
         [0022]    The region-based WP can be extended to block based or macro-block (MB) based weighted prediction with marginal computation increase, but without any increase of memory bandwidth. In another embodiment, macro-block based weighted prediction are calculated with marginal computation increase, but without any increase of memory bandwidth. Thus, global brightness change detection is improved. Improving global brightness change detection (GBD) in order to produce a more accurate decision. If there is no brightness change or global brightness change, on/off weighted prediction switch is switched globally. If we detect local brightness change, we apply weighted prediction with reference list reordering to put the same reference in multiple reference list. With reference reordering, both region-based WP and MB-based weighted prediction are enabled. 
         [0023]    As a result, the quality loss by improved GBD method is minimized. The improved GBD is also critical to region-based WP. By carefull decision on the use of region-based WP, the coding gain is maximized by region-based WP and the overhead bits to encoder reference index is minimized. The proposed MB-based WP with best MV cost check requires motion estimation only with 1 reference for each block, but have the similar performance with microblock based WP with motion estimation with 2 references. Hence, better quality is achieved with marginal increase of computational complexity. 
         [0024]    MB-based WP with final MV cost check: If Fade=1 and N min &lt;Rcount&lt;N max , we reorder the reference frame so that the same reference frames sits in two places, refidx=0 and refidx=1, respectively. 
         [0025]    WP parameters are calculated and associated to refidx equal to 0 and associate zero WP parameters to refidx equal to 1. The encoder should send reference reordering syntaxes for decoders. When encoding each MB, motion estimation with zero WP parameters (refidx=1) are calculated first and get the best MV and its MV cost, which are denoted by MV 1  and MVCost 1 . Then, for MV 1 , MV cost w.r.t. non-zero WP parameters (refidx=0) are calculated (denoted by MV 0  and MVCost 0 ). Decide the best reference index by comparing MVCost 1  and MVCost 0 . 
         [0026]    In the above algorithm, this is an issue on which reference index we will use for motion estimation. Motion estimation gets better when the actual content is not hampered by brightness change. Based on this fact, we can do motion estimation with non-zero WP in case of fade-in (i.e. v cur (n)−v ref (n)&gt;=0). The above MB-based WP can be modified to use 2 references for motion estimation instead of final MV cost check. It is supposed to show the best performance. But the computational complexity will increase. We use this 2-ref MB-based WP as bench mark to evaluate the region-based WP and the (1-ref) MB-based WP with final MV cost check. 
         [0027]    While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.