Patent Publication Number: US-2006018557-A1

Title: Apparatus and method for filtering blocking effect in an image

Description:
BACKGROUND OF THE INVENTION  
      This application claims the priority of Korean Patent Application No. 10-2004-0057140, filed on Jul. 22, 2004, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference.  
      1 . Field of the Invention  
      The present invention relates to a blocking effect in an image, and more particularly, to an apparatus and method for filtering a blocking effect in an image, which removes a blocking effect that may occur in an image that is compressed in block units in compression and/or decompression of a moving image.  
      2. Description of the Related Art  
      In compression and/or decompression of a moving image, a blocking effect may occur around boundaries between blocks (hereinafter, referred to as block boundaries) in an image that is compressed in block units. Thus, a filtering method for removing a blocking effect in an image is required. A blocking effect filtering method specified in MPEG-4 (Moving Picture Expert Group) indicates two types of domain modes, i.e., a DC offset mode and a default mode, using 10 pixels around block boundaries and selectively performing filtering on 8 pixels around the block boundaries in each mode. Determination of the two types of domain modes is performed using Equation 1. 
 
 f =Φ( v 0 −v 1)+Φ( v 1 −v 2)+Φ( v 2− v 3)+Φ( v 3− v 4)+Φ( v 4 −v 5)+Φ( v 5 −v 6)+Φ( v 6 −v 7)+Φ( v 7 −v 8)+Φ( v 8 −v 9)  (1). 
 
      where Φ(a)=1 if (|a|&lt;=TH1) else 0.  
      Using f obtained using Equation 1, the number of cases where the difference between adjacent pixels among the 10 pixels around the block boundaries is less than a first threshold value TH1 is obtained. When the obtained number is less than a second threshold value TH2, the domain mode is determined to be the DC-offset mode. When the obtained number is not less than the second threshold value TH2, the domain mode is determined to be the default mode. When a domain is determined to be in the DC-offset mode, it is determined to be a flat domain in which the blocking effect is removed using a low pass filter having a predetermined filter coefficient. When a domain is determined to be in the default mode, frequency information around block boundaries is obtained and the size of discontinuous components included in the block boundaries is adjusted to a minimal value using the frequency information, thereby removing the blocking effect.  
      However, since the blocking effect filtering method specified in MPEG-4 performs filtering on domains in the same manner regardless of whether the domains are excessively compressed or little compressed, blurring may occur.  
      When at least one of two adjacent macroblocks (i.e., a current macroblock and a macroblock that is located above the current macroblock) in a frame picture is coded in a field mode, i.e., when at least one of them is coded using field-DCT (discrete cosine transform) or field-prediction, its top-field line and bottom-field line are coded separately. In this case, the blocking effect filtering method specified in MPEG-4, which uses Equation 1, cannot effectively remove the blocking effect in a macroblock that is coded in a field mode.  
     SUMMARY OF THE INVENTION  
      Therefore, a need exists for filtering a blocking effect in an image that is compressed in block units in compression and/or decompression of a moving image and also prevents blurring.  
      A method is provided for filtering a blocking effect in an image, by which the blocking effect is effectively removed even when at least one of two adjacent macroblocks in a frame picture is coded in a field mode.  
      According to one aspect of the present invention, there is provided an apparatus for filtering a blocking effect in an image. The apparatus comprises a single blocky pattern searching unit, a first filtering unit, a double blocky pattern searching unit, a second filtering unit, and a third filtering unit.  
      The single blocky pattern searching unit searches for a single blocky pattern in a predetermined section adjacent to the boundary between image blocks. When there is a single blocky pattern in the predetermined section, the first filtering unit filters three pixels on either side of the boundary if a first filtering condition is satisfied and filters two pixels on either side of the boundary if the first filtering condition is not satisfied.  
      The double blocky pattern searching unit searches for a double blocky pattern in the predetermined section if there is no single blocky pattern in the predetermined section. When there is a double blocky pattern in the predetermined section, the second filtering unit filters three pixels on either side of the boundary if a second filtering condition is satisfied and filters two pixels on either side of the boundary if the second filtering condition is not satisfied.  
      When there is neither a single blocky pattern nor a double blocky pattern in the predetermined section or the difference between two pixels in the predetermined section is more than two times a quantization scale factor, the third filtering unit filters one pixel on either side of a boundary between 8×8 blocks if a third filtering condition is satisfied.  
      A single blocky pattern is where there exists a step height between two sides of the boundary between blocks and the step height is the largest around the boundary among step heights of flat patterns. Also, a single blocky pattern is where: (a) there are two pixels on either side of the boundary; (b) a step height of less than two times the quantization scale factor exists between two pixels, each being on either side of the boundary; and (c) the difference between the two pixels is less than a predetermined threshold value.  
      A double blocky pattern is where at least two boundaries overlap and a step height exists over two pixels and is the largest around the boundaries among step heights of flat patterns. More specifically, a double blocky pattern is where: (a) there is one pixel in the boundary region; (b) there are two pixels on either side of the boundary region; (c) a step height of less than two times the quantization scale factor exists between two pixels, each being on either side of the boundary region; and (d) the difference between the two pixels is less than a predetermined threshold value.  
      According to another aspect of the present invention, there is provided a method for filtering a blocking effect in an image. The method comprises (a) searching for a single blocky pattern in a predetermined section adjacent to a boundary between image blocks, (b) filtering three pixels on either side of the boundary if a first filtering condition is satisfied and filtering two pixels on either side of the boundary if the first filtering condition is not satisfied, when there is a single blocky pattern in the predetermined section, (c) searching for a double blocky pattern in the predetermined section if there is no single blocky pattern in the predetermined section, (d) filtering three pixels on either side of the boundary if a second filtering condition is satisfied and filtering two pixels on either side of the boundary if the second filtering condition is not satisfied, when there is a double blocky pattern in the predetermined section, and (e) filtering one pixel on either side of a boundary between 8×8 blocks if a third filtering condition is satisfied, when there is neither a single blocky pattern nor a double blocky pattern in the predetermined section or a difference between two pixels in the predetermined section is more than two times a quantization scale factor.  
      The method further comprises (b′) repeating (a) through (e) on the pixels filtered in (b).  
      A single blocky pattern is where there exists a step height between two sides of the boundary between blocks and the step height is the largest around the boundary among step heights of flat patterns. More specifically, the degree of flatness is the quantization scale factor divided by 8 plus 1 and the step height is less than two times the quantization scale factor. Also, a single blocky pattern is where: (a) there are two pixels on either side of the boundary; (b) a step height of less than two times the quantization scale factor exists between two pixels, each being on either side of the boundary; and (c) the difference between the two pixels is less than a predetermined threshold value.  
      A double blocky pattern is where at least two boundaries overlap and a step height exists over two pixels and is the largest around the boundaries among step heights of flat patterns. More specifically, a double blocky pattern is where: (a) there is one pixel in the boundary region; (b) there are two pixels on either side of the boundary region; (c) a step height of less than two times the quantization scale factor exists between two pixels, each being on either side of the boundary region; and (d) the difference between the two pixels is less than a predetermined threshold value.  
      According to yet another aspect of the present invention, there is provided a method for filtering a blocking effect of an image in which at least one of two adjacent blocks is coded in a field mode. The method comprises filtering two pixels included in a top field of one of the two adjacent blocks on one side of the boundary between the two adjacent blocks and two pixels included in the top field of the other block on the other side of the boundary and filtering two pixels included in the bottom field of one of the two adjacent blocks on one side of the boundary between the two adjacent blocks and two pixels included in the bottom field of the other block on the other side of the boundary. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
      The above and other features of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:  
       FIG. 1  shows a schematic block diagram of an apparatus for filtering a blocking effect in an image according to an exemplary embodiment of the present invention;  
       FIG. 2  shows a flowchart illustrating a method for filtering a blocking effect in an image according to an exemplary embodiment of the present invention, which is implemented in the apparatus for filtering a blocking effect in an image shown in  FIG. 1 ;  
       FIG. 3  shows pixel distribution around a horizontal block boundary and a vertical block boundary;  
       FIG. 4  shows a single blocky pattern and a first filtering condition;  
       FIG. 5  shows a double blocky pattern and a second filtering condition;  
       FIG. 6  shows a flowchart illustrating a method for filtering a blocking effect in an image according to an exemplary embodiment of the present invention when at least one of two adjacent macroblocks is coded in a field mode. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       FIG. 1  shows a schematic block diagram of an apparatus for filtering a blocking effect in an image according to an exemplary embodiment of the present invention, and  FIG. 2  shows a flowchart illustrating a method for filtering a blocking effect in an image according to an exemplary embodiment of the present invention, which is implemented in the apparatus for filtering a blocking effect in an image shown in  FIG. 1 .  
      Referring to  FIG. 1 , the apparatus for filtering a blocking effect in an image according to the present invention includes a single blocky pattern searching unit  11 , a first filtering unit  13 , a double blocky pattern searching unit  15 , a second filtering unit  17 , and a third filtering unit  19 .  
      Hereinafter, the method for filtering a blocking effect in an image, which is implemented in the apparatus for filtering a blocking effect in an image shown in  FIG. 1 , will be described in detail with reference to the flowchart illustrated in  FIG. 2 .  
      The single blocky pattern searching unit  11  searches for a single blocky pattern in a predetermined section adjacent to a boundary in steps S 1  and S 2 . The predetermined section includes 10 pixels V 0  through V 9  as shown in  FIG. 3  that shows the distribution of pixels adjacent to the horizontal and vertical block boundaries.  
      Here, a single blocky pattern is as shown in  FIG. 4  such that a step height X of less than 2QP exists between two sides of a boundary and is the largest around the boundary among step heights of flat patterns, where QP indicates a quantization scale factor. The single blocky pattern is caused by a blocking effect. The degree of flatness is determined by QP and is QP/8+1. More specifically, a single blocky pattern is where: (a) there are two pixels on either side of the boundary, i.e., two pixels V 2  and V 3  are on one side of the boundary and two pixels V 4  and V 5  are on the other; (b) the step height X of less than 2QP exists between two pixels, i.e., between pixels V 2  and V 5  and between pixels V 3  and V 4 ; and (c) the differences Y between pixels V 2  and V 3  and between pixels V 4  and V 5  are each less than a predetermined threshold value TH.  
      Thus, in the method for filtering a blocking effect in an image according to the present invention, by tightening the standards for a pattern as QP decreases, less filtering takes place in a little-compressed image in which QP is small. As a result, blurring does not occur in delicate portions of an image. The reason why the step height X is less than 2QP in the single blocky pattern is that it is possible to determine that a single blocky pattern is generated due to a blocking effect caused by quantization only when the step height X is less than 2QP. Thus, a pattern in which the step height X is less than 2QP is subject to filtering.  
      A pattern in which the step height X is less than 2QP is mainly generated due to a quantization error between blocks when intra-coded. When the generated pattern is inter-coded and is thus motion-compensated, a single blocky pattern is generated as a block boundary moves. When the generated pattern is motion-compensated one more iteration, a plurality of single blocky patterns may be generated in the predetermined section including pixels V 0  through V 9 . However, in the method for filtering a blocking effect according to the present invention, as will be described later, since filtering is performed on 6 pixels at a time, all sections in which a blocking effect occurs can be filtered by searching for two single blocky patterns in the predetermined section including pixels V 0  through V 9  and filtering the found two single blocky patterns.  
      Referring back to  FIGS. 1 and 2 , when a single blocky pattern is present in the predetermined section including pixels V 0  through V 9 , the first filtering unit  13  filters three pixels on either side of the boundary, and if a first filtering condition is not satisfied, the first filtering unit  13  filters two pixels on either side of the boundary, in steps S 3  through S 5 . In other words, if the first filtering condition is satisfied, the first filtering unit  13  filters three pixels on either side of the boundary, i.e., three pixels V 1 , V 2 , and V 3  on one side of the boundary and three pixels V 4 , V 5 , and V 6  on the other. If the first filtering condition is not satisfied, the first filtering unit  13  filters two pixels on either side of the boundary, i.e., two pixels V 1  and V 2  on one side of the boundary and two pixels V 4  and V 5  on the other. Here, the first filtering condition is that differences Z between pixels V 2  and V 3  and between pixels V 5  and V 6  are each less than a predetermined threshold value TH, as shown in  FIG. 4 .  
      After filtering is performed in steps S 3  through S 5 , steps S 1  through S 5  are repeated with respect to the filtered pixels. At this time, a search is performed for another single blocky pattern that is located in another location in the predetermined section including pixels V 0  through V 9 .  
      If another single blocky pattern is not present in the predetermined section V 0  through V 9 , the double blocky pattern searching unit  15  searches for a double blocky pattern in the predetermined section including pixels V 0  through V 9 , in steps S 6  and S 7 .  
      Here, a double blocky pattern is as shown in  FIG. 5  such that at least two boundaries overlap and a step height X exists over two pixels and is the largest around the boundaries among step heights of flat patterns. The double blocky pattern is caused by a blocking effect. A double blocky pattern is mainly generated due to an overlap of boundaries when a motion-compensated block is motion-compensated again after inter-coding. The degree of flatness is determined by QP and is QP/8+1, like with the single blocky pattern. More specifically, the double blocky pattern is where: (a) there is a pixel V 4  in a boundary region; (b) there are two pixels on either side of the boundary region, i.e., two pixels V 2  and V 3  are on one side of the boundary region and two pixels V 5  and V 6  are on the other; (c) a step height X of less than 2QP exists between two pixels, i.e., between pixels V 2  and V 6  and between pixels V 3  and V 5 ; and (d) differences Y between pixels V 2  and V 3  and between pixels V 5  and V 6  are each less than the predetermined threshold value TH.  
      Referring back to  FIGS. 1 and 2 , when a double blocky pattern is present in the predetermined section including V 0  through V 9 , the second filtering unit  17  filters three pixels on either side of the boundary if a second filtering condition is satisfied and filters two pixels on either side of the boundary if the second filtering condition is not satisfied, in steps S 8 , S 4 , and S 5 . In other words, if the second filtering condition is satisfied, the second filtering unit  17  filters three pixels on either side of the boundary, i.e., the three pixels V 1 , V 2 , and V 3  on one side of the boundary and the three pixels V 5 , V 6 , and V 7  on the other. If the second filtering condition is not satisfied, the second filtering unit  17  filters two pixels on either side of the boundary, i.e., the two pixels V 2  and V 3  on one side of the boundary and the two pixels V 5  and V 6  on the other. Here, the second filtering condition is that differences Z between pixels V 1  and V 2  and between pixels V 6  and V 7  are each less than the predetermined threshold value TH, as shown in  FIG. 5 .  
      When there is no double blocky pattern in the predetermined section including pixels V 0  through V 9 , i.e., neither a single blocky pattern nor a double blocky pattern exist, or the step height X between two pixels is more than 2QP, if a third filtering condition is satisfied, the third filtering unit  19  filters one pixel on either side of an 8×8 block boundary, in steps S 9  through S 11 . In other words, the third filtering unit  19  filters one pixel V 4  on one side of the vertical or horizontal block boundary shown in  FIG. 3  and one pixel V 5  on the other.  
      Since a blocking effect in an image may occur due to a quantization error caused by residual components in an 8×8 block boundary when motion compensation is performed after inter-coding, steps S 9  through S 11  are performed to remove such a blocking effect that is not removed by performing filtering with respect to the single blocky pattern and double blocky pattern. In this case, the blocking effect is not serious, and filtering is only performed on one pixel on either side of the 8×8 block boundary.  
       FIG. 6  shows a flowchart illustrating a method for filtering a blocking effect in an image according to an exemplary embodiment of the present invention when at least one of two adjacent macroblocks is coded in a field mode. When at least one of two macroblocks is coded in a field mode, a method for filtering a blocking effect in an image according to the present invention filters two pixels V 1  and V 3  included in a top field of a macroblock on one side of a macroblock boundary (indicated as MB-boundary in  FIG. 6 ) and two pixels V 5  and V 7  included in a top field of a macroblock on the other and filters two pixels V 2  and V 4  included in a bottom field of the macroblock on one side of the macroblock boundary and two pixels V 6  and V 8  included in a bottom field of the macroblock on the other.  
      More specifically, in the case of an interlacing picture, a macroblock can be coded in a field or a frame mode. However, when a macroblock is coded in a field mode, if the method for filtering a blocking effect in an image shown in  FIG. 2  is used, patterns do not meet the first and second filtering conditions for a single blocky pattern and a double blocking pattern. As a result, filtering is not performed well.  
      Also, when a macroblock is coded in a field mode, i.e., the macroblock is coded using field-DCT or field-prediction, pixels included in a top-field line of the macroblock and pixels included in a bottom-field line of the macroblock are coded separately. Thus, in this case, differences between the pixels included in the top-field line and the pixels included in the bottom-field line are large. As a result, the filtering method prescribed in MPEG-4 that uses Equation 1 cannot effectively remove a blocking effect in the macroblock that is coded in a field mode.  
      Therefore, when a macroblock is coded in a field mode, as shown in  FIG. 6 , pixels V 1 , V 3 , V 5 , and V 7  included in the top field and pixels V 2 , V 4 , V 6 , and V 8  included in the bottom field should be filtered separately. In other words, pixels V 1 , V 3 , V 5 , and V 7  included in the top field are filtered, and separately from them, pixels V 2 , V 4 , V 6 , and V 8  included in the bottom field should be filtered.  
      However, considering the bus-bandwidth of hardware, it is necessary to receive decompressed picture data from a compression/decompression unit and to deblock the decompressed picture data on the fly, rather than storing picture data that is compressed and decompressed by the compression/decompression unit in an external memory, retrieving the stored picture data, and then deblocking the picture data. Thus, in this case, to use the minimum hardware resources, only a macroblock boundary is filtered.  
      Hereinafter, an example of filtering performed using a method for filtering a blocking effect in an image according to the present invention shown in  FIGS. 2 through 6  will be described. When a block boundary exists on the left side of or above a predetermined pixel V n  in a predetermined section including pixels V 0  through V 9 , if a filtering condition is satisfied, filtering is performed using Equations 2 through 4. If a filtering condition is not satisfied, filtering is performed using Equations 5 and 6. 
 
 V   n ′=( V   n−2 +2 V   n−1 +2 V   n +2 V   n+1   +V   n+2 )/8  (2) 
 
 V   n−1   ′ ′=(2 V   n−2 +2 V   n−1 +2 V   n +2 V   n+1 )/8  (3) 
 
 V   n−2 ′=(2 V   n−3 +3 V   n−2   +V   n−1   +V   n   +V   n+1 )/8  (4) 
 
 V   n ′=(2 V   n−1 +4 V   n +2 V   n+1 )/8  (5) 
 
 V   n−1 ′=(4 V   n−1 +3 V   n   +V   n+1 )/8  (6) 
 
      In Equations 2 through 6, V n−3 , V n−2 , V n−1 , V n , V n+1 , and V n+2  indicate the values of pixels before filtering and V n−2 ′, V n−1 ′, and V n ′ indicate the values of pixels after filtering.  
      When a block boundary exists on the right side of or below a predetermined pixel V n+1  in a predetermined section including pixels V 0  through V 9 , if a filtering condition is satisfied, filtering is performed using Equations 7 through 9. If a filtering condition is not satisfied, filtering is performed using Equations 10 and 11. 
 
 V   n+1 ′=( V   n+3 +2 V   n+2 +2 V   n+1 +2 V   n   +V   n−1 )/8  (7) 
 
 V   n+2 ′=(2 V   n+3 +2 V   n+2 +2 V   n+1 +2 V   n )/8  (8) 
 
 V   n+3 ′=(2 V   n+4 +3 V   n+3   +V   n+2   +V   n+1   +V   n )/8  (9) 
 
 V   n+1 ′=(2 V   n+2 +2 V   n+1 +2 V   n )/8  (10) 
 
 V   n+2 ′=(4 V   n+2 +3 V   n+1   +V   n )/8  (11) 
 
      In Equations 7 through 11, V n−1 , V n , V n+1 , V n+2 , V n+3 , and V n+4  indicate values of pixels before filtered and V n+1 ′, V n+2 ′, and V n+3 ′ indicate values of pixels after filtered.  
      As described above, in an apparatus and method for filtering a blocking effect in an image according to the present invention, in the case of a blocky pattern, a threshold value varies with QP. Thus, since QP is large in an excessively compressed region, more deblocking filtering is performed. Since QP is small in a little compressed region, less deblocking filtering is performed. Also, deblocking filtering is performed only around a boundary where a blocking effect occurs, thereby effectively removing the blocking effect and preventing excessive blurring.  
      Also, in a method for filtering a blocking effect in an image according to the present invention, a blocking effect can be effectively removed even when at least one of two adjacent macroblocks in a frame picture is coded in a field mode.  
      While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as set forth in the following claims.