Abstract:
Provided are a highly efficient image encoding device which is achieved by quantization of each sub-block in accordance with visual characteristics, and a quantization method that uses the image encoding device. An input image is divided into macro-blocks having a prescribed first size, each of the divided macro-blocks is divided into sub-blocks having a prescribed second size, and each of the sub-blocks is encoded using a same or different quantization parameter for each of the sub-blocks. As a result, by executing quantization for each of the sub-blocks in accordance with the visual characteristics, highly efficient image encoding and image decoding devices can be provided.

Description:
FIELD OF THE INVENTION 
     The present invention relates to an image encoding technique to compress video data based on pixel correlation and an image decoding technique to expand the encoded data into the video data, and particularly to a quantization technique used to reduce the amount of information. 
     BACKGROUND OF THE INVENTION 
     In an image encoding technique such as MPEG-2 and MPEG-4, an encoding is performed by dividing an input picture into macroblocks of 16×16 pixels, the macroblock (hereinafter referred to as “MB”) being a basic processing unit. As the encoding performed on the MB basis, a prediction, a transformation, a quantization, and an entropy encoding are well known. Among them, the quantization related to the present invention is performed on each coefficient of an input block based on an arbitrary quantization step size. When setting the quantization step size as Qstep, the input coefficient as C, and the quantization result as Z, an arithmetic expression of a general quantization is represented by Eq. 1 as follows:
 
 Z =round( C/Q step)  Eq. 1
 
     A compression ratio is improved by increasing the quantization step size Qstep. In this case, however, loss of information is increased. The influence of the information loss on image quality degradation depends on the pattern of the MB of interest. Specifically, in a region with a simple pattern such as sky and wall or a region with less motion, it is easy to perceive the image quality degradation. On the other hand, in a region with a complex pattern or a region with intense motion, it is difficult to perceive the image quality degradation. By using such visual characteristics, subjective image quality can be improved by setting a larger quantization step size in the region where it is difficult to perceive the image quality degradation, and conversely setting a smaller quantization step size in the region where it is easy to perceive the image quality degradation (see Patent Document 1 to Patent Document 3) 
     A conventional control of the quantization step size will be described with reference to  FIG. 1 .  FIG. 1  shows a processing block diagram of a conventional encoding device. Reference numeral “ 100 ” denotes an encoding device. Reference numeral “ 101 ” denotes a MB division unit. Reference numeral “ 102 ” denotes a degradation cost evaluation unit. Reference numeral “ 103 ” denotes a quantization step size determination unit. Reference numeral “ 104 ” denotes a prediction unit. Reference numeral “ 105 ” denotes a transformation unit. Reference numeral “ 106 ” denotes a quantization unit. Reference numeral “ 107 ” denotes an entropy encoding unit. Reference numeral “ 108 ” denotes an inverse quantization unit. Reference numeral “ 109 ” denotes an inverse transformation unit. Reference numeral “ 110 ” denotes a reconstruction unit. 
     In  FIG. 1 , first, the encoding device  100  inputs an input image to the MB division unit  101 . The MB division unit  101  divides the input image into blocks (MB) of 16×16 pixels, the block being called a macroblock (MB), and outputs the MBs to the degradation cost evaluation unit  102  and the prediction unit  104 . The degradation cost evaluation unit  102  calculates the image quality degradation cost of each of the input MBs, and outputs it to the quantization step size determination unit  103 . For example, when a variation in pixel values is defined as the image quality degradation cost, the degradation cost evaluation unit  102  calculates the image quality degradation cost by using the following Eq. 2 and Eq. 3: 
     
       
         
           
             
               
                 
                   DC 
                   = 
                   
                     round 
                     ⁡ 
                     
                       ( 
                       
                         
                           ( 
                           
                             
                               ∑ 
                               
                                 y 
                                 = 
                                 0 
                               
                               15 
                             
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               
                                 ∑ 
                                 
                                   x 
                                   = 
                                   0 
                                 
                                 15 
                               
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 MB 
                                 ⁡ 
                                 
                                   [ 
                                   
                                     x 
                                     , 
                                     y 
                                   
                                   ] 
                                 
                               
                             
                           
                           ) 
                         
                         / 
                         256 
                       
                       ) 
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                   ⁢ 
                   2 
                 
               
             
           
         
       
     
     
       
         
           
             
               
                 
                   
                     COST 
                     = 
                     
                       
                         ∑ 
                         
                           y 
                           = 
                           0 
                         
                         15 
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         
                           ∑ 
                           
                             x 
                             = 
                             0 
                           
                           15 
                         
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         
                           abs 
                           ⁡ 
                           
                             ( 
                             
                               DC 
                               - 
                               
                                 MB 
                                 ⁡ 
                                 
                                   [ 
                                   
                                     x 
                                     , 
                                     y 
                                   
                                   ] 
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                   , 
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                   ⁢ 
                   3 
                 
               
             
           
         
       
     
     where DC represents an average pixel value in the MB, and COST is the sum of absolute values of differences between the DC and the pixel values and is the image quality degradation cost in this example. 
     First, the quantization step size determination unit  103  determines a reference quantization step size according to a target bit rate that is inputted from the outside. Subsequently, a quantization step size, which makes the image quality uniform, is obtained based on the image quality degradation cost inputted from the degradation cost evaluation unit  102 . In order to determine the quantization step size based on the input image quality degradation cost, for example, a table  10  as shown in  FIG. 10  is prepared. In the table  10 , Qstep indicates the reference quantization step size. The quantization step size determination unit  103  outputs the determined quantization step size to the quantization unit  106 . Further, the quantization step size is set for each MB. 
     The prediction unit  104  generates a prediction image by using the correlation with neighboring pixels of the MB or the correlation between the current frame and frames before and after the current frame, and outputs a differential image between the prediction image and the MB to the transformation unit  105 . The transformation unit  105  transforms the input differential image into 4×4 blocks or 8×8 blocks by using orthogonal transformation such as two-dimensional discrete cosine transform (DCT), and outputs them to the quantization unit  106 . The quantization unit  106  quantizes an input transform coefficients based on the quantization step size inputted from the quantization step size determination unit  103 , and outputs the quantized transform coefficients to the entropy encoding unit  107  and the inverse quantization unit  108 . 
     The entropy encoding unit  107  transforms encoded control information such as the input quantized transform coefficients and the quantization step size into a bit stream. Further, the entropy encoding unit  107  outputs the amount of codes generated when the information is transformed into the bit stream (generated code amount) to the quantization step size determination unit  103 . The quantization step size determination unit  103  monitors whether the generated code amount is equal to a target bit rate and controls to make the generated code amount equal to the target bit rate by finely adjusting the reference quantization step size if the generated code amount is not equal to the target bit rate. Further, a reconstructed image is generated from the quantized transform coefficients through inverse quantization by the inverse quantization unit  108 , inverse transformation by the inverse transformation unit  109  and reconstruction by the reconstruction unit  110 , and is outputted to the prediction unit  104 . 
     Patent Document 1: International Publication No. WO 2011/064926 
     Patent Document 2: Japanese Patent Publication No. 4146444 
     Patent Document 3: Japanese Patent Publication No. 4768779 
     In image encoding techniques such as MPEG-2 and MPEG-4, the quantization step size is controlled on a MB basis. However, the image to be encoded is an image regardless of boundary of the MB. Accordingly, in the MB located at the boundary of an object present in the image, a complex region and a simple region may be mixed. When setting a smaller quantization step size in the MB located at the boundary of the object, the code amount of the complex region increases and the compression ratio decreases. Conversely, when setting a larger quantization step size in the MB, the image quality degradation of the simple region may be significant. 
     For example,  FIG. 2  is a diagram showing an original image before encoding, and  FIG. 3  is a diagram showing an image after encoding. In the encoding with reference to  FIGS. 2 and 3 , the quantization is performed with a larger quantization step size when a variation in pixel values is large in the MB, and the quantization is performed with a smaller quantization step size when a variation in pixel values is small in the MB. When viewing the image after encoding, degradation of a leaf portion is not significant, but a block noise due to an encoding may be checked at a boundary portion between sky and leaves (portion inside a dashed ellipse). 
     SUMMARY OF THE INVENTION 
     In view of the above, the present invention provides an image encoding device and an image encoding method with high efficiency by performing a quantization in each sub-block depending on visual characteristics. 
     In order to achieve the above object, according to a first aspect of the present invention, an image encoding device is configured to divide an input image into macroblocks each having a predetermined first size, divide each of the macroblocks into sub-blocks each having a predetermined second size, and perform an encoding with a same or different quantization parameter for each of the sub-blocks. Further, a decoding device includes a unit configured to extract quantization step size information on a sub-block basis which is multiplexed into a bit stream, and a unit configured to perform an inverse quantization on a sub-block basis based on the extracted quantization step size information. 
     According to a second aspect of the present invention, the image encoding device according to the first aspect of the present invention may include an evaluation unit configured to evaluate a degradation cost in each of the sub-blocks, a determination unit configured to determine a quantization step size for an image area of said each of the sub-blocks based on the evaluation unit, and a quantization unit configured to quantize the image area based on the determined quantization step size. 
     According to a third aspect of the present invention, the image encoding device of the first or the second aspect of the present invention may further include a multiplexing unit configured to multiplex encoded control information with a same or different quantization parameter for each of the sub-blocks into a bit stream. 
     Further, in order to achieve the above object, according to a fourth aspect of the present invention, an image encoding method includes dividing an input image into macroblocks each having a predetermined first size, dividing each of the macroblocks into sub-blocks each having a predetermined second size, evaluating a degradation cost in each of the sub-blocks, determining a quantization step size for an image area of said each of the sub-blocks based on said evaluating, and quantizing the image area based on the determined quantization step size to perform an encoding with a same or different quantization parameter for each of the sub-blocks. 
     According to a fifth aspect of the present invention, the image encoding method according to the fourth aspect of the present invention may further include multiplexing encoded control information with a same or different quantization parameter for each of the sub-blocks into a bit stream. 
     According to the present invention, by performing a quantization in each of sub-blocks depending on the visual characteristics, it is possible to provide an image encoding device and image decoding device with high efficiency. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  shows a processing block diagram of a conventional encoding device. 
         FIG. 2  is a diagram illustrating an original image before a conventional encoding. 
         FIG. 3  is a diagram illustrating an image after the conventional encoding. 
         FIG. 4  is a diagram illustrating an encoded image according to the present invention. 
         FIG. 5  shows an exemplary processing block diagram of an encoding device of the present invention. 
         FIG. 6  is a diagram illustrating second_qp_delta_map when second_qp_delta_mode of a picture header is 1. 
         FIG. 7  is a diagram illustrating second_qp_delta_map when second_qp_delta_mode of the picture header is 2. 
         FIG. 8  is a diagram showing an example of a picture parameter set to which the syntax of second_qp_delta_mode is added. 
         FIG. 9  is a diagram showing an example of a MB Layer to which the syntax of second_qp_delta_mode map shown in each of  FIGS. 6 and 7  is added. 
         FIG. 10  is a table for determining a quantization step size based on a conventional image quality degradation cost. 
         FIG. 11  is a table for determining a sub-block quantization step size based on a sub-block image quality degradation cost value of the present invention. 
     
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
     An encoding device in accordance with an embodiment of the present invention includes a unit configured to set a quantization step size in a MB on the basis of a sub-block of, e.g., 8×8 pixels and performing a quantization. Further, the encoding device includes a unit configured to multiplex the quantization step size set on a sub-block basis into a bit stream. Further, a decoding device includes a unit configured to extract quantization step size information on the sub-block basis which is multiplexed into the bit stream, and a unit configured to perform an inverse quantization on the sub-block basis based on the extracted quantization step size information. One embodiment of the present invention will be described with reference to the drawings. Further, the following description is for the purpose of explaining an exemplary embodiment of the present invention, and is not intended to limit the scope of the present invention. Therefore, since embodiments in which individual elements or all the elements thereof are replaced with equivalent ones can be employed by those skilled in the art, these embodiments are also included in the scope of the present invention. Further, in the following description of the drawings including the drawings described above, components having a common function are denoted by the same reference numeral, and redundant description thereof will be omitted. 
     An encoding device in accordance with an embodiment of the present invention will be described with reference to  FIG. 5 .  FIG. 5  is an exemplary processing block diagram of the encoding device of the present invention. Reference numeral “ 500 ” denotes the encoding device, reference numeral “ 502 ” denotes a sub-block degradation cost evaluation unit, reference numeral “ 503 ” denotes a sub-block quantization step size determination unit, reference numeral “ 507 ” denotes an entropy encoding unit, and reference numeral “ 511 ” denotes a sub-block quantization information multiplexing unit. A configuration of the encoding device  500  of the present invention is different from the configuration of the conventional encoding device  100  in that the encoding device  500  includes the sub-block degradation cost evaluation unit  502  configured to evaluate the degradation cost on a sub-block basis, the sub-block quantization step size determination unit  503  configured to determine the quantization step size on the sub-block basis, and the sub-block quantization information multiplexing unit  511  configured to multiplex the quantization information provided on the sub-block basis in the entropy encoding unit  507 . 
     In  FIG. 5 , the sub-block degradation cost evaluation unit  502  divides an input MB into sub-blocks each having a size of, e.g., 8×8 pixels, evaluates the degradation cost on a sub-block basis by using the following Eq. 4 and Eq. 5, and outputs the evaluated degradation cost to the sub-block quantization step size determination unit  503 . That is, the sub-block degradation cost evaluation unit  502  divides the macroblock into sub-blocks each having a size of 8×8 pixels, and an encoding is performed with a same or different quantization parameter for each of the sub-blocks. The sub-block degradation cost evaluation unit  502  outputs the evaluated degradation cost to the sub-block quantization step size determination unit  503 . 
     
       
         
           
             
               
                 
                   DC 
                   = 
                   
                     round 
                     ⁡ 
                     
                       ( 
                       
                         
                           ( 
                           
                             
                               ∑ 
                               
                                 y 
                                 = 
                                 0 
                               
                               7 
                             
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               
                                 ∑ 
                                 
                                   x 
                                   = 
                                   0 
                                 
                                 7 
                               
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 Subblock 
                                 ⁡ 
                                 
                                   [ 
                                   
                                     x 
                                     , 
                                     y 
                                   
                                   ] 
                                 
                               
                             
                           
                           ) 
                         
                         / 
                         64 
                       
                       ) 
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                   ⁢ 
                   4 
                 
               
             
             
               
                 
                   COST 
                   = 
                   
                     
                       ∑ 
                       
                         y 
                         = 
                         0 
                       
                       7 
                     
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       
                         ∑ 
                         
                           x 
                           = 
                           0 
                         
                         7 
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         abs 
                         ⁡ 
                         
                           ( 
                           
                             DC 
                             - 
                             
                               Subblock 
                               ⁡ 
                               
                                 [ 
                                 
                                   x 
                                   , 
                                   y 
                                 
                                 ] 
                               
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   Eq 
                   . 
                   
                       
                   
                   ⁢ 
                   5 
                 
               
             
           
         
       
     
     The sub-block quantization step size determination unit  503  determines the quantization step size on the sub-block basis from the input degradation cost on the sub-block basis, and outputs the quantization step size to the quantization unit  106 . Correspondence of the degradation cost on the sub-block basis and the quantization step size on the sub-block basis is obtained from a table  11  shown in  FIG. 11 . It is preferable that a threshold Th used in the table  11  shown in  FIG. 11  is about one fourth of a threshold Th shown in the conventional example (see  FIG. 10 ). Further, the quantization step size on the sub-block basis is set for each of the sub-blocks of each MB. The quantization unit  106  quantizes the input transform coefficients based on the quantization step size on the sub-block basis inputted from the sub-block quantization step size determination unit  503 , and outputs the quantized transform coefficients to the entropy encoding unit  507  and the inverse quantization unit  108 . 
     The entropy encoding unit  507  transforms encoded control information such as the input quantized transform coefficients and the quantization step size into a bit stream. Further, the entropy encoding unit  507  outputs the amount of codes generated when the information is transformed into the bit stream to the quantization step size determination unit  103 . That is, the entropy encoding unit  507  multiplexes and outputs control information encoded with a same or different quantization parameter for each of the sub-blocks in the bit stream. 
     Subsequently, the sub-block quantization information multiplexing unit  511  of the entropy encoding unit  507  will be described. The sub-block quantization information multiplexing unit  511  multiplexes information of Δ value for the quantization step size and information of a sub-block in the MB to which the Δ value will be applied so that they can be correctly decoded in the decoding. First, it is preferable that the Δ value for the quantization step size is multiplexed in a picture header, and fixed in each picture. In an extended embodiment to the H.264 coding standard, syntax of second_qp_delta_mode_flag and second_qp_delta are added to a picture parameter set (see  FIG. 8 ). In this case, second_qp_delta_mode_flag indicates the mode information of quantization control using the Δ value on the sub-block basis. Further, second_qp_delta syntax indicates the Δ value with respect to reference quantization parameter (QP) in the picture. 
     Subsequently, it is preferable that the information of a sub-block in the MB to which the Δ value will be applied is multiplexed in an MB header. An extended embodiment to the H.264 coding standard is shown in  FIG. 9 . Two types of syntax such as second_qp_delta_flag and second_qp_delta_map are added to the MB header (see  FIG. 9 ). In this case, second_qp_delta_flag is multiplexed when second_qp_delta_mode of the picture header is not 0. If second_qp_delta_flag is 0, it indicates that quantization in the MB is performed with common QP. If second_qp_delta_flag is 1, it indicates that quantization is performed with the quantization parameter of the sub-block indicated by second_qp_delta_map to be described later, wherein the value of second_qp_delta is added to the quantization parameter. In this case, second_qp_delta_map is a syntax indicating the position of the sub-block which is quantized with a value of QP+Δ and has a code length of 3 bits or 4 bits. 
     If second_qp_delta_mode of the picture header is 1, second_qp_delta_map becomes a syntax of 3 bits indicating, e.g., eight patterns, and the mapping is performed as shown in  FIG. 6 . In the mapping shown in  FIG. 6 , the eight patterns includes four patterns each having one sub-block to which the Δ value is not applied, and the other four patterns each having two lengthwise adjacent sub-blocks to which the Δ value is not applied or two transversely adjacent sub-blocks to which the Δ value is not applied. 
     If second_qp_delta_mode of the picture header is 2, second_qp_delta_map becomes a syntax of 4 bits indicating, e.g., fourteen patterns, and the mapping is performed as shown in  FIG. 7 . 
     By using the above-described embodiment of the present invention, it is possible to control the quantization parameters on a sub-block basis, and encode the MB located at the object boundary with high quality.  FIG. 4  shows an encoded image according to the present invention. In the encoding in this embodiment, the MB is divided into sub-blocks of 8×8 pixels, and a variation in pixel values is evaluated on a sub-block basis to control the quantization step size. As a result, by performing the quantization of each sub-block depending on the visual characteristics, an image encoding device and an image encoding method with high efficiency are realized. 
     INDUSTRIAL APPLICABILITY 
     The present invention is broadly applicable to video and broadcasting fields and the like requiring an image encoding technique to compress video data by using pixel correlation and an image decoding technique to decompress the compressed encoded data into the video data. 
     EXPLANATION OF REFERENCE SYMBOLS 
     
         
           100 : encoding device 
           101 : MB division unit 
           102 : degradation cost evaluation unit 
           103 : quantization step size determination unit 
           104 : prediction unit 
           105 : transformation unit 
           106 : quantization unit 
           107 : entropy encoding unit 
           108 : inverse quantization unit 
           109 : inverse transformation unit 
           110 : reconstruction unit 
           500 : encoding device 
           502 : sub-block degradation cost evaluation unit 
           503 : sub-block quantization step size determination unit 
           507 : entropy encoding unit 
           511 : sub-block quantization information multiplexing unit