Abstract:
Image coding having high coding efficiency is provided with a small computational complexity by properly determining a prediction mode and a quantization parameter. When an image is coded by a prediction mode selected from plural prediction modes every any coding unit, an undetermined multiplier λ is first calculated from a quantization parameter (function S 301 ) Subsequently, the cost of each of R-D points (pairs of number of coded bits and coding distortion that correspond to plural combinations of prediction modes and quantization parameters) is calculated (function S 305 ) on the basis of the undetermined multiplier λ while generation and estimation of the R-D points and deletion of the points (functions S 302 -S 304 ) are repeated every coding unit, for example, every macroblock, and the optimal combination of the prediction mode and the quantization parameter is determined on the basis of the R-D point providing the minimum cost.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2004-372357, filed on Dec. 23, 2004; the entire contents of which are incorporated herein by reference.  
       TECHNICAL FIELD  
       [0002]     The present invention relates image coding control method and device for motion pictures or still images.  
       BACKGROUND OF THE INVENTION  
       [0003]     According to a motion picture coding standard system such as ITU-T H.261, H.263, H.264, ISO/IEC MPEG-2, MPEG-4 part 2 or the like, coding is carried out every coding unit called as a macroblock while switching a prediction mode. It is said that it greatly affects the coding performance how the switching operation is carried out.  
         [0004]     One of the methods of switching the prediction mode is disclosed in TMN9 which is a test model of ITU-T H. 263 (see Non-patent Document 1, ITU-T, Study Group 16: “Video Codec Test Model, Near-term, Version 9 (TMN9)”, Document: Q15-C-15, 1997) In the Non-patent Document 1, threshold value processing is carried out by using the absolute value difference sum between an input picture and a predictive picture achieved by motion compensation, and the absolute value difference sum between the input picture and the macroblock of the input picture, whereby three prediction modes of an inter-frame prediction mode of 8×8 blocks, an inter-frame prediction mode of 16×16 blocks and an intra-frame prediction mode are switched to one another.  
         [0005]     Furthermore, a method of selecting the prediction mode of each macroblock on the basis of Lagrange&#39;s undetermined multiplier method is introduced in a non-patent document 2 (Gary J. Sullivan and Thomas Wiegand, “Rate-Distortion Optimization for Video Compression”, IEEE Signal Processing Magazine, Vol. 15, No. 6, pp 74-90, November 1998). More specifically, coding is actually carried out in each prediction mode to determine a number of coded bits (rate) and a coding distortion. Thereafter, the cost of each prediction mode to the same quantization parameter is calculated under the condition that the Lagrange&#39;s undetermined multiplier is assumed as the function of the quantization parameter, and the prediction mode in which the cost is minimum is selected. This publication reports that the coding efficiency is more greatly enhanced as compared with TMN9 by this method.  
         [0006]     The switching method of the prediction mode disclosed in the non-patent document 1 is an easy method, and the switching operation of the prediction mode can be performed with small computational complexity. However, this method pays no attention to the actual number of coded bits (rate) and the distortion, and thus it can be hardly said that the switching operation of the prediction mode which is optimal in the relationship between the number of coded bits and the distortion is carried out. Accordingly, this method does not greatly enhance the coding efficiency.  
         [0007]     According to the method disclosed in the non-patent document 2, a number of coded bits—a coding distortion function having the same gradient with respect to the same quantization parameter is assumed. Therefore, when the quantization parameter is varied or when the shape of the number of coded bits—coding distortion function is different from the assumed shape, it is impossible to carry out accurate comparison estimation on the coding cost, and thus a proper prediction mode cannot be selected. Furthermore, it is also impossible to compare the coding cost when the quantization parameter is varied under the combination with the number of coded bits control. Accordingly, it cannot be expected that the coding efficient is greatly enhanced by this method.  
         [0008]     Therefore, Japanese Patent Application No. 2004-177190 discloses a method of selecting not only an optimal prediction mode, but also an optimal quantization parameter. According to this method, not only the prediction mode of each macroblock, but also the value of the quantization parameter is varied to calculate the cost, and the prediction mode and the quantization parameter that provides the minimum cost are selected. This method more greatly enhances the coding efficiency as compared with the non-patent document 2. However, the frequency at which the coding is actually carried out by using each prediction mode and each quantization parameter is large, and thus the computational complexity may be increased.  
         [0009]     Therefore, an object of the invention is to provide image coding control method and device in which image coding having high coding efficiency can be carried out with a small computational complexity by properly determining a prediction mode and a quantization parameter.  
       BRIEF SUMMARY OF THE INVENTION  
       [0010]     According to embodiments of the invention, an image coding control method for controlling a quantization parameter by using one prediction mode selected from plural prediction mode every any coding unit, comprises: calculating an undetermined multiplier corresponding to a parameter used for a cost function from a given reference quantization parameter; selecting a set of secondary number of coded bits—coding distortion pairs from a set of primary number of coded bits—coding distortion pairs that correspond to plural combinations of the prediction mode and the quantization parameter every coding unit; selecting number of coded bits—coding distortion pairs providing the minimum cost function minimum as the optimal number of coded bits—coding distortion pairs from the set of the secondary number of coded bits—coding distortion pairs every coding unit by using the undetermined multiplier thus calculated; and selecting a prediction mode and a quantization parameter used for the coding from the optimal number of coded bits—coding distortion pairs, wherein when the set of the secondary number of coded bits—coding distortion pairs is selected every coding unit,  
         [0011]     (1) a convex hull comprising the set of the primary number of coded bits—coding distortion pairs is formed on a two-dimensional orthogonal coordinate system in which the number of coded bits is set as one axis and the coding distortion is set as the other axis,  
         [0012]     (2) a set on the convex hull that connect the position of a number of coded bits—coding distortion pair providing the minimum number of coded bits and the position of a number of coded bits—coding distortion pair providing the minimum coding distortion is determined, and  
         [0013]     (3) a set of number of coded bits—coding distortion pairs that is on the convex hull and exists at a nearer side to the origin of the two-dimensional orthogonal coordinate system is selected as a set of the secondary number of coded bits—coding distortion pairs.  
         [0014]     According to the embodiments of the invention, by forming the set of the secondary number of coded bits—coding distortion pairs for sequentially forming the convex hull, the prediction mode and the quantization parameter to be used for the coding can be determined from the optimal number of coded bits—coding distortion pairs while considering the overall computational complexity. Accordingly, the proper prediction mode and the quantization parameter are determined with a small computational complexity, whereby an image coding operation can be performed with high coding efficiency.  
         [0015]     Furthermore, when a set of secondary number of coded bits—coding distortion pairs is selected, the set of the primary number of coded bits—coding distortion pairs located in the convex hull has a low probability to minimize the cost, and thus they are sequentially deleted from candidates, whereby the computational complexity is expected to be reduced without lowering the coding efficiency. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0016]      FIG. 1  is a block diagram showing the construction of an image coding device according to an embodiment of the invention;  
         [0017]      FIG. 2  is a block diagram showing the construction of a frame memory/predictive image creator in  FIG. 1 ;  
         [0018]      FIG. 3  is a flowchart showing the processing flow of a coding controller and a mode selector of  FIG. 1 ;  
         [0019]      FIG. 4  is a diagram showing an R-D point that is encoded by a reference quantization parameter;  
         [0020]      FIG. 5  is a diagram showing a specific example of a method of finding out a lower left convex hull to determine an R-D point set;  
         [0021]      FIGS. 6A and 6B  are diagrams showing a specific example of a method of finding out a lower left convex hull to determine the R-D point set;  
         [0022]      FIG. 7  is a diagram showing the processing of deleting a prediction mode that does not exist at the lower left convex hull in the R-D point set;  
         [0023]      FIG. 8  is a diagram showing a process of estimating a number of coded bits and a coding distortion from actually-measured values;  
         [0024]      FIG. 9  is a diagram showing a process of selecting an R-D point having the minimum cost from the R-D point set; and  
         [0025]      FIG. 10  is a diagram showing an R-D plane to explain a left convex hull. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0026]     Embodiments according to the invention will be described hereunder with reference to the accompanying drawings.  
         [0000]     (1) Construction of Image Coding Device  
         [0027]      FIG. 1  shows an image coding device for coding a motion picture according to an embodiment of the invention.  
         [0028]     A motion picture signal is input as an image signal  100  to the image coding device on a frame basis.  
         [0029]     The difference between an input image signal  100  and a predictive image signal  102  is taken by a subtracter  101  to generate a prediction error signal  103 . On the prediction error signal  103  thus generated, orthogonal transformation, for example, discrete cosine transformation (DCT) is conducted by an orthogonal transformer  104 . Orthogonal transformation coefficient information  105 , for example, DCT coefficient information is achieved in the orthogonal transformer  104 . The orthogonal transformation efficiency information  105  is quantized by a quantizer  106  to achieve quantized orthogonal transformation coefficient information  107 . The quantized orthogonal transformation coefficient information  107  is input to both an entropy encoder  108  and an inverse quantizer  109 .  
         [0030]     The quantized orthogonal transformation coefficient information  107  input to the inverse quantizer  109  is successively subjected to the opposite processing to the processing of the quantizer  106  and the orthogonal transformer  104  by the inverse quantizer  109  and the inverse orthogonal transformer  110  to achieve a signal similar to the prediction error signal, and then the signal is added with the predictive image signal  102  in an adder  110 , thereby generating a local decoded image signal  112 . The local decoding image signal  112  is input to the frame memory/predictive image generator  108 .  
         [0031]     The frame memory/predictive image generator  113  generates the predictive image signal from the input image signal  100  and the local decoded image signal  112 . The details of the frame memory/predictive image generator  113  will be described later.  
         [0032]     In the entropy encoder  108 , the quantized orthogonal transformation coefficient information  107  and motion vector information/prediction mode information  114  are subjected to entropy encoding. Respective codes thus generated are multiplexed by a multiplexer  116 , and then smoothened by an output buffer  117 .  
         [0033]     The encoded data  118  output from the output buffer  117  are transmitted to a transmission system or accumulation system (not shown).  
         [0034]     A mode selector  121  directly controls a switch in the frame memory/predictive image generator  113  to select a prediction mode, and also controls the quantization parameter. The details of the mode selector  121  will be described later.  
         [0035]     A coding controller  119  controls a coding portion  115  containing elements from the subtracter  101  to the frame memory/predictive image generator  113 . In this embodiment, a number of coded bits is allocated every coding unit while considering the buffer amount of the buffer  117 . This embodiment uses a macroblock unit as the coding unit, for example, however, the invention is not limited to this mode.  
         [0000]     ( 2 ) Frame Memory/Predictive Image Generator  113   
         [0036]      FIG. 2  shows a specific example of the frame memory/predictive image generator  113  for generating the predictive image signal  102  from the input image signal  100  and the local decoded image signal  112 .  
         [0037]     The local decoded image signal  112  from the adder  111  of  FIG. 1  is temporarily accumulated in a frame memory  200 .  
         [0038]     A motion vector detector  201  carries out the matching (block matching) between the input image signal  100  and the local decoded image signal accumulated in the frame memory  200  every block in the frame to detect a motion vector.  
         [0039]     An inter-frame predictor  202  subjects motion compensation to the local decoded image signal in the frame memory  200  on the basis of the motion vector detected in the motion vector detector  201  to generate a predictive image signal based on the inter-frame prediction.  
         [0040]     On the other hand, a intra-frame predictor  203  generates a predictive image signal based on the intra-frame prediction from the input image signal  100  and the local decoded image signal of an area which has been already coded in a frame of the frame memory  200 .  
         [0041]     The inter-frame predictor  202  has inter-frame prediction modes of K (K represents an integer of two or more), and the intra-frame predictor  203  has intra-frame prediction modes of L (L represents an integer of two or more). A switch  204  is connected to the outputs of the inter-frame predictor  202  and the intra-frame predictor  203 . The switch  204  is controlled by the mode selector  121 , and outputs a predictive image signal based on a prediction mode selected from the inter-frame prediction modes of K or a predictive image signal based on a prediction mode selected from the intra-frame prediction modes of L.  
         [0042]     The motion vector information output from the motion vector information/prediction mode  114 , that is, the motion vector detector  201  and the prediction mode information indicating the prediction mode selected in the switch  204  by the mode selector  121  are transmitted to the entropy encoder  108 . The motion vector information is output from the motion vector detector  201  only when the inter-frame prediction mode is selected.  
         [0000]     ( 3 ) Coding Controller  119  and Mode Selector  121   
         [0043]      FIG. 3  is a flowchart showing the flow of the operation of the coding controller  119  and the mode selector  121  in  FIG. 1 .  
         [0044]     First, the coding controller  119  controls the coding portion  115  to successively carry out the coding operation in plural prediction modes (the inter-frame prediction modes of K and the intra-frame prediction modes of L) and also switch the quantization parameter in each prediction mode.  
         [0045]     An undetermined multiplier λ is calculated from the reference quantization parameter Q 0  given from the coding controller  109  (step S 301 ).  
         [0046]     Subsequently, three reference quantization points are set to carry out the coding operation in each prediction mode, and a set of number of coded bits R—coding distortion D pairs (hereinafter referred to as “R-D point”) is created (step S 302 ).  
         [0047]     A lower left convex hull set is formed by the R-D point set thus created, and a prediction mode which does not exist on the lower left convex hull plane is deleted (step S 303 ). Here, the “lower left convex hull set” will be described. A convex hull is formed from an R-D point set existing on a two-dimensional orthogonal coordinate system in which the number of coded bits R is set to the abscissa axis and the coding distortion D is set to the ordinate axis. The lower left convex hull set is an R-D point set that exists on a convex hull connecting the position of the R-D point providing the minimum number of coded bits R in the convex hull and the position of the R-D point providing the minimum coding distortion D and also exists at a nearer side to the origin of the two-dimensional orthogonal coordinate system. The details will be described later.  
         [0048]     An R-D point of a quantization parameter which is not coded in the residual R-D points is estimated(step S 304 ).  
         [0049]     An R-D point providing the minimum cost is selected from the R-D point set and set as a new reference quantization parameter (step S 305 ).  
         [0050]     When coding is carried out at a predetermined coding frequency or it is impossible to set the reference quantization point because coding has been already carried out, the processing is finished, and the prediction mode and the quantization parameter providing the minimum cost that are selected in the step S 305  are output. If not so, the processing returns to step S 302  to repeat the processing.  
         [0051]     Next, specific examples of the processing of each of the steps S 301  to S 306  of  FIG. 3  will be described.  
         [0000]     [Step S 301 ] 
         [0052]     The undetermined multiplier λ is estimated from the reference quantization parameter Q 0  given from the coding controller  109  by using the following equation (1), for example. 
 
λ= f× exp( gQ   0 )  (1) 
 
         [0053]     Here, f and g represent constants determined on the basis of the prediction structure of picture or slice, such as a picture type and a slice type.  
         [0000]     [Step S 302 ] 
         [0054]     The coding is carried out on the basis of totally three quantization parameters, that is, the reference quantization parameter Q 0  given from the coding controller  109  and the two quantization parameters Q min  and Q max  before and after the value of the reference quantization parameter every prediction mode to determine a number of coded bits R and a coding distortion D (see  FIG. 4 ).  
         [0055]     Here, the coding distortion D is calculated on the basis of the square error between an input image signal (org) and a local decoded image signal (coded) of each of the brightness signal Y and the color difference signals Cb and Cr every macroblock.  
             D   =         ∑     i   =   0     15     ⁢       ∑     j   =   0     15     ⁢     (         Y   org     ⁡     (     i   ,   j     )       -       Y   coded     ⁡     (     i   ,   j     )         )         +       ∑     i   =   0     7     ⁢       ∑     j   =   0     7     ⁢     (         Cb   org     ⁡     (     i   ,   j     )       -       Cb   coded     ⁡     (     i   ,   j     )         )         +       ∑     i   =   0     7     ⁢       ∑     j   =   0     7     ⁢     (         Cr   org     ⁡     (     i   ,   j     )       -       Cr   coded     ⁡     (     i   ,   j     )         )                   (   2   )             
 
 In place of the square errors of the brightness signal Y and the color difference signals Cb and Cr, the coding distortion may be the square error of only the bright signal, or any value such as an absolute value differential sum, a value weighted on the basis of a visual characteristic or the like insofar as it is a value representing the coding distortion. 
 
 [Step S 303 ]
 
         [0056]     Subsequently, a lower left convex hull is formed by the R-D point set, and a prediction mode which does not exist on the convex hull plane is deleted.  
         [0057]     The set of the lower left convex hull will be hereunder described on the R-D plane corresponding to the two-dimensional orthogonal coordinate system shown in  FIG. 10  in which the number of coded bits R is set to the abscissa axis and the coding distortion D is set to the ordinate axis.  
         [0058]     A convex hull comprising the set of the primary number of coded bits—coding distortion pairs is formed on the two-dimensional orthogonal coordinate system. This convex hull is a set having a shape nearer to a closed elliptical shape.  
         [0059]     There is considered a set located on the convex hull connecting the position A of the number of coded bits—coding distortion pair providing the minimum number of coded bits R and the position B of the number of coded bits—coding distortion pair providing the minimum coding distortion D. Since the shape of the convex hull is nearer to the elliptical shape, two kinds of sets, that is, a set existing at a side nearer to the origin of the two-dimensional orthogonal coordinate system (the set of the lower left convex hull and it is indicated by a solid line of  FIG. 10 ), and a set existing at a side farther from the origin (the set of the upper right convex hull and it is indicated by a dashed line) exist as the set on the convex hull.  
         [0060]     Therefore, if the set of the number of coded bits—coding distortion pairs existing at the side near to the origin is a set of secondary number of coded bits—coding distortion pair, the set of the lower left convex hull could be selected.  
         [0061]     The set existing at the side nearer to the origin will be further described while considering the cost function (J=D+λ·R) described later.  
         [0062]     First, the position A of the minimum number of coded bits pair corresponding to the number of coded bits—coding distortion pair providing the minimum number of coded bits R in the convex hull is determined.  
         [0063]     Subsequently, the position B of the minimum coding distortion pair corresponding to the number of coded bits—coding distortion pair providing the minimum coding distortion D in the convex hull is determined.  
         [0064]     Next, a straight line L (J=D+λ·R) passing through the position A of the minimum number of coded bits pair and the position B of the minimum coding distortion pair thus determined is considered. When the gradient λ of the line L is represented by λ 1  and the intercept J of the line L is represented by J 1 , the condition: J 1 ≧J=D+λ 1 ·R is satisfied at the lower side of the lower left convex hull. Accordingly, the set of the number of coded bits—coding distortion pairs at which the value Q of the cost function determined by the common undetermined multiplier λ 1  corresponding to the above gradient is equal to or lower than the value of the minimum number of coded bits pair or the minimum coding distortion pair is set as the set of the secondary number of coded bits—coding pairs. This set corresponds to the set of the lower left convex hull.  
         [0065]     A method of finding out the lower left convex hull as shown in  FIG. 7  will be described with reference to  FIGS. 5 and 6 .  
         [0066]     First, the primary R-D point set is rearranged in an increasing or decreasing order with the value of the coding distortion D as a reference, for example. Actually, the coding distortion D of each prediction mode is increased as the quantization parameter is larger. Therefore, the rearrangement is unnecessary, and the primary R-D point set may be merged in increasing or decreasing order.  FIG. 6  shows a case where the primary R-D points of  FIG. 5  are ordered on the basis of the value of the coding distortion D, and the coding distortion D is increased in the connection order of the primary R-D points connected to one another by a broken line.  
         [0067]     Next, as shown in  FIGS. 6A and 6B , each point of the primary R-D point set is successively added with the next one point in the increasing order or decreasing order of the coding distortion D, and the shape of a broken line connecting the latest three points is checked. Here, if the shape of the broken line is a convex shape under the view from the lower left side as shown in  FIG. 6A , the three points are left as points of the secondary R-D point set, and the next point is added. On the other hand, if the shape of the broken line is a concave shape under the view from the lower side as shown in  FIG. 6B , the point in the middle of the three points is deleted.  
         [0068]     The processing as described above is carried out on all the points of the R-D point set, and the residual R-D points at the time when the processing concerned is finished are set as a lower left convex hull plane set. By checking the shape of the broken line at the frequency corresponding to the number of all the points, it can be judged whether the processing on all the points of the R-D point set is finished.  
         [0069]     At this time, with respect to prediction modes which never belong to the lower left convex hull plane set, they are deleted from the subsequent candidates of the prediction modes. Specifically, a mode  1  and a mode  4  shown in  FIG. 7  never belong to the lower left convex hull plane set, and thus they are deleted from the candidates at this time point.  
         [0000]     [Step S 304 ] 
         [0070]     Subsequently, with respect to the residual prediction modes, the number of coded bits R and coding distortion D of a quantization parameter which has not been coded are estimated from the points of the actually coded quantization parameter before and after the non-coded quantization parameter concerned as shown in  FIG. 8 . It is assumed that the number of coded bits R and the coding distortion D satisfy the relationship shown by the equations (3) and (4) with the quantization parameter represented by Q. In the following equations, a, b, c, d represents coefficients. 
 
log R=aQ+b   (3) 
 
log D=cQ+d   (4) 
 
         [0071]     The coefficients a, b, c, d assumed in the equations (3) and (4) are estimated from the following equations (5) to (8) by using the numbers of coded bits R 1 , R 2  and the coding distortions D 1 , D 2  of the actually-coded two points before and after the non-coded quantization parameter.  
             a   =         log   ⁢           ⁢     R   2       -     log   ⁢           ⁢     R   1             Q   2     -     Q   1                 (   5   )               b   =           Q   2     ⁢   log   ⁢           ⁢     R   1       -       Q   1     ⁢   log   ⁢           ⁢     R   2             Q   2     -     Q   1                 (   6   )               c   =         log   ⁢           ⁢     D   2       -     log   ⁢           ⁢     D   1             Q   2     -     Q   1                 (   7   )               d   =           Q   2     ⁢   log   ⁢           ⁢     D   1       -       Q   1     ⁢   log   ⁢           ⁢     D   2             Q   2     -     Q   1                 (   8   )             
 
 From these coefficients a, b, c, d, the number of coded bits R and the coding distortion D of each quantization parameter which skip some values are estimated from the four coefficients a, b, c, d according to the following equations (9) and (10). 
 
 {circumflex over (R)} ( Q )=exp( aQ+b )  (9) 
 
 {circumflex over (D)} ( Q )=exp( cQ+d )  (10) 
 
 Referring to  FIG. 8 , white circular points represent actually measured values, and white rectangular points represent estimation values estimated in the above step. The points thus estimated are also added to the R-D point set. 
 
 [Step S 305 ]
 
         [0072]     Subsequently, the R-D point providing the minimum cost is selected from the R-D point set. Here, the cost calculation of each point is carried out according to the cost function of the equation (11). λ represents the undetermined multiplier determined in the step S 301 . 
 
 J=D+λR   (11) 
 
         [0073]     The quantization parameter of the point providing the minimum cost in the R-D point set as shown in  FIG. 9  is set as a new reference quantization parameter Q 0 .  
         [0000]     [Step S 306 ] 
         [0074]     In this case, when the coding frequency for the R-D point calculation exceeds a predetermined frequency, or when the new reference quantization parameter corresponding to a point which has been already coded, it is judged that the condition is satisfied, the processing is finished, and the quantization parameter and the prediction mode of the point selected in step S 305  are output.  
         [0075]     If the condition is not satisfied, the processing returns to step S 302 , and the processing of steps S 302  to S 306  is repeated until the above condition is satisfied.  
         [0000]     (4) Modification  
         [0076]     In the above embodiment, the macroblock is set as the coding unit, and a pair of prediction mode and quantization parameter is determined at every macroblocks. However, the coding unit may be set to plural macroblocks, or it may be set to another unit such as slice, field, frame, picture or GOP.  
         [0077]     Furthermore, the above embodiment has been described by taking the motion picture coding as an example. However, the invention may be applied to still picture coding.