Image processing apparatus

Abstract: The invention is intended to improve image quality in a process of image coding. In an apparatus for quantizing conversion data resulted from converting image information into frequency regions and then coding the quantized data, a coefficient to be multiplied by a chrominance quantizing matrix is set depending on a coefficient to be multiplied by a luminance quantizing matrix for an improvement in color reproducibility.

Description: The present invention relates to an image processing method and apparatus, and more particularly to an image processing method and apparatus for quantizing frequency conversion data and coding the quantized conversion data. A number of color image compressing techniques have been proposed in the past. As a typical one of color still image coding techniques, there is proposed an adaptive discrete cosine transform coding method (hereinafter referred to as an ADCT method). Fig. 3 is a conceptual view showing arrangement of an image coding apparatus using the ADCT method. It is assumed that an input image is given by data converted into 8 bits, i.e., 256 gradations, for each color in a not-shown analog-to-digital converter (hereinafter referred to as an A/D converter) and the number of colors is three, i.e., luminance Y, color P B and color P R , in this case. The digital data of the luminance signal Y inputted through an input terminal 101 in Fig. 3 is subjected to two-dimensional discrete cosine transform (hereinafter abbreviated as DCT) in an N x N DCT circuit 102 in units of subblocks each comprising N x N pixels. Thereafter, the conversion coefficients are linearly quantized in a linear quantizer 113 with a quantizing step size being different for each of the conversion coefficients. The quantizing step sizes for respective conversion coefficients are given by values resulted from multiplying 2 S by quantization matrix elements of N x N in consideration of the fact that visual sensitivity for quantizing noise is different for each of the conversion coefficients. Here, S is 0 or a positive or negative integer and called a scaling factor. The image quality or the data to be generated is controlled depending on the value of S. One example of the quantization matrix elements is shown in Fig. 5 in the case of 8 x 8. A quantization matrix generation circuit 111 is controlled by a control circuit 302 and generates a quantization matrix for the luminance signal Y. A switch 114 is changed over to select the side a for supplying the quantization matrix for the luminance signal Y to a multiplier 110. Using the scaling factor S supplied from a scaling factor generation circuit 301, the multiplier 110 multiplies 2 S by each quantized element of the quantization matrix. The multiplied results are supplied to the linear quantizer 103 where the conversion coefficients are linearly quantized by using the supplied results. After the quantization, direct current conversion coefficients (hereinafter referred to as DC components) are supplied to a one-dimensional predictor 104 to calculate prediction values using several subblocks adjacent to each other, and prediction errors are subjected to Huffman coding in a Huffman encoder 105. More specifically, after dividing quantized outputs of the prediction errors into groups, the ID numbers of the groups to which the respective prediction errors belong are first subjected to Huffman coding, and which values in each group correspond to the respective prediction errors are then represented by using equi-length codes. The conversion coefficients other than the above DC components, i.e., the alternating current conversion coefficients (hereinafter referred to as AC components), are supplied to a zigzag scan circuit 106 in which the AC components are scanned in a zigzag manner with two-dimensional frequencies from low-frequency component to high-frequency component as shown in Fig. 4, followed by Huffman coding in a Huffman encoder 107. More specifically, those conversion coefficients for which the quantized outputs are not 0 (hereinafter referred to as significant coefficients) are divided into groups depending on their values. The ID numbers of the divided groups and the numbers of those conversion coefficients which are present between every two significant coefficients and for which the quantized outputs are 0 (hereinafter referred to as insignificant coefficients) are subjected in pair to Huffman coding. Subsequently, which values in each group correspond to the significant coefficients are represented by using equi-length codes. Respective code strings of the DC components and the AC components are multiplexed in a multiplexer 108 and outputted from an output terminal 109. Next, when the digital data of the chrominance signals P B , P R are inputted through the input terminal 101, the control circuit 302 controls the quantization matrix generation circuit 111 to generate quantization matrices for the chrominance signals P B , P R . The switch 114 is changed over to select the side b for supplying the quantization matrices for the chrominance signals to the multiplier 110. Subsequently, the chrominance signals processed in a like manner to the luminance signal Y as mentioned above for Huffman coding. When the compression rate requires to be increased (for a higher degree of compression) from limitations in a transmission path or other reasons, control information is entered from a input terminal 303 so that the control circuit 302 controls a scaling factor generation circuit 301 to increase the scaling factor S. Conversely, in order to obtain a high-quality image, the scaling factor S is made smaller. However, because the scaling factor S of the same value is used when coding the luminance signal Y and the chrominance signals P B , P R in the above prior art, there has accompanied a shortcoming that image quality in color areas (particularly in a red area to which human eyes are more sensitive) deteriorates to a remarkable extent, when the value of the scaling factor S is increased to obtain a high degree of compression. The present invention is concerned with providing an image processing apparatus which can realize satisfactory reproducibility of colors. European Patent Application No. EP-A-0323363 discloses image processing apparatus in which quantizing and encoding are carried out in parallel on luminance and colour data. Accordingly, in one aspect the present invention provides colour image coding apparatus as set out in claim 1. From a second aspect the present invention comprises a colour image coding method as set out in claim 12. In order that the present invention may be more readily understood, embodiments thereof will now be described by way of example and with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing a first embodiment of the present invention; Fig. 2 is a block diagram showing a second embodiment of the present invention; Fig. 3 is a block diagram showing the prior art; Fig. 4 is a diagram representing a zigzag scan; Fig. 5 is a diagram showing one example of quantization matrix elements; and Fig. 6 is a block diagram showing an application example of the present invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS In the following embodiments of the present invention, a coefficient factor to be multiplied by quantization matrix elements for a chrominance signal is controlled depending on a scaling factor value S 0 of a luminance signal Y, thereby reducing an extent of deterioration in image quality even when the scaling factor value is increased for a higher compression rate. Fig. 1 is a block diagram showing a first embodiment of the present invention. In the drawing, denoted by reference numeral 101 is an input terminal for digital image data, 102 is a DCT ciruit per N x N pixels, 103 is a linear quantizer for linearly quantizing conversion coefficients, 104 is a one-dimensional predictor for DC components, 105 is a. Huffman encoder, 106 is a zigzag scan circuit for scanning AC components in a zigzag pattern, 107 is a Huffman encoder, 108 is a multiplexer for multiplexing codes of the DC and AC components, and 109 is an output terminal. 110 is a multiplier for multiplying quantization matrix elements for luminance or chrominance by a scaling factor, 111 is a quantization matrix generation circuit for generating quantization matrices for luminance and chrominance, 112 is a control circuit for controlling the quantization matrix generation circuit and a scaling factor generation circuit, 113 is a scaling factor generation circuit for generating scaling factors or coefficients to be multiplied for a luminance signal Y and chrominance signals P B , P R , 114, 115 are select switches, and 116 is an input terminal through which control information (such as an amount of coded data) is entered to the control circuit. Note that components having the same functions as those in Fig. 3 are denoted by the same reference numerals. The following description will be made about only different parts between the prior art and this embodiment. The scaling factor generation circuit 113 is controlled by the control circuit 112 to generate the scaling factor value S 0 to be multiplied by the quantization matrix elements for the luminance signal Y and, at this time, the switch 115 is changed over to select the contact a . When conversion coefficients of the chrominance signal P B are linearly quantized by the linear quantizer 103 after coding the luminance signal Y, the scaling factor generation circuit 113 is controlled by the control circuit 112 to generate a scaling factor S B for P B , and the switch 115 is changed over to select the contact b. In this case, S B is generated as a function of S 0 , for example, as expressed below: S B = f B (S 0 ) When conversion coefficients of the chrominance signal P R are linearly quantized by the linear quantizer 103, the scaling factor generation circuit 113 is controlled by the control circuit 112 to generate a scaling factor S R for P R , and the switch 115 is changed over to select the contact c. In this case, S R is generated as a function of S 0 , for example, as expressed below: S R = f R (S 0 ) Example of function forms of S B , S R usable in this embodiment are enumerated below: i) S B = f B (S 0 ) = S 0 + XB, S R = f R (S 0 ) = S 0 - XR where XB, XR are positive constants where KB 1 , KB 2 , KR 1 , KR 2 are constants iii) S B = f B (S 0 ) = S 0 x K B (S 0 ), S R = f R (S 0 ) = S 0 x K R (S 0 ) where K B (S 0 ), K R (S 0 ) are functions of S 0 where C, A are constants Optimum ones of the above functions and constants are selected depending on the compression rate required and the nature of an input image so that deterioration of image quality is minimized. This selection can be achieved by, for example, repeating the coding process several times with different quantization characteristics or performing the coding processes with different quantization characteristics in parallel as explained later. As an alternative method of controlling respective quantization steps for the chrominance signals P B , P R , the coefficients to be multiplied by the quantization matrix elements for chrominance signals may be themseleves controlled rather than generating the scaling factors. In this case, the scaling factor generation circuit 113 generates coefficients M B , M R to be multiplied for the chrominance signals P B , P R in place of the scaling factors. M B , M R are functions of S 0 as expressed below: M B = g B (S 0 ), M R = g R (S 0 ) Example of function forms of M B , M R usable in this case are enumerated below: v) M B = g B (S 0 ) = 2 S 0 + TB, M R = g R (S 0 ) = 2 S 0 - TR where TB, TR are positive constants vi) M B = g B (S 0 ) = CB x 2 S 0 , M R = g R (S 0 ) = CR x 2 S 0 where CB, CR are constants vii) M B = g B (s 0 ) = 2 S 0 x C B (S 0 ), M R = g R (S 0 ) = 2 S 0 x C R (S 0 ), where C B (S 0 ), C R (S 0 ) are functions of S 0 Optimum ones of the above functions and constants are selected depending on the compression rate required and the nature of an input image so that deterioration of image quality is minimized. The compression rate can be changed by entering control information to the control circuit 112 through the input terminal 116 and changing the scaling factor value S 0 . The mode for the method of deciding the quantizing steps for the chrominance signals P B , P R can also be changed by entering mode information to the control circuit 112 through the input terminal 116. As mentioned above, since the scaling factor values of the luminance signal Y and the chrominance signals P B , P R are set by controlling the scaling factor values of the chrominance signals P B , P R depending on the scaling factor value S 0 of the luminance signal Y (such that in need of increasing the compression rate, a smaller scaling factor value is set for P R which has higher visual sensitivity and a larger scaling factor value is set for P B ), an image with less deterioration in image quality can be obtained even when the compression rate is increased. Further, by controlling the coefficients to be multiplied by the quantization matrix elements for the chrominance signals using function such as f B , f R , g B , g R , the coding can be performed more efficiently than the case of setting those coefficients individually. Moreover, even with the image data having other input form (such as GBR inputs) than the foregoing, the similar operating effect can also be achieved by weighing the coefficients to be multiplied by the quantization matrix elements in a like manner to the above case in match with the visual sensitivity of human eyes (e.g., such that in the case of GBR inputs, weighing is applied to the G input near a luminance signal, the B input and the R input in this order). (Another Embodiment) Fig. 2 shows a second embodiment of the present invention. Components having the same functions as those in the prior art are denoted by the same reference numerals and will not be described here. Blocks a, b and c circumscribed by broken lines all function in a like manner. However, an input to the block a is only the luminance signal Y, an input to the block b is only the chrominance signal P B , and an input to the block c is only the chrominance signal P C . Thus, the luminance signal and the chrominance signals are separately processed in parallel for coding. Accordingly, a control circuit 222 supplies a control signal suitable for parallel processing to a quantization matrix generation circuit 221 so that the quantization matrices for the luminance signal and the chrominance signals are generated almost at the same timing. Likewise, a scaling factor generation circuit 220 is controlled by the control circuit 222 to generate scaling factors S Y , S B , S R for the luminance signal Y and the chrominance signals P B , P R almost at the same timing. S Y , S B , S R are functions of S 0 as expressed below: S Y = S 0 , S B = f B (S 0 ), S R = f R (S 0 ) Alternatively, the scaling factor generation circuit 220 may generate coefficients to be directly multiplied by the quantization matrix elements for the chrominance signals. In this case, coefficients M B , M R used for the chrominance signals M B , M R are given by: M B = g B (S 0 ), M R = g R (S 0 ) By so arranging, it is possible to not only obtain an image with less deterioration in image quality even at the increased compression rate, but also adapt to high-speed coding through parallel processing, which permits real-time coding of moving images and the like. As explained above, by setting the scaling factor values or the coefficients themselves to be multiplied by the quantization matrix elements for the luminance signal Y and the chrominance signals P B , P R separately, and controlling the scaling factor values or the coefficients themselves to be multiplied by the quantization matrix elements for the chrominance signals P B , P R depending on the scaling factor value S 0 of the luminance signal Y, there can be obtained an image with less deterioration in image quality corresponding to the compression rate (namely, an image with less deterioration in quality of color images can be obtained even at the increased compression rate through such a control, for example, that in need of increasing the compression rate, a smaller scaling factor value than that of the luminance signal Y is set for the chrominance P R which has higher visual sensitivity and a larger scaling factor value is set for the chrominance signal P B ). Further, by controlling the coefficients to be multiplied by the quantization matrix elements for the chrominance signals using functions, the coding can be performed more efficiently than the case of setting those coefficients individually. Thus, according to the present invention, it is possible to provide an image encoding apparatus and method with which a reproduced image of high quality can be obtained. An example of the present invention applied to a moving image encoding apparatus will be described below in detail. Fig. 6 is a block diagram of the apparatus. In the drawing, denoted by 20 is an input terminal for an analog TV signal. The TV signal sensed by using a CCD sensor and inputted through the terminal 20 is converted by an A/D converter 22 into a digital signal of 8 bits. The digital signal is divided into blocks each comprising (8 x 8) pixels by an (8 x 8) block formation circuit 26 which operates in a like manner to the (N x N) DCT circuit 102 in Fig. 1, the blocks being supplied to a DCT circuit 28 one by one. Pixel data D 11 to D 88 of each block are converted by the DCT circuit 28 into a data matrix X 11 to X 88 for two-dimensional frequency regions as with the case of Fig. 1, followed by being supplied to quantization circuits 32a to 32d and a 1F (one-frame) delay circuit 34. A quantization matrix generation circuit 36 generates similar matrices to those generated from the aforesaid quantization matrix generation circuit 111. Supplied to multipliers 38a to 38d are initial coefficients C 1 to C 4 as control coefficients (scaling factors) C from a coefficient calculator 48. As with the case of Fig. 1, the multipliers 38a to 38d multiply outputs of the quantization matrix generation circuit 36 and the control coefficients C X for respective components, i.e., (W ij x C X /C 0 ), the multiplied results being outputted to the quantization circuits 32a to 32d. The quantization circuits 32a to 32d respectively quantize conversion coefficients with the quantizing steps controlled by the control coefficients C 1 to C 4 and output the quantized values to VLCs (Variable Length Coders) 40a to 40d. In this embodiment, the VLCs 40a to 40d do not output the actual coded data, but output only information nb1 to nb4 of total bit number for each image as resulted from the similar processing to an ordinary VLC. Using the information nb1 to nb4 outputted- from the VLCs 40a to 40d and the control coefficients C 1 to C 4 , the coefficient calculator 48 calculates a control coefficient C 5 desired for the total bit number and outputs it to a multiplier 38e. The coefficient calculator 48 also outputs control coefficients C 6 , C 7 as an approximate value of C 5 to multipliers 38f, 38g and, on the other hand, outputs the control coefficients C 5 to C 7 to terminals 44a to 44c, respectively. It is here assumed that when C n ≦ C 5 ≦ C n+1 holds for C 5 , C 6 and C 7 meet the relationships C n ≦ C 6 < C 5 and C 5 < C 7 ≦ C n+1 , respectively, and n meets the relationship 1 ≦ n ≦ 3. The multipliers 38e to 38g respectively multiply outputs of the quantization matrix generation circuit 36 and the control coefficients C 5 to C 7 as with the multipliers 38a to 38d, the multiplied results being outputted to quantization circuits 32e to 32g within a vertical blanking period. Meanwhile, those DCT conversion coefficients resulted from delaying DCT conversion coefficients through one frame by the 1F delay circuit 34 are inputted to the quantization circuits 32e to 32g. The conversion coefficients are quantized in the quantization circuits 32e to 32g with the quantizing steps controlled by the control coefficients C 5 to C 7 , and the quantized values are outputted to VLCs 40e to 40g. In this embodiment, the VLCs 40e to 40g output the actual coded data to entry buffers 42a to 42c and also output the total bit numbers nb5 to nb7 to a judgment circuit 50. The judgment circuit 50 delivers such an output to switches 44 and 46 as to select the quantization result which is less than the desired total bit number and nearest thereto. The entry buffers 42a to 42c store the coded data until the judgment circuit 50 issues the judgment result. The switch 44 selects one of the control coefficients in accordance with the judgment result and outputs it to a multiplexer 52. Also, the switch 46 selects one of the coded data and outputs it to the multiplexer 52. The multiplexer 52 multiplexes the control coefficient and the coded data, followed by outputting to an output terminal 54. The scaling factor for quantizing the chrominance data may be controlled independently rather than being dependent on the scaling factor for quantizing the luminance data as stated before. In other words, the luminance data and the chrominance data may be coded in a completely parallel manner. The scaling factor may be fixed in one frame of an image as mentioned before, or may be varied per block of N x N pixels. Further, the scaling factor may be controlled depending on the amount of coded data resulted from the coding, or may be controlled depending on whether the original image is a line image including characters and the like or a natural image. Additionally, while the component Y is extracted as luminance data and the components P B , P R are extracted as chrominance data in the above embodiments, those components may be (Y, C R , C B ), (L*, a*, b*), (L*, u*, v*), (Y, u, v) or the like. Moreover, the functions of the control circuit 112, the scaling factor generation circuit 113 and the quantization matrix generation circuit 111 can be executed by a CPU and a RAM, a ROM, etc. connected to the CPU. It should be understood that the present invention is not limited to the embodiments as set forth above and may be variously changed and modified within the scope of the invention defined in the attached claims.

Claims: A colour image coding apparatus comprising: input means (20) for inputting image data representing a colour image and including luminance and chrominance data; quantizing means (32e-32g) for quantizing the image data; and encoding means (40e-40g) for encoding the quantized image data, characterized in that said quantizing means includes a first quantizer (32e) for quantizing the image data by using a first quantizing parameter and a second quantizer (32g) for quantizing the same image data by using a second quantizing parameter different from the first quantizing parameter each of said first and second quantizing being operative to quantize said luminance data and chrominance data, said encoding means includes a first encoder (40g) for encoding the image data quantized by said first quantizer and a second encoder (40e) for encoding the image data quantized by said second quantizer, and said first and second quantizers quantize the same image data in parallel and said first and second encoders perform encoding operations in parallel. Apparatus according to claim 1 and further comprising means (50) for selecting between the outputs of said first and second encoders so as to select for output the quantization result which is both less than and nearest to a desired data amount. Apparatus according to claim 1 or claim 2, including means for selecting quantizing parameters by evaluating a corresponding parameter which relates to the contents of the colour image. Apparatus according to any preceding claim, further comprising means (48) for generating said first and second quantizing parameters. Apparatus according to any one of the preceding claims, further comprising evaluating means for evaluating amounts of encoded image data to be output from said first and second encoding means. Apparatus according to claim 3, wherein said evaluating means include: first and second initial quantizers (32a-32d); a quantization matrix generation circuit (36) adapted to generate initial control coefficients for said initial quantizers; first and second initial encoding means associated with the outputs of said first and second initial quantizers and adapted to supply bit number information to a coefficient calculator (48) adapted to calculate control coefficients for said first and second quantizers. Apparatus according to any one of the preceding claims, further comprising converting means (28) for converting said image data into spatial frequency components and wherein the output of said converting means is supplied to said first and second quantizers. Apparatus according to any one of the preceding claims, further comprising generating means (36) for generating a quantizing matrix for said first and second quantizers. Apparatus according to any one of the preceding claims, wherein said encoders are adapted to perform variable-length coding. Apparatus according to any one of the preceding claims, wherein said input means comprise a CCD sensor. Apparatus according to any one of the preceding claims and adapted to encode a moving colour image. A method of encoding a colour image comprising: inputting image data representing a colour image and including luminance and chrominance data; quantizing the image data; and encoding the quantized image data, characterized in that the quantizing step includes quantizing the image data by utilising a first quantizer which uses a first quantizing parameter and quantizing in parallel the same image data by utilising a second quantizer which uses a second quantizing parameter different from the first quantizing parameter, each of said first and second quantizer quantizing said luminance and said chrominance data, and encoding in parallel the image data quantized using said first quantizing parameter and the image data quantized using said second quantizing parameter. A method according to claim 12 and including selecting (50) between the encoded first and second image data which has been encoded in parallel so as to select for output the quantization result which is both less than and nearest to a desired data amount. A method according to claim 12 or 13, further comprising the step of generating (48) said first and second quantizing parameters. A method according to any one of claims 12, 13 or 14, and including selecting quantizing parameters by evaluating a corresponding parameter which relates to the contents of the colour image. A method according to any one of claims 12 to 15 comprising: converting said image data into spatial frequency components and supplying the spatial frequency components for quantization using said first and second quantizing parameters via a delay circuit. A method according to any one of claims 12 to 16, wherein encoding is carried out by variable-length encoders. A method according to any one of claims 12 to 17, comprising inputting the image data representing a colour image by means of a CCD sensor. A method according to any one of claims 12 to 18, wherein the input image data represents a moving colour image.

Inventors: FUJII AKIO, FUJII, AKIO, FUJII, AKIO, C/O CANON KABUSHIKI KAISHA
Applicants: CANON KK, CANON KABUSHIKI KAISHA
Application year: 1993
Main classification: H04N11
IPC codes: G06T9, H04N11, H04N7