Image encoding apparatus, image encoding method, and image encoding program

An image encoding apparatus, an image encoding method, and an image encoding program capable of minimizing image degradation, controlling a code amount in units smaller than a picture, and ensuring that encoding of the picture is completed within a certain time period is provided. A symbol number estimating device estimates the total bin number of a plurality of macro blocks constituting a picture. An entropy encoding selector outputs the entropy encoding mode selecting signal to an entropy encoder using the inputted bin number, in order to select one of a CABAC device or a VLC device. A CPU performs image encoding by executing a control program stored in a program storing portion.

This application is the National Phase of PCT/JP2008/058166, filed Apr. 28, 2008, which is based upon and claims priority from Japanese Patent Application No. 2007-134162 filed May 21, 2007.

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates to an image encoding apparatus, an image encoding method, and an image encoding program using image encoding technology. In particular, the present invention relates to an image encoding apparatus, an image encoding method, and an image encoding program capable of selecting an entropy encoding scheme.

2. Background Art

An image encoding apparatus encodes external input image data in accordance with a predetermined image encoding scheme to generate a bit stream. H.264/AVC is known as one of image encoding schemes used for such encoding process (see Non-patent document 1). This scheme conforms to MPEG (Moving Picture Experts Group)-4 Written Standards Part 10, and Joint model scheme is known as its encoding reference model. An image encoding apparatus based on the Joint model scheme, which is related to the present invention, is called “related art image encoding apparatus” hereinafter.

FIG. 9shows a structure of a related art image encoding apparatus. The related art image encoding apparatus100includes an image frame buffer102to successively store image frames which constitute a target image data101to be compressed. An image data103is divided into and outputted as macro blocks having a predetermined image area size from the image frame buffer102, and inputted to a macro block encoder104in which the image data is encoded in units of macro blocks. A code amount controller105and a decoded picture buffer106are connected to the macro block encoder104. The macro block encoder104outputs an encoded bit stream107. The definitions of the macro block and picture is explained later.

The macro block encoder104of such image encoding apparatus100includes a macro block buffer111to receive the image data103, a predicting device112connected to the macro block buffer111, a calculating device113to subtract the output from the predicting device112from the output of the macro block buffer111, a conversion and quantization device114to convert and quantize the calculation result from the calculating device113under the control of the code amount controller105, an entropy encoder115and inverse-conversion and inverse-quantization device116arranged on the output side of the conversion and quantization device114, and an adder117arranged on the output side of the inverse-conversion and inverse-quantization device116.

Assume that the image data101inputted to the image encoding apparatus100has a QCIF (Quarter Common Intermediate Format). The QCIF is one of image signal formats defined in the ITU (International Telecommunication Union).

FIG. 10shows an image frame in the QCIF image format. The image flame in the QCIF is composed of macro blocks of 176 blocks in wide by 144 blocks in high. One image frame is composed of one frame picture in the progressive scanning. Furthermore, one image frame is composed of two field pictures in the interlace scanning. They are called simply “picture(s)” in the following explanation.

Each macro block, which is the unit constituting the picture, is composed of brightness pixels in 16×16 pixels, and color-difference pixels of Cr (color-difference signals) and Cb (color-difference signals), each in 8×8 pixels.FIG. 10shows a brightness position (x) and color-difference position (o) of an 8×8 pixel block on a pixel-by-pixel basis when the macro block is divided into 16 parts.

The macro block encoder104shown inFIG. 9encodes the image data103in units of macro blocks. In this case, the encoding is successively performed with raster scanning diagonally from the upper left of the picture to the lower right in similar manner to the raster scan for a television system.

Firstly, the macro block buffer111of the macro block encoder104reads a target image data to be encoded in macro blocks, temporally stores it, and supplies it to the conversion and quantization device114arranged at the subsequent stage. At this point, the calculating device113subtracts a prediction image122outputted from the predicting device112from the image121read from the macro block buffer111in macro blocks, and the supplies the prediction error image123(i.e., the calculation result) to the conversion and quantization device114.

Incidentally, there are two types of the prediction error image123, i.e., a prediction image generated based on inter-frame prediction and a prediction image generated based on intra-frame prediction. The inter-frame prediction is generated using the correlation between separate image frames, i.e., using the image of an image frame which was encoded and reconstructed before and which has a different display time from the current target image frame to be encoded. On the other hand, the intra-frame prediction is generated using the correlation within the image frame, i.e., using the image of an image frame which was encoded and reconstructed before the current target image frame to be encoded and has the same display time as the current target image frame.

A set (slice) of macro blocks which can be encoded using the intra-frame prediction alone is called “I-slice” hereinafter. Furthermore, a slice of macro blocks which can be encoded using both the intra-frame prediction and inter-frame prediction is called “P-slice”. Furthermore, a slice of macro blocks which can be encoded using the inter-frame prediction where not only the image of one image frame but also the images of two image frames can be used simultaneously is called “B-slice”.

Furthermore, a picture which can be encoded using the I-slice is called “I-picture”, and a picture which can be encoded using both the I-slice and P-slice is called “P-picture”. Furthermore, a picture which can be encoded using not only the I-slice and P-slice but also the B-slice is called “B-picture”.

The conversion and quantization device114frequency-converts the prediction error image123in smaller units than macro blocks. It converts the prediction error image123from a spatial domain to a frequency domain. In the AVC (Advanced Video Coding) Standards, a frequency conversion in units of 8×8 blocks or 4×4 blocks is applicable to brightness pixels. A prediction error image converted to a frequency domain is called “conversion coefficient” hereinafter. This conversion coefficient is quantized based on a parameter125supplied from the code amount controller105, and supplied to the entropy encoder115as code data126. This quantized conversion coefficient is called “quantized value” in the specification.

The code data126is also supplied to the inverse-conversion and inverse-quantization device116. The inverse-conversion and inverse-quantization device116inverse-quantizes the quantized value supplied from the conversion and quantization device114, and further inverse-frequency-converts it to the original spatial domain. Then, the adder117adds the prediction image122supplied from the predicting device112to the prediction error image which was restored to the spatial domain to obtain a decoded image128. This decoded image128is stored in the decoded picture buffer106for subsequent encoding.

The entropy encoder115entropy-encodes the inputted code data126and outputs a bit stream107. The term “entropy-encoding” means compression of data in which a code having different length is assigned depending on the occurring probability of the data. Since the present invention closely relates to the entropy encoder115, the detail of it will be explained later.

The predicting device112supplies a generating parameter of a prediction image to the entropy encoder115as code data129. The generating parameter may includes, for example, a prediction mode indicating the type of prediction such as inter-frame prediction and intra-frame prediction, a index of a decoded frame used in inter-frame prediction, a motion vector used in inter-frame prediction, and intra-frame prediction direction used in intra-frame prediction.

As explained above, the decoded picture buffer106stores the decoded image128supplied from the inverse-conversion and inverse-quantization device116. Then, it manages decoded image pictures reconstructed from the decoded image128(which is simply called “decoded pictures” hereinafter).

The code amount controller105monitors a bit stream131outputted from the entropy encoder115to encode a picture with a desired bit number. Then, if the bit number of the bit stream131is larger than the desired bit number, it outputs a parameter indicating the increase of a quantizing step size as a quantizing parameter125. On the other hand, if the bit number of the bit stream131is smaller than the desired bit number, it outputs a parameter indicating the decrease of a quantizing step size as the quantizing parameter125.

The entropy encoder115also monitors a symbol number (bin number) which is inputted to an arithmetic encoder (which is explained later) in the case where CABAC (Context-based Adaptive Binary Arithmetic Coding) is used as entropy encoding with a entropy encoding selecting signal132(the detail of the CABAC is explained later). Then, the quantizing parameter will be adjusted such that the ratio between a bit number and a bin number satisfies the ratio specified in the above-mentioned AVC Standards.

FIG. 11shows the specific structure of this entropy encoder. The entropy encoder115includes a first selector141to receive the image code data126outputted from the conversion and quantization device114shown inFIG. 9, a CABAC device142having an input side connected to one of the output sides of the first selector141, a VLC (Variable Length Coding) device143having an input side connected to the other output side of the first selector141, and a second selector144to selectively receive one of the outputs from the CABAC device142and VLC device143which are used as the devices for these two types of coding.

An entropy encoding mode selecting signal132is provided to the entropy encoder115for the switching of the first selector141and second selector144. The entropy encoding mode selecting signal132is a signal to select one of the CABAC device142and VLC device143. In this manner, in the AVC (Advanced Video Coding) Standards, entropy encoding is performed on the code data126of macro blocks on a picture-by-picture basis by selecting one of the coding by CABAC device142or the coding by VLC device143.

In the entropy encoder115shown inFIG. 11, the CABAC device142receives a code data1261outputted from the first selector141, and outputs a bit stream1071to the second selector144. The CABAC device142also outputs a bin number data145representing a bin number. The specific structure of the CABAC device142is explained later.

The VLC device143receives a code data1262outputted from the first selector141, and outputs a bit stream1072to the second selector144. The specific structure of the VLC device143is explained later. The second selector144also outputs a bit number data146representing a bit number as well as the bit stream107.

Incidentally, the CABAC device142achieves higher encoding efficiency than the VLC device143. However, the CABAC device142requires larger processing effort than the VLC device143. Therefore, in general, the CABAC device142is used for a higher profile (e.g., High profile or Main profile) which supports complicated process. Meanwhile, the VLC device143is used for a lower profile (e.g., Base-line profile) which does not support complicated process. However, since code data in higher level layers than a macro block layer has a relatively smaller code data amount which occupies a bit stream, the VLC device143is used for code data in such layers to prioritize comparability among each profile.

FIG. 12shows the specific structure of a CABAC device. The CABAC device142includes a binarization device151to receive a code data1261through the first selector141shown inFIG. 11and converting it to binary, and a switch153to switch the output of the binary outputted from the binarization device151. A bin155outputted from the switch153as a binary symbol is supplied to an arithmetic encoder156and a context calculator157.

The binarization device151is adapted to convert inputted code data1261to a binary string in accordance with procedure specified in the AVC Standards and output it as a binary string data152. The arithmetic encoder156encodes the binary string of the bin155which is successively supplied from the switch153to binary arithmetic code by using a dominant symbol158and a state number159supplied from the context calculator157. Furthermore, it successively supplies an updated dominant symbol158and an updated state number159to the context calculator157. The term “state number159” means a table number of a table storing a value corresponding to the occurring probability of dominant symbol specified in the AVC Standards.

The context calculator157supplies stored dominant symbol158and state number159corresponding to the bin155which is successively supplied from the switch153as a symbol. Furthermore, it also stores the dominant symbol158and state number159which are updated by the binary arithmetic encoding at the arithmetic encoder156.

The switch153outputs a bin number data145to the outside of the CABAC device142. Furthermore, the arithmetic encoder156outputs a bit stream1071to the outside of the CABAC device142. As shown inFIG. 11, the bit stream1071is supplied to the second selector144in parallel to a bit stream1072outputted from the VLC device143.

FIG. 13shows the specific structure of a VLC device. The VLC device143includes a variable length encoder161to receive a code data1262through the first selector141shown inFIG. 11, and a table selector162.

The variable length encoder161encodes the code data1262to variable length code in accordance with a table specified by the table selector162, and outputs a bit stream1072. The table selector162contains variable length encoding tables (not shown) corresponding to the types of code data1262such as a prediction mode, a quantized value, and the like. Then, it supplies a table163selected from variable length encoding tables to the variable length encoder161.

In accordance with the image encoding apparatus100described above which is related to the present invention, when the VLC device143(FIG. 11) is used for entropy encoding, the completion time of the picture encoding by this image encoding apparatus100is determined by the amount of inputted code data.

On the other hand, when the CABAC device142shown inFIG. 12is used as the block for entropy encoding, the completion time of the picture encoding by this image encoding apparatus100is determined by the number of the bin155inputted to the arithmetic encoder156. Incidentally, in the case where a block other than the CABAC device142is used, the process will be always completed within finite length of time since the amount of input data is finite.

In the H.264/AVC Standards, the following two items are used as restrictions to limit the processing amount of entropy encoding per picture on the decoding side. (a) Encoding must be controlled such that the bit number of a picture becomes equal to or less than a value specified in the H.264/AVC Standards. (b) Encoding must be controlled such that the ratio between the bit number and bin number of a picture becomes equal to or less than a value specified in the H.264/AVC Standards.

Since the VLC device143shown inFIG. 13allows process in units of the code data1262, its process is simpler than that of the CABAC device142shown inFIG. 12. Therefore, assuming that the process is always completed within a certain time period, encoding of the image encoding apparatus100needs to be controlled such that the bin number of a picture does not exceeds the maximum bin number for which the arithmetic encoder156can process during the processing time.

A following method is one of exemplary methods for such encoding controlling. (i) For a picture having a large bin number, there is a method of encoding using PCM (Pulse-Code Modulation) mode in which a input image in macro blocks is directly outputted as a bit stream (Patent document 1 and Non-patent document 2). (ii) There is another method in which CABAC device142and a VLC device143are operated in parallel for the code data of a picture. Then, if the process of the CABAC device142is completed within a predetermined time, the encoding output from this CABAC device142is selected for the encoding. If not, the encoding output from this VLC device143is selected for the encoding (Non-patent document 3).[Patent document 1] Japanese unexamined application publication No. 2004-135251 (paragraphs 121-124, and FIG. 1).[Non-patent document 1] ISO/IEC 14496-10 Advanced Video Coding.[Non-patent document 2] “PCM ENCODING METHOD CONFORMING TO H.264 MB UPPER LIMIT BIT USING LOCAL DECODE IMAGE” by Chono et al, PCSJ (Picture Coding Symposium of Japan), pp. 5-17, November 2006.[Non-patent document 3] “ENTROPY ENCODING TECHNIQUE IN H.264/AVC FOR HIGH-RESOLUTION” by Okamoto et al, The Institute of Electronics, Information and Communication Engineers General Conference, D-11-2, March 2007.

SUMMARY

The former method using a PCM mode is effective for applications in which the bin number of one macro block is restricted. However, there is a problem associated with degraded images in such applications in which the bin number of a picture is restricted (i.e., the number of bins of a plurality of macro blocks is restricted). Because when the PCM mode is heavily used in a picture to implement the method stated in the above item (i), a macro block which is encoded after the heavy use of the PCM mode is coarsely quantized since the PCM mode has a large output bit number.

Meanwhile, in the latter method in which a CABAC device142and a VLC device143are operated in parallel, and one of the encoding outputs is selected, it becomes after-the-event selection. Therefore, it poses a critical problem in the image encoding apparatus100that the encoding cannot be controlled in units smaller than the picture. Because the bit number for the code data126inputted to the entropy encoder115is unknown until the selected encoding output becomes known.

Incidentally, Patent document 1 proposes a method in which when the bin number of code data encoded from an image is larger than a predetermined value, an alternative bit stream of code data separately encoded from the above-mentioned image with a different encoding parameter (e.g., different quantizing parameter) (such that its bin number becomes smaller than the predetermined value) is outputted. However, since this method is also the after-the-event selection, it has a critical problem similar to the above-mentioned problem.

The present invention has been made in view of these circumstances, and provides an image encoding apparatus, an image encoding method, and an image encoding program capable of minimizing image degradation, controlling a code amount in units smaller than pictures, and ensuring that encoding of the picture is completed within a certain time period.

In accordance with one aspect of the present invention, an image encoding apparatus includes: (a) a code data generator to transform and quantize an image block to generate code data; (b) a variable length code encoder to encode the code data into bit-string data on the basis of applying variable length encoding; (c) an arithmetic encoder to encode the code data into bit-string data on the basis of converting the code data into binary symbols and applying binary arithmetic encoding; (d) a binary symbol number estimator to estimate the total number of the binary symbols that are converted from the code data generated from a plurality of image blocks; and (e) a encoding selector to select, for the encoding of the code data, either the variable length code encoder or the arithmetic encoder for the encoding of the code data on the basis of the total number estimated by the binary symbol number estimator.

That is, the present invention in one aspect achieves the above-mentioned object by estimating the bin number of the image of a target picture to be encoded; and controlling entropy encoding and the target bit number (assigned picture rate) of the picture using the estimated bin number and compression ratios of the variable length encoder and arithmetic encoder.

In accordance with another aspect of the present invention, an image encoding method includes: (a) transforming and quantizing an image block to generate code data; (b) estimating the total number of the binary symbols that are converted from the code data generated from a plurality of image blocks; and (c) selecting the encoding method of the code data from variable length encoding and arithmetic encoding on the basis of the total number estimated, the variable length encoding being adapted to encode the code data into bit-string data on the basis of applying variable length encoding, and the arithmetic encoding being adapted to encode the code data into bit-string data on the basis of converting the code data into binary symbols and applying binary arithmetic encoding.

That is, the present invention in one aspect expresses similar technical idea to the above-mentioned image encoding apparatus as the transitions of processes.

In accordance with another aspect, the present invention provides a computer-readable recording medium recording an image encoding program to instruct a computer to execute: (a) a code data generating process to transform and quantize an image block to generate code data; (b) a binary symbol number estimating process to estimate the total number of the binary symbols that are converted from the code data generated from a plurality of image blocks; and (c) an encoding selecting process to select the encoding method of the code data from variable length encoding and arithmetic encoding on the basis of the total number estimated by the binary symbol number estimating process, the variable length encoding being adapted to encode the code data into bit-string data on the basis of applying variable length encoding, and the arithmetic encoding being adapted to encode the code data into bit-string data on the basis of converting the code data into binary symbols and applying binary arithmetic encoding.

That is, the present invention in one aspect expresses similar technical idea to the above-mentioned image encoding apparatus as a program executed by a computer.

EXEMPLARY EMBODIMENT

Exemplary embodiments in accordance with the present invention are explained hereinafter.

First Embodiment

FIG. 1shows the structure of an image encoding apparatus in accordance with a first embodiment of the present invention. An image encoding apparatus200in accordance with a first embodiment includes an image frame buffer202to successively store target image data201to be compressed. The image frame buffer202outputs image data203in units of macro blocks in similar manner to the above-explained image encoding apparatus100(FIG. 9) in the related art to the present invention. This image data203is inputted to a macro block encoder204, and encoded in units of macro blocks. A code amount controller205and a decoded picture buffer206are connected to the macro block encoder204.

A symbol number estimating device208which estimates the bin number of macro blocks of a target picture to be encoded, i.e., the total bin number of a plurality of macro blocks constituting the picture, is connected to the image frame buffer202. The symbol number estimating device208reads the image of a target picture to be encoded which is stored in the image frame buffer202, and an assigned picture rate for the target picture to be encoded at the205, and performs the above-mentioned estimation.

The bin number of the macro blocks of the target picture to be encoded which was estimated by the symbol number estimating device208is inputted to an entropy encoding selector210. The entropy encoding selector210outputs an entropy encoding mode selecting signal232, which is used to select a entropy encoding mode, based on the inputted bin number209.

The macro block encoder204includes, in comparison to the macro block encoder104shown inFIG. 9, a macro block buffer211to receive the image data203, a predicting device212connected to the macro block buffer211, a calculating device213to subtract the output from the predicting device212from the output of the macro block buffer211, a conversion and quantization device214to convert and quantize the calculation result from the calculating device213under the control of the code amount controller205, an entropy encoder215and inverse-quantization and inverse-conversion device216arranged on the output side of the conversion and quantization device214, and an adder217arranged on the output side of the inverse-conversion and inverse-quantization device216. The macro block encoder204outputs an encoded bit stream207.

The image encoding apparatus200in accordance with this embodiment includes a CPU (Central Processing Unit)271, a program storing portion272such as a disk drive and an optical disk to store a control program executed by this CPU271, and a working memory273such as semiconductor memory to temporally store various data during the time when the CPU271is executing the control program stored in the program storing portion272. At least some part of the components shown inFIG. 1may be achieved by software operations by the CPU271executing the control program stored in the program storing portion272.

In the image encoding apparatus200having structure explained above, the code amount controller205specifies an assigned picture rate depending on the indicated content of the entropy encoding mode selecting signal232established by the entropy encoding selector210. Therefore, its operations are different from the code amount controller105shown inFIG. 9. That is, the code amount controller105shown inFIG. 9is set up in accordance with an instruction manual, and in general, one of the CABAC device242or VLC device243which corresponds to the CABAC device142and VLC device143respectively inFIG. 11and is selected in a fixed manner in accordance with a profile.

The macro block encoder204in accordance with the first embodiment of the present invention has substantially the same components such as the macro block buffer211and entropy encoder215as those of the macro block encoder104. Therefore, the structures shown inFIGS. 11-13are applied to this embodiment without any modification. However, in this embodiment, the number in hundreds place of the sign representing each component is changed from “1” to “2” in the following explanation.

Incidentally, the image encoding apparatus200of this embodiment determines the content of the entropy encoding mode selecting signal232by operating the symbol number estimating device208and entropy encoding selector210at the start of the picture encoding. The image encoding apparatus200of this embodiment updates the assigned picture rate by operating the code amount controller205after the decision of the content of the entropy encoding mode selecting signal232and before the start of the encoding of macro blocks within a target picture to be encoded. After the picture rate update, the image encoding apparatus200successively encodes macro blocks within the target picture to be encoded by operating the macro block encoder204. Then, after the whole macro blocks within the picture are encoded, the image encoding apparatus200performs predetermined processes and starts the encoding of the next picture.

FIG. 2shows the operation of the image encoding apparatus in accordance with this embodiment from the start to the end of picture encoding. The following explanation is made with reference toFIG. 1andFIGS. 9-13.

Firstly, the image encoding apparatus200of this embodiment starts the picture encoding with symbol number estimating process using the symbol number estimating device208(Step S301). Assume that the brightness pixel data of the target picture to be encoded which is stored at the image frame buffer202is src[y][x] (wherein 0≦x≦width−1, 0≦y≦height−1) at the start of the picture encoding when an image data201is sent from a image data source (not shown).

In the formulas, “x” is a horizontal position of a pixel within the picture, and “y” is a vertical position of the pixel. Furthermore, “width” is the horizontal pixel count of the picture, and “height” is the vertical pixel count.

In the symbol number estimating process of the step S301, the symbol number estimating device208estimates the bin number from the brightness pixel data src[y][x]. To this end, an activity mb_act[i] is calculated for each macro block with the following equation (1) using this brightness pixel data src[y][x].

In the equation (1), “ave[i] is expressed by the following equation (2). As seen from the equation (1), the activities represent variations of the pixel values of macro blocks. Therefore, it is possible to use the standard deviation or variance of the pixel values of the macro blocks.

In the equations (1) and (2), “i” is a macro block address in order of raster scanning within a picture. Furthermore, “mbx(i)” is a horizontal position of a macro block within the picture corresponding to the macro block address i, and “mby(i)” is a vertical position of the macro block within the picture corresponding to the macro block address i.

Then, in the symbol number estimating process of the step S301, estimated bin number complexity of the target picture to be encoded epic_bin_x is calculated with the following equation (3) using the activity for each macro block mb_act[i].
epic_bin—x=α×pic_act+β  (3)

In the equation, “pic_act” is expressed by the following equation (4).

In the equations (3) and (4), “α” and β represent the gradient and intercept respectively of the regression line of I picture bin number complexity for the activity. Furthermore, “pic_size_mbs” is the number of macro blocks contained in the picture (c=(width*height)/256). The term “I picture bin number complexity pic_bin_x” means the product of the picture bin number pic_bin and the average quantizing step size of the macro block pic_qs (i.e., pic_bin_x=pic_bin*pic_qs) when the target picture to be encoded is encoded as I picture.

At the end of the symbol number estimating process of the step S301, estimated bin number of the target picture to be encoded epic_bin is calculated with the following equation (5) using the estimated bin number complexity of the target picture to be encoded epic_bin_x and the assigned picture rate for the target picture to be encoded pic_rate.
epic_bin=max(epic_bin—x/qs—rc,γ×pic_rate)  (5)

In the equation, “qs_rc” is expressed by the following equation (6).
qs—rc=pic—x/pic_rate  (6)

In the equations (5) and (6), the picture complexity pic_x is the product of the occurring picture rate of the picture which has the same picture type as the target picture to be encoded (which is explained later) and is encoded lastly i.e., last_pic_rate and the average quantizing step size of the macro block last_pic_qs. Furthermore, “γ” is the ratio between the symbol number bin and the bit number bit (γ=bin/bit), In the equation, “γ” is larger number than 1. The “γ” may be a statistical ratio for each picture type, or a ratio calculated from the data on a lastly encoded picture for each picture type.

After the symbol number estimating process of the step S301is completed in this manner, an entropy encoding selecting process is performed (Step S302). At the step S302, the entropy encoding selector210selects the entropy encoding mode selecting signal232using the estimated bin number of the target picture to be encoded epic_bin. Then, it calculates the average quantizing step size qs_cabac using the following equation (7) for the case where the target picture is encoded by the CABAC device242(seeFIG. 12)
qs—cabac=γ×pic—x/min(epic_bin,ppic_bin)  (7)

In the equation, “ppic_bin” is expressed by the following equation (8).
ppic_bin=pmb_bin×pic_size—mbs(8)

In the equations (7) and (8), “pmb_bin” is the bin number for which the arithmetic encoder of the CABAC device242(seeFIG. 12) of the entropy encoder215can process within the encoding time of I macro block. Therefore, the average quantizing step size qs_cabac is the image quality by the CABAC device242under the condition where the upper limit bit number is imposed by the processible bin number within a predetermined time period. The smaller the average quantizing step size qs_cabac, the better the image quality becomes.

Then, in the entropy encoding selecting process of the step S302, it calculates the average quantizing step size qs_vlc using the following equation (9) for the case where the target picture is encoded by the VLC device243.
qs—vlc=γ×pic—x/pic_rate  (9)

In the equation (9), “γ” is the compression ratio between the bit number when the code data of a picture is processed by the VLC device243vlc_bit and the bit number when it is arithmetically encoded cabac_bit (γ=vlc_bit/cabac_bit). Furthermore, “γ” is larger number than 1, and a statistical ratio for each picture type. As seen from the definition of “γ”, the average quantizing step size qs_vlc is the image quality by the VLC device243under the condition where the reduction in compression efficiency by the VLC device243is imposed.

At the end of the entropy encoding selecting process of the step S302, the entropy encoding mode selecting signal232is determined by the following equation (10) using the average quantizing step size qs_cabac and the average quantizing step size qs_vlc.

After the entropy encoding selecting process of the step S302is completed in this manner, a picture rate setting process is performed (Step S303). In this picture rate setting process, the code amount controller205updates the assigned picture rate of the target picture to be encoded pic_rate based on the entropy encoding mode selecting signal232(mode) using the following equation (11).

Incidentally, the quantizing step size qs_rc to achieve the assigned picture rate pic_rate can be calculated with the following equation (12).

After the picture rate setting process of the step S303is completed in this manner, a macro block encoding process is performed (Step S304). In this macro block encoding process, the macro block encoder204encodes one macro block of the target picture to be encoded using the decided entropy encoding mode selecting signal232.

At this point, similar to the code amount controller105of the image encoding apparatus100shown inFIG. 9, the code amount controller205monitors a bit stream207outputted from the entropy encoder215. Then, if the bit number of the bit stream207outputted from the entropy encoder215is larger than the assigned picture rate, it outputs a quantizing parameter indicating the increase of a quantizing step size. On the other hand, if the bit number of the bit stream207is smaller than the assigned picture rate, the code amount controller205outputs a quantizing parameter indicating the decrease of a quantizing step size.

After the macro block encoding process of the step S304is performed in this manner, it proceeds to the next step (Step S305). In the step S305, it determines whether or not the whole macro blocks in the target picture to be encoded are encoded. If it is not completed (N), it returns to the step S304to continue the process. If the encoding of the whole macro blocks in the target picture to be encoded were completed (Step S305: Y), the picture complexity pic_x is updated with the following equation (13) depending on the decided entropy encoding mode selecting signal232(mode) at the end of the picture encoding (Step S306), and a series of the picture encoding process is completed (END).

In the equation, “pic_act_rate” is the occurring bit number of a picture which is encoded at that time. Furthermore, “qs_pic” is the average quantizing step size of the macro block of the picture which is encoded at that time. As already explained above, the picture complexity pic_x stores its value for each picture type.

As explained above, the image encoding apparatus200in accordance with the first embodiment of the present invention can generate a bit stream207by successively applying the processes to encode a picture as shown inFIG. 2to externally inputted image data201. That is, the following encoding processes are performed on each picture in the image encoding apparatus200of this embodiment.

(a) A case where the estimated symbol number corresponding to the assigned picture rate is equal to or less than the symbol number for which the arithmetic encoder can process within the predetermined time period in the processes before the step S303. In this case, the encoding is performed by the CABAC device242in the entropy encoder215as a usual case.

(b) A case where the estimated symbol number corresponding to the assigned picture rate is larger than the symbol number for which the arithmetic encoder can process within the predetermined time period, and the image quality by the CABAC device242which is restricted by the bit number corresponding to the symbol number for which the arithmetic encoder can process within the predetermined time period is better than the image quality of the VLC device243in the processes before the step S303.

In this case, the encoding is performed by the CABAC device242which is restricted by the assigned picture rate corresponding to the symbol number for which the arithmetic encoder can process within the predetermined time period.

(c) The other case, namely, a case where the estimated symbol number corresponding to the assigned picture rate is larger than the symbol number for which the arithmetic encoder can process within the predetermined time period, and the image quality by the CABAC device242which is restricted by the bit number corresponding to the symbol number for which the arithmetic encoder can process within the predetermined time period is worse than the image quality of the VLC device243in the processes before the step S303.

In this case, the encoding is performed by the VLC device243.

As explained above, the image encoding apparatus200of this embodiment can minimize image degradation, control a code amount in units smaller than pictures, and ensure that encoding of the picture is completed within a certain time period.

Second Embodiment

FIG. 3shows the structure of an image encoding apparatus in accordance with a second embodiment of the present invention. An image encoding apparatus400in accordance with a second embodiment includes a first image frame buffer4021to successively store target image data401to be compressed. A first image data4031which is outputted from this first image frame buffer4021in units of macro blocks is inputted to a preliminary encoder404and a second image frame buffer4022. A second image data4032which is outputted from the second image frame buffer4022in units of macro blocks is inputted to a main encoder405.

The preliminary encoder404includes a preliminary macro block encoder411to receive the first image data4031, and a preliminary code amount controller412connected to the preliminary macro block encoder411. Furthermore, a decoded picture buffer413is arranged on the outside of the preliminary encoder404and connected to the preliminary macro block encoder411. Furthermore, a VLC device selecting signal414is supplied to the preliminary macro block encoder411.

The main encoder405includes a macro block encoder421and a code amount controller422connected to the macro block encoder421. A decoded picture buffer423connected to the macro block encoder421, and a symbol number estimating device424and a entropy encoding selector425both of which are connected to the code amount controller422are arranged on the outside of the main encoder405. The macro block encoder421outputs an encoded bit stream426.

The image encoding apparatus400in accordance with the second embodiment includes a CPU431, a program storing portion432such as a disk drive and an optical disk to store a control program executed by this CPU431, and a working memory433such as semiconductor memory to temporally store various data during the time when the CPU431is executing the control program stored in the program storing portion432. At least some part of the components shown inFIG. 3may be achieved by software operations by the CPU431executing the control program stored in the program storing portion432.

The image encoding apparatus400of the second embodiment has the combined circuit structure of the image encoding apparatus100shown inFIG. 9and the image encoding apparatus200in accordance with the first embodiment of the present invention. That is, the preliminary macro block encoder411has the same circuit structure as that of the macro block encoder104shown inFIG. 9. Specifically, the first image data403outputted from the first image frame buffer4021shown inFIG. 3is supplied to the same circuit (not shown) in the preliminary macro block encoder411as the macro block buffer111shown inFIG. 9. The preliminary code amount controller412has the same circuit as the code amount controller105shown inFIG. 9, and is connected to the conversion and quantization device (not shown) and the entropy encoder (not shown) (both of which are the same as the conversion and quantization device114and entropy encoder115respectively in theFIG. 9). A bit stream441outputted from the entropy encoder (not shown), which is encoded in the same manner as the bit stream107outputted from the entropy encoder115shown inFIG. 9, is outputted to the outside of the preliminary macro block encoder411and inputted to the symbol number estimating device424. Furthermore, the decoded picture buffer413which is identical to the decoded picture buffer106shown inFIG. 9is connected to the adder117A (not shown) and predicting device112A (not shown) (which are identical to the adder117and predicting device112respectively inFIG. 9).

However, this embodiment is different from the structure shown inFIG. 9in that while the entropy encoding mode selecting signal132is supplied to the entropy encoder115shown inFIG. 9, the VLC device selecting signal414is supplied to the entropy encoder in this embodiment. In this manner, in the preliminary encoder404of this embodiment, it is set to select the VLC device (not shown) which corresponds to the VLC device143(FIG. 11) in which the entropy encoding process is always completed within a certain time period. Incidentally, it is also possible to select the CABAC device (not shown) which corresponds to the CABAC device142(FIG. 12) instead of the VLC device. However, in such case, the bin number needs to be monitored to control the code amount such that the arithmetic encoding is completed within a predetermined time period.

Meanwhile, the macro block encoder421of the main encoder405has the same structure as that of the macro block encoder204shown inFIG. 1. That is, the second image data4032outputted from the second image frame buffer4022is inputted to the macro block buffer211A (not shown) corresponding to the macro block buffer211shown inFIG. 1in units of macro blocks in similar manner to the image data203inFIG. 1. Furthermore, similar to the code amount controller205shown inFIG. 1, the code amount controller422is connected the conversion and quantization device214A (not shown) corresponding to the conversion and quantization device214and the entropy encoder215A (not shown) corresponding to the entropy encoder215. Furthermore, similar to the decoded picture buffer206shown inFIG. 1, the decoded picture buffer423is connected the predicting device212A (not shown) corresponding to the predicting device212and the inverse-conversion and inverse-quantization device216A (not shown) corresponding to the inverse-conversion and inverse-quantization device216. Then, the entropy encoder215A corresponding to the entropy encoder215shown inFIG. 1outputs a bit stream426corresponding to the bit stream207inFIG. 1.

With these structures of the image encoding apparatus400of this embodiment, the preliminary encoder404successively encodes macro blocks of the target picture to be encoded, which are outputted from the first image frame buffer4021, and output a bit stream441. The bit number of the bit stream441outputted from the preliminary encoder404is called “preceding encoding information” in the specification.

The decoded picture buffer413stores decoded images from the preliminary encoder404, and manages decoded image pictures reconstructed from the decoded images.

The symbol number estimating device424which has the same structure as the symbol number estimating device208shown inFIG. 1counts the bit number of a bit stream441supplied from the preliminary encoder404. Then, after the preliminary encoder404completes the encoding (preliminary encoding) of the whole macro blocks, it estimates the bin number of the macro blocks of the target picture to be encoded, i.e., the total bin number of a plurality of the macro blocks constituting the picture based on this counted bit number.

The entropy encoding selector425which has the same structure as the entropy encoding selector210shown inFIG. 1brings the entropy encoding mode selecting signal428to one of the states using the above-mentioned bin number estimated by the symbol number estimating device424. The code amount controller422establishes a assigned picture rate based on the state of the entropy encoding mode selecting signal428selected by the entropy encoding selector425.

After the preliminary encoder404completes the encoding of the whole macro blocks, the second image frame buffer4022reads the target image to be encoded from the first image frame buffer4021, and prepares for the start of the encoding (main encoding) of the target picture to be encoded.

The main encoder405which is composed of the macro block encoder421and code amount controller422successively encodes macro blocks of the target picture to be encoded of the second image frame buffer4022, and output a bit stream426.

The overall operation of the image encoding apparatus400of the second embodiment is briefly explained hereinafter.

The image encoding apparatus400performs the main encoding of a target picture to be encoded by operating the first image frame buffer4021and preliminary macro block encoder411at the start of the picture encoding. After the preliminary encoding of the target picture to be encoded, the image encoding apparatus400determines the entropy encoding mode selecting signal428by operating the symbol number estimating device424and entropy encoding selector425at the start of the picture encoding. The image encoding apparatus400of updates an assigned picture rate by operating the code amount controller422after the decision of the entropy encoding mode selecting signal428and before the start of the main encoding of macro blocks of the target picture to be encoded. After the picture rate update, the image encoding apparatus400performs the main encoding of macro blocks of the target picture to be encoded by operating the second image frame buffer4022and main encoder405. After the whole macro blocks within the picture are encoded, the image encoding apparatus400performs predetermined processes and starts the encoding of the next picture.

FIG. 4shows the operation of the image encoding apparatus for preliminary picture encoding. The following explanation is made with reference toFIG. 3.

At the start of the preliminary encoding of a picture, an assigned picture rate of the preliminary picture encoding pre_pic_rate is established at the preliminary code amount controller412(Step S501). The quantizing step size qs_rc to achieve this assigned picture rate pre_pic_rate can be calculated with the following equation (14).
qs—rc=λ×pic—x/pre_pic_rate  (14)

After the parameter is established in this manner, the preliminary encoder404performs the preliminary encoding of one macro block of a target picture to be encoded by using the above-mentioned VLC device which corresponds to the VLC device143(FIG. 11) as an entropy encoding mode (Step S502). Next, the preliminary encoder404determines whether or not the whole macro blocks in the target picture to be encoded are encoded (Step S503). If the preliminary encoding is not completed (N), it returns to the step S502to continue the process.

On the other hand, if the preliminary encoding were completed (Step S503: Y), the preliminary picture encoding complexity pre_pic_x is updated with the following equation (15), and the preliminary picture encoding process is completed (END).
pre_pic—x=pic_act_rate×qs_pic  (15)

In the equation, “pic_act_rate” is the occurring bit number of a picture which is encoded at that time, and “qs_pic” is the average quantizing step size of the macro block of the picture which is encoded at that time.

FIG. 5shows the operation of the image encoding apparatus for main picture encoding. The following explanation is made with reference toFIG. 3.

At the start of the main encoding of the picture, the second image frame buffer4022reads the target picture to be encoded which is stored in the first image frame buffer4021. The symbol number estimating device424calculates an estimated bin number with the following equation (16) using the preliminary encoding picture complexity pre_pic_x which is updated at the step S503inFIG. 4(Step S521). As explained above, it may be recalculated from the above-mentioned equation (15) by counting the bit number of the bit stream441supplied from the preliminary encoder404for each macro block.

In the equation, “qs_rc” is calculated from the following equation (17).
qs—rc=pic—x/pic_rate  (17)

After the symbol number is estimated in this manner, the entropy encoding selector425selects an entropy encoding mode using the estimated bin number of the target picture to be encoded epic_bin (Step S522).

Firstly, it calculates the average quantizing step size qs_cabac using the above-mentioned equation (7) for the case where the target picture is encoded by the CABAC device (not shown) which corresponds to the CABAC device142(FIG. 12).

In the equation, similar to the first embodiment, “qs_cabac” is the image quality by the CABAC device under the condition where the upper limit bit number is imposed by the processible bin number within a predetermined time period. The smaller the value “qs_cabac”, the better the image quality becomes.

Then, it calculates the average quantizing step size qs_vlc using the above-mentioned equation (9) for the case where the target picture is encoded by the VLC device. Similar to the first embodiment, the average quantizing step size qs_vlc is the image quality by the VLC device under the condition where the reduction in compression efficiency of the VLC device is imposed.

Finally, the entropy encoding mode selecting signal428(mode) is determined with the equation (10) using the average quantizing step size qs_cabac and average quantizing step size qs_vlc.

After the step S522is completed in this manner, the code amount controller422updates the assigned picture rate of the target picture to be encoded pic_rate based on the entropy encoding mode selecting signal428(mode) using the above-mentioned equation (11).

Incidentally, the quantizing step size qs_rc to achieve the assigned picture rate pic_rate can be calculated with the above-mentioned equation (12).

After the step S523is completed in this manner, the main encoder405performs the main encoding of one macro block of the target picture to be encoded using the decided entropy encoding mode selecting signal428(mode) (Step S524).

At this point, similar to the first embodiment, the code amount controller422monitors a bit stream107A (not shown) outputted from the entropy encoder215A which corresponds to the entropy encoder115shown inFIG. 9. Then, if the bit number of the outputted bit stream107A is larger than the assigned picture rate, it outputs a quantizing parameter indicating the increase of a quantizing step size. On the other hand, if the bit number of the outputted bit stream107A is smaller than the assigned picture rate, it outputs a quantizing parameter indicating the decrease of a quantizing step size.

After the step S524is completed in this manner, the main encoder405determines whether or not the whole macro blocks in the target picture to be encoded are encoded (Step S525). If it is not completed, it returns to the step S524to continue the process.

On the other hand, if the main encoding of the whole macro blocks in the target picture to be encoded were completed (Step S525: Y), the picture complexity pic_x is updated with the above-mentioned equation (13) depending on the decided entropy encoding mode selecting signal428(mode) (Step S306), the main picture encoding process is completed (END).

In the equation (13), “pic_act_rate” is the occurring bit number of a picture which is encoded at that time, and “qs_pic” is the average quantizing step size of the macro block of the picture which is encoded at that time. As already explained above, the picture complexity pic_xstores its value for each picture type.

As explained above, the image encoding apparatus400in accordance with the second embodiment of the present invention can generate a bit stream426by successively applying the processes to encode a picture to an externally inputted image data401. Moreover, since it can use the preliminary information by the preliminary encoder404which uses the VLC, the main encoder405can estimate the bin number of the image of a target picture to be encoded more accurately than the first embodiment. In this manner, it can make image degradation smaller than the image encoding apparatus200of the first embodiment, and ensure that encoding of the picture is completed within a certain time period.

Third Embodiment

FIG. 6shows the structure of an image encoding apparatus in accordance with a third embodiment of the present invention. In the example of the second embodiment explained above, one picture is encoded by using one pair of the preliminary macro block encoder411and macro block encoder421as shown inFIG. 3. An image encoding apparatus600in accordance with a third embodiment of the present invention encodes one picture by using several pairs of a preliminary macro block encoder and a macro block encoder.

FIG. 6shows one example in which one picture is encoded by using three pairs of a preliminary macro block encoder and a macro block encoder. The image encoding apparatus600includes a first image frame buffer6021to successively store target image data601to be compressed. A first image data6031which is outputted from this first image frame buffer6021in units of macro blocks is inputted to a preliminary parallel encoder604and a second image frame buffer6022. A second image data6032which is outputted from the second image frame buffer6022in units of macro blocks is inputted to a main parallel encoder605.

The preliminary parallel encoder604includes first to third preliminary macro block encoders6111-6113, each of which has the same structure as the preliminary macro block encoder411of the second embodiment, and a preliminary code amount controller612which operates in the same manner as the preliminary code amount controller412of the second embodiment. A first image data6031which is outputted from the first image frame buffer6021in units of macro blocks is inputted in parallel to the first to third preliminary macro block encoders6111-6113in the preliminary parallel encoder604. The first to third preliminary macro block encoders6111-6113are arranged on the outside of the preliminary parallel encoder604, and connected to a decoded picture buffer606corresponding to the decoded picture buffer413of the second embodiment.

The main parallel encoder605includes first to third main macro block encoders6211-6213, each of which has the same structure and operates in the same manner as the entropy encoding selector425of the second embodiment, and a main code amount controller622which corresponds to the code amount controller422of the second embodiment. A second image data6032which is outputted from the second image frame buffer6022in units of macro blocks is inputted in parallel to the first to third main macro block encoders6211-6213in the main parallel encoder605. The first to third main macro block encoders6211-6213are arranged on the outside of the main parallel encoder605, and connected to a decoded picture buffer607corresponding to the decoded picture buffer423of the second embodiment.

Encoded bit streams6311-6313outputted from the first to third preliminary macro block encoders6111-6113respectively in the preliminary parallel encoder604are inputted to a symbol estimating device608corresponding to the symbol number estimating device424of the second embodiment to perform a symbol estimating process. A bin number obtained by the symbol estimating device608is inputted to an entropy encoding selector609corresponding to the entropy encoding selector425of the second embodiment, and a bit stream outputted from the entropy encoding selector609is supplied to the main code amount controller622.

Furthermore, first to third bit streams6331-6333outputted from the first to third main macro block encoders6211-6213respectively in the main parallel encoder605are inputted to a multiplexer610. The multiplexer610multiplexes the first to third bit streams6331-6333, and outputs a bit stream634.

The image encoding apparatus600in accordance with the third embodiment includes a CPU671, a program storing portion672such as a disk drive and an optical disk to store a control program executed by this CPU671, and a working memory673such as semiconductor memory to temporally store various data during the time when the CPU671is executing the control program stored in the program storing portion672. At least some part of the components shown inFIG. 6may be achieved by software operations by the CPU671executing the control program stored in the program storing portion672.

Each of the first to third preliminary macro block encoders6111-6113encodes their respective image portions of a target picture to be encoded of the first image frame buffer6021after the picture is divided into the three image portions. For example, assume that the image signal format of a target picture to be encoded is the above-mentioned QCIF. In such case, the first preliminary macro block encoder6111encodes the first successive 33 macro blocks (in order of raster scanning). Then, the second preliminary macro block encoder6112encodes the next successive 33 macro blocks, and the third preliminary macro block encoder6113encodes the final successive 33 macro blocks.

The preliminary parallel encoder604encodes three macro blocks of the target picture to be encoded of the first image frame buffer6021in parallel at a time. Then, it outputs the first to third bit streams6331-6333from the first to third preliminary macro block encoders6111-6113. At this point, in the preliminary parallel encoder604, it is set to select the VLC device (not shown) in which the entropy encoding process is always completed within a certain time period. It is also possible to select the CABAC device (not shown) instead of the VLC device. However, in such case, the bin numbers of their respective first to third preliminary macro block encoders6111-6113need to be monitored to control the code amounts such that all the arithmetic encodings in these first to third preliminary macro block encoders6111-6113are completed within a predetermined time period.

Incidentally, the bit numbers of bit streams outputted from the preliminary parallel encoder604are called “preceding encoding information” in the specification. Furthermore, the decoded picture buffer606stores decoded images from the preliminary parallel encoder604, and manages decoded image pictures reconstructed from the decoded images.

The symbol estimating device608counts the bit numbers of bit streams6311-6313of the first to third preliminary macro block encoders6111-6113supplied from the preliminary parallel encoder604. Then, after the preliminary parallel encoder604completes the encoding of the whole macro blocks, i.e., the preliminary encoding, it estimates the bin numbers of the divided images for which their respective first to third main macro block encoders6211-6213will perform the encoding based on their respective bit numbers. In the above-mentioned example, each of the first to third main macro block encoders6211-6213estimates the bin numbers for their respective 33 macro blocks. In general, it estimates the total bin number of a plurality of macro blocks constituting a divided image.

The entropy encoding selector609selects an entropy encoding mode using each estimated bin number of the symbol estimating device608. The main code amount controller622establishes an assigned picture rate based on the entropy encoding mode selecting signal628established by the entropy encoding selector609. Furthermore, as explained in detail later, the macro block minimum quantizing parameter (which is also called “macro block minimum QP” in this specification) is calculated such that each of the first to third main macro block encoders6211-6213completes the entropy encoding within a predetermined time period.

After the preliminary parallel encoder604completes the encoding of the whole macro blocks, the second image frame buffer6022reads the target image to be encoded from the first image frame buffer6021, and prepares for the start of the encoding of the target picture to be encoded.

The first main macro block encoder6211which corresponds to the first preliminary macro block encoder6111encodes a divided image of the image in the second image frame buffer6022. Likewise, the second main macro block encoder6212which corresponds to the second preliminary macro block encoder6112encodes the corresponding divided image of the image in the second image frame buffer6022, and the third main macro block encoder6213which corresponds to the third preliminary macro block encoder6113encodes the corresponding divided image of the image in the second image frame buffer6022.

The main parallel encoder605encodes three macro blocks of the target picture to be encoded of the second image frame buffer6022in parallel at a time. Then, it outputs the first to third bit streams6331-6333as bit streams of the first to third main macro block encoders6211-6213respectively. However, the first to third main macro block encoders6211-6213perform entropy encoding with the entropy encoding mode established through the main code amount controller622. Furthermore, the first to third main macro block encoders6211-6213perform the encoding while compensating the quantizing parameter supplied from the main code amount controller622using the macro block minimum qp established through the main code amount controller622. The multiplexer610multiplexes the first to third bit streams6331-6333outputted from the first to third main macro block encoders6211-6213, and outputs it externally.

The overall operation of the image encoding apparatus600of the third embodiment is briefly explained hereinafter. The image encoding apparatus600performs the main encoding of a target picture to be encoded by operating the first image frame buffer6021and preliminary parallel encoder604at the start of the picture encoding. After the preliminary encoding of the target picture to be encoded, the image encoding apparatus600determines an entropy encoding mode by the entropy encoding mode selecting signal628by operating the symbol estimating device608and entropy encoding selector609at the start of the picture encoding. The image encoding apparatus600updates an assigned picture rate and calculates the macro block minimum qp by operating the main code amount controller622after the decision of the entropy encoding mode and before the start of the main encoding of macro blocks of the target picture to be encoded. After the picture rate update, the image encoding apparatus600performs the main encoding of macro blocks of the target picture to be encoded by operating the second image frame buffer6022and main parallel encoder605. After the whole macro blocks within the picture are encoded, the image encoding apparatus600performs predetermined processes and starts the encoding of the next picture.

FIG. 7shows the operation of the image encoding apparatus for preliminary picture encoding. The following explanation is made with reference toFIG. 6.

At the start of the preliminary encoding of a picture, an assigned picture rate of the preliminary picture encoding pre_pic_rate is established at the preliminary parallel encoder604(Step S701). The quantizing step size qs_rc to achieve this assigned picture rate pre_pic_rate can be calculated with the above-mentioned equation (14).

After the parameter is established in this manner, the preliminary parallel encoder604performs the preliminary encoding of three macro blocks of a target picture to be encoded by using the above-mentioned VLC device which corresponds to the VLC device143(FIG. 11) as an entropy encoding mode (Step S702). Next, the preliminary parallel encoder604determines whether or not the whole macro blocks in the target picture to be encoded are encoded (Step S703). If the preliminary encoding is not completed (N), it returns to the step S702to continue the process. On the other hand, if the preliminary encoding were completed (Step S703: Y), the preliminary picture encoding process is completed (END).

FIG. 8shows the operation of the image encoding apparatus for main picture encoding. The following explanation is made with reference toFIG. 6.

At the start of the main encoding of the picture, the second image frame buffer6022reads the target picture to be encoded which is stored in the first image frame buffer6021. The symbol estimating device608calculates the estimated bin numbers of the divided images for which their respective first to third main macro block encoders6211-6213will perform the encoding (e_bin[i], 0≦i≦2) with the following equation (18) using the bit numbers of the first to third preliminary macro block encoders6111-6113(pre_rate[i], 0≦i≦2).

In the equation, “qs_rc” is calculated from the above-mentioned equation (17).

After the symbol number is estimated in this manner, the entropy encoding selector609determines an entropy encoding mode using the estimated bin numbers of the divided images for which the first to third main macro block encoders6211-6213will perform the encoding e_bin[i] (Step S722).

Firstly, it calculates the average quantizing step size qs_cabac using the above-mentioned equation (7) for the case where the target picture is encoded by the CABAC device which corresponds to the CABAC device142(FIG. 12).

In the equation, “epic_bin” is calculated from the following equation (19).

In the equation, “ppic_bin” is expressed by the above-mentioned equation (8).

As seen from these equations (7), (19) and (8), “qs_cabac” is the image quality by the CABAC device under the condition where the upper limit bit number is imposed by the average processible bin number within a predetermined time period.

Then, it calculates the average quantizing step size qs_vlc using the equation (9) for the case where the target picture is encoded by the VLC device.

Similar to the second embodiment, “qs_vlc” is the image quality by the VLC device under the condition where the reduction in compression efficiency of the VLC device is imposed.

Finally, it brings the entropy encoding mode selecting signal628to one of the states with the equation (10) using the average quantizing step size qs_cabac and average quantizing step size qs_vlc. With these operations, the step S722is completed.

At the next step S723, the main code amount controller622updates the assigned picture rate of the target picture to be encoded pic_rate based on the entropy encoding mode selecting signal628(mode) using the equation (11).

Incidentally, the quantizing step size qs_rc to achieve the assigned picture rate pic_rate can be calculated with the equation (12).

After the process of the step S723is completed, the main code amount controller622calculates the macro block minimum qp (mb_min_qp) with the following equation (29) at the next step S724.

In the equation, “qs”, “max_emb_bin”, “m_emb_bin” and “emb_bin[i]” are expressed by the following equations (21)-(24).

Furthermore, the function qs2qp(x) is a function to calculates a quantizing parameter corresponding to a quantizing step size x, and “min_qs” is a quantizing step size corresponding to the minimum quantizing parameter (qp=0). From the calculation of the equation (20), the macro block minimum qp which is used as the restriction such that it does not exceeds the bin number for which the first to third main macro block encoders6211-6213can process within a predetermined time can be obtained.

After the process of the step S724is completed, the main parallel encoder605encodes three macro blocks of the target picture to be encoded by using the decided entropy encoding mode selecting signal628at the next step S725.

Similar to the second embodiment, the main code amount controller622monitors a bit stream outputted from the above-mentioned entropy encoder215A (not shown) which corresponds to the entropy encoder115shown inFIG. 9. Then, if the bit number of the outputted is larger than the assigned picture rate, it outputs a quantizing parameter indicating the increase of a quantizing step size. On the other hand, if the bit number of the bit stream outputted from the entropy encoder215A is smaller than the assigned picture rate, it outputs a quantizing parameter indicating the decrease of a quantizing step size.

However, first to third main macro block encoders6211-6213perform the encoding while compensating the quantizing parameter supplied from the main code amount controller622mb_qp using the macro block minimum QP established through the main code amount controller622.

With this compensation, the number of bins which are inputted to the first to third main macro block encoders6211-6213can be controlled such that they are all processed within a predetermined time period.

After the encoding process of macro blocks of the step S725is completed in this manner, it determines whether or not the whole macro blocks in the picture for which the main encoding is performed are encoded (Step S726). If the encoding of macro blocks is not completed, it returns to the step S725to continue the process.

If the main encoding of the whole macro blocks were completed, the picture complexity pic_x is updated with the equation (13) depending on the decided entropy encoding mode (mode) (Step S727), and the main picture encoding process is completed (END).

In the equation (13), “pic_act_rate” is the occurring bit number of a picture which is encoded at that time, i.e., the sum of the output bit numbers of the first to third main macro block encoders6211-6213, and “qs_pic” is the average quantizing step size of the macro block of the picture which is encoded at that time. The picture complexity pic_x at the step S727stores its value for each picture type.

In accordance with the image encoding apparatus600of the third embodiment described above, it can establish an entropy encoding mode and a picture rate, for which the main parallel encoder605can achieve with its average processing capacity, based on the bit numbers of the bit streams of the first to third preliminary macro block encoders6111-6113of the preliminary parallel encoder604. Furthermore, the number of bins which are inputted to the first to third main macro block encoders6211-6213can be controlled such that they are all processed within a predetermined time period by compensating the quantizing parameter of the first to third main macro block encoders6211-6213based on the macro block minimum qp calculated from the bit streams of the first to third preliminary macro block encoders6111-6113. In this manner, even in the case where one picture is divided to several screens and they are encoded in parallel using several macro block encoders, the image encoding apparatus600can minimize image degradation, and ensure that encoding of a picture is completed within a certain time period.

The present invention is not limited to First to the third embodiments explained above, and various modifications can be made to the embodiments. For example, in the second embodiment, when the preliminary encoder404uses the CABAC device (not shown) to perform entropy encoding, the bin number can be used as substitute for the bit number, and still the same advantageous effects of the present invention is achieved.

INDUSTRIAL APPLICABILITY

The present invention is utilized, for example, in image encoding technology, in particular, encoding apparatus capable of selecting an entropy encoding scheme.