Picture encoding device, picture decoding device, and picture communication system

In a picture encoding device and a picture decoding device, the access to a reference frame memory is suppressed. The picture encoding device is comprised of a first encoder for intra picture encoding, a second encoder for inter picture encoding, and an intermediate buffer. A local decoded picture generated by the first encoder is stored as a reference picture in the intermediate buffer, and the inter picture encoding by the second encoder is performed by referring to the local decoded picture in the intermediate buffer. A picture decoding device is comprised of a first decoder for intra picture decoding, a second decoder for inter picture decoding, and an intermediate buffer. A local decoded picture generated by the first decoder is stored as a reference picture in the intermediate buffer, and the inter picture decoding by the second decoder is performed by referring the local decoded picture in the intermediate buffer.

CROSS-REFERENCE TO RELATED APPLICATIONS

The disclosure of Japanese Patent Application No. 2014-139488 filed on Jul. 7, 2014 including the specification, drawings and abstract is incorporated herein by reference in its entirety.

BACKGROUND

The present invention relates to a picture encoding device, a picture decoding device, and a picture communication system, and is suitably applicable to picture encoding and picture decoding with low delay at low cost.

Pictures have a huge amount of information; however, there is a strong correlation between pictures closely located within the same frame or between pictures located in the same coordinates in adjacent frames. Accordingly, this correlation is used for compression of the amount of code. For example, as typical international standards, the compression encoding and decoding methods of moving pictures are standardized, such as MPEG-2 (ITU-T Rec. H.262|ISO/IEC 13818-2), MPEG-4 (ISO/IEC 14496-2), and H.264 (ITU-T Rec. H.264|ISO/IEC 14496-10). According to these coding methods, in the intra frame coding, an original picture is orthogonal-transformed, and the result is quantized, variable-length-encoded, and transmitted. On the other hand, inverse quantization and inverse orthogonal transformation are performed to create a reference picture, and the reference picture are stored in a reference frame memory in preparation to be used for subsequent inter frame coding. In the inter frame coding, a subtraction is calculated between the original picture of an inputted frame as an encoding target and the reference picture of a frame in the past, or in the future in some cases, stored in the reference frame memory, and the subtraction result is encoded. A picture encoded using the picture information of only the frame as an encoding target is called an “I picture”, a picture encoded using the picture information of the past frame in addition to the picture information of the present frame as an encoding target is called a “P picture”, and a picture encoded also using the picture information of the frame of the future is called a “B picture.” Various encoding and decoding methods which combine an I picture, a P picture, and also a B picture are adopted. The compression efficiency of an I picture is lower than others, however, the I picture is certainly necessary as a starting point of other inter frame coding in order to complete encoding and decoding with the picture information of the frame alone. In order to improve encoding efficiency, one I picture is combined with one or more P pictures and also one or more B pictures.

Patent Document 1 discloses an encoding device (a picture encoding device) which can reduce the frequency of access to a reference memory to store reference pictures. As illustrated inFIG. 1of the Patent Document 1, the encoding device100is comprised of a motion compensation unit003and a frequency conversion unit004. The motion compensation unit003is coupled to the reference memory010via an intermediate memory009. The encoding device100reads, from the reference memory010, the information of the reference frame which the motion compensation unit003uses in the encoding, and stores it in the intermediate memory009. The reference memory010is arranged in a common use SDRAM (Synchronous Dynamic Random Access Memory) in the system, and the intermediate memory009is arranged as a dedicated memory of the encoding device100. Compared with the case where the intermediate memory009is not arranged, the present configuration can reduce the frequency of access to the reference memory010. For example, as illustrated in FIG. 3 of Patent Document 1, when P5(a fifth inputted frame to be encoded as a P picture) is encoded, I2in the reference memory010is once read and stored in the intermediate memory009, and is used for the encoding of P5. When B3and B4are encoded, I2and P5in the reference memory010are stored in the intermediate memory009, and are used for the encoding of both B3and B4simultaneously. Originally it is necessary to read I2and P5from the reference memory010when encoding B3, and it is necessary to read I2and P5again when encoding B4. However, when B3and B4refer to the same region of the same frame (I2and P5in the present case), it is possible to use the same data by encoding B3and B4simultaneously; accordingly, it is possible to reduce the access to the reference memory010which is otherwise necessary in the encoding of B4.

Patent Document 2 discloses a picture encoding device and a picture decoding device which can reduce the encoding arithmetic amount and the decoding arithmetic amount, suppressing deterioration in the encoding efficiency. An input picture is divided into blocks of size n×m. Furthermore each block is divided into K sub-blocks of size n1×m1. A divided picture Pk (k=1−K) is created by collecting sub-blocks in the same position in a block. The divided picture P0is intra-picture-encoded and the divided pictures P1-PK are inter-picture-encoded. The reference picture of the sub-block Bk of Pk is created from the sub-block B0of the surrounding P0with a filter specified by the relative position of the pixel. Accordingly, it is possible to suppress the coding arithmetic amount to a smaller value than in the intra picture prediction coding in the past.(Patent Document 1) Japanese Unexamined Patent Application Publication No. 2009-111797(Patent Document 2) Japanese Unexamined Patent Application Publication No. 2012-175332

SUMMARY

As a result of the examination of Patent Documents 1 and 2 by the present inventors, it turned out that there are new issues as follows.

Patent Document 1 discloses the picture encoding device which can reduce the frequency of access to the reference memory010. However, it turned out that it is difficult to reduce the access to the reference memory010to zero for example, at the time of encoding of a B frame; accordingly the effect of reducing the memory access is not enough.

The following will explain a solution to such an issue. The other issues and new features of the present invention will become clear from the description of the present specification and the accompanying drawings.

One embodiment according to the present application goes as follow.

A picture encoding device which encodes plural time-series pictures is comprised of an intra picture encoder, an inter picture encoder, and an intermediate buffer. A local decoded picture created by the intra picture encoder is stored as a reference picture in the intermediate buffer. The inter picture encoding which refers to the local decoded picture is performed while the local decoded picture concerned is stored in the intermediate buffer. When the local decoded picture concerned does not need to be referred to any more after that, the reference frame memory for storing the local decoded picture is omitted. Similarly, a picture decoding device is comprised of an intra picture decoder, an inter picture decoder, and an intermediate buffer. A local decoded picture created by the intra picture decoder is stored as a reference picture in the intermediate buffer. The inter picture decoding which refers to the local decoded picture is performed while the local decoded picture concerned is stored in the intermediate buffer. When the local decoded picture concerned does not need to be referred to anymore after that, the reference frame memory for storing the local decoded picture is omitted.

The effect obtained by one embodiment described above is explained briefly as follows.

That is, the frequency of access to the reference frame memory is suppressed to a low level, and in some cases, it is possible to adopt the configuration in which the reference frame memory is omitted.

DETAILED DESCRIPTION

1. Outline of Embodiment

First, an outline of a typical embodiment disclosed in the present application is explained. A numerical symbol of the drawing referred to in parentheses in the outline explanation about the typical embodiment only illustrates what is included in the concept of the component to which the numerical symbol is attached.

The typical embodiment disclosed in the present application is a picture encoding device (100-700) which encodes plural time-series pictures. The picture encoding device is comprised of a first encoder (an intra picture encoder001), a second encoder (an inter picture encoder002), an intermediate buffer (005,006), and an encoding target picture controller (003,004,007).

The first encoder uses picture information in a picture of an encoding target to encode the picture information of the picture concerned, creates a reference picture from the encoding result, and writes the reference picture in the intermediate buffer.

The second encoder refers to picture information in a picture of an encoding target and the reference picture stored in the intermediate buffer to encode the picture information of the picture concerned.

The encoding target picture controller makes the second encoder start encoding the picture with reference to the reference picture concerned, before the following reference picture is written in the intermediate buffer by the first encoder.

According to this configuration, it is possible to keep the access to the reference frame memory (103) at a minimum, and in another embodiment, it is possible to omit the reference frame memory.

In Paragraph 1, each of the time-series pictures is a frame composed of plural pixel lines, and the frame includes a first field composed only of even-numbered pixel lines and a second field composed only of odd-numbered pixel lines.

The encoding target picture controller (043,044) supplies the first field and the second field to the first encoder (040) and the second encoder (041), respectively.

The first encoder uses picture information in a field of an encoding target to encode the picture information of the picture concerned, and creates a reference picture from the encoding result, and writes the reference picture in the intermediate buffer (045).

The second encoder refers to picture information in a field of an encoding target and the reference picture stored in the intermediate buffer to encode the picture information of the field concerned.

According to this configuration, it is possible to omit the reference frame memory (103).

(3) <Field Division in a Smaller Unit (Embodiment 5)>

In Paragraph 1, each of the time-series pictures is a frame composed of plural pixel lines, and the frame is composed of plural macroblocks. Each of the plural time-series pictures includes a first field composed only of even-numbered pixel lines included in two macroblocks which adjoin mutually in an orthogonal direction to a pixel line among the macroblocks and a second field composed only of odd-numbered pixel lines included in the two macroblocks concerned.

The encoding target picture controller (053,054) supplies one of the first field and the second field to the first encoder (050) and the other to the second encoder (051).

The first encoder refers to picture information in a field of an encoding target to encode the picture information of the picture concerned, creates a reference picture from the encoding result, and writes the reference picture in the intermediate buffer (052).

The second encoder refers to picture information in a field of an encoding target and the reference picture stored in the intermediate buffer to encode the picture information of the field concerned.

According to this configuration, it is possible to compose the intermediate buffer (052) with a memory element of smaller storage capacity, and at the same time it is possible to suppress the delaying amount of the encoding to a low level.

In Paragraph 3, the encoding target picture controller (053,058,059) supplies selectively one of the first field and the second field to the first encoder (050) and the other to the second encoder (051), by making the selection for every two macroblocks concerned.

According to this configuration, it is possible to suitably choose the reference direction with higher encoding efficiency.

In Paragraph 1, each of the plural time-series pictures is a frame including plural pixel lines, each composed of plural pixels, and the frame is composed of plural macroblocks. Each of the plural time-series pictures includes a first column picture composed of only even-numbered pixels of each pixel line included in two macroblocks adjoining mutually in the extension direction of the pixel line among the macroblocks and a second column picture composed of only odd-numbered pixels of each pixel line included in the two macroblocks concerned.

The encoding target picture controller (063,064) supplies one of the first column picture and the second column picture to the first encoder (060), and the other to the second encoder (061).

The first encoder refers to picture information in a column picture of an encoding target to encode the picture information of the picture concerned, creates a reference picture from the encoding result, and writes the reference picture in the intermediate buffer (065).

The second encoder refers to picture information in a column picture of an encoding target and the reference picture stored in the intermediate buffer to encode the picture information of the column picture concerned.

According to this configuration, it is possible to compose the intermediate buffer (065) with a memory element of smaller storage capacity, and at the same time it is possible to suppress the delaying amount of the encoding to a still lower level than in Paragraph 3.

In Paragraph 5, the encoding target picture controller (063,064,068) supplies selectively one of the first column picture and the second column picture and the other to the second encoder, by making the selection for every two macroblocks concerned.

According to this configuration, it is possible to suitably choose the reference direction with higher encoding efficiency.

In Paragraph 1, the first encoder is an I-picture encoder (030) and the second encoder is a P-picture encoder (031), and the encoding target picture controller (033,034) supplies two consecutive pictures to the first encoder and the second encoder and makes the encoders perform encoding in parallel.

According to this configuration, in the picture encoding device which creates an encoded stream including an I picture and a P picture alternately, it is possible to omit the reference frame memory.

In Paragraph 7, each of the plural time-series pictures includes plural macroblocks.

The first encoder (030) performs the encoding for every macroblock, creates a reference picture from the encoding result, and writes the reference picture in the intermediate buffer (035). The second encoder (031) refers to the reference picture of the macroblock within a prescribed range among the reference pictures to perform the encoding.

The encoding target picture controller (032) makes the second encoder (031) start the encoding after the completion of the writing of the reference picture of the macroblock within the prescribed range, from the first encoder (030) to the intermediate buffer (035).

According to this configuration, the latency time with which the P-picture encoder starts the encoding can be suppressed to necessary minimum; accordingly, it is possible to provide an picture encoding device with low delay as a whole.

(9) <An I Encoder+ a P Encoder+ a B Encoder+ an Intermediate Buffer (Embodiment 2)>

In Paragraph 1, the first encoder is an I-picture encoder (020) and the second encoder includes a P-picture encoder (021) and a B-picture encoder (022). The intermediate buffer operates as a first intermediate buffer (025), and the picture encoding device (200) is further provided with a second intermediate buffer (026).

The I-picture encoder (020) encodes picture information of a first picture supplied, creates a first local decoded picture from the encoding result, and writes the first local decoded picture in the first intermediate buffer (025).

The P-picture encoder (021) refers to picture information in a second picture supplied and the first local decoded picture to encode the picture information of the second picture, creates a second local decoded picture from the encoding result, and writes the second local decoded picture in the second intermediate buffer (026).

The B-picture encoder (022) refers to picture information in a third picture supplied, the first local decoded picture, and the second local decoded picture, to encode the picture information of the third picture.

The encoding target picture controller (027) makes the P-picture encoder (021) start encoding the picture with reference to the reference picture concerned, before the following first local decoded picture is written in the first intermediate buffer (025) from the first I-picture encoder (020). The encoding target picture controller (028) makes the B-picture encoder (022) start encoding of the picture with reference to the reference picture concerned, before the following second local decoded picture is written in the second intermediate buffer (026) from the P-picture encoder (021).

According to this configuration, in the picture encoding device (200) which creates an encoded stream including an I picture and a P picture alternately, it is possible to omit the reference frame memory (103). At this time, the P picture refers to the local decoded picture of the I picture as a reference picture, and the B-picture refers to the local decoded picture of the I picture and the local decoded picture of the P picture as a reference picture. The encoding of the I picture, the P picture, and the B picture is performed in parallel. It is preferable to include plural B-picture encoders which refer to the local decoded picture of the same I picture and the local decoded picture of the P picture as a reference picture.

In Paragraph 9, each of the plural time-series pictures includes plural macroblocks.

The I-picture encoder (020) performs the encoding for every macroblock, creates a first local decoded picture from the encoding result, and writes the first local decoded picture in the first intermediate buffer (025).

The P-picture encoder (021) performs the encoding, referring to the reference picture of a macroblock within a prescribed range among the first local decoded pictures, creates a second local decoded picture from the encoding result, and writes the second local decoded picture in the second intermediate buffer (026).

The B-picture encoder (022) performs the encoding, referring to the reference picture of a macroblock within the prescribed range, out of the first local decoded picture and the second local decoded picture.

The encoding target picture controller (027) makes the P-picture encoder start the encoding after the completion of the writing of the first local decoded picture of the macroblock within the prescribed range, from the I-picture encoder to the first intermediate buffer. The encoding target picture controller (028) makes the B-picture encoder (022) start the encoding after the completion of the writing of the second local decoded picture of the macroblock within the prescribed range, from the P-picture encoder (021) to the second intermediate buffer (026).

According to this configuration, the latency time with which the P-picture encoder (021) and the B-picture encoder (022) start the encoding can be suppressed to a necessary minimum, and accordingly, it is possible to provide the picture encoding device (200) with low delay.

In Paragraph 1, the picture encoding device (100) further comprises the first encoder (001) operating as an I-picture encoder or a P-picture encoder; the second encoder (002) operating as a B-picture encoder; the intermediate buffer operating as a first intermediate buffer (006); a third intermediate buffer (005); and a reference memory (103) externally coupled.

When operating as the I-picture encoder, the first encoder (001) encodes picture information of a first picture supplied, creates a first local decoded picture from the encoding result, and writes the first local decoded picture in the first intermediate buffer (006) and the reference memory (103).

The encoding target picture controller reads the first local decoded picture from the reference memory (103) and stores it in the third intermediate buffer (005).

When operating as the P-picture encoder, the first encoder (001) refers to the first local decoded picture stored in the third intermediate buffer (005) to encode picture information of a second picture supplied, creates a second local decoded picture from the encoding result, and writes the second local decoded picture in the first intermediate buffer (006).

The B-picture encoder refers to the first local decoded picture and the second local decoded picture to encode the third picture supplied.

The encoding target picture controller (007) makes the B-picture encoder (002) start encoding of the picture with reference to the reference picture concerned, after the second local decoded picture is written from the first encoder (001) to the first intermediate buffer (006) and before the following first local decoded picture or the following second local decoded picture is written.

According to this configuration, it is possible to perform the encoding of the I picture and the encoding of the B picture, and the encoding of the P picture and the encoding of the B picture, respectively in parallel, and accordingly, it is possible to reduce the frequency of access to the reference memory (103). The local decoded picture of the I picture is once stored in the reference memory and read to the third intermediate buffer (005), referred to as a reference picture in the encoding of the P picture, and also referred to in parallel as a reference picture in the encoding of the B picture. The local decoded picture of the P picture is stored in the first intermediate buffer and referred to as a reference picture in the encoding of the B picture. In this way, the encoding of the I picture or the encoding of the P picture, and the encoding of the B-picture which refers to the local decoded picture of the same I picture or the local decoded picture of the P picture are performed in parallel. Accordingly, it is possible to suppress the number of read-out of the local decoded picture from the reference memory to a minimum.

In Paragraph 11, each of the time-series pictures is composed of plural macroblocks.

The first encoder (001) performs encoding for every macroblock, and when operating as the P-picture encoder, the first encoder creates a second local decoded picture from the encoding result and writes the second local decoded picture in the first intermediate buffer (006).

The B-picture encoder (002) refers to the reference picture of the macroblock within the respectively prescribed range, out of the first local decoded picture and the second local decoded picture, to perform the encoding.

The encoding target picture controller (007) makes the B-picture encoder (002) start the encoding after the completion of the writing of the second local decoded picture of the macroblock within the prescribed range, from the first encoder (001) to the first intermediate buffer (006).

According to this configuration, the latency time with which the B-picture encoder (002) starts the encoding can be suppressed to necessary minimum; accordingly, it is possible to provide the picture encoding device (100) with low delay.

In Paragraph 1, the time-series pictures include plural time-series pictures respectively seen from multi-view points, and a time-series picture seen from one viewpoint is defined as a main picture and a time-series picture seen from another viewpoint is defined as a sub picture.

The first encoder (071) sets the main picture as an encoding target, refers to picture information in the main picture of the encoding target to encode the picture information of the main picture concerned, creates a reference picture from the encoding result, and writes the reference picture in the intermediate buffer (076).

The second encoder (072) sets the sub picture temporally corresponding to the main picture as an encoding target, and refers to picture information in the sub picture of the encoding target and the reference picture stored in the intermediate buffer, to encode the picture information of the sub picture concerned.

According to this configuration, in the picture encoding device (700) which performs the multi-view coding, it is possible to omit the reference memory (103).

(14) <Multi-View Coding=an I/P Encoder for a Main Picture+ a B Encoder for a Sub Picture+ a First and a Third Intermediate Buffer+ a Reference Frame Memory (Embodiment 7)>

In Paragraph 1, the time-series pictures include plural time-series pictures respectively seen from multi-view points, a time-series picture seen from one viewpoint is defined as a main picture and a time-series picture seen from another viewpoint is defined as a sub picture. The picture encoding device is further comprised of the first encoder (071-1) operating as a base view encoder; the second encoder (071-2) operating as an inter-view encoder; the intermediate buffer operating as a first intermediate buffer (076-1); a third intermediate buffer (075); and a reference memory (103) externally coupled.

The first encoder (071-1) encodes picture information of a first main picture supplied, creates a first local decoded picture from the encoding result, and writes the first local decoded picture in the first intermediate buffer (076-1) and the reference memory (103).

The encoding target picture controller reads the first local decoded picture from the reference memory (103) and stores it in the third intermediate buffer (075).

When encoding a second main picture next to the first main picture, the first encoder (071-1) refers to the first local decoded picture stored in the third intermediate buffer to encode picture information of the second main picture supplied, creates a second local decoded picture from the encoding result, and writes the second local decoded picture in the first intermediate buffer (076-1).

The second encoder (071-2) refers to the second local decoded picture to encode picture information of a first sub picture corresponding to the first main picture supplied.

The encoding target picture controller (077-2) makes the second encoder (071-2) start encoding of the picture with reference to the reference picture concerned, after the second local decoded picture is written from the first encoder to the first intermediate buffer (076-2) and before the following local decoded picture is written.

According to this configuration, it is possible to perform the I-picture encoding of the main picture and the B-picture encoding of the following main picture, and the inter-view encoding of the sub picture and the inter-view encoding of the following sub picture, respectively in parallel; accordingly, it is possible to reduce the frequency of access to the reference memory (103), as in the case with Paragraph 11.

The typical embodiment disclosed in the present application is a picture decoding device (800,810) which decodes plural time-series pictures from a first and a second encoded stream supplied. The picture decoding device is comprised of a first decoder (intra picture decoder081); a second decoder (inter picture decoder082); an intermediate buffer (086); and a decoding target image controller (083,084,087).

The first decoder (081) refers to the first encoded stream to decode picture information of a picture corresponding to the first encoded stream, creates a reference picture from the decoding result, and writes the reference picture to the intermediate buffer (086).

The second decoder (082) refers to the second encoded stream and the reference picture stored in the intermediate buffer to decode picture information of a picture corresponding to the second encoded stream.

The decoding target image controller (087) makes the second decoder start decoding a picture with reference to the reference picture concerned, before the following reference picture is written in the intermediate buffer from the first decoder.

According to this configuration, in the picture decoding device, it is possible to keep the access to the reference memory (803) to a minimum. In another embodiment, it is possible to omit the reference memory.

In Paragraph 15, the first decoder is an I-picture decoder (088), and the second decoder is a P-picture decoder (089).

The decoding target image controller (083,084) supplies a first encoded stream and a second encoded stream, corresponding to two consecutive pictures, to the first decoder (088) and the second decoder (089), respectively, and makes the two decoders perform decoding in parallel.

According to this configuration, in the picture decoding device which decodes an encoded stream including an I picture and a P picture alternately, it is possible to omit the reference memory.

In Paragraph 15, the picture decoding device (800) is further comprised of the first decoder (081) operating as an I-picture decoder or a P-picture decoder; the second decoder (082) operating as a B-picture decoder; the intermediate buffer operating as a first intermediate buffer (086); a third intermediate buffer (085); and a reference memory (803) externally coupled.

When operating as the I-picture decoder, the first decoder (081) decodes an encoded stream corresponding to a first picture, creates a first local decoded picture from the decoding result concerned, and writes the first local decoded picture in the first intermediate buffer (086) and the reference memory (803).

The decoding target image controller reads the first local decoded picture from the reference memory (803) and stores it in the third intermediate buffer (085).

When operating as the P-picture decoder, the first decoder (081) refers to the first local decoded picture stored in the third intermediate buffer (085) to decode an encoded stream corresponding to a second picture, creates a second local decoded picture from the decoding result concerned, and writes the second local decoded picture in the first intermediate buffer (086).

The B picture decoder (082) refers to the first local decoded picture and the second local decoded picture to decode an encoded stream corresponding to a third picture supplied.

The decoding target image controller (087) makes the B-picture decoder (082) start decoding of a picture with reference to the reference picture concerned, after the second local decoded picture is written from the first decoder (081) to the first intermediate buffer and before the following first local decoded picture is written.

According to this configuration, it is possible to perform the decoding of the I picture and the decoding of the B picture, and the decoding of the P picture and the decoding of the B picture, respectively in parallel, and it is possible to reduce the frequency of access to the reference memory, as is the case with the picture encoding device described in Paragraph 11.

The typical embodiment disclosed in the present application is comprised of a transmitter (1000,1100,1200,1300) including a picture encoding device (100) and an output control device (104); a transmission line (1002); and a receiver (1001,1101,1201,1301) including a picture decoding device (800,805,806,807).

The picture encoding device (100) encodes plural time-series pictures, and is comprised of a first encoder (001), a second encoder (002), an intermediate buffer (005,006), and an encoding target picture controller (003,004,007).

The first encoder refers to picture information in a picture of an encoding target to encode the picture information of the picture concerned, outputs a first encoded bit string, creates a reference picture from the encoding result, and writes the reference picture in the intermediate buffer.

The second encoder refers to picture information in a picture of an encoding target and the reference picture stored in the intermediate buffer, to encode the picture information of the picture concerned and outputs a second encoded bit string.

The encoding target picture controller makes the second encoder start encoding a picture with reference to the reference picture concerned, before the following reference picture is written in the intermediate buffer by the first encoder.

The output control device applies time-division multiplexing to the first encoded bit string and the second encoded bit string, and sends them out to the transmission line as an encoded stream.

The receiver supplies the encoded stream inputted from the transmission line to the picture decoding device, and makes the picture decoding device decode the encoded stream.

According to this configuration, it is possible to provide the picture communication system with low delay. As is the case with Paragraph 1, it is possible to keep the access to the reference memory to a minimum in the picture encoding device, and it is possible to omit the reference memory in another embodiment.

(19) <Multiplexing of an Encoded Bit String (Embodiment 10)>

In Paragraph 18, the output control device (104) applies time-division multiplexing to the first encoded bit string and the second encoded bit string in a unit finer than a picture, and sends them out to the transmission line as an encoded stream.

The receiver is further comprised of an input control device (801).

The input control device applies demultiplexing (833) to the encoded stream inputted from the transmission line, divides the encoded stream into a first encoded bit stream corresponding to the first encoded bit string, and a second encoded bit stream corresponding to the second encoded bit string, and supplies the first encoded bit stream and the second encoded bit stream to the picture decoding device (805).

According to this configuration, it is possible for the receiver to reconstruct the first and the second encoded bit stream which are in conformity to the coding standard, from the encoded stream received from the transmission line. Therefore, it is possible to decode an image with the use of a general-purpose picture decoding device.

(20) <Multiplexing of an Encoded Bit String (Embodiment 12)>

In Paragraph 19, the picture decoding device is comprised of a first decoder (081), a second decoder (082), a first intermediate buffer (086), a third intermediate buffer (085), a decoding target image controller (083,084,087), and a reference memory (803) externally coupled.

The first decoder (081) operates as an I-picture decoder or a P-picture decoder, and when operating as the I-picture decoder, the first decoder decodes the first encoded stream corresponding to a first picture, creates a first local decoded picture from the decoding result concerned, and writes the first local decoded picture in the first intermediate buffer (086) and the reference memory (803).

The decoding target image controller reads the first local decoded picture from the reference memory (803) and stores it in the third intermediate buffer (085).

When operating as the P-picture decoder, the first decoder (081) refers to the first local decoded picture stored in the third intermediate buffer (085) to decode the first encoded stream corresponding to a third picture, creates a second local decoded picture from the decoding result concerned, and writes the second local decoded picture in the first intermediate buffer (086).

The second decoder operates as a B-picture decoder, and refers to the first local decoded picture and the second local decoded picture to decode the second encoded stream corresponding to a second picture.

The decoding target image controller (087) makes the B-picture decoder start decoding of a picture with reference to the local decoded picture concerned, after the second local decoded picture is written from the first decoder (081) to the first intermediate buffer and before the following first local decoded picture is written.

The decoding target image controller (087) supplies the first encoded stream, the second encoded stream, and the third encoded stream, respectively corresponding to consecutive three pictures, the first picture, the second picture, and the third picture, to the first decoder and the second decoder sequentially, and makes the two decoders perform decoding in parallel.

According to this configuration, it is possible to provide the picture communication system with low delay which combines the picture encoding device (100) described in Paragraph 11 and the picture decoding device (800) described in Paragraph 17. In the picture encoding device, it is possible to perform the encoding of the I picture and the encoding of the B picture, and the encoding of the P picture and the encoding of the B picture, respectively in parallel, and as is the case with Paragraph 11, it is possible to reduce the frequency of access to the reference memory (103). Also in the picture decoding device, it is possible to perform the decoding of the I picture and the decoding of the B picture, and the decoding of the P picture and the decoding of the B picture, respectively, in parallel, and as is the case with Paragraph 17, it is possible to reduce the frequency of access to the reference memory (803).

2. Details of Embodiment

Embodiment is further explained in full detail.

A typical embodiment disclosed in the present application is a picture encoding device which encodes plural time-series pictures. The picture encoding device is comprised of an intra picture encoder, an inter picture encoder, an intermediate buffer, and an encoding target picture controller. The term “picture” indicates not only a frame and a field but also a block and a macroblock which compose them, that is, it indicates a batch of images which is used as the unit of encoding or decoding.

The intra picture encoder refers to picture information in a picture as an encoding target to encode the picture information of the picture concerned, creates a reference picture such as a local decoded picture from the encoding result, and writes the reference picture in the intermediate buffer. For example, an encoder of an I picture corresponds to this. The intra picture encoder may further be provided with the function of inter picture encoding (for example, encoding of a P picture or a B picture) which refers to the reference picture (local decoded picture) created from pictures other than the picture as the encoding target and encodes the picture information of the picture concerned.

The inter picture encoder refers to picture information in a picture as an encoding target and the reference picture (local decoded picture) stored in the intermediate buffer and encodes the picture information of the picture concerned. For example, an encoder of a P picture or a B picture accompanied by motion compensation corresponds to this.

The encoding target picture controller makes the inter picture encoder start encoding the picture with reference to the reference picture concerned, before a reference picture created for the encoding of the next picture is written from the intra picture encoder to the intermediate buffer.

According to this configuration, after the reference picture (local decoded picture) is once stored in the reference frame memory, it is not necessary to read anew the reference picture for reference before the encoding which refers to it (inter picture encoding). Therefore, it is possible to keep the access to the reference frame memory at a minimum, and it is possible to omit the reference frame memory in another embodiment. When plural pieces of encoding which refer to the same reference picture are performed in parallel, it is possible to keep the access to the reference frame memory at a minimum, without reading the same reference picture from the reference frame memory repeatedly.

The above is a fundamental technical thought and it can be applied to various embodiments. The present invention is explained in concrete forms in Embodiment 1 through Embodiment 7 in the following; however, the present invention is not restricted to these embodiments. Based on the corresponding technical thought, a picture decoding device is illustrated in Embodiment 8 and a picture communication system is illustrated in Embodiments 9-12. As for the picture decoding device and the picture communication system, the present invention is not restricted to the embodiments to be disclosed and can be materialized in various other forms.

The details are explained in the following with concrete examples.

FIG. 1is a block diagram illustrating an example of the entire configuration of a picture encoding device according to Embodiment 1.FIG. 2is a block diagram illustrating a configuration example of an output control device104. A signal line in each block diagram quoted by the present description is implemented with a signal line with one bit or plural bits. However, the drawing of a bus is omitted.

The entire configuration of the picture encoding device according to Embodiment 1 includes an input-picture supply device101, an input picture memory102, an encoding device A (100), a reference frame memory103, and an output control device104. The input-picture supply device101supplies an original picture as an encoding target and it is exemplified by an imaging device, such as a camera. The input picture memory102stores an original picture supplied from the input-picture supply device101, and the reference frame memory103stores a reference picture, such as a local decoded picture. Although the input picture memory102and the reference frame memory103are shown as separate memories, they may be implemented as one memory. For example, by the address mapping scheme, they may be implemented in an external SDRAM which is shared by the whole system. The encoding device A (100) reads the original picture as an encoding target from the input picture memory102in units of encoding and in order of encoding, and performs encoding. The encoding device A (100) stores a reference picture such as a local decoded picture created in the process, in the reference frame memory103, and reads for reference a reference picture necessary in encoding from the reference frame memory103. The encoding device A (100) outputs plural encoded bit strings as a result of encoding, and the output control device104creates an encoded stream from the plural encoded bit strings. A first encoded bit string and a second encoded bit string are illustrated inFIG. 1. However, further more encoded bit strings may be created and outputted. As illustrated inFIG. 2, the output control device104is comprised of a first output buffer010which stores the first encoded bit string temporarily, a second output buffer011which stores the second encoded bit string temporarily, and a switch012which switches the first and the second encoded bit string to create an encoded stream.

FIG. 3is a block diagram illustrating a configuration example of a picture encoding device A (100) according to Embodiment 1. The input-picture supply device101, the input picture memory102, the reference frame memory103, and the output control device104are also illustrated in the figure. The same symbol is attached to the same component as that inFIG. 1, and the repeated explanation thereof is omitted. In the input picture memory102and the reference frame memory103, original pictures0,1, . . . ,2n−1,2n, . . . , and reference frames X and Y are illustrated typically. Here, n is arbitrary natural number, and2n−1 indicates an odd-numbered frame and2nindicates an even-numbered frame.

The encoding device A (100) is comprised of an I/P encoder001, a B encoder002, an input controller003, a switch004, a third intermediate buffer005, a first intermediate buffer006, and a position controller007. The I/P encoder001encodes an I picture or a P picture, and the B encoder002encodes a B picture. The I picture, the P picture, and the B picture are described below. The input controller003controls the switch004, reads an original picture from the input picture memory102, and supplies it to the I/P encoder001and the B encoder002. The I/P encoder001writes a reference picture created in encoding into the reference frame memory103, and at the same time stores it in the first intermediate buffer006temporarily. The third intermediate buffer005stores temporarily a reference picture (reference frame) suitably read from the reference frame memory103. When the I/P encoder001encodes a P picture, a reference picture (reference frame) stored in the third intermediate buffer005is referred to. When the B encoder002encodes a B picture, a reference picture (reference frame) stored in the first intermediate buffer006temporarily and a reference picture (reference frame) stored at the third intermediate buffer005are referred to. The position controller007controls the processing position of the I/P encoder001and the B encoder002, that is, the position of the picture data of the processing target in encoding. The details thereof will be described below.

The I/P encoder001does not need to be an encoder which performs only the I-picture encoding and the P-picture encoding, but it may be replaced with an I/P/B encoder which performs encoding including the B-picture encoding. Similarly, the B encoder002does not need to be an encoder which performs only the B-picture encoding, but it may be replaced with an I/P/B encoder. The number of B encoders may not be one. By mounting N-piece B encoders, it is possible to set the number of B pictures between two P pictures (or I pictures) to N sheets. In that case, the reference region X and the reference region Y used as an input are common to the N-piece B encoders, and the N-piece B encoders can share the third intermediate buffer005and the first intermediate buffer006. The plural (N sheets) B pictures inserted between the same I picture or P picture group, refer in common to the reference picture (local decoded picture) created in the encoding of the I picture or P picture group concerned. Accordingly, during the period when the encoding of the plural (N sheets) B pictures is performed, it is only necessary to store the reference picture (local decoded picture) to be referred to in the encoding, in the third intermediate buffer005and the first intermediate buffer006, and it is not necessary to access the reference frame memory103to read another reference picture. An example of configuration in the case of N=2 is illustrated inFIG. 4.

FIG. 4is a block diagram illustrating another configuration example of the picture encoding device A (100) according to Embodiment 1. As is the case withFIG. 3, the input-picture supply device101, the input picture memory102, the reference frame memory103, and the output control device104are also illustrated in the figure. The encoding device A (100) is comprised of the I/P encoder001, the input controller003, the switch004, the third intermediate buffer005, the first intermediate buffer006, and the position controller007. What is different fromFIG. 3is that two B encoders002-1and002-2are provided corresponding to N=2. Two B encoders002-1and002-2refer to the same reference picture. Accordingly, the picture data of the same reference frame is supplied from the first intermediate buffer006and the third intermediate buffer005, respectively. Two B encoders002-1and002-2output a B encoded bit string1and a B encoded bit string2to the output control device104, respectively. Unlike inFIG. 3, the output control device104receives three inputs, and creates an encoded stream composed of the I/P encoded bit string, the B encoded bit string1, and the B encoded bit string2. If the number of the B pictures inserted between the same I picture or P picture group increases, it is possible to meet it by increasing the number of the B encoder and by increasing the number of inputs to the output control device104.

The operation of the picture encoding device is explained.

First, an I picture, a P picture, and a B picture are explained.FIG. 5-FIG. 7are explanatory drawings about the encoding of an I picture, a P picture, and a B picture, respectively. The I picture, the P picture, and the B picture are picture types adopted in a moving image coding system. The I picture is a picture type which performs encoding by the intra picture prediction with the use of the picture information in the picture. The P picture is a picture type which performs encoding by the inter picture prediction with the use of one reference frame. The B picture is a picture type which performs encoding by the inter picture prediction with the use of two reference frames. InFIG. 5-FIG. 7, the pictures are shown in the order of photographing, with the past on the left and the future on the right. In the I picture I0illustrated inFIG. 5, the encoding is performed by the intra picture prediction which uses picture information in the picture. In the P picture P3illustrated inFIG. 6, the encoding is performed by the inter picture prediction which uses I0as a reference frame. In the B picture B1illustrated inFIG. 7, the encoding is performed by the inter picture prediction which uses the past picture I0as a reference frame and the inter picture prediction which uses the future picture P3as a reference frame. Here, although not shown in the figure, the B picture B2uses I0and P3as the reference picture. The P picture refers to one past picture and the B picture refers to one past picture and one future picture. The number of the P pictures between the I pictures and the number of the B pictures between the I pictures or the P pictures are arbitrary respectively. However, depending on the kind of coding standards, such as H.264, it is possible to refer to two past pictures. Accordingly, in the present embodiment and other embodiments disclosed by the present application, the order of encoding is not necessarily specified. When actually performing encoding by the encoding device according to the present application, the order of encoding is changed to input the picture into the encoding device so that the picture to be referred to is encoded earlier than the picture to refer to.

The following explains the operation of the picture encoding device illustrated inFIG. 3when the encoding is performed so that one B picture is inserted between I pictures or P pictures like IBPBPB . . . . The input picture memory102is supplied with an original picture as the encoding target on a time-series basis from the input-picture supply device101; such as an original picture0, an original picture1, . . . , an original picture (2n−1), an original picture (2n), . . . . When encoding like IBPBPB . . . , the original picture0is encoded as the I picture, the original picture1as the B picture, the original picture2as the P picture, and henceforth the odd-numbered original picture (2n−1) is encoded as the B picture and the even-numbered original picture (2n) as the I picture or the P picture (n is one or a greater integer). It is assumed that the original picture (2n) is encoded as a P picture, the P picture (2n) uses the picture (2n−2) as a reference frame, and the B picture (2n−1) uses the picture (2n) and the picture (2n−2) as a reference frame.

The encoding device A (100) encodes every two pictures in parallel. Generally, encoding is performed in units of blocks each of which is a divided rectangle of a picture. This processing unit block is called a “macroblock” (hereinafter abbreviated as “MB”). It is possible to restrict the reference frame used for the inter picture encoding to a certain region in a picture, and it is called as a reference region. In the present application, by processing in units of MB, the data of an MB which the I/P encoder001has finished encoding can be used in an MB which the B encoder is going to encode, allowing the parallel encoding of two pictures.

The order of the encoding in the I/P encoder001is explained.

(1) An encoding target MB of the original picture (2n) is inputted into the I/P encoder001from the input picture memory102.

(2) A reference region X to be used for the P-picture encoding is inputted into the third intermediate buffer005from a reference frame X in the reference frame memory103.

(3) The I/P encoder001performs the P-picture encoding by the inter picture prediction from the original picture (2n) MB and the reference region X in the third intermediate buffer005, and outputs a first encoded bit string to the output control device104. Subsequently, a picture which is decoded from the encoded bit string (hereinafter called a local decoded picture) is written and stored in the first intermediate buffer006.

(4) The local decoded picture stored in the first intermediate buffer006is written in the reference frame Y of the reference frame memory103, in order to be used as a reference frame by the subsequent encoding.

After the above processing (1)-(4) is performed for all the MBs in the original picture (2n), the encoding of one picture is completed. When performing the I-picture encoding, the I-picture encoding is performed by the intra picture prediction in (3). At this time, the reference region X in the third intermediate buffer005is not used. However, even when the I-picture encoding is performed, the reference region X is stored into the third intermediate buffer005in (2).

The order of the encoding in the B encoder002is explained.

(5) An encoding target MB of the original picture (2n−1) is inputted into the B encoder002from the input picture memory102.

(6) The B encoder002performs the B-picture encoding by the inter picture prediction, from the original picture (2n−1) MB, the reference region X in the third intermediate buffer005, and the reference region Y composed of the local decoded picture in the first intermediate buffer006, and outputs the second encoded bit string to the output control device104.

After the above processing (5)-(6) is performed for all the MBs in the original picture (2n−1), the encoding of one picture is completed.

The processing of (1)-(4) by the I/P encoder001and the processing of (5)-(6) by the B encoder002are performed in parallel (two-picture parallel encoding).

The processing order of the encoding of the I picture, the P picture, and the B picture is illustrated inFIG. 8-FIG. 10typically.FIG. 8illustrates the processing order of the encoding in a comparative example.FIG. 9andFIG. 10illustrate two kinds of processing order which can be employed in Embodiment 1. Period N (N=0, 1, . . . , 2n+1, . . . ) is a processing time of one picture. A smaller N expresses the anterior (past) time, and a larger N expresses the posterior (future) time. The numeric character attached to the characters I, P, and B expresses the order of the pictures being captured.

As illustrated inFIG. 8, in the comparative example, only the encoding of one kind of picture type is performed in one period. Encoding is performed for I0at Period0, a B picture captured before I0at Period1, P2at Period2, B1at Period3, P4at Period4, and B3at Period5. Subsequently, encoding is performed for P (2n−2) at Period (2n−2), B (2n−3) at Period (2n−1), P (2n) at Period (2n), and B (2n−1) at Period (2n+1). The arrow in the figure expresses the reference relation of the reference picture. That is, when encoding the picture at the tip of an arrow, the reference picture (local decoded picture) created by the encoding of the picture at the origin of the arrow is referred to. When performing the encoding of P4at Period4, the reference picture (local decoded picture) created by the encoding of P2performed at Period2is referred to. When performing the encoding of B3at Period5, the reference picture (local decoded picture) created by the encoding of P2performed at Period2is referred to, and in addition the reference picture (local decoded picture) created by the encoding of P4performed at Period4is referred to. In this way, the original picture is read from the input picture memory102not in the order of capturing but in the order of encoding, and is supplied. Although B3is a picture between P2and P4in the order of capturing, it is the B picture and uses not only the past reference picture of P2but the reference picture of P4of the future; therefore B3is inputted after P4. Hereafter, in the encoding of P (2n) performed at Period (2n), the reference picture (local decoded picture) created by the encoding of P (2n−2) performed at Period (2n−2) is referred to. When performing the encoding of B (2n−1) at Period (2n+1), the reference picture (local decoded picture) created by the encoding of P (2n−2) performed at Period (2n−2) is referred to, and in addition, the reference picture (local decoded picture) created by the encoding of P(2n) performed at Period (2n) is referred to. Here, the reference picture (local decoded picture) created by the encoding of P2performed at Period2is written in the reference frame memory at Period2. After that, the reference picture concerned is read for the encoding of P4at Period4, and is read again for the encoding of B3at Period5. Generally, the reference picture (local decoded picture) created by the encoding of P (2n−2) performed at Period (2n−2) is written in the reference frame memory at Period (2n−2). After that, the reference picture concerned is read for the encoding of P(2n) at Period (2n), and is read again for the encoding of B(2n−1) at Period (2n+1). In this way, the reference frame memory is accessed once at write and at least twice at read, per one piece of reference picture.

The encoding device A (100) illustrated in Embodiment 1 can perform the parallel operation of the I/P encoder001and the B encoder002simultaneously. Accordingly, it is possible to perform the encoding of an I picture or a P picture and the encoding of a B picture in parallel.

FIG. 9is an explanatory drawing illustrating an example of the processing order of the encoding in Embodiment 1. At Period0, the I/P encoder001performs the encoding of I0, and the B encoder002performs the encoding of the B picture captured before I0. Subsequently, the encoding is performed in parallel for P2and B1at Period2, P4and B3at Period4, P(2n−2) and B (2n−3) at Period (2n−2), and P (2n) and B (2n−1) at Period (2n), respectively. Here, the reference picture (local decoded picture) created by the encoding of P2performed at Period2is written in the reference frame memory at Period2. After that, the reference picture concerned is read for the encoding of P4at Period4, and is used for the encoding of B3at the same Period4. Generally, the reference picture (local decoded picture) created by the encoding of P (2n−2) performed at Period (2n−2) is written in the reference frame memory at Period (2n−2). After that, the reference picture concerned is read for the encoding of P (2n) at Period (2n), and is used for the encoding of B(2n−1) at the same Period (2n). Accordingly, it does not need to be read again. In this way, the frequency of access to the reference frame memory is suppressed to once at write and once at read, per one piece of reference picture. In the encoding of B3at Period4, the reference picture created by the encoding of P4performed at the same Period4is used. Therefore, the reference picture needed in the encoding of B3is already created in the encoding of P4, and is written in the first intermediate buffer006. After that, the encoding of B3can be started. Such control is performed by the position controller007as will be described later.

FIG. 10is an explanatory drawing illustrating another example of the processing order of the encoding in Embodiment 1. At Period0, the I/P encoder001performs the encoding of I0, and the B encoder002performs the encoding of the B picture captured before I0. Subsequently, the encoding is performed in parallel for P2and B1at Period1, P4and B3at Period2, P(2n−2) and B(2n−3) at Period (n−1), and P(2n) and B(2n−1) at Period (n), respectively. Here, the reference picture (local decoded picture) created by the encoding of P2performed at Period1is written in the reference frame memory at Period1. After that, the reference picture concerned is read for the encoding of P4at Period2, and is used for the encoding of B3at the same Period2. Generally, the reference picture (local decoded picture) created by the encoding of P(2n−2) performed at Period (n−1) is written in the reference frame memory at Period (n−1). After that, the reference picture concerned is read for the encoding of P(2n) at Period (n), and is used for the encoding of B(2n−1) at the same Period (n). Accordingly, it does not need to be read again. In this way, as is the case with the example described above inFIG. 9, the frequency of access to the reference frame memory is suppressed to once at write and once at read, per one piece of reference picture. In the example illustrated inFIG. 9, in order to perform encoding at the same interval as the frame interval of capturing, the encoding of two pictures is performed at two picture periods. Therefore, it is possible to lower the operating frequency and to suppress the power consumption. Or after performing the encoding of two pictures at one picture period, the clock is stopped till the end of two picture periods, thereby suppressing the power consumption. On the other hand, in the example illustrated inFIG. 10, the encoding of two pictures is performed at one picture period. In this case, it becomes possible to improve the processing capability twice.

Next, the operation of the position controller007is explained. As described above, in the encoding of B3at Period4illustrated inFIG. 9, the reference picture created by the encoding of P4performed at the same Period4is used. Accordingly, it must be controlled such that the reference picture needed in the encoding of B3is already created in the encoding of P4and is written in the first intermediate buffer006and then the encoding of B3is started.

FIG. 11andFIG. 12are explanatory drawings illustrating the example of control by the position controller007;FIG. 11illustrates the state of waiting instruction, andFIG. 12illustrates the state of processing permission. In the present explanation, for simplicity, a picture horizontally composed of six MBs is used, and a reference region is a 2×2 4-MB region with a starting point of the same position as the encoding target MB. For example, the number of MBs which compose one picture, such as one frame, and the size of the reference region are arbitrary, and are prescribed by the standard of picture encoding or prescribed as specifications of an image system. The reference region X of the I/P encoder001, the reference region X of the B encoder002, and the reference region Y do not need to be the same size, respectively.

FIG. 11illustrates the I/P encoder001, the B encoder002, the position controller007, the first intermediate buffer005, and the third intermediate buffer006. An original picture (2n) inputted into the I/P encoder001is illustrated for every MB, and an original picture (2n−1) inputted into the B encoder002is illustrated for every MB. A hatched MB expresses an already inputted MB and a thick-framed MB expresses an encoding target MB at present.FIG. 11illustrates an example in which the B encoder002is about to start the processing of MB0and the I/P encoder001is currently performing the processing of MB7. MB0-MB6are already inputted into the I/P encoder001, the encoding is already finished, and the local-decoded MB0-MB6are stored in the first intermediate buffer006. The necessary reference picture of the reference frame is read from the reference frame memory103and stored in the third intermediate buffer005. As illustrated inFIG. 11, MB7of the original picture (2n) is inputted into the I/P encoder001, encoding is performed with reference to 4 blocks of MB7, MB8, MB13, and MB14as the reference regions X, from the third intermediate buffer005, and the local-decoded MB7is written to the first intermediate buffer006. The I/P encoder001notifies the position controller007of the information expressing that “MB7is in process.” On the other hand, the B encoder002sends a “processing request for MB0” to the position controller007. The position controller007manages whether the data of the reference region Y required by the B encoder002is ready, and notifies the B encoder002of “waiting instruction” or “processing permission.”FIG. 11illustrates the state where the “waiting instruction” is notified because the I/P encoder001has not completed the processing of MB7and the reference region Y (MB0/MB1/MB6/MB7) necessary for the processing of MB0by the B encoder002is not ready. Upon receiving the “waiting instruction”, the B encoder002goes into a waiting state, and does not start the encoding.

FIG. 12illustrates an example in which the B encoder002is starting the processing of MB0while the I/P encoder001has finished the processing of MB7and is performing the processing of MB8. In this case, the I/P encoder001has completed the processing of MB7and the reference region Y (MB0/MB1/MB6/MB7) necessary for the processing of MB0by the B encoder002is ready in the first intermediate buffer006; accordingly, the position controller007notifies “processing permission” to the B encoder002. Upon receiving the “processing permission”, the B encoder002starts the encoding using MB0of the original picture (2n−1), (MB0/MB1/MB6/MB7) of the reference region X, and (MB0/MB1/MB6/MB7) of the reference region Y.

As explained inFIG. 9andFIG. 10, the I/P encoding and the B encoding are macroscopically performed in parallel at the same picture period. However, as already explained with reference toFIG. 11andFIG. 12, microscopically, the control (processing-position control) is performed to start the encoding after the completion of the encoding of MB of the necessary reference region, in units of MB as the processing unit of the encoding. The reference region is determined depending on the range and size of a motion vector which are allowed in the motion compensation, for example, and is less than one frame at the maximum. When the reference region is the greatest one frame, after the I/P encoding of the referenced side is completed for one frame, the “processing permission” is notified to the B encoding on the referring side. Accordingly, for example, the processing of B3inFIG. 9is postponed until Period5. Even in this case, the B encoding on the referring side is uniformly delayed. Therefore, it is possible to perform in parallel the encoding of two pictures in one picture period, as illustrated inFIG. 10. On the other hand, it is preferable that the B encoding on the referring side is made start before the contents of the intermediate buffer, especially the first intermediate buffer006, are overwritten by the reference frame (local decoded picture) by the encoding of the next picture (frame). According to this configuration, it is possible to suppress the storage capacity of the intermediate buffer to a value equal to one picture (one frame). On the other hand, when the reference region is restricted, the storage capacity of the intermediate buffer can be made still smaller.

The picture encoding device according to Embodiment 1 encodes the first picture by the I/P encoder001, stores the local decoded picture created by the encoding in the first intermediate buffer006, the B encoder002encodes the second picture in parallel with reference to the local decoded picture, and the position controller007performs the control between both encoders. As described above, the present configuration is effective in reducing the frequency of access to the reference frame memory103. In the present configuration, the B encoder002which encodes the B picture uses the data already stored in the third intermediate buffer005and the first intermediate buffer006, as the reference region. Accordingly, the access to the reference frame memory103does not take place at the time of the B-picture encoding. Accordingly, it is only at the time of the P-picture encoding that data is read out from the reference frame memory103.

In the picture encoding device disclosed by Patent Document 1, it is necessary to read data equal to one picture at the time of the P-picture encoding, and in addition to it, it is necessary to read data equal to two pictures at the time of the B-picture encoding. On the contrary, in the present configuration, it is necessary to read only data equal to one picture at the time of the P-picture encoding; accordingly, the present configuration is effective in reducing the memory zone to one third. Compared with the general past art which Patent Document 1 attempted to solve, the present configuration is effective in reducing the memory zone even to one fifth.

As other effects, it is possible to reduce the capacity of the reference frame memory103. In the past, the capacity for three pictures were required, however, the present configuration requires the capacity equal to one picture for read and one picture for write, totaling two pictures.

The capacity of the third intermediate buffer005added to the encoding device A (100) just stores the data of the region where the I/P encoder001refers to plus the data of the region where the B encoder002refers to; therefore, several-MB line at maximum and three-MB line at minimum is sufficient. The capacity of the first intermediate buffer006just stores the data of the region where the I/P encoder001writes the decoded MB plus the data of the region where the B encoder002refers to; therefore, several-MB line at maximum and two-MB line at minimum is sufficient.

Note that it is possible to apply the present configuration to almost all the moving picture coding standards.

An I Encoder+ a P Encoder+ a B Encoder+ an Intermediate Buffer

It has been shown in Embodiment 1 that the access zone of the reference frame memory can be reduced by providing two encoders and two intermediate buffers. However, the access zone cannot be reduced to zero but the reference frame memory is as necessary as ever. For example, when the reference frame memory is assumed to be installed in an external shared memory, the capacity and non-zero access zone of the reference frame memory will increase system cost.

Accordingly, Embodiment 2 illustrates the configuration which does not need the reference frame memory.

FIG. 13is a block diagram illustrating an example of the entire configuration of a picture encoding device according to Embodiment 2.

The entire configuration of the picture encoding device according to Embodiment 2 includes an input-picture supply device101, an input picture memory102, an encoding device B (200), and an output control device104. Unlike the picture encoding device according to Embodiment 1, the encoding device B (200) is not provided with the reference frame memory103. The encoding device B (200) encodes the original picture inputted from the input picture memory102in conformity with a certain picture coding system, without using the reference frame memory. The encoding device B (200) outputs three or more encoded bit strings.

FIG. 14is a block diagram illustrating a configuration example of the picture encoding device according to Embodiment 2. The input-picture supply device101, the input picture memory102, and the output control device104are also illustrated in the figure. It is typically shown in the figure that the input picture memory102stores original pictures0,1, . . . ,3n,3n+1,3n+2, . . . .

The encoding device B (200) is comprised of an I encoder020, a P encoder021, a B encoder022, an input controller023, a switch024, a first intermediate buffer025, a second intermediate buffer026, a first position controller027, and a second position controller028. The I encoder020encodes an I picture, creates a first encoded bit string, outputs it to the output control device104, and outputs a local decoded picture to the first intermediate buffer025as a reference picture. The P encoder021performs the P-picture encoding with reference to the local decoded picture stored in the first intermediate buffer025, creates a second encoded bit string, outputs it to the output control device104, and outputs a local decoded picture to the second intermediate buffer026as a reference picture. The B encoder performs the B-picture encoding with reference to the local decoded picture stored in the first intermediate buffer025and the local decoded picture stored in the second intermediate buffer026, creates a third encoded bit string, and outputs it to the output control device104.

The I encoder020, the P encoder021, and the B encoder022do not need to bean encoder only for the respective picture type, and they may be an I/P/B encoder. The number of B encoders may not be one. By mounting N-piece B encoders, it is possible to set the number of B pictures between I pictures and P pictures to N sheets.

The operation of the picture encoding device according to Embodiment 2 is explained. The following explains the operation of the picture encoding device illustrated inFIG. 14when the encoding is performed so that three picture types, an I picture, a B picture, and a P picture, are repeated every three pictures like IBPIBPIBP . . . . The input picture memory102is supplied with an original picture as the encoding target on a time-series basis from the input-picture supply device101; such as an original picture0, an original picture1, . . . , an original picture (3n), an original picture (3n+1), an original picture (3n+2), . . . . When encoding like IBPIBPIBP . . . , the original picture0is encoded as the I picture, the original picture1as the B picture, the original picture2as the P picture, and henceforth the original picture (3n) is encoded as the I picture, the original picture (3n+1) as the B picture, and the original picture (3n+2) as the P picture (n is an integer equal to or greater than 0).

The encoding device B (200) encodes every three pictures in parallel.

The order of the encoding in the I encoder020is explained.

(1) An encoding target MB of the original picture (3n) is inputted into the I encoder020from the input picture memory102.

(2) The I encoder020performs the I-picture encoding for the target MB of the original picture (3n) by the intra picture prediction, and outputs the first encoded bit string to the output control device104. Subsequently, the local decoded picture is written and stored in the first intermediate buffer025.

After the above processing (1)-(2) is performed for all the MBs in the original picture (3n), the encoding of one picture is completed.

The order of the encoding in the P encoder021is explained.

(3) An encoding target MB of the original picture (3n+2) is inputted into the P encoder021from the input picture memory102.

(4) The P encoder021performs the P-picture encoding by the inter picture prediction from the reference region X composed of the target MB of the original picture (3n+2) and the local decoded picture by the I encoder020in the first intermediate buffer025, and outputs the second encoded bit string to the output control device104. Subsequently, the local decoded picture is written and stored in the second intermediate buffer026.

After the above processing (3)-(4) is performed for all the MBs in the original picture (3n+2), the encoding of one picture is completed.

The order of the encoding in the B encoder022is explained.

(5) An encoding target MB of the original picture (3n+1) is inputted into the B encoder022from the input picture memory102.

(6) The B encoder022performs the B-picture encoding by the inter picture prediction from the target MB of the original picture (3n+1), the reference region X composed of the local decoded picture by the I encoder020in the first intermediate buffer025, and the reference region Y composed of the local decoded picture by the P encoder021in the second intermediate buffer026, and outputs the third encoded bit string to the output control device104.

After the above processing (5)-(6) is performed for all the MBs in the original picture (3n+1), the encoding of one picture is completed.

The processing of (1)-(2) by the I encoder020, the processing of (3)-(4) by the P encoder021, and the processing of (5)-(6) by the B encoder022are performed in parallel (three-picture parallel encoding).

The P encoder021refers to the local decoded picture by the I-picture encoding of the past original picture (3n) for the P-picture encoding of the original picture (3n+2). However, the local decoded picture is stored in the first intermediate buffer025. Therefore, it is not necessary to access the reference frame memory. The B encoder022refers to the local decoded picture by the I-picture encoding of the past original picture (3n) and the local decoded picture by the P-picture encoding of the future original picture (3n+2), for the B-picture encoding of the original picture (3n+1). However, those reference pictures are stored in the first intermediate buffer025and the second intermediate buffer026, respectively. Therefore, it is not necessary to access the reference frame memory. Accordingly, the picture encoding device according to Embodiment 2 does not need to be provided with the reference frame memory.

Next, the operation of the first position controller027and the second position controller028is explained.FIG. 15is an explanatory drawing illustrating an example of control by the position controllers027and028according to Embodiment 2. In the present explanation, for simplicity, as is the case withFIG. 11andFIG. 12, a picture horizontally composed of six MBs is used, and a reference region is a 2×2 4-MB region with a starting point of the same position as the encoding target MB.

InFIG. 15, the I encoder020, the P encoder021, the B encoder022, the first position controller027, the second position controller028, the first intermediate buffer025, and the second intermediate buffer026are illustrated. The original picture (3n) inputted into the I encoder020, the original picture (3n+2) inputted into the P encoder021, and the original picture (3n+1) inputted into the B encoder022are illustrated for every MB, respectively. A hatched MB expresses an already inputted MB and a thick-framed MB expresses an encoding target MB at present.

MB0-MB15of the original picture (3n) are already inputted into the I encoder020, the encoding is already finished, and the local-decoded MB0-MB15are stored in the first intermediate buffer025. At present, the I encoder020is performing the encoding for MB16as the target, and the information expressing “MB16is in process” is notified to the first position controller027and the second position controller028.

MB0-MB7of the original picture (3n+2) are already inputted into the P encoder021, the encoding is already finished, and the local-decoded MB0-MB7are stored in the second intermediate buffer026. The P encoder021has sent “processing request for MB8” to the first position controller027, in order to perform the encoding for MB8at present. The first position controller027manages whether the data of the reference region X required by the P encoder021is ready, and notifies the P encoder021of “waiting instruction” or “processing permission.” In the example illustrated inFIG. 15, The first position controller027makes the P encoder021start the encoding of MB8, by notifying the “processing permission” to the P encoder021, after the reference region X (MB8/MB9/MB14/MB15) which the P encoder021refers to for the encoding of MB8is written in the first intermediate buffer025from the I encoder020. Upon receiving the “processing permission”, the P encoder021starts the encoding using MB8of the original picture (3n+2) and (MB8/MB9/MB14/MB15) of the reference region X. When the encoding is started, the information expressing “MB8is in process” is notified to the second position controller028.

MB0of the original picture (3n+1) is inputted into the B encoder022. In order to perform the encoding for it, the “processing request for MB0” is notified to the second position controller028. The I encoder020writes the local-decoded MB16to the first intermediate buffer025, and notifies the second position controller028of the information expressing “MB16is in process.” The P encoder021writes the local-decoded MB8to the second intermediate buffer026, and notifies the second position controller028of “MB8is in process.” The second position controller028manages whether the data of the reference region X and the reference region Y which is required by the B encoder022is ready, and notifies the B encoder022of “waiting instruction” or “processing permission.” In the example illustrated inFIG. 15, the I encoder020and the P encoder021both have completed processing of MB8. Accordingly, the reference region X (MB0/MB1/MB6/MB7) and the reference region Y (MB0/MB1/MB6/MB7), which are required for the encoding of MB0by the B encoder022, are all ready, and “processing permission” is notified. When putting in other words, the second position controller028notifies “processing permission” to the B encoder022, after the reference region X (MB0/MB1/MB6/MB7) has been written in the first intermediate buffer025from the I encoder020, and the reference region Y (MB0/MB1/MB6/MB7) has been written in the second intermediate buffer026from the P encoder021. Upon receiving the “processing permission”, the B encoder022starts the encoding using MB0of the original picture (3n+1), (MB0/MB1/MB6/MB7) of the reference region X, and (MB0/MB1/MB6/MB7) of the reference region Y. When the processing position of the P encoder021alone can guarantee that the reference region in the first intermediate buffer025composed of the local-decoded MB of the I encoder020is ready, it is possible to omit the notice of the processing position to the second position controller028from the I encoder020.

While Embodiment 1 adopts one-stage combination in which one intermediate buffer is provided between two encoders, Embodiment 2 adopts two-stage combination in which two intermediate buffers are provided among three encoders. Accordingly, it is possible to delete the reference frame memory103which has been necessary in Embodiment 1.

Embodiment 1 adopts the one-stage combination in which one intermediate buffer is provided between two encoders such as the I/P encoder001, the B encoder002, the position controller007, and the first intermediate buffer006. By storing temporarily the local decoded picture at the first stage in the intermediate buffer, and referring to it by the encoding by the second stage encoder, the frequency of access to the reference frame memory can be reduced. However, when the first stage I/P encoder001performs the encoding of the P picture, it is necessary to access the reference frame memory. As compared with this, Embodiment 2 adopts the two-stage combination in which two intermediate buffers are provided among three encoders, such as the I encoder020, the first intermediate buffer025, the first position controller027, the P encoder021, the second intermediate buffer026, the second position controller028, and the B encoder022. The first picture is encoded by the I encoder020, a local decoded picture created by the encoding is stored in the first intermediate buffer025, and the second picture is encoded in parallel by the P encoder021referring to the local decoded picture in the first intermediate buffer025. A local decoded picture created when the P encoder021encodes the second picture is stored in the second intermediate buffer026, and the B encoder022encodes the third picture in parallel with reference to the local decoded picture of the first intermediate buffer025and the local decoded picture of the second intermediate buffer026. The encoding of the first picture by the I encoder020, the encoding of the second picture by the P encoder021, and the encoding of the third picture by the B encoder022are performed in parallel macroscopically. Control to enable each of the encoders to start respective encoding is performed by the first position controller027and the second position controller028. In the present configuration, the P encoder021which encodes the P picture uses the data already stored in the first intermediate buffer025as the reference region, and the B encoder022which encodes the B picture uses the data already stored in the first intermediate buffer025and the second intermediate buffer026as the reference region. Accordingly, no access to the reference frame memory103takes place at all. Therefore, in the present configuration, it is possible to reduce the memory zone and memory capacity of the reference frame memory to zero; accordingly it is possible to reduce the cost of the system greatly.

An I Encoder+ a P Encoder+ an Intermediate Buffer

Embodiment 2 illustrates the configuration which is composed of one I encoder, one P encoder, and one or more B encoders, but not having a reference frame memory, thereby reducing the system cost. However, in Embodiment 2, it is necessary to provide three or more encoders, and hardware cost increases compared with Embodiment 1. Generally, the encoding of pictures in which a B picture exists between I pictures or P pictures yields an improved efficiency of encoding. On the other hand, it is necessary to rearrange an input picture according to the order of encoding. Accordingly, there exists the demerit that encoding delay becomes large. This fact produces forcefully the demerit that the delay from an input to a display becomes large also in the decoding. Therefore, there are also needs of the coding which does not include a B picture.

Embodiment 3 illustrates the configuration which enables reduction of hardware cost and reduction of encoding delay by removing a B encoder from the configuration of Embodiment 2.

FIG. 16is a block diagram illustrating an example of the entire configuration of a picture encoding device according to Embodiment 3.

The entire configuration of the picture encoding device according to Embodiment 3 includes an input-picture supply device101, an input picture memory102, an encoding device C (300), and an output control device104. Unlike the picture encoding device according to Embodiment 1, the encoding device C (300) is not provided with the reference frame memory103. The encoding device C (300) encodes the original picture inputted from the input picture memory102in conformity with a certain picture coding system, without using the reference frame memory. Unlike the picture encoding device according to Embodiment 2, the encoding device C (300) outputs two encoded bit strings.

FIG. 17is a block diagram illustrating a configuration example of the picture encoding device according to Embodiment 3. The input-picture supply device101, the input picture memory102, and the output control device104are also illustrated in the figure. It is typically shown in the figure that the input picture memory102stores original pictures0,1, . . . ,2n−1,2n,2n+1, . . . .

The encoding device C (300) is comprised of an I encoder030, a P encoder031, an input controller033, a switch034, a first intermediate buffer035, and a first position controller032. The I encoder030encodes an I picture, creates a first encoded bit string, outputs it to the output control device104, and outputs a local decoded picture to the first intermediate buffer035as a reference picture. The P encoder031performs the P-picture encoding with reference to the local decoded picture stored in the first intermediate buffer035, creates a second encoded bit string, and outputs it to the output control device104. The I encoder030and the P encoder031do not need to be an encoder only for the respective picture type, and both may be an I/P/B encoder.

The operation of the picture encoding device according to Embodiment 3 is explained. The following explains the operation of the picture encoding device illustrated inFIG. 16when the encoding is performed so that two picture types, an I picture and a P picture, are repeated every two pictures like IPIPIP . . . . The input picture memory102is supplied with an original picture as the encoding target on a time-series basis from the input-picture supply device101; such as an original picture0, an original picture1, . . . , an original picture (2n−1), an original picture (2n), an original picture (2n+1). When encoding like IPIPIP . . . , the original picture0is encoded as an I picture, the original picture1as a P picture, and henceforth the original picture (2n−1) is encoded as an I picture, the original picture (2n) as a P picture, and the original picture (2n+1) as an I picture (n is an integer equal to or greater than 0).

The encoding device C (300) encodes every two pictures in parallel.

The order of the encoding in the I encoder030is explained.

(1) An encoding target MB of the original picture (2n−1) is inputted into the I encoder030from the input picture memory102.

(2) The I encoder030performs the I-picture encoding for the target MB of the original picture (2n−1) by the intra picture prediction, and outputs the first encoded bit string to the output control device104. Subsequently, the local decoded picture is written and stored in the first intermediate buffer035.

After the above processing (1)-(2) is performed for all the MBs in the original picture (2n−1), the encoding of one picture is completed.

The order of the encoding in the P encoder031is explained.

(3) An encoding target MB of the original picture (2n) is inputted into the P encoder031from the input picture memory102.

(4) The P encoder031performs the P-picture encoding by the inter picture prediction from the reference region X composed of the target MB of the original picture (2n) and the local decoded picture by the I encoder030in the first intermediate buffer035, and outputs the second encoded bit string to the output control device104.

After the above processing (3)-(4) is performed for all the MBs in the original picture (2n), the encoding of one picture is completed.

The processing of (1)-(2) by the I encoder030and the processing of (3)-(4) by the P encoder031are performed in parallel (two-picture parallel encoding).

The P encoder031refers to the local decoded picture by the I-picture encoding of the past original picture (2n−1) for the P-picture encoding of the original picture (2n). However, the local decoded picture is stored in the first intermediate buffer035. Therefore, it is not necessary to access the reference frame memory. Accordingly, the picture encoding device according to Embodiment 3 does not need to be provided with the reference frame memory.

Next, the operation of the first position controller032is explained.

FIG. 18is an explanatory drawing illustrating the example of control by the first position controller032according to Embodiment 3. In the present explanation, for simplicity, as is the cases withFIG. 11,FIG. 12, andFIG. 15, a picture horizontally composed of six MBs is used, and a reference region is a 2×2 4-MB region with a starting point of the same position as the encoding target MB.

FIG. 18illustrates the I encoder030, the P encoder031, the first position controller032, and the first intermediate buffer035. The original picture (2n−1) inputted into the I encoder030and the original picture (2n) inputted into the P encoder031are illustrated for every MB, respectively. A hatched MB expresses an already inputted MB and a thick-framed MB expresses an encoding target MB at present.

MB0-MB7of the original picture (2n−1) are already inputted into the I encoder030, the encoding is already finished and the local-decoded MB0-MB7are stored in the first intermediate buffer035. At present, the I encoder030is performing the encoding for MB8as the target, and the information expressing “MB8is in process” is notified to the first position controller032.

The P encoder031has sent “processing request for MB0” to the first position controller027, in order to perform the encoding for MB8of the original picture (2n) as a target. The first position controller032manages whether the data of the reference region X required by the P encoder031is ready, and notifies the P encoder031of “waiting instruction” or “processing permission.” In the example illustrated inFIG. 18, the first position controller032makes the P encoder031start the encoding of MB0, by notifying the “processing permission” to the P encoder031, after the reference region X (MB0/MB1/MB6/MB7) which the P encoder031refers to for the encoding of MB0is written in the first intermediate buffer035from the I encoder030. Upon receiving the “processing permission”, the P encoder031starts the encoding using MB0of the original picture (2n) and (MB0/MB1/MB6/MB7) of the reference region X.

In Embodiment 3, the B encoder022, the second intermediate buffer026, and the second position controller028according to Embodiment 2 are deleted. Consequently, the present configuration is effective in reducing the hardware cost and reducing the encoding delay. It is also effective in reducing the decoding delay in the decoding. These effects lead to the reduction of the system cost.

Field Division

In Embodiments 1-3, in order to supply two or more pieces of the original pictures simultaneously (in parallel), the input picture memory102is necessary. Even if the reference frame memory103is deleted as in Embodiment 3, the input picture memory102cannot be deleted. Accordingly, it is difficult to eliminate the access to an external shared memory, posing an issue of increase of the system cost. In addition, it is necessary to store the original picture once in the input picture memory102from the input-picture supply device101; accordingly there arises an issue of increased delay in the processing from input to output.

Embodiment 4 illustrates the configuration which enables deletion of the input picture memory102by performing a picture division in an encoding device.

FIG. 19is a block diagram illustrating an example of the entire configuration of a picture encoding device according to Embodiment 4.

The entire configuration of the picture encoding device according to Embodiment 4 includes an input-picture supply device101, an encoding device D (400), and an output control device104. Unlike the picture encoding devices according to Embodiments 1-3, the picture encoding device according to Embodiment 4 is not provided with the input picture memory102, but the original picture data as an encoding target is directly inputted into the encoding device D (400) from the input-picture supply device101.

FIG. 20is a block diagram illustrating a configuration example of the picture encoding device according to Embodiment 4. The input-picture supply device101and the output control device104are also illustrated in the figure.

The encoding device D (400) is comprised of an I encoder040, a P encoder041, a position controller042, an input controller043, a field divider044, an intermediate buffer045, an input buffer046, a first MB buffer047, and a second MB buffer048. The I encoder040performs the I picture encoding, creates a first encoded bit string, outputs it to the output control device104, and outputs a local decoded picture to the intermediate buffer045as a reference picture. The P encoder041performs the P-picture encoding with reference to the local decoded picture stored in the intermediate buffer045, creates a second encoded bit string, and outputs it to the output control device104. The I encoder040and the P encoder041do not need to be an encoder only for the respective picture type, and it may be an I/P/B encoder.

The encoding device D (400) is different from the encoding device C (300) according to Embodiment 3 in a point that the original picture is inputted directly from the input-picture supply device101, a point that the field divider044which divides and supplies an original picture is provided instead of the switch034, and a point that the input buffer046which once stores the original picture and the first MB buffer047and the second MB buffer048which once store the output of the field divider044are provided. That is, in the encoding device D (400), the input buffer046once stores the original picture of a frame structure inputted from the input-picture supply device101, and the field divider044decomposes the original picture into a field structure and supplies it to one of the I encoder040and the P encoder041. The field divider044divides and reconstructs a picture for every MB which is the unit of encoding of each encoder, and supplies the picture data as the encoding target to the I encoder040via the first MB buffer047and to the P encoder041via the second MB buffer048, respectively.

In order to explain the operation of the encoding device D (400), the frame structure and the field structure are explained first.

FIG. 21is an explanatory drawing illustrating the frame structure of a picture.FIGS. 22 and 23are explanatory drawings illustrating the field structure of a picture of top field first and a picture of bottom field first, respectively.

In the frame structure illustrated inFIG. 21, all the pixel lines that compose one picture exist in the identical data structure. On the other hand, in the field structure illustrated inFIG. 22, one picture is composed of two data structures of the top field which includes only the even-numbered (0, 2, 4, . . . ) pixel lines of the frame structure, and the bottom field which includes only the odd-numbered (1, 3, 5, . . . ) pixel lines.FIG. 22illustrates the top-field-first field structure which is composed of a top field as the first field and a bottom field as the second field. As illustrated inFIG. 23, it is also possible to define the bottom-field-first field structure which is composed of a bottom field as the first field and a top field as the second field.

Many moving picture coding standards, such as H.264 define the coding method with respect to the field structure.

The operation of the picture encoding device according to Embodiment 4 is explained. The following explains the operation of the picture encoding device illustrated inFIG. 19when the encoding is performed so that two picture types, an I picture and a P picture, are repeated every two pictures like IPIPIP . . . .

FIG. 24is an explanatory drawing illustrating field division of a picture. The original picture of the frame structure inputted from the input-picture supply device101is divided into a field structure by the field divider044, and the first field of an I picture and the second field of a P picture are encoded alternately as illustrated inFIG. 24.

The field division operation of the encoding device D (400) is explained.

In the field divider044, the field division is performed as follows, for example. It is assumed that the input is performed line by line. The field division system depends on input form and is not necessarily restricted to the following.

(1) The pixel line inputted from the input-picture supply device101is stored in the input buffer046.

(2) After 32 lines are stored in the input buffer046, according to the parity of the line number of the pixel line, the pixel of the target MB position of an even-numbered line is stored in the first MB buffer047, and the pixel of the target MB position of an odd-numbered line is stored in the second MB buffer048.

(3) When 16 lines have accumulated in the first MB buffer047, it is possible to output the pixels as the MB of the top field. Similarly, when 16 lines have accumulated in the second MB buffer048, it is possible to output the pixels as the MB of the bottom field.

(4) When the last MB of 32 lines is outputted to the field divider047, the 32 lines can be deleted from the input buffer046.

Any of the top and the bottom can also be set as the first field, by changing the storage destination of the even-numbered line and the odd-numbered line.

It is considerable that without providing the input buffer046, the field divided data are stored directly in the first MB buffer047and the second MB buffer048. In the present case, the first MB buffer047and the second MB buffer048are provided with a capacity equivalent to the picture width by 16 lines at least, respectively.

The encoding operation of the encoding device D (400) is explained.

When the original picture of one frame is inputted, the encoding device D (400) divides the original picture into two fields and encodes the two fields in parallel.

In the I encoder040, the encoding is performed as follows.

(1) The encoding target MB of the first field is inputted to the I encoder040from the first MB buffer047.

(2) The I encoder040performs the I-picture encoding for the target MB of the first field by the intra picture prediction, and outputs a first encoded bit string to the output control device104. Subsequently, the I encoder040creates a local decoded picture and stores it in the intermediate buffer045.

After the above processing (1)-(2) is performed for all the MBs in the first field, the encoding of the first field is completed. In the P encoder041, the encoding is performed as follows.

(3) The encoding target MB of the second field is inputted to the P encoder041from the second MB buffer048.

(4) The P encoder041performs the P-picture encoding by the inter picture prediction from the reference region X composed of the target MB of the second field and the local decoded picture by the I encoder040in the intermediate buffer045, and outputs a second encoded bit string to the output control device104.

After the above processing (3)-(4) is performed for all the MBs in the second field, the encoding of the second field is completed.

The processing (1)-(2) by the I encoder040and the processing (3)-(4) by the P encoder041are performed for every two fields in parallel.

Next, the operation of the position controller042is explained.

FIG. 25is an explanatory drawing illustrating an example of control by the position controller042according to Embodiment 4. In the present explanation, for simplicity, as is the cases withFIG. 11,FIG. 12, andFIG. 15, a picture horizontally composed of six MBs is used, and a reference region is a 2×2 4-MB region with a starting point of the same position as the encoding target MB.

FIG. 25illustrates the I encoder040, the P encoder041, the position controller042, and the intermediate buffer045. The first field to be inputted into the I encoder040and the second field to be inputted into the P encoder041are shown together with MBs, respectively. A hatched MB expresses an already inputted MB and a thick-framed MB expresses an encoding target MB at present.

MB0-MB7of the first field are already inputted into the I encoder040, the encoding is already finished, and the local-decoded MB0-MB7are stored in the intermediate buffer045. At present, the I encoder040is performing the encoding for MB8as the target, and the information expressing “MB8is in process” is notified to the position controller042.

The P encoder041has sent “processing request for MB0” to the position controller042, in order to perform the encoding for MB8of the second field. The position controller042manages whether the data of the reference region X required by the P encoder041is ready, and notifies the P encoder041of “waiting instruction” or “processing permission.” In the example illustrated inFIG. 25, the position controller042makes the P encoder041start the encoding of MB0, by notifying the “processing permission” to the P encoder041, after the reference region X (MB0/MB1/MB6/MB7) which the P encoder041refers to for the encoding of MB0is written in the intermediate buffer045from the I encoder040. Upon receiving the “processing permission”, the P encoder041starts the encoding using MB0of the second field and (MB0/MB1/MB6/MB7) of the reference region X.

In Embodiment 4, the field divider044which divides the inputted original picture of one frame into two fields is provided, accordingly, it is possible to delete the input picture memory102and to eliminate the access to the external shared memory. Therefore, the present configuration is effective in reducing the cost of the system greatly. It is not necessary to once store the original picture in the input picture memory102from the input-picture supply device101. Accordingly, the present configuration is also effective in greatly reducing the delay in the processing from input to output, as another effect. The configuration is not restricted to the one which is not provided with the input picture memory102, however it may be provided with the input picture memory102as is the case withFIG. 16.

By controlling the processing position between the I encoder040and the P encoder041, the capacity of the intermediate buffer045added to the inside of the encoding device D (400) does not require the one picture's worth of buffer amount and can be realized only with the amount of several MB lines at most. The P encoder can performs the motion prediction and motion compensation from the first field as the target of the I encoding. On the other hand, the first field and the second field are created by the division of the identical frame; therefore, it is also possible to perform the encoding assuming that “motion” is not present. That is, it is possible to consider the implementation in which no motion prediction is performed and the motion vector is always set to 0 in the P encoding. In this case, the capacity of the intermediate buffer045can be theoretically realized with two MBs.

Field Division in Smaller Units

Embodiment 4 illustrates the configuration in which the picture of one frame is divided into two fields and the second field refers to the first field, thereby enabling the deletion of the input picture memory102. Embodiment 5 illustrates an example in which the field division is performed in smaller units (for example, two vertical MBs), the encoding by the interpolation processing is performed with one divided block referring to the other divided block, and multiplexing in units of divided blocks is performed to compose the encoded stream of one picture.

The entire configuration of the picture encoding device according to Embodiment 5 is the same as the configuration example illustrated inFIG. 19, and includes an input-picture supply device101, an encoding device Ea (500), and an output control device104. In Embodiment 5, it is not necessary to provide the input picture memory102as is the case with Embodiment 4, however, the configuration is not restricted to the one which is not provided with the input picture memory102, but may be the one which is provided with the input picture memory102as is the case withFIG. 16.

FIG. 26is the block diagram illustrating a configuration example of a picture encoding device according to Embodiment 5. The input-picture supply device101and the output control device104are also illustrated in the figure.

The encoding device Ea (500) is comprised of an intra MB encoder050, an inter MB encoder051, an input controller053, a field divider054, an intermediate buffer052, an input buffer055, a first MB buffer056, and a second MB buffer057. The input buffer055once stores the original picture of a frame structure inputted from the input-picture supply device101. By the control of the input controller053, the field divider054decomposes the original picture of the frame structure stored in the input buffer055into the field structure (a first field and a second field), and supplies them to either of the encoders (the intra MB encoder050and the inter MB encoder051), via the first MB buffer056and the second MB buffer057. The intra MB encoder050performs the intra picture encoding of the first field, creates a first encoded bit string, outputs it to the output control device104, and outputs a local decoded picture as a reference picture to the intermediate buffer052. The inter MB encoder051performs the inter picture encoding with reference to the local decoded picture stored in the intermediate buffer052, creates a second encoded bit string, and outputs it to the output control device104.

Here, a position controller is not necessary for the encoding device Ea (500). As described later, it is possible to delete the position controller by setting the unit of the field division to two MBs in the vertical positional relationship.

The operation of the encoding device Ea (500) is explained.

FIG. 27is an explanatory drawing illustrating operation in encoding of a field-divided picture. The field divider054, the intra MB encoder050, the inter MB encoder051, and the intermediate buffer052are illustrated in the figure. The MB pair of a frame structure to be inputted into the field divider054and the first and the second field MB which are field-divided are also illustrated. The MB pair of a frame structure to be inputted into the field divider054are two MBs which adjoin mutually in the vertical direction in a frame. The MB is a processing unit of the encoding by the intra MB encoder050and the inter MB encoder051. The MB is comprised of plural pixel lines. The field divider054divides the MB pair (two vertical MBs) of a frame structure into the first field MB and the second field MB, and inputs them to the intra MB encoder050and the inter MB encoder051, respectively. As an example, the first field MB is a field (top field) reconstructed only from the even-numbered (0, 2, 4, . . . ) lines of the MB pair of a frame structure, and the second field MB is a field (bottom field) reconstructed only from the odd-numbered (1, 3, 5, . . . ) lines of the MB pair of a frame structure. Their size is one MB, respectively. The intra MB encoder050encodes the first field MB and at the same time stores a local-decoded MB in the intermediate buffer052. The inter MB encoder051encodes the second field MB by the inter picture encoding technique with the intermediate buffer052as the reference MB. The output control device104multiplexes the first encoded bit string and the second encoded bit string in units of MB, to compose and output an encoded stream.

The field division performed by the field divider054may be the same as the field division performed by the field divider044according to Embodiment 4.

The processing unit may be the unit in which the two vertical MBs are divided into two fields as described above, or may be the unit smaller than it, for example, the unit of 16×8 pixels to which one MB is divided. In the latter case, the delay for two MB lines is unnecessary within the field divider054, and it is possible to realize lower delay and smaller buffer capacity.

In the encoding device Ea (500), the field division is performed in smaller units (for example, two vertical MBs), the encoding by the interpolation processing is performed with one divided block referring to the other divided block, and multiplexing in units of divided blocks is performed to compose the encoded stream of one picture.

FIG. 28is an explanatory drawing illustrating an example of the direction of field reference in encoding. The first field MB (top field), as one of the blocks divided from two vertical MBs, undergoes the intra field encoding, and the second field MB (bottom field), as the other of the blocks, undergoes the inter field encoding referring to the first field MB. The relation between the top field and the bottom field is arbitrary, and the first field MB may be the bottom field and the second field MB may be the top field. By using the present configuration example illustrated inFIG. 26, the coding method as illustrated inFIG. 28can be realized. That is, when a block structure like MBAFF (Macroblock Adaptive Field Frame) specified by H.264 is assumed, it is possible to perform the encoding by the interpolation processing which the second field refers to the first field.

The present configuration example is effective in realizing the intra picture encoding with better encoding efficiency than the existing intra picture encoding.

Compared with Embodiment 4, the intermediate buffer052may have smaller capacity and the position controller is not necessary; accordingly the present configuration is effective in reducing the cost.

When the field division is performed in smaller units, for example when one MB is divided in units of 16×8 pixels, the delay for two MB lines becomes unnecessary within the field divider054; therefore, the present configuration is effective in realizing more reduced delay and smaller buffer capacity.

A modified example of Embodiment 5 is explained. The entire configuration of the picture encoding device according to the modified example of Embodiment 5 is the same as the configuration example illustrated inFIG. 19, and includes an input-picture supply device101, an encoding device Eb (501), and an output control device104. In the modified example, the symbol of the encoding device is changed from Ea (500) to Eb (501).

FIG. 29is a block diagram illustrating another configuration example of the picture encoding device according to Embodiment 5.

The encoding device Eb (501) is comprised of a switch058which switches the storing destination of the output of the field divider054, and a switch059which switches the output destination of two encoders (050and051), in addition to the configuration of the encoding device Ea (500) illustrated inFIG. 26. The input controller053controls these switches. The other configuration is the same as that of the encoding device Ea (500) illustrated inFIG. 26, therefore, the explanation thereof is omitted.

The operation of the encoding device Eb (501) is explained.

FIG. 30is an explanatory drawing illustrating operation of the encoding device Eb (501) in the encoding of the picture after the field division. In the figure, the switch058and the switch059are added when compared withFIG. 27. The switch058can switch selectively between a position A and a position B. In the position A, the top field divided by the field divider054is fed as the first field MB and the bottom field is fed as the second field MB. Conversely, in the position B, the bottom field is fed as the first field MB and the top field is fed as the second field MB. The switch059can switch selectively between a position C and a position D. In the position C, the output of the intra MB encoder050is fed as the first encoded bit string and the output of the inter MB encoder051is fed as the second encoded bit string. Conversely, in the position D, the output of the intra MB encoder050is fed as the second encoded bit string and the output of the inter MB encoder051is fed as the first encoded bit string.FIG. 30illustrates the state where the diagonal path (expressed by thick lines) of the switch058is chosen. In the present state, the bottom-field of the output of the field divider054is fed as the first field MB, and the top-field is fed as the second field MB. The bottom-field undergoes the intra field encoding and the top-field undergoes the inter field encoding, respectively. After that, the order is returned by the diagonal path (expressed by thick lines) of the switch059.

FIG. 31is an explanatory drawing illustrating an example of the direction of field reference in encoding. Through the above procedure, it is possible to perform the reference from the bottom field MB to the top field MB as illustrated inFIG. 31.

FIG. 32is an explanatory drawing illustrating another example of the direction of field reference in encoding. By performing the switching of the switch058and the switch059adaptively, it is possible to change the reference direction adaptively, as illustrated inFIG. 32.

The output control device104multiplexes the first encoded bit string and the second encoded bit string in units of MB, to create and output an encoded stream. It is preferable to include the function of the switch059in the output control device104.

The field division performed by the field divider054may be the same as the field division performed by the field divider044according to Embodiment 4.

The processing unit may be the unit in which the two vertical MBs are divided into two fields as described above, or may be the unit smaller than it, for example, the unit of 16×8 pixels to which one MB is divided. In the latter case, the delay for two MB lines is unnecessary within the field divider054, and it is possible to realize lower delay and smaller buffer capacity.

As is the case with the encoding device Ea (500), the present configuration example is effective in realizing the intra picture encoding with better encoding efficiency than the existing intra picture encoding. Compared with Embodiment 4, the intermediate buffer052may have smaller capacity and the position controller is not necessary; accordingly the present configuration is effective in reducing the cost. When the field division is performed in smaller units, for example when one MB is divided in units of 16×8 pixels, the delay for two MB lines becomes unnecessary within the field divider054; the present configuration is effective in realizing more reduced delay and smaller buffer capacity.

Furthermore, it is possible to change arbitrarily the reference direction between field blocks. That is, it is possible to apply the present configuration example to MBAFF specified by H.264, for example.

Division in the Vertical Direction

Embodiments 4 and 5 illustrate the configuration in which the picture of one frame is divided into two fields, each composed of the odd-numbered pixel lines or the even-numbered pixel lines, and the encoding of one field refers to the other field, thereby enabling the deletion of the input picture memory102. However, the division may not necessarily be in the units of fields or every pixel line. Embodiment 6 illustrates the example in which a frame is divided in the vertical direction (column direction) orthogonal to the pixel line.

The entire configuration of the picture encoding device according to Embodiment 6 is the same as the configuration example illustrated inFIG. 19, and includes an input-picture supply device101, and an encoding device F (600), and an output control device104. In Embodiment 6, in order to distinguish from the encoding device of other embodiments, the symbol of the present encoding device is changed to F (600).

FIG. 33is a block diagram illustrating the configuration example of the encoding device F (600) according to Embodiment 6. The input-picture supply device101and the output control device104are also illustrated in the figure.

The encoding device F (600) is comprised of an I encoder060, a P encoder061, a position controller062, an input controller063, a picture vertical divider064, an intermediate buffer065, an input buffer066, a first MB buffer067-1, and a second MB buffer067-2. The I encoder060performs the I picture encoding, creates a first encoded bit string, outputs it to the output control device104, and outputs a local decoded picture to the intermediate buffer065as a reference picture. The P encoder061performs the P-picture encoding with reference to the local decoded picture stored in the intermediate buffer065, creates a second encoded bit string, and outputs it to the output control device104. The I encoder060and the P encoder061do not need to be an encoder only for the respective picture type, and both may be an I/P/B encoder.

The encoding device F (600) is different from the encoding device D (400) according to Embodiment 4 in the point that the encoding device F (600) includes, instead of the field divider044, a picture vertical divider064which divides an original picture in the vertical direction and supplies it.

The division in the vertical direction (column direction) of a picture is explained.

FIG. 34andFIG. 35are explanatory drawings of the vertical division of a picture;FIG. 34illustrates the frame structure andFIG. 35illustrates a vertically divided right and left column picture. In contrast to the picture of the frame structure illustrated inFIG. 34, the vertical division structure illustrated inFIG. 35can be divided into the left column picture (the first picture) which collects the even-numbered (0, 2, 4, . . . ) columns for every vertical column, and the right column picture (the second picture) which collects the odd-numbered (1, 3, 5, . . . ) columns.

The operation of the picture encoding device F according to Embodiment 6 is explained. The following explains the operation of the picture encoding device F illustrated inFIG. 33when the encoding is performed so that two picture types, an I picture and a P picture, are repeated every two vertically divided pictures like IPIPIP . . . .

FIG. 36is an explanatory drawing illustrating vertical division of a picture. The original picture of a frame structure inputted from the input-picture supply device101is divided into the vertical division structure by the picture vertical divider064, and the divided original pictures are encoded alternately as the first field of an I picture and the second field of a P picture, as illustrated inFIG. 36

The field division operation of the encoding device F (600) is explained.

In the picture vertical divider064, the vertical division of a picture is performed as follows, for example. It is assumed that the input is performed line by line. It is assumed that one MB is comprised of 16 lines. The field division system depends on input form and is not necessarily restricted to the following.

(1) The pixel line inputted from the input-picture supply device101is stored in the input buffer066.

(2) After 16 lines of one MB are stored in the input buffer066, according to the parity of the column number of the pixel, the pixel of the target MB position of an even-numbered column is stored in the first MB buffer067-1, and the pixel of the target MB position of an odd-numbered column is stored in the second MB buffer067-2.

(3) When 16 lines have accumulated in the first MB buffer067-1, it is possible to output the pixels as the MB of the left column. Similarly, when 16 lines have accumulated in the second MB buffer067-2, it is possible to output the pixels as the MB of the right column.

(4) When the last MB of 16 lines is outputted to the picture vertical divider064, the 16 lines can be deleted from the input buffer066.

It is possible to set any of the right column and the left column as the first vertical divided picture by changing the storage destination of the even-numbered column and the storage destination of the odd-numbered column.

It is considerable that without providing the input buffer066, the vertically divided data are stored directly in the first MB buffer067-1and the second MB buffer067-2. In this case, the first MB buffer067-1and the second MB buffer067-2have a capacity equivalent to (a picture width/2)×(16 lines) at least, respectively.

The encoding operation of the encoding device F (600) is explained.

When the original picture of one frame is inputted, the encoding device F (600) divides it into two vertically divided pictures, and performs encoding for the vertically divided pictures in parallel.

In the I encoder060, the encoding is performed as follows.

(1) An encoding target MB of the first picture is inputted into the I encoder060from the first MB buffer067-1.

(2) The I encoder060performs the I-picture encoding for the target MB of the first picture by the intra picture prediction, and outputs a first encoded bit string to the output control device104. Subsequently, the I encoder060creates a local decoded picture and stores it in the intermediate buffer065.

After the above processing (1)-(2) is performed for all the MBs in the first vertically divided picture, the encoding of the first vertically divided picture is completed. In the P encoder061, the encoding is performed as follows.

(3) The encoding target MB of the second vertically divided picture is inputted to the P encoder061from the second MB buffer067-2.

(4) The P encoder061performs the P-picture encoding by the inter picture prediction from the reference region X composed of the target MB of the second field and the local decoded picture by the I encoder060in the intermediate buffer065, and outputs a second encoded bit string to the output control device104.

After the above processing (3)-(4) is performed for all the MBs in the second vertically divided picture, the encoding of the second vertically divided picture is completed.

The processing (1)-(2) by the I encoder060and the processing (3)-(4) by the P encoder061are performed for every two vertically divided pictures in parallel.

Next, the operation of the position controller062is explained.

FIG. 37is an explanatory drawing illustrating an example of control by the position controller062according to Embodiment 6. In the present explanation, for simplicity, as is the cases withFIG. 11,FIG. 12, andFIG. 15, a picture (vertically divided picture) horizontally composed of six MBs is used, and a reference region is a 2×2 4-MB region with a starting point of the same position as the encoding target MB.

FIG. 37illustrates the I encoder060, the P encoder061, the position controller062, and the intermediate buffer065. The first vertically divided picture to be inputted into the I encoder060and the second vertically divided picture to be inputted into the P encoder061are shown together with MBs, respectively. A hatched MB expresses an already inputted MB and a thick-framed MB expresses an encoding target MB at present.

MB0-MB7of the first vertically divided picture are already inputted into the I encoder060, the encoding is already finished, and the local-decoded MB0-MB7are stored in the intermediate buffer065. At present, the I encoder060is performing the encoding for MB8as the target, and the information expressing “MB8is in process” is notified to the position controller062.

The P encoder061has sent “processing request for MB0” to the position controller042, in order to perform the encoding for MB8of the second vertically divided picture. The position controller062manages whether the data of the reference region X required by the P encoder061is ready, and notifies the P encoder061of “waiting instruction” or “processing permission.” In the example illustrated inFIG. 37, the position controller062makes the P encoder061start the encoding of MB0, by notifying the “processing permission” to the P encoder061, after the reference region X (MB0/MB1/MB6/MB7) which the P encoder061refers to for the encoding of MB0is written in the first intermediate buffer065from the I encoder060. Upon receiving the “processing permission”, the P encoder061starts the encoding using MB0of the second vertically divided picture, and (MB0/MB1/MB6/MB7) of the reference region X.

In Embodiment 6, the picture vertical divider064which divides one frame of the inputted original picture into two vertically divided pictures is provided. Accordingly, it is possible to delete the input picture memory102and to eliminate the access to the external shared memory, as is the case with Embodiment 4. Therefore, the present configuration is effective in greatly reducing the cost of the system. It is not necessary to once store the original picture in the input picture memory102from the input-picture supply device101. Accordingly, the present configuration is also effective in greatly reducing the delay in the processing from input to output. Note that the present embodiment is not restricted to the configuration which is not provided with the input picture memory102, however, the input picture memory102may be provided, as is the case withFIG. 16.

In Embodiment 4, the division of a picture is completed after the input picture of two vertical MBs is inputted. On the contrary, in Embodiment 6, the picture information necessary for the division of a picture becomes completed only with the inputting of the input picture of one MB in the vertical direction. Therefore, the input buffer046according to Embodiment 4 requires the capacity equivalent to the number of lines of two MBs; however, in Embodiment 6, it is sufficient for the input buffer066of the encoding device F (600) to have the capacity equivalent to the number of lines of one MB. Therefore, the present configuration is effective in reducing not only the cost but the encoding delay. Since the picture width which each encoder processes is set to one half, the present configuration is also effective in reducing the line memories for storing the peripheral block information necessary in the encoding.

By controlling the processing position between the I encoder060and the P encoder061, the capacity of the intermediate buffer065added to the inside of the encoding device F (600) does not require the one picture's worth of buffer amount and can be realized only with the amount of several MB lines at most. The P encoder061can performs the motion prediction and motion compensation from the first field as the target of the I encoding. On the other hand, the first vertically divided picture and the second vertically divided picture are created by the division of the identical frame. Therefore, it is also possible to perform the encoding assuming that “motion” is not present. That is, it is possible to consider the implementation in which no motion prediction is performed and the motion vector is always set to 0 in the P encoding. In this case, the capacity of the intermediate buffer065can be theoretically realized with two MBs.

<Vertical Picture Division in Smaller Units>

In the above-described configuration example, the picture of one frame is divided into two vertically divided pictures and the second vertically divided picture refers to the first vertically divided picture, thereby enabling the deletion of the input picture memory102. In Embodiment 6, it is possible to adopt another configuration like Embodiment 5 corresponding to Embodiment 4. The following illustrates another example of configuration in which the vertical picture division is performed in smaller units (for example, two horizontal MBs), the encoding by the interpolation processing is performed with one divided block referring to the other divided block, and multiplexing in units of divided blocks is performed to compose the encoded stream of one picture.

The entire configuration of the picture encoding device according to the present configuration is the same as the configuration example illustrated inFIG. 19, and includes an input-picture supply device101, an encoding device Fa (601), and an output control device104. In the present configuration example, it is not necessary to provide the input picture memory102as is the case with the configuration example illustrated inFIG. 33. However, the present configuration is not restricted to the configuration in which the input picture memory102is not provided, however, it may adopt a configuration in which the input picture memory102is provided as illustrated inFIG. 16.

FIG. 38is a block diagram illustrating the encoding device Fa (601) as another configuration example of Embodiment 6. The input-picture supply device101and the output control device104are also illustrated in the figure.

The encoding device Fa (601) is comprised of an intra MB encoder060, an inter MB encoder061, an input controller063, a picture vertical divider064, an intermediate buffer065, an input buffer066, a first MB buffer067-1, and a second MB buffer067-2. The input buffer066once stores the original picture of a frame structure inputted from the input-picture supply device101. By the control of the input controller063, the picture vertical divider064divides, in the column direction (vertical direction), the original picture of the frame structure stored at the input buffer066into a first block and a second block, and supplies them to either of the encoders (the intra MB encoder060and the inter MB encoder061), via the first MB buffer067-1and the second MB buffer067-2. The intra MB encoder060performs the intra picture encoding of the first block, creates a first encoded bit string, outputs it to the output control device104, and outputs a local decoded picture as a reference picture to the intermediate buffer065. The inter MB encoder061performs the inter picture encoding with reference to the local decoded picture stored in the intermediate buffer065, creates a second encoded bit string, and outputs it to the output control device104.

The division in the vertical direction (column direction) of a picture is the same as the above-described explanation, the frame structure is as illustrated inFIG. 34, and the right column picture and the left column picture after the vertical division are those illustrated inFIG. 35.

FIG. 39andFIG. 40are explanatory drawings illustrating the case where the similar vertical division of a picture is performed in units of MB.FIG. 39illustrates the MB pair of a frame structure, andFIG. 40illustrates the MB pair of the vertical division structure. The MB pair of the frame structure is two MBs, each as the unit of encoding, adjoining in the horizontal direction as illustrated inFIG. 39. The vertical division structure MB pair illustrated inFIG. 40can be divided into the left column (the first MB) which collects the even-numbered columns for every vertical column, and the right column (the second MB) which collects the odd-numbered columns. Here, as illustrated inFIG. 39andFIG. 40, by setting the unit of the vertical division of a picture as two MBs which are aligned horizontally, it is possible to delete the position controller in the encoding device. The encoding device Fa (601) illustrated inFIG. 38adopts the present division method to delete the position controller.

The operation of the encoding device Fa (601) is explained.

FIG. 41is an explanatory drawing illustrating operation in encoding of a vertically divided picture. The figure illustrates the picture vertical divider064, the intra MB encoder060, the inter MB encoder061, and the intermediate buffer065. The figure also illustrates the MB pair of a frame structure inputted into the picture vertical divider064, and the first MB and the second MB after the vertical division. The MB pairs of a frame structure inputted into the picture vertical divider064are two MBs which adjoin mutually in the horizontal (lateral) direction in a frame. The MB is a processing unit of the encoding by the intra MB encoder060and the inter MB encoder061. The MB is composed of plural pixel columns. The picture vertical divider064divides the MB pair (two horizontal MBs) of a frame structure into the first MB and the second MB, and inputs them into the intra MB encoder060and the inter MB encoder061, respectively. The first MB is the left column block reconstructed only from the even-numbered columns of the MB pair of a frame structure, for example, and the second MB is the right column block reconstructed only from the odd-numbered columns of the MB pair of a frame structure, for example. Their size is one MB, respectively. The intra MB encoder060encodes the first MB and at the same time stores a local-decoded MB in the intermediate buffer065. The inter MB encoder061encodes the second MB by the inter picture encoding technique with the intermediate buffer065as the reference MB. The output control device104multiplexes the first encoded bit string and the second encoded bit string in units of MB, to compose and output an encoded stream.

The vertical division operation of a picture by the encoding device Fa (601) is explained.

In the picture vertical divider064, the vertical division of a picture is performed as follows, for example. It is assumed that the input is performed line by line. It is assumed that one MB is comprised of 16 lines. The field division system depends on input form and is not necessarily restricted to the following.

(1) The pixel line inputted from the input-picture supply device101is stored in the input buffer066.

(2) After 16 lines of one MB are stored in the input buffer066, according to the parity of the column number of the pixel, the pixel of the target MB position of an even-numbered column is stored in the first MB buffer067-1, and the pixel of the target MB position of an odd-numbered column is stored in the second MB buffer067-2.

(3) When 16 lines have accumulated in the first MB buffer067-1, it is possible to output the pixels as the MB of the left column. Similarly, when 16 lines have accumulated in the second MB buffer067-2, it is possible to output the pixels as the MB of the right column.

(4) When the last MB of 16 lines is outputted to the picture vertical divider064, the 16 lines can be deleted from the input buffer066.

The processing unit may be the unit in which the two horizontal MBs are divided into the first MB and the second MB for every right and left column, or may be the unit smaller than it, for example, the unit of 8×16 pixels to which one MB is divided.

In the encoding device Fa (601) according to the present configuration example, a picture is divided vertically, and the encoding of one of the divided blocks is performed by the interpolation processing with reference to a reference picture created by the encoding of the other of the divided blocks.

The present configuration example is effective in realizing the intra picture encoding with better encoding efficiency than the existing intra picture encoding. In addition, compared with the encoding device F (600) illustrated inFIG. 33, the intermediate buffer062may have smaller capacity and the position controller is not necessary; accordingly the present configuration is effective in reducing the cost. Since the picture width is set to one half, the present configuration is also effective in reducing the line memories for storing the peripheral block information necessary in the encoding.

In Embodiment 5, the division of a picture is completed after the input picture of two vertical MBs is inputted. On the contrary, in the present configuration example, the picture information necessary for the division of a picture becomes completed only with the inputting of the input picture of one MB in the vertical direction. Therefore, the input buffer055according to Embodiment 5 requires the capacity equivalent to the number of lines of two MBs; however, in the present configuration example, for the input buffer066of the encoding device Fa (601), the number of lines of one MB is sufficient. Therefore, the present configuration is effective in reducing not only the cost but the encoding delay.

Further another modified example is explained. The entire configuration of the picture encoding device according to the present modified example is the same as the configuration example illustrated inFIG. 19, and includes an input-picture supply device101, and an encoding device Fb (602), and an output control device104. In the present modified example, the symbol of the encoding device is changed from Fa (601) to Fb (602).

FIG. 42is a block diagram illustrating further another configuration example of the picture encoding device according to Embodiment 6.

The encoding device Fb (602) is comprised of a switch068which switches the storing destination of the output of the picture vertical divider064, and a switch069which switches the output destination of two encoders (060and061), in addition to the configuration of the encoding device Fa (601) illustrated inFIG. 38. The input controller063controls these switches. The other configuration is the same as that of the encoding device Fa (601) illustrated inFIG. 38; therefore, the explanation thereof is omitted.

The operation of the encoding device Fb (602) is explained. In the encoding device Fa (601), one of the left column MB and the right column MB is inputted into one of the intra MB encoder060and the inter MB encoder061, and the reference direction in which the encoding of one side refers to a local decoded picture of the other side is fixed. On the contrary, in the encoding device Fb (602), it is possible to reverse the reference direction or to change it adaptively for every two MBs. In the encoding device Fb (602), the change of the reference direction is enabled with the added switches068and069. The switch068can switch selectively between a position A and a position B. In the position A, the left column MB divided by the picture vertical divider064is fed as the first MB and the right column MB is fed as the second MB. Conversely, in the position B, the right column MB is fed as the first MB and the left column MB is fed as the second MB. The switch069can switch selectively between a position C and a position D. In the position C, the output of the intra MB encoder060is fed as the first encoded bit string and the output of the inter MB encoder061is fed as the second encoded bit string. Conversely, in the position D, the output of the intra MB encoder060is fed as the second encoded bit string and the output of the inter MB encoder061is fed as the first encoded bit string.

FIG. 43-FIG. 45are explanatory drawings illustrating examples of the reference direction in encoding of a vertically divided picture.FIG. 43illustrates a typical example in which the left column MB is fed as the first MB and is encoded by the intra MB encoder060, and the right column MB is fed as the second MB and is encoded by the inter MB encoder061with reference to the local decoded picture of the left column MB.FIG. 44illustrates an example in which the local decoded picture of the right column MB is conversely referred to in the encoding of the left column MB.FIG. 45illustrates an example in which the reference direction is switched adaptively every two MBs.

As is the case with the encoding device Fa (601), the encoding device Fb (602) divides a picture vertically and performs the encoding of one of the divided blocks by the interpolation processing, with reference to a reference picture created by the encoding of the other of the divided blocks. The present configuration example is effective as well in realizing the intra picture encoding with better encoding efficiency than the existing intra picture encoding. Compared with the encoding device F (600) illustrated inFIG. 33, the intermediate buffer065may have smaller capacity and the position controller is not necessary; accordingly the present configuration is effective in reducing the cost. Since the picture width is set to one half, the present configuration is also effective in reducing the line memories for storing the peripheral block information necessary in the encoding. This point is also the same as that of the encoding device Fa (601).

Furthermore, by adopting the configuration of the encoding device Fb (602), it is possible to realize the coding method which switches the reference direction adaptively every two MBs as described above. That is, when assuming a block structure in which the MBAFF specified by H.264 is transposed horizontal to vertical, the encoding by the interpolation processing is performed with the reference from the right column MB to the left column MB (FIG. 43). It is also possible to perform the encoding with reference from the left column MB to the right column MB (FIG. 44), and it is furthermore possible that both directions of reference coexist in a picture (FIG. 45).

FIG. 46is a block diagram illustrating an example of the entire configuration of a picture encoding device according to Embodiment 7.FIG. 46is an example in which the configuration of Embodiment 1 is applied to the multi-view coding method.

The entire configuration of the picture encoding device according to Embodiment 7 includes a first input-picture supply device701which supplies an original picture of a first view, a second input-picture supply device702which supplies an original picture of a second eye, an input picture memory102, an encoding device Ga (700), a reference frame memory103, and an output control device104. The first input-picture supply device701and the second input-picture supply device702supply the multi-view original picture as an encoding target, and are exemplified by an imaging device, such as a multi-view camera used for an on-vehicle around view monitor. The input picture memory102stores the original picture supplied from the first input-picture supply device701and the second input-picture supply device702. The reference frame memory103stores a reference picture, such as a local decoded picture. Although the input picture memory102and the reference frame memory103are shown as a separate memory in the figure, they may be implemented as one memory. For example, by the address mapping scheme, they may be implemented in an external SDRAM which is shared by the whole system. The encoding device Ga (700) encodes the original picture as an encoding target in conformity with a specified multi-view coding method. In accordance with the unit of the encoding and the order of the encoding, an original picture is read from the input picture memory102and encoded. Reference pictures, such as a local decoded picture created in the process, are stored in the reference frame memory103. A reference picture necessary in the process of the encoding is read from the reference frame memory103for reference. The encoding device Ga (700) outputs multiple encoded bit strings as a result of the encoding, and the output control device104creates a multi-view encoded stream from these encoded bit strings.FIG. 46illustrates the first encoded bit string and the second encoded bit string; however, it is preferable to adopt the configuration in which more encoded bit strings are created and outputted.

FIG. 47is a block diagram illustrating a configuration example of the encoding device Ga (700) illustrated inFIG. 46. For convenience of explanation, the first input-picture supply device701, the second input-picture supply device702, the input picture memory102, the reference frame memory103, and the output control device104are also illustrated in the figure. The figure illustrates a typical state where as the original pictures, main pictures0,1, . . . , n−1, n, . . . , and sub pictures0,1, . . . , n−1, n, . . . (n is an arbitrary natural number) are stored in the input picture memory102, and the reference frames X and Y are stored in the reference frame memory103.

The encoding device Ga (700) is comprised of a base view encoder071, an inter-view encoder072, an input controller073, a switch074, a third intermediate buffer075, a first intermediate buffer076, and a position controller077. The base view encoder071encodes the main picture, and the inter-view encoder072encodes the sub picture. The input controller073controls the switch074, reads the original picture from the input picture memory102, and supplies it to the base view encoder071and the inter-view encoder072. The base view encoder071writes a reference picture created in encoding into the reference frame memory103and at the same time stores it in the first intermediate buffer076temporarily. The third intermediate buffer005stores temporarily a reference picture (reference frame) suitably read from the reference frame memory103. The inter-view encoder072refers to the first intermediate buffer076, when encoding the sub picture, but does not refer to the third intermediate buffer075, unlike with Embodiment 1.

The explanation of the present embodiment illustrates an example of two views composed of one sub picture. However, it is possible to adopt a configuration of multi views composed of two or more sub pictures. That case can be realized by adding the group of the inter-view encoder072and the position controller077.

The operation of the encoding device Ga (700) is explained.

The base view encoder071performs the base view encoding of the ordinary multi-view coding which uses the reference frame memory103. In that case, a local decoded MB is stored in the first intermediate buffer076. The inter-view encoder072encodes the sub picture with reference to the local decoded MB of the first intermediate buffer076.

The encoding device Ga (700) encodes in parallel every two pictures, that is, a set of a main picture and a sub picture.

In the base view encoder071, the encoding is performed as follows.

(1) An encoding target MB of the main picture (n) is inputted into the base view encoder071from the input picture memory102.

(2) A reference region X to be used for the base view encoding is inputted into the third intermediate buffer075from a reference frame X in the reference frame memory103.

(3) The base view encoder071performs the base view encoding from the main picture (n) MB and the reference region X in the third intermediate buffer075, and outputs a first encoded bit string to the output control device104. Subsequently, the base view encoder071creates a local decoded MB and writes it in the first intermediate buffer076.

(4) The local decoded MB stored in the first intermediate buffer076is written in the reference frame memory103, in preparation to be used as a reference frame in the subsequent base view encoding.

After the above processing (1)-(4) is performed for all the MBs in the main picture (n), the base view encoding of one picture is completed. The base view encoding here can include the I/P encoding of each picture. When encoding the I picture, the base view encoder071does not refer to the reference region X in the third intermediate buffer075in the step (3).

In the inter-view encoder072, the encoding is performed as follows.

(5) An encoding target MB of the sub picture (n) is inputted into the inter-view encoder072from the input picture memory102.

(6) The inter-view encoder072performs the inter-view encoding by the inter-view prediction from the sub picture (n) MB and the reference region Y composed of the local decoded MB of the main picture in the first intermediate buffer076, and outputs a second encoded bit string to the output control device104.

After the above processing (5)-(6) is performed for all the MBs in the sub picture (n), the sub-picture encoding of one picture is completed. The inter-view encoding here performs only the encoding using the inter-view prediction which refers to the main picture or the intra picture prediction.

The processing of (1)-(4) by the base view encoder071and the processing of (5)-(6) by the inter-view encoder072are performed in parallel for one set of the main picture (n) and the sub picture (n).

Since the data of the third intermediate buffer075is not used by the inter-view encoder072and can be deleted in accordance with the processing of the base view encoder071. The third intermediate buffer075may be omitted depending on the configuration of the base view encoder071. The reference frame memory103may be omitted similarly. For example, when the base view encoder071performs only the encoding of the I picture, the third intermediate buffer and the reference frame memory are not necessary.

The position controller077performs synchronous control of the processing position of the base view encoder071and the inter-view encoder072. The base view encoder071outputs “processing position”, the inter-view encoder072outputs “processing request”, and the position controller077notifies “waiting instruction” or “processing permission” to the inter-view encoder072. The control system is the same as the method which is explained with reference toFIG. 11andFIG. 12in Embodiment 1.

In the present configuration, the inter-view encoder072encodes the sub picture with reference to the local decoded MB of the base view encoder071, with the help of the first intermediate buffer076and the position controller077. Accordingly, it is possible to make no access to the reference frame memory103at the time of the encoding of the sub picture. Therefore, the present configuration is effective in reducing the memory capacity and memory zone of the reference frame memory103. It is also possible to perform the inter-view encoding in parallel with the base view encoding. Accordingly, the present configuration is effective in reducing the encoding delay.

A Modified Example of Embodiment 7; Application to the Multi-View Coding Method of Embodiment 4

FIG. 48is a block diagram illustrating another example of the entire configuration of the picture encoding device according to Embodiment 7.FIG. 48is an example of applying the configuration of Embodiment 4 to the multi-view coding method.

The entire configuration of the picture encoding device illustrated inFIG. 48includes a first input-picture supply device701, a second input-picture supply device702, an encoding device Gb (703), and an output control device104. Differing from the picture encoding device illustrated inFIG. 46, the present picture encoding device illustrated inFIG. 48is not provided with the input picture memory102and the reference frame memory103, and the original picture data as the encoding target is directly inputted into the encoding device Gb (703) from the first input-picture supply device701and the second input-picture supply device702.

FIG. 49is a block diagram illustrating a configuration example of the encoding device Gb (703) illustrated inFIG. 48. Compared with the configuration of the encoding device Ga (700) illustrated inFIG. 47, the third intermediate buffer075is omitted. The switch074acts so that the first input-picture supply device701inputs the main picture into the base view encoder071directly and the second input-picture supply device702inputs the sub picture into the inter-view encoder072directly. The input controller073and the switch074may be simplified, or may be omitted. There is no output from the first intermediate buffer to the reference frame memory.

The configuration example illustrated inFIG. 49is for two views with one sub picture. However, it is possible to adopt a configuration of multi views composed of two or more sub pictures. That case can be realized by adding the group of the inter-view encoder072and the position controller077.

The operation of the encoding device Gb (703) is explained.

The base view encoder071performs the base view encoding of the multi-view coding, without using the reference frame memory103. That is, the encoding is only the I-picture encoding which performs only the intra picture prediction. In that case, a local decoded MB is stored in the first intermediate buffer076. The inter-view encoder072encodes the sub picture with reference to the local decoded MB of the first intermediate buffer076.

The encoding device Gb (703) encodes in parallel every two pictures, that is, a set of a main picture and a sub picture.

In the base view encoder071, the encoding is performed as follows.

(1) An encoding target MB of the main picture (n) is inputted into the base view encoder071from the first input-picture supply device701.

(2) The base view encoder071performs the base view encoding for the main picture (n) and outputs a first encoded bit string to the output control device104. Subsequently, the base view encoder071creates a local decoded MB and stores it in the first intermediate buffer076.

After the above processing (1)-(2) is performed for all the MBs in the main picture (n), the base view encoding of one picture is completed. The base view encoding here is the encoding of the I picture.

In the inter-view encoder072, the encoding is performed as follows.

(3) An encoding target MB of the sub picture (n) is inputted into the inter-view encoder072from the second input-picture supply device702.

(4) The inter-view encoder072performs the inter-view encoding by the inter-view prediction from the sub picture (n) MB and the reference region Y composed of the local decoded MB of the main picture in the first intermediate buffer076, and outputs a second encoded bit string to the output control device104.

After the above processing (3)-(4) is performed for all the MBs in the sub picture (n), the sub-picture encoding of one picture is completed. The inter-view encoding here performs only the encoding using the inter-view prediction which refers to the main picture or the intra picture prediction.

The processing of (1)-(2) by the base view encoder071and the processing of (3)-(4) by the inter-view encoder072are performed in parallel for one set of the main picture (n) and the sub picture (n).

The position controller077performs synchronous control of the processing position of the base view encoder071and the inter-view encoder072. The control method is the same as that of the position controller077in the above-described encoding device Ga (700).

As described above, in the modified example of Embodiment 7 illustrated by the encoding device Gb (703), the sub picture is encoded by the inter-view encoder072with reference to the local decoded MB of the base view encoder071, with the help of the first intermediate buffer076and the position controller077. The input picture memory102and the reference frame memory103are not used owing to the direct entry of the original picture from the first input-picture supply device701and the second input-picture supply device702, and by restricting the base view encoder071to the intra picture encoding.

According to the present configuration, it is possible to omit the input picture memory102and the reference frame memory103at the time of the multi-view coding. Accordingly, the present configuration is effective in reducing the system cost.

In addition, it is possible to perform the base view encoding and the inter-view encoding in parallel. Accordingly, the present configuration is effective in reducing the encoding delay.

A Further Modified Example of Embodiment 7; the Base-View Encoding Including a B Picture

The base view encoder071of the encoding device Ga (700) performs the encoding of the I picture and the P picture and the base view encoder071of the encoding device Gb (703) encodes only the I picture. However, it is possible to adopt a configuration in which the base view encoder071performs the encoding of the B picture in addition to the I picture and the P picture. In the present case, two pictures for the main picture and two pictures for the sub picture, four pictures in total are inputted, the portion of the base view encoder071is replaced with the group of the I/P encoder and the B encoder, as is the case with Embodiment 1, and the portion of the inter-view encoder072is replaced with the group of an encoder which refers to the output of the I/P encoder and an encoder which refers to the output of the B encoder, thereby realizing the configuration. The following explains a configuration example of the encoding device Gc (704) configured based on such technical thought.

FIG. 50is a block diagram illustrating a configuration of the encoding device Gc (704).

The encoding device Gc (704) is comprised of an input controller073, a switch074, an I/P base view encoder071-1, a B base view encoder071-2, a first inter-view encoder072-1, a second inter-view encoder072-2, a first intermediate buffer076-1, a second intermediate buffer076-2, a third intermediate buffer075, and a position controller077-1, a position controller077-2, and a position controller077-3.

The input controller073controls the switch074to supply the main pictures (2n) and (2n−1) to the I/P base view encoder071-1and the B base view encoder071-2, respectively, and to supply the sub pictures (2n) and (2n−1) to the first inter-view encoder072-1and the second inter-view encoder072-2, respectively.

The I/P base view encoder071-1, the B base view encoder071-2, the first intermediate buffer076-1, the third intermediate buffer075, and the position controller077-1correspond respectively to the I/P encoder001, the B encoder002, the first intermediate buffer006, the third intermediate buffer005, and the position controller007, of the encoding device A (100) according to Embodiment 1. The operation is the same as that of the operation of the encoding device A (100) explained in Embodiment 1. To the main picture, the encoding is performed so that one B picture may enter between I pictures or P pictures like IBPBPB, for example. The I/P base view encoder071-1and the B base view encoder071-2output a first encoded bit string and a second encoded bit string, respectively.

The local decoded picture of the main picture (2n) is stored at the first intermediate buffer076-1temporarily, referred to in the B-picture encoding of the main picture (2n−1), and also supplied to the first inter-view encoder072-1. The local decoded picture of the main picture (2n−1) encoded by the B base view encoder071-2is stored in the second intermediate buffer076-2temporarily, and supplied to the second inter-view encoder072-2. The first inter-view encoder072-1performs the encoding of the sub picture (2n), referring to the local decoded picture of the main picture (2n), and outputs a third encoded bit string. At this time, the processing request and the processing permission about the MB as the processing target are controlled by the position controller077-2. The second inter-view encoder072-2performs the encoding of the sub picture (2n−1), referring to the local decoded picture of the main picture (2n−1), and outputs a fourth encoded bit string. At this time, the processing request and the processing permission about the MB as the processing target are controlled by the position controller077-3.

Also in the multi-view coding, by the above configuration, it is possible to perform the encoding which includes the B picture in the base view; accordingly, it is possible to improve the encoding efficiency and the image quality.

The above explains the configuration example in which the base view encoding is combined with the encoding device illustrated in Embodiment 1. However, it is possible to combine with various encoding devices illustrated in Embodiments 2, 3, 4, 5, and 6. For example, if various encoding devices illustrated in Embodiments 3, 4, 5, and 6 are combined, it is possible to compose the multi-view encoding device which does not need the access to the reference frame memory, as is the cases withFIG. 45andFIG. 49.

A Picture Decoding Device

To the various picture encoding devices illustrated in each of the above-described embodiments, it is possible to configure and provide a picture decoding device corresponding to each, based on the same technical thought.

A Picture Decoding Device Corresponding to Embodiment 1

A configuration example of a picture decoding device corresponding to Embodiment 1 is explained. In a system mounting a decoding device, when the number of the B picture between the I/P pictures and the fact that the reference region of the B picture is restricted to a fixed range are known beforehand, the decoding device of the present embodiment can be utilized. For example, a case such as a so-called self-recording and replaying system can be considered.

FIG. 51is a block diagram illustrating an entire configuration example of a picture decoding device according to Embodiment 8a.

The entire configuration of the picture decoding device according to Embodiment 8a includes an input control device801, an input code memory802, a reference frame memory803, a decoding device Ha (800), and a picture processing device804. The input control device801writes an encoded stream inputted, for example from a transmission line such as a network, into the input code memory802. The input code memory802stores the inputted encoded stream, and the reference frame memory803stores a reference frame. The decoding device Ha (800) reads an encoded stream from the input code memory802, performs decoding with reference to the reference frame stored in the reference frame memory803, and creates and outputs plural decoded pictures. The picture processing device804performs reconstruction processing for two or more decoded pictures which the decoding device Ha (800) outputs. The input code memory802and the reference frame memory803are assigned to an external SDRAM shared by the system, for example.

FIG. 52is a block diagram illustrating a configuration example of the decoding device Ha (800) according to Embodiment 8a. The input control device801, the input code memory802, the reference frame memory803, and the picture processing device804are also illustrated in the figure. The input code memory802and the reference frame memory803show schematically that the code streams0,1, . . . ,2n−1,2n, . . . , and reference frames X and Y are stored respectively.

The decoding device Ha (800) is comprised of an I/P decoder081, a B decoder082, an input controller083, a switch084, a third intermediate buffer085, a first intermediate buffer086, and a position controller087. The I/P decoder081decodes an I picture or a P picture, and the B decoder082decodes a B picture. The input controller083controls the switch084and reads a code stream from the input code memory802and supplies it to the I/P decoder081and the B decoder082. The I/P decoder081decodes an I picture or a P picture, and outputs a first decoded picture to the picture processing device804, writes a reference frame created in process of decoding in the reference frame memory803and at the same time stores it in the first intermediate buffer086temporarily. The third intermediate buffer085stores temporarily the reference frame suitably read from the reference frame memory803. When the I/P decoder081decodes a P-picture, a reference frame stored in the third intermediate buffer085is referred to. The B decoder082refers to the reference frame stored in the first intermediate buffer086temporarily, and the reference frame stored in the third intermediate buffer085, decodes the B picture, and outputs a second decoded picture to the picture processing device804. The position controller087controls the processing position of the I/P decoder081and the B decoder082, that is, the position of the picture as the processing target in decoding.

The I/P decoder081does not need to be a decoder which performs only the I-picture decoding and the P-picture decoding, but it may be replaced with an I/P/B decoder which performs decoding including the B-picture decoding. Similarly, the B decoder082does not need to be a decoder which performs only the B-picture decoding, but it may be replaced with an I/P/B decoder.

The number of B decoders may not be one. By mounting N-piece B decoders, it is possible to set the number of the B pictures between two P pictures (or I pictures) to N sheets. In that case, the reference region X and the reference region Y used as an input can share the first intermediate buffer085and the second intermediate buffer086.

The operation of the decoding device Ha (800) is explained. In the present embodiment, it is assumed to decode the encoded stream in which one B picture enters between I pictures or P pictures, such as IBPBPB . . . .

It is assumed that the encoded stream is stored in the transmitted order in the input code memory802. The encoded stream is a binary bit stream. There is no break of a picture; however, for convenience, it is illustrated such as a code stream0, a code stream1, . . . , a code stream (2n−1), a code stream (2n) for a picture.

By the input controller083and the switch084, the code stream of the I picture or the P picture is inputted into the I/P decoder081, and the code stream of the B picture is inputted into the B decoder082, and the decoding is performed in parallel. Determination of I/P/B can be made by searching the header of a bit string which composes a code stream.

The decoding device Ha (800) decodes two pictures in parallel.

(1) A code stream of I/P is inputted into the I/P decoder081.

(2) A reference region X necessary in order to decode a code stream is inputted into the third intermediate buffer085from the reference frame X in the reference frame memory803. The necessary reference region is decided along with the decoding.

(3) The I/P decoder081performs the I-picture decoding or the P-picture decoding from the first code stream and the reference region X, and outputs a first decoded picture to the picture processing device804. Subsequently, the decoded picture is stored in the first intermediate buffer086.

(4) The decoded picture stored in the first intermediate buffer086is written in the reference frame Y of the reference frame memory803, in preparation for being used as a reference frame in the subsequent decoding.

After the above processing (1)-(4) is performed for all the MBs of the first code stream, the decoding of one picture is completed. When performing the I-picture decoding, the I-picture decoding is performed by the intra picture prediction in (3). At this time, the reference region X in the first intermediate buffer085is not used.

In the B decoder082, the decoding is performed as follows.

(5) A code stream of B is inputted into the B decoder082.

(6) The B decoder082performs the B-picture decoding by the inter picture prediction, from the first code stream, the reference region X in the third intermediate buffer085, and the reference region Y composed of the decoded picture in the first intermediate buffer086, and outputs a second decoded picture to the picture processing device804.

After the above processing (5)-(6) is performed for all the MBs of the second code stream, the decoding of one picture is completed.

The processing of (1)-(4) by the I/P decoder081and the processing of (5)-(6) by the B decoder082are performed in parallel (two-picture parallel decoding).

The position controller087synchronizes the processing position of the I/P decoder081and the B decoder082. The I/P decoder081outputs “processing position”, the B decoder082outputs “processing request”, and the position controller087notifies “waiting instruction” or “processing permission” to the B decoder082. The present control is the same as the method which is explained with reference toFIG. 11andFIG. 12in Embodiment 1. At this time, it is the requisite that the greatest range of the reference region is fixed.

As described above, the decoding device Ha (800) decodes the B picture with reference to the decoded MB of the I/P decoder081by the B decoder082, with the help of the first intermediate buffer086and the position controller087. According to this configuration, when the number of the B picture between the I/P pictures and the fact that the reference region of the B picture is restricted to a fixed range are known beforehand, no access to the reference frame memory803is necessary in the B picture decoding. Therefore, the present configuration is effective in reducing the memory capacity and memory zone of the reference frame memory803.

A Picture Decoding Device Corresponding to Embodiment 3

The present embodiment is application of Embodiment 2, Embodiment 3, and Embodiment 4. Here, a configuration example of the picture decoding device corresponding to Embodiment 3 is explained.

In a system mounting a decoding device, when it is known beforehand that the I picture and the P picture are alternately encoded such as IPIPIP . . . , and that the reference region of the P picture is restricted to a fixed range, the decoding device according to the present embodiment can be utilized. For example, a case such as a self-recording and replaying system which uses the encoding device C (300) according to Embodiment 3 can be considered.

FIG. 53is a block diagram illustrating an example of the entire configuration of a picture decoding device according to Embodiment 8b.

The entire configuration of the picture decoding device according to Embodiment 8b includes an input control device801, a decoding device Hb (810), and a picture processing device804. The input control device801and the picture processing device804are the same as described above with reference toFIG. 51; therefore, the explanation thereof is omitted. Embodiment 8b is different from Embodiment 8a in the point that the input code memory802and the reference frame memory803are not provided. A code stream is supplied directly to the decoding device Hb (810) from the input control device801, and the decoding device Hb (810) performs decoding, without using the reference frame memory803.

FIG. 54is a block diagram illustrating a configuration example of the decoding device Hb (810) according to Embodiment 8b. The input control device801and the picture processing device804are also illustrated in the figure.

The decoding device Hb (810) is comprised of an I decoder088, a P decoder089, an input controller083, a switch084, an intermediate buffer086, and the position controller087. The I decoder088decodes an I picture, and the P decoder089decodes a P picture. The input controller083controls the switch084to supply directly the first code stream and the second code stream to the I decoder088and the P decoder089from the input control device801, respectively. The I decoder088decodes the I picture, outputs a first decoded picture to the picture processing device804, and stores temporarily a reference frame created in process of the decoding in the intermediate buffer086. There is no output from the intermediate buffer086to the reference frame memory. The P decoder089decodes a P picture with reference to the reference frame stored in the intermediate buffer086temporarily, and outputs a second decoded picture to the picture processing device804. The position controller087controls the processing position of the I decoder088and the P decoder089, that is, the position of the picture of the processing target in decoding.

The explanation of the present embodiment illustrates the example in which the input control device801inputs the first code stream and the second code stream separately; however, it is possible to adopt a configuration in which the input controller083and the switch084separate them.

The I decoder088does not need to be a decoder which performs only the I-picture decoding, but it may be replaced with an I/P/B decoder or an I/P decoder. Similarly, the P decoder089does not need to be a decoder which performs only the P-picture decoding, but it may be replaced with an I/P/B decoder or an I/P decoder.

For example, when the encoding order is fixed such as IBPIBPIBP . . . and when it is known beforehand that the reference region of the P picture and the B picture is restricted to a fixed range (for example, a self-recording and replaying system using the encoding device B (200) according to Embodiment 2), it is possible to realize the corresponding picture decoding device, by adding a B decoder, a second intermediate buffer relevant thereto, and a second position controller.

It is similarly possible to realize the picture decoding device corresponding to the code stream of a field structure such as IPIP . . . (for example, a self-recording and replaying system using the encoding device D (400) according to Embodiment 4).

The operation of the decoding device Hb (810) is explained. In the present embodiment, it is assumed to decode an encoded stream in which an I picture and a P picture are present alternately such as IPIP . . . .

The decoding device Hb (810) decodes two pictures in parallel.

In the I decoder088, the decoding is performed as follows.

(1) The I-picture code stream (the first code stream) is inputted into the I decoder088.

(2) The I decoder088performs the I-picture decoding from the first code stream, and outputs a first decoded picture to the picture processing device804. Subsequently, the I decoder088stores the decoded picture in the intermediate buffer086.

After the above processing (1)-(2) is performed for all the MBs of the first code stream, the decoding of one picture is completed.

In the P decoder089, the decoding is performed as follows.

(3) The P-picture code stream (the second code stream) is inputted into the P decoder089.

(4) The P decoder089performs the P-picture decoding by the inter picture prediction from the second code stream and the reference region composed of the decoded picture in the intermediate buffer086, and outputs a second decoded picture to the picture processing device804.

After the above processing (3)-(4) is performed for all the MBs of the second code stream, the decoding of one picture is completed.

The processing of (1)-(2) by the I decoder088and the processing of (3)-(4) by the P decoder089are performed in parallel (two-picture parallel decoding).

The position controller087synchronizes the processing position of the I decoder088and the P decoder089. The control method is the same as in Embodiment 8a. At this time, it is the requisite that the greatest range of the reference region is fixed.

As described above, the decoding device Hb (810) decodes the P picture by the P decoder089with reference to the decoded MB of the I decoder088, with the help of the intermediate buffer086and the position controller087. When the I picture and the P picture are encoded alternately such as IPIPIP . . . , and when it is known beforehand that the reference region of the P picture is restricted to a fixed range, no access to the reference frame memory803is necessary. Accordingly, the present configuration is effective in reducing the system cost.

Embodiment 8 explains the picture decoding device corresponding to the encoding device illustrated in Embodiments 1-4. However, it is possible to realize the picture decoding device corresponding to the encoding device illustrated in other embodiments, based on the same technical thought.

A Picture Communication System

A picture communication system can be configured by mounting the picture encoding device and the picture decoding device according to the embodiments described above in a transmitter, a receiver, or both.

FIG. 55is a block diagram illustrating an example of the entire configuration of a picture communication system according to Embodiment 9.

A transmitter1000and a receiver1001are coupled via a transmission line1002. The transmitter1000is comprised of the input-picture supply device101, the encoding device A (100), and the output control device104which are illustrated in Embodiment 1. The output control device104is comprised of the first output buffer010which once stores the first encoded bit string, the second output buffer011which once stores the second encoded bit string, and the switch012which selects the bit string to transmit. The receiver1001is comprised of the input control device801including the input buffer830, a general-purpose decoding device805, and the picture processing device804.

As the encoding device in the transmitter1000, the encoding device A (100) illustrated in Embodiment 1 is described. However, a similar system configuration is possible by adopting the encoding device B (200) according to Embodiment 2, the encoding device C (300) according to Embodiment 3, or the encoding device D (400) according to Embodiment 4. The input frame memory102and the reference frame memory103are omitted in the figure.

The transmitter1000buffers the first encoded bit string and the second encoded bit string respectively outputted by the encoding device A (100), and outputs them alternately in units of pictures, thereby enabling the transmission of an encoded stream which is in conformity with the coding standard. When the encoding device is the encoding device D (400) according to Embodiment 4, the output becomes an interlaced encoded stream.

In the receiver1001, the encoded stream is once buffered by the input buffer830and is decoded by the general-purpose decoding device805. The picture processing device804is exemplified by an image display apparatus, but any kind of an image processing device can be employed.

The transmitter1000according to Embodiment 9 transmits the output of the encoding device A (100), the encoding device B (200), the encoding device C (300), or the encoding device D (400), alternately in units of pictures. The present embodiment is effective in reducing the memory access on the transmitting side. Moreover, the receiving side does not require any special device, and can be comprised of existing general-purpose decoding devices which are in conformity with the coding standard.

A Picture Communication System

FIG. 56is a block diagram illustrating an example of the entire configuration of a picture communication system according to Embodiment 10.

As is the case with the picture communication system illustrated inFIG. 55, a transmitter1100and a receiver1101are coupled via a transmission line1002. The transmitter1100is comprised of the input-picture supply device101, the encoding device A (100), and the output control device104, The receiver1101is comprised of the input control device801, the general-purpose decoding device805, and the picture processing device804. In the transmitter1100, the output control device104is comprised of a multiplexer013in lieu of the switch012, and in the receiver1101, the input control device801is comprised of a demultiplexer833, a first input buffer831, a second input buffer832, and a switch834. What is described above is different from the picture communication system illustrated inFIG. 55.

As the encoding device in the transmitter1100according to Embodiment 10, the encoding device A (100) illustrated in Embodiment 1 is adopted, as is the case with Embodiment 9. However, a similar system configuration is possible by adopting the encoding device B (200) according to Embodiment 2, the encoding device C (300) according to Embodiment 3, or the encoding device D (400) according to Embodiment 4. The input frame memory102and the reference frame memory103are omitted in the figure.

The multiplexer013in the output control device104of the transmitter1100multiplexes the first encoded bit string and the second encoded bit string, which are outputted by the encoding device, in the finer unit in a picture, and transmits them. As the unit of multiplexing, the unit such as a slice specified by the coding standard may be employed, or a uniquely smaller unit may be employed.

A demultiplexer833in the input control device801of the receiver1101performs demultiplexing in the same units used by the multiplexer013of the transmitter1100, and restores the first encoded bit string and the second encoded bit string. The restored first encoded bit string and second encoded bit string are selected by the switch834so that it may become a stream of the coding standard, and supplied to the general-purpose decoder805.

The transmitter1100according to Embodiment 10 multiplexes the output of the encoding device A (100), the encoding device B (200), the encoding device C (300), or the encoding device D (400), in the finer unit in a picture, and transmits it, and the receiver1101reconstructs and decodes to the stream of the coding standard. The present embodiment has the following effects. That is, it is possible to reduce the capacity of the first output buffer010and the second output buffer011of the transmitter1100; accordingly, the present configuration is effective in reducing the output delay on the transmitting side. The present configuration is also effective in easy smoothing of the bit rate.

A Picture Communication System

FIG. 57is a block diagram illustrating an example of the entire configuration of a picture communication system according to Embodiment 11.

As is the case with the picture communication system according to Embodiment 10 illustrated inFIG. 56, a transmitter1200and a receiver1201are coupled via a transmission line1002. The transmitter1200is comprised of the input-picture supply device101, the encoding device A (100), and the output control device104. Differing from the receiver1101in the picture communication system according to Embodiment 10, the receiver1201is comprised of an input control device801and a picture processing device804, and in addition, a first general-purpose decoding device806, and a second general-purpose decoding device807. Accordingly, the switch834in the input control device801is deleted.

As the encoding device in the transmitter1200according to Embodiment 11, the encoding device A (100) according to Embodiment 1 is adopted as is the case with Embodiment 9 and Embodiment 10. However, a similar system configuration is possible by adopting the encoding device B (200) according to Embodiment 2, the encoding device C (300) according to Embodiment 3, or the encoding device D (400) according to Embodiment 4. The input frame memory102and the reference frame memory103are omitted in the figure.

The first general-purpose decoding device806of the receiver1201inputs and decodes the first encoded stream in the first input buffer831. In parallel with it, the second general-purpose decoding device807inputs and decodes the second encoded stream in the second input buffer832. If necessary, the picture processing device804reconstructs the output picture of the first general-purpose decoding device806and the output picture of the second general-purpose decoding device807.

In addition to the feature of Embodiment 10, the picture communication system according to Embodiment 11 has the feature that the decoding devices of the receiving side in the receiver1201operate in parallel. The present embodiment is effective in reducing the code buffering amount in both the transmission and the reception. The receiver1200can perform processing with low delay; therefore, the present embodiment is effective in realizing the configuration suitable for a system which demands low delay performance.

A Picture Communication System

FIG. 58is a block diagram illustrating an example of the entire configuration of a picture communication system according to Embodiment 12.

As is the case with the picture communication system according to Embodiment 11 illustrated inFIG. 57, a transmitter1300and a receiver1301are coupled via a transmission line1002. The transmitter1300is comprised of the input-picture supply device101, the encoding device A (100), and the output control device104. The receiver1301is comprised of an input control device801and a picture processing device804. Differing from the receiver1201in the picture communication system according to Embodiment 11, the receiver1301is further comprised of the decoding device Ha (800) according to Embodiment 8 in lieu of the first general-purpose decoding device806and the second general-purpose decoding device807. At this time, it is a premise that the number of sheets of the B picture and the specification of the B picture reference region restrictions are decided identically beforehand, in the encoding device A (100) and the decoding device Ha (800)

A group of the encoding device A (100) and the decoding device Ha (800) can be replaced with a group of the encoding device C (300) according to Embodiment 3 and the decoding device Hb (810) according to Embodiment 8b, or a group of the encoding device D (400) according to Embodiment 4 and the decoding device Hb (810) according to Embodiment 8b. It is a premise also in these cases that the specification of the P picture reference region restrictions of the encoding device is decided beforehand.

The decoding device Ha (800) of the receiver1301decodes the first encoded stream and the second encoded stream according to the number of the B picture and the specification of B picture reference region restrictions decided beforehand.

The picture communication system according to Embodiment 12 is a system in which the receiving side knows the coding restrictions of the transmitting side beforehand, and the encoding device and the decoding device according to the present application are arranged on both the transmitting side and the receiving side. According to the present embodiment, there are following effects. It is possible to reduce the zone of the external memory and the capacity of the external memory, in the decoding device on the receiving side, as well as in the encoding device on the transmitting side.

In the case of the group of the encoding device A (100) and the decoding device Ha (800), no memory access in the B-picture encoding and the B-picture decoding is necessary.

In the case of the group of the encoding device C (300) and the decoding device Hb (810), no external memory access is necessary on the receiving side. It is also possible to reduce the system cost further.

In the case of the group of the encoding device D (400) and the decoding device Hb (810), no external memory access is necessary in all the transmission and the reception. It is also possible to reduce the system cost further.

As described above, the invention accomplished by the present inventors has been concretely explained based on the embodiments. However, it cannot be overemphasized that the present invention is not restricted to the embodiments as described above, and it can be changed variously in the range which does not deviate from the gist.

For example, the encoding device, the decoding device, other devices, the controller may be realized, for example by the dedicated hardware over a semiconductor integrated circuit, or they may be realized as a part of the function of the software operating on a processor.