Moving image decoding apparatus, moving image coding apparatus, moving image decoding circuit, and moving image decoding method

A moving image decoding apparatus which enables reduction in the memory bandwidth and the memory access latency for the motion compensation filter coefficients for use in inter-picture prediction involving motion compensation using variable coefficients includes: a decoding unit (101) which decodes, from a coded stream, a plurality of motion compensation filter coefficients; a memory (109) for holding the motion compensation filter coefficients included in the coded stream; a filter coefficient storage unit (103) for holding at least one of the motion compensation filter coefficients which is required for the motion compensation; a motion compensation unit (107) which performs motion compensation using the required motion compensation filter coefficient held in the filter coefficient storage unit; and a filter coefficient transfer control unit (102) which writes, in the memory, the motion compensation filter coefficients decoded by the decoding unit, and transfers the required motion compensation filter coefficient from the memory to the filter coefficient storage unit, only when the required coefficient is not yet stored therein.

TECHNICAL FIELD

The present invention relates to image decoding apparatuses, image coding apparatuses, image decoding circuits, and image decoding methods, and particularly to an image decoding apparatus, an image coding apparatus, an image decoding circuit, and an image decoding method using filter coefficients that are used in inter-picture prediction involving motion compensation using variable coefficients in order to decode a coded video stream.

BACKGROUND ART

Recently, there have been widely-used standards for video compression techniques. Examples of such standards include H.261 and H.263 by the ITU-T (International Telecommunication Union Telecommunication Standardization Sector), MPEG (Moving Picture Experts Group)-1, MPEG-2, MPEG-4, etc. by the ISO/IEC (International Organization for Standardization/International Electrotechnical Commission), and H.264/MPEG-4 AVC (Advanced Video Coding) by the JVT (Joint Video Team) as a joint team of the ITU-T and the MPEG. Furthermore, the next generation video compression technique is now being studied by the ITU-T, the ISO/IEC, etc.

In general, one of the important elements of a video compression technique is inter-picture prediction involving motion compensation intended to compress the amount of information by reducing temporal redundancies between plural consecutive pictures that make up a video. Here, the inter-picture prediction involving motion compensation is a coding method involving (i) detecting the amount and direction of a motion in a reference picture located forward or backward of a current picture that is to be coded in units of a macroblock or a sub-macroblock (hereinafter also referred to as a “macroblock or the like”), (ii) generating a prediction image, and (iii) coding a difference value between the prediction image and the current picture. It is to be noted that the information indicating how much and to what direction a macroblock or the like in the current picture to be coded is moved in the reference picture located forward or backward of the current picture is referred to as a motion vector. In addition, a picture to be referred to at this time is referred to as a reference picture.

In the decoding of a video stream coded using motion-compensation inter-picture prediction, decoded pictures are held in a frame memory and used as reference pictures in the decoding of the following pictures to be decoded.

In addition, prediction images of macroblocks coded using motion-compensation inter-picture prediction are generated as described below. First, a motion vector in the coded video stream is decoded. A reference pixel area indicated by the motion vector is obtained from a frame memory holding reference pictures. A prediction image is generated by, as necessary, filtering the reference pixel area. In order to generate a prediction image having a sub-pixel accuracy, integer pixels are filtered.

For example, a prediction image having a ½ pixel accuracy is generated using a 6-tap FIR filter (filter coefficients: 1, −5, 20, 20, −5, and 1) that has fixed filter coefficients defined in the 11.264 standard. Next, a prediction image having a ¼ pixel accuracy is generated using a 2-tap average-value filter (filter coefficients: ½ and ½) that has fixed filter coefficients defined in the standard. At this time, filter coefficients that have same values (fixed values) irrespective of image characteristics are used to generate prediction images having a sub-pixel accuracy.

On the other hand, a motion compensation technique (hereinafter, also referred to as “motion compensation using variable coefficients”) has been proposed (for example, see NPLs 1, 2, and 3). The motion compensation technique is intended to adaptively change filter coefficients for generating prediction images having a sub-pixel accuracy according to the image characteristics in order to achieve a higher coding efficiency. This motion compensation using variable coefficients has been proposed by the ITU-T, ISO/IEC, etc. as the next generation video compression technique, more specifically as the next generation motion-compensation inter-picture prediction technique.

For example, NPL 1 has proposed a technique for adaptively changing filter coefficients that are used to generate prediction images having a sub-pixel accuracy according to image characteristics, instead of using conventional filter coefficients having fixed values. In addition, for example, NPL 2 has proposed a technique for performing filtering when generating prediction images using, for each of values in the decimal part of each of motion vectors, a different filter coefficient defined in a coded stream. In addition, NPL 3 has proposed a technique for switching filter coefficients that are used for macroblock-based filtering in inter-picture prediction involving motion compensation, focusing on a fact that the optimum filter coefficient varies depending on the places even within a picture.

CITATION LIST

Patent Literature

Non Patent Literature

SUMMARY OF INVENTION

Technical Problem

However, the inter-picture prediction involving motion compensation proposed as the next generation video compression technique requires frequently selecting filtering coefficients from among numerous candidate filtering coefficients and using the selected filtering coefficients in the filtering for generating prediction images. For this reason, in general, it is predicted that an extremely large memory capacity, memory bandwidth, and memory access latency are required for a memory for holding such filter coefficients.

The motion-compensation decoding of a video stream coded using motion-compensation inter-picture prediction requires storing decoded pictures in a frame memory and reading out reference pixel areas from the frame memory. For this reason, in the case of using the motion-compensation inter-picture prediction proposed as the next generation video compression technique, in general, it is predicted that an extremely large memory bandwidth and memory access latency are required for the frame memory because of the need of the storage of decoded pictures and reading-out of reference pixel areas and access for, for example, the display of the decoded pictures.

To prevent this, various techniques have been proposed so far as methods for reducing the memory capacity, memory bandwidth, and/or memory access latency required for a frame memory (for example, PTL 1). For example, PTL 1 transfers a plurality of reference pixel areas from a frame memory to a local memory all together when performing motion-compensation inter-picture prediction in the case where the collective transfer of the reference pixel areas increases the coding efficiency. This eliminates the necessity that each of reference areas which is commonly used for a plurality of blocks should be transferred each time one of the blocks is processed. Thus, it is possible to reduce the number of times of access to the frame memory. In this way, it is possible to reduce the memory bandwidth, memory access latency, and the number of processing cycles for the frame memory.

However, PTL 1 merely discloses a configuration for reducing the memory bandwidth and memory access latency for the frame memory storing reference images. In other words, PTL 1 does not disclose descriptions of memory access for filter coefficients that are used in the inter-picture prediction involving motion compensation using variable coefficients proposed as the next generation video compression technique. Accordingly, with the configuration disclosed in PTL 1, it is impossible to reduce the memory bandwidth and memory access latency for the filter coefficients in the case of performing the inter-picture prediction involving motion compensation using variable coefficients.

In view of the aforementioned situations, the present invention has been conceived with an aim to provide a moving image decoding apparatus, a moving image coding apparatus, a moving image decoding circuit, and a moving image decoding method which enable reduction in the memory bandwidth and memory access latency for motion compensation filter coefficients (filter coefficients of a motion compensation filter) which are used to perform inter-picture prediction involving motion compensation using variable coefficients.

Solution to Problem

In order to solve the aforementioned conventional problem, a moving image coding apparatus according to the present invention performs motion-compensation decoding involving motion compensation of a stream of motion-compensation coded moving images, and includes: a decoding unit configured to decode, from the stream, a plurality of motion compensation filter coefficients (filter coefficients of a motion compensation filter) for use in the motion-compensation decoding; a memory for holding the motion compensation filter coefficients included in the stream decoded by the decoding unit; a filter coefficient storage unit for holding at least one of the motion compensation filter coefficients required for the motion compensation; a motion compensation unit configured to perform the motion compensation using the at least one motion compensation filter coefficient held in the filter coefficient storage unit; and a transfer control unit configured to write, to the memory, the motion compensation filter coefficients decoded by the decoding unit, and, only when the filter coefficient storage unit does not hold the at least one motion compensation filter coefficient, transfer the at least one motion compensation filter coefficient from the memory to the filter coefficient storage unit.

With this structure, it is possible to implement a moving image decoding apparatus capable of reducing the memory bandwidth and reducing the memory access latency for motion compensation filter coefficients used to perform inter-picture prediction involving motion compensation using variable coefficients.

In order to solve the aforementioned conventional problem, a moving image coding apparatus according to the present invention performs motion-compensation coding using a current image to be coded and a reference image and includes: a generation unit configured to generate a plurality of motion compensation filter coefficients for use in the motion-compensation coding; a memory for holding the motion compensation filter coefficients generated by the generation unit; a filter coefficient storage unit for holding at least one of the motion compensation filter coefficients held in the memory, the at least one motion compensation filter coefficient being required for the motion compensation; a motion estimation unit configured to generate a prediction image by performing motion compensation between the current image to be coded and the reference image, using the at least one motion compensation filter Coefficient held in the filter coefficient storage unit; and a transfer unit configured to write, to the memory, the motion compensation filter coefficients generated by the generation unit, and transfer the at least one motion is compensation filter coefficient from the memory to the filter coefficient storage unit, only when the filter coefficient storage unit does not hold the at least one motion compensation filter coefficient.

With this structure, it is possible to implement a moving image coding apparatus capable of reducing the memory bandwidth and reducing the memory access latency for motion compensation filter coefficients used to perform inter-picture prediction involving motion compensation using variable coefficients.

It is to be noted that the present invention can be implemented not only as an apparatus but also as an integrated circuit including the processing units of the apparatus and as a method including the steps corresponding to the processing units of the apparatus.

Advantageous Effects of Invention

According to the present invention, it is possible to implement an image decoding apparatus, an image coding apparatus, an image decoding circuit, and an image decoding method for enabling reduction in the memory bandwidth and reduction in the memory access latency for motion compensation filter coefficients used to perform inter-picture prediction involving motion compensation using variable coefficients.

DESCRIPTION OF EMBODIMENTS

FIG. 1is a structural diagram of a decoding apparatus according to Embodiment 1 of the present invention. Each ofFIG. 2AandFIG. 2Bis an illustration showing schematic structures of a coded stream generated according to a standard of a video compression technique.FIG. 2Cis an illustration showing an example of a coded stream according to the present invention.FIG. 3is an example of information held in a management table for storage states of filter coefficients.

A decoding apparatus100as shown inFIG. 1is a moving image decoding apparatus which performs motion-compensation decoding of a coded stream generated by performing motion compensation of moving images. The decoding apparatus100includes a decoding unit101, a filter coefficient transfer control unit102, a filter coefficient storage unit103, a management table for storage states of filter coefficients104, a reference image transfer control unit105, a reference image storage unit106, a motion compensation unit107, an adder108, and a memory109.

The decoding unit101decodes, from the coded stream, at least a plurality of motion compensation filter coefficients for use in the motion-compensation decoding. More specifically, the decoding unit101has a function of decoding a coded stream generated according to a standard of a moving image compression technique, and outputting at least header information and a prediction error signal.

Here, the coded stream according to the standard of the moving image compression technique is explained with reference toFIGS. 2A and 2B. As shown inFIG. 2A, a sequence of images (video) in the coded stream has a hierarchical structure. A sequence of plural pictures (or a GOP that is a Group Of Pictures) is given here as an example. Each of the pictures that make up the sequence is divided into slices, and is further divided into macroblocks each composed of 16×16 pixels. It is to be noted that a picture may not be divided into slices. The decoding apparatus100performs decoding in units of a slice or a macroblock.

As shown inFIG. 2B, these units are coded hierarchically in the coded stream. The coded stream is configured to include a sequence header for controlling a sequence, a picture header for controlling a picture, a slice header for controlling a slice, and macroblock data. The macroblock data is classified into (i) coded information of a marcroblock type, an intra-picture prediction (intra prediction) mode, motion vector information, quantized parameters, and (ii) coefficient information corresponding to each pixel data. In the H.264 standard, a sequence header is referred to as Sequence Parameter Set (SPS) and a picture header is referred to as Picture Parameter Set (PPS).

For example, a structure as shown inFIG. 2Cis also possible. In other words, it is assumed that a GOP or a picture header of each of the pictures in the sequence includes all of (i) motion compensation filter coefficients for all of the pictures and (ii) pieces of information (hereinafter referred to as “motion compensation filter coefficient ID) each identifying the motion compensation filter coefficient for use in motion compensation of a current corresponding one of the slices or the macroblocks to be decoded. Furthermore, it is assumed that a slice header in a picture includes only pieces of motion compensation filter coefficient ID. Here, the motion compensation filter coefficient ID corresponds identification information in the CLAIMS according to the present application.

The motion compensation filter coefficients and the pieces of motion compensation filter coefficient ID may be provided in units of a GOP or a sequence, instead of in units of a picture. Alternatively, motion compensation filter coefficients and the pieces of motion compensation filter coefficient ID may be included in units of a slice, and the pieces of motion compensation filter coefficient ID may be included in units of a macroblock. In other words, the aforementioned units may be arbitrarily combined, or another combination of other units may be used as such a unit instead.

The filter coefficient transfer control unit102corresponds to a transfer control unit in the CLAIMS of the present application. The filter coefficient transfer control unit102writes, to the memory109, a plurality of motion compensation filter coefficients decoded by the decoding unit101, and transfers at least one of the motion compensation filter coefficients from the memory109to the filter coefficient storage unit103, only when the filter coefficient storage unit103does not hold the at least one motion compensation filter coefficient required for use in the current motion compensation process. More specifically, the filter coefficient transfer control unit102has a function of writing, to the memory109, the motion compensation filter coefficients for use in the motion compensation processes. The motion compensation filter coefficients are defined in the coded stream. In addition, the filter coefficient transfer control unit102has a function of referring to information held in the management table for storage states of filter coefficients104, based on the motion compensation filter coefficient ID that is information indicating the motion compensation filter coefficient for use in a current motion compensation process performed by the decoding unit101. The filter coefficient transfer control unit102checks whether or not the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID is held in the filter coefficient storage unit103, based on the information in the management table for storage states of filter coefficients104.

For example, when the filter coefficient transfer control unit102finds out that the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID is not held in the filter coefficient storage unit103, the filter coefficient transfer control unit102reads out, from the memory109, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, and writes the read-out motion compensation filter coefficient in the filter coefficient storage unit103(transfers it thereto). On the other hand, when the filter coefficient transfer control unit102finds out that the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID is held (stored) in the filter coefficient storage unit103, the filter coefficient transfer control unit102does not read out, from the memory109, the motion compensation filter coefficient (does not transfer it thereto).

It is to be noted that the filter coefficient transfer control unit102may issue a write instruction to the decoding unit101so that the decoding unit101writes the motion compensation filter coefficient to the memory109. Likewise, the filter coefficient transfer control unit102may issue a read instruction to the filter coefficient storage unit103so that the filter coefficient storage unit103reads out, from the memory109, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, and holds the read out motion compensation filter coefficient.

The filter coefficient storage unit103is for holding the required motion compensation filter coefficients for use in the motion compensation among the plurality of motion compensation filter coefficients held in the memory109. More specifically, the filter coefficient storage unit103has a function of holding at least two kinds of motion compensation filter coefficients transferred from the memory109, and a function of setting, in the motion compensation unit107, the motion compensation filter coefficients indicated by the ID of the motion compensation filter coefficients.

The management table for storage states of filter coefficients104corresponds to a storage state management unit in the CLAIMS of the present application. The management table for storage states of filter coefficients104manages information indicating whether or not the filter coefficient storage unit103holds the motion compensation filter coefficients indicated by the motion compensation filter coefficient ID. More specifically, the management table for storage states of filter coefficients104has a function of receiving, as inputs, the pieces of motion compensation filter coefficient ID, and outputs, to the filter coefficient storage unit103, the information indicating whether or not the filter coefficient storage unit103holds each of the motion compensation filter coefficients indicated by the motion compensation filter coefficient ID. The management table for storage states of filter coefficients104holds, for example, information as shown inFIG. 3. More specifically, the management table for storage states of filter coefficients104holds all the pieces of the motion compensation filter coefficient ID included in the header information such as sequence headers and picture headers in the coded stream to be decoded by the decoding unit101. Furthermore, the management table for storage states of filter coefficients104holds, as the storage states, information indicating whether or not the filter coefficient storage unit103holds the motion compensation filter coefficients indicated by the respective pieces of motion compensation filter coefficient ID.

The reference image transfer control unit105has a function of reading out, from the memory109, reference pixels required for the motion compensation, based on the header information including motion vectors and reference picture information output by the decoding unit101, and writing the reference image to the reference image storage units106.

It is to be noted that the reference image transfer control unit105may issue a read instruction to the reference image storage unit106based on the header information including the motion vectors and reference picture information output by the decoding unit101, so that the reference image storage unit106reads out, from the memory109, the reference pixels required for the motion compensation and holds the read-out reference pixels.

The reference image storage unit106has a function of holding the reference pixels transferred from the memory109.

The motion compensation unit107performs motion compensation using at least the motion compensation filter coefficients held in the filter coefficient storage unit103. More specifically, the motion compensation unit107has a function of obtaining (i) header information including motion vectors and the ID of motion compensation filter coefficients output by the decoding unit101, (ii) reference pixels required for motion compensation indicated by the motion vectors held in the reference image storage unit106, and (iii) motion compensation filter coefficients indicated by the motion compensation filter coefficient ID held in the filter coefficient storage unit103. The motion compensation unit107has a function of generating a prediction image by performing motion compensation using these pieces of information and outputting these pieces of information to the adder108.

The adder108has a function of adding the prediction error signal output by the decoding unit101and the prediction image output by the motion compensation unit107, and outputs the outcome as the decoded image, and a function of transferring the decoded image to the memory109.

The memory109is for holding at least the plurality of motion compensation filter coefficients included in the bit stream decoded by the decoding unit101. More specifically, the memory109has a function of holding the motion compensation filter coefficients and reference pictures that are referred to by the motion compensation unit107. It is to be noted that the memory109may hold only the motion compensation filter coefficients. In such a case, it is only necessary that the decoding apparatus100additionally includes a frame memory for holding the reference pictures that are referred to by the motion compensation unit107.

The decoding apparatus100is structured as described above.

Next, a description is given of decoding operations performed by the decoding apparatus100.FIG. 4is a flowchart of the decoding operations performed by the decoding apparatus according to Embodiment 1 of the present invention.

As shown inFIG. 4, in the decoding apparatus100, the decoding unit101decodes coded header information in an input coded stream (S101), and outputs, as decoded header information, at least motion compensation filter coefficients. Next, the filter coefficient transfer control unit102writes, to the memory109, all the motion compensation filter coefficients decoded, in Step S101, by the decoding unit101(S102).

Next, the decoding unit101determines whether a current image to be decoded has a picture type of P, B or I (S103). When the current image has a picture type of P or B (the case of P or B in S103), the decoding unit101determines that motion compensation is required, decodes coded header information in the coded stream and a prediction residual signal (S104), and outputs, as decoded header information, at least the motion compensation filter coefficient ID, the motion vector information, and the prediction residual signal. On the other hand, when the current image has a picture type of I (the case of I in S103), the decoding unit101determines that motion compensation is not required, proceeds to S114and performs intra-picture decoding, and then proceeds to S113.

Next, the filter coefficient transfer control unit102checks whether or not the filter coefficient storage unit103holds the motion compensation filter coefficients indicated by the pieces of motion compensation filter coefficient ID decoded in S104, with reference to the management table for storage states of filter coefficients (S105).

When the filter coefficient storage unit103does not hold the motion compensation filter coefficients indicated by the motion compensation filter coefficient ID (the case of No in S105), the filter coefficient transfer control unit102reads out, from the memory109, the motion compensation filter coefficients indicated by the motion compensation filter coefficients ID. Next, the filter coefficient transfer control unit102writes the motion compensation filter coefficients in an area which is of the filter coefficient storage unit103and holds the motion compensation filter coefficient read out from the memory109at earliest time (S106). Next, the filter coefficient transfer control unit102updates the management table for storage states of filter coefficients104(S107). More specifically, the filter coefficient transfer control unit102updates, to “held”, the storage state which is of the read out motion compensation filter coefficient and is indicated in the management table for storage states of filter coefficients104, and updates the storage state of the erased motion compensation filter coefficient to “not held”.

On the other hand, when the filter coefficient-storage unit103holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of Yes in S105), the filter coefficient transfer control unit102proceeds to S108without reading out the motion compensation filter coefficient from the memory109.

Next, the reference image transfer control unit105obtains a reference image (S108). More specifically, the reference image transfer control unit105reads out, from the memory109, the reference image for use in the motion compensation, based on the header information including the motion vector and the reference picture information output by the decoding unit101in S104. Next, the reference image transfer control unit105writes the read-out reference image in the reference image storage unit106.

Next, the motion compensation unit107sets the motion vector output by the decoding unit101in S104(S109). Next, the motion compensation unit107determines the motion compensation filter coefficient for use in the motion compensation based on the motion compensation filter coefficient ID and the motion vector output by the decoding unit101in S104, and reads out the motion compensation filter coefficient from the filter coefficient storage unit103(S110). Next, the motion compensation unit107reads out the reference image held in the reference image storage unit106, generates a prediction image by performing motion compensation, and outputs the prediction image to the adder108(S111).

Next, the adder108adds the prediction image output by the motion compensation unit107in S111and the prediction residual signal output by the decoding unit101in S104, and outputs the outcome (S112).

Next, the decoding unit101determines whether or not all of the motion compensation blocks that need to be motion-compensation using the motion compensation filter coefficients decoded in S101(S113) are already decoded. When the answer is YES (Y in S113), the decoding unit101completes the decoding. On the other hand, when there remains any motion compensation block that is not yet decoded (N in S113), the decoding unit101returns to S103and repeats the following processing.

In this way, the decoding apparatus100performs the decoding operations.

It is to be noted that, when the decoding apparatus100decodes the images (pictures) included in a sequence or a GOP, the decoding apparatus100writes, in S102, all of the motion compensation filter coefficients decoded by the decoding unit101in Step S101to the memory109without writing all of the motion compensation filter coefficients in the management table for storage states of filter coefficients104. By repeating the processes from S103to S113, some of the motion compensation filter coefficients are is held in the filter coefficient storage unit103.

According to Embodiment 1, when it is found, with reference to the management table for storage states of filter coefficients104, that the filter coefficient storage unit103stores the at least one motion compensation filter coefficient indicated by motion compensation filter coefficient ID, the at least one motion compensation filter coefficient is not read out from the memory109. For this reason, it is possible to reduce the number of times of access to the memory109. For this reason, it is possible to reduce the memory bandwidth and the memory access latency related to the motion compensation filter coefficient.

In the above case, when the filter coefficient storage unit103does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, the filter coefficient transfer control unit102reads out the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID from the memory109. The above description is given of a case of writing the motion compensation filter coefficient to an area which is of the filter coefficient storage unit103and stores the motion compensation filter coefficient read out from the frame memory at the earliest time. However, the area to which the motion compensation filter coefficient is written is not limited thereto. For example, the motion compensation filter coefficient may be written to an area which stores the motion compensation filter coefficient read out from the memory most recently, or may be written to an area which stores a filter coefficient and is selected at random. Any area selection method is possible as long as the method enables reduction in the access to the memory109and reduction in the capacity of the filter coefficient storage unit103.

The above description is given of a case where the motion compensation filter coefficient ID is header information different from a motion vector, but the motion compensation filter coefficient ID is not limited thereto. For example, the motion vector may be the motion compensation filter coefficient ID. Alternatively, information including the motion vector not only the header information different from the motion vector may be the motion compensation filter coefficient ID.

The above description is given of a case where the motion vector is set in the motion compensation unit107, but the decimal part of the motion vector may be set.

In the above description, the decoding apparatus100includes the decoding unit101, the filter coefficient transfer control unit102, the filter coefficient storage unit103, the management table for storage states of filter coefficients104, the reference image transfer control unit105, the reference image storage unit106, the motion compensation unit107, the adder108, and the memory109. However, the structure of the decoding apparatus100is not limited thereto. As shown inFIG. 5, it is only necessary for the decoding apparatus100to include a decoding apparatus unit10as its essential element. Specifically, it is only necessary for the decoding apparatus100to include the decoding apparatus unit10including the decoding unit101, the filter coefficient transfer control unit102, the filter coefficient storage unit103, the memory109, and the motion compensation unit107.

More specifically, the decoding apparatus unit10may be a moving image decoding apparatus which performs motion-compensation decoding of a coded stream generated by performing motion compensation of a moving image and may include: a decoding unit101which decodes, from the coded stream, a plurality of motion compensation filter coefficients for use in the motion-compensation decoding; a memory109for holding the motion compensation filter coefficients included in the coded stream decoded by the decoding unit101; a filter coefficient storage unit103for holding at least one motion compensation filter coefficient required for the motion compensation from among the plurality of motion compensation filter coefficients held in the memory; a motion compensation unit107which performs motion compensation using the at least one motion compensation filter coefficient held in the filter coefficient storage unit103; and a filter coefficient transfer control unit102which writes, to the memory109, the plurality of motion compensation filter coefficients decoded by the decoding unit101, and transfers the at least one required motion compensation filter coefficient from the memory109to the filter coefficient storage unit103only when the filter coefficient storage unit103does not hold the at least one required motion compensation filter coefficient.

In the decoding apparatus100including the decoding apparatus unit10as its essential element, the filter coefficient transfer control unit102transfers, to the filter coefficient storage unit, at least one of the motion compensation filter coefficients stored in the memory109and for use in the motion-compensation decoding, and the filter coefficient storage unit stores the transferred motion compensation filter coefficient. In this way, it is possible to reduce the number of times of direct access to the memory109.

FIG. 6is a structural diagram of a decoding apparatus according to Embodiment 2 of the present invention. Each ofFIG. 7AandFIG. 7Bis an example of information held in the management table for reference history of filter coefficients. InFIG. 6, the same structural elements as those inFIG. 1are assigned with the same reference signs, and the same descriptions thereof are not repeated.

A decoding apparatus200as shown inFIG. 6includes a decoding unit101, a filter coefficient transfer control unit102, a filter coefficient storage unit103, a management table for storage states of filter coefficients104, a reference image transfer control unit105, a reference image storage unit106, a motion compensation unit107, an adder108, a memory109, and a management table for reference history of filter coefficients201. Unlike the decoding apparatus100according to Embodiment 1, the decoding apparatus200as shown inFIG. 6includes the management table for reference history of filter coefficients201.

The management table for reference history of filter coefficients201corresponds to a reference management unit in the CLAIMS of the present application. The management table for reference history of filter coefficients201manages, for each of the motion compensation filter coefficients included in the coded stream, use history information indicating the number of times of reference from the start of the decoding. More specifically, the management table for reference history of filter coefficients201has a function of receiving, as inputs, the pieces of motion compensation filter coefficient ID, and providing, as an output, the number of times of reference to the motion compensation filter coefficient indicated by a corresponding one of the pieces of motion compensation filter coefficient ID from the start of the decoding. The management table for reference history of filter coefficients201holds, for example, information as shown inFIG. 7A. More specifically, the management table for reference history of filter coefficients201holds, as the filter coefficient ID, the motion compensation filter coefficient ID included in, for example, the header information of the coded stream to be decoded by the decoding unit101. Furthermore, the management table for reference history of filter coefficients201holds, as the number of times of reference by which each of the motion compensation filter coefficients identified by a corresponding one of the motion compensation filter coefficient ID has been referred to in the stream so far.

It is to be noted that the management table for reference history of filter coefficients201may have a function of receiving, as an input, the motion compensation filter coefficient ID, and providing, as an output, a reference order indicating how many times before the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID was referred to in the sequential motion compensation processing currently being performed by the motion compensation unit107. In this case, the management table for reference history of filter coefficients201holds, for example, information as shown inFIG. 7B. More specifically, the management table for reference history of filter coefficients201holds, as the filter coefficient ID, the motion compensation filter coefficient ID included in, for example, the header information of the coded stream to be decoded by the decoding unit101. Furthermore, the management table for reference history of filter coefficients201holds, as the reference order, information (a value) indicating how many times before the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID was referred to last in the sequential motion compensation processing that is currently being performed by the motion compensation unit107.

Next, a description is given of decoding operations performed by the decoding apparatus200configured as mentioned above.FIG. 8is a flowchart of the decoding operations performed by the decoding apparatus according to Embodiment 2 of the present invention.

As shown inFIG. 8, in the decoding apparatus200, the decoding unit101decodes coded header information in an input coded stream (S201), and outputs, as decoded header information, at least motion compensation filter coefficients. Next, the filter coefficient transfer control unit102writes, to the memory109, all the motion compensation filter coefficients decoded by the decoding unit101(S202).

Next, the decoding unit101determines whether a current image to be decoded has a picture type of P, B or I (S203). When the current image has a picture type of P or B (the case of P or B in S203), the decoding unit101determines that motion compensation is required, decodes the coded header information in the coded stream and a prediction residual signal (S204), and outputs, as the decoded header information, at least the motion compensation filter coefficient ID, the motion vector information, and the prediction residual signal. On the other hand, when the current image has a picture type of I (the case of I in S203), the decoding unit101determines that motion compensation is not required, proceeds to S215and performs intra-picture decoding, and then proceeds to S214.

Next, the filter coefficient transfer control unit102checks whether or not the filter coefficient storage unit103holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID decoded in S204, with reference to the management table for storage states of filter coefficients104(S205).

When the filter coefficient storage unit103does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of No in S205), the filter coefficient transfer control unit102reads out, from the memory109, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID. In addition, the filter coefficient transfer control unit102writes the motion compensation filter coefficient to an area which is of the filter coefficient storage unit103and stores the motion compensation filter coefficient referred to least frequently so far (S206). Here, the filter coefficient transfer control unit102identifies the motion compensation filter coefficient referred to least frequently with reference to the management table for reference history of filter coefficients201.

Next, the filter coefficient transfer control unit102increments the number of times of reference of the motion compensation filter coefficient referred to in the management table for reference history of filter coefficients (S207).

In addition, the filter coefficient transfer control unit102updates the management table for storage states of filter coefficients104(S208). More specifically, the filter coefficient transfer control unit102updates, to “held”, the storage state which is of the read out motion compensation filter coefficient and is indicated in the management table for storage states of filter coefficients104, and updates the storage state of the erased motion compensation filter coefficient to “not held”.

On the other hand, when the filter coefficient storage unit103holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of Yes in S205), the filter coefficient transfer control unit102proceeds to S209without reading out the motion compensation filter coefficient from the memory109.

Next, the reference image transfer control unit105obtains a reference image (S209). More specifically, the reference image transfer control unit105reads out, from the memory109, the reference image for use in the motion compensation, based on the header information including the motion vector and the reference picture information output by the decoding unit101in S204. Next, the reference image transfer control unit105writes the read-out reference image in the reference image storage unit106.

Next, the motion compensation unit107sets the motion vector output by the decoding unit101in S204(S210). Next, the motion compensation unit107determines the motion compensation filter coefficient for use in the motion compensation based on the motion compensation filter coefficient ID and the motion vector output by the decoding unit101in S204, and reads out the motion compensation filter coefficient from the filter coefficient storage unit103(S211). Next, the motion compensation unit107reads out the reference image held in the reference image storage unit106, generates a prediction image by performing motion compensation, and outputs the prediction image to the adder108(S212).

Next, the adder108adds the prediction image output by the motion compensation unit107in S212and the prediction residual signal output by the decoding unit101in S204, and outputs the outcome (S213).

Next, the decoding unit101determines whether or not all of the motion compensation blocks need to be motion-compensation using the motion compensation filter coefficients decoded in S201(S214). When the answer is YES (Y in S214), the decoding unit101completes the decoding. On the other hand, when there remains any motion compensation block that is not yet decoded (N in S214), the decoding unit101returns to S203and repeats the following processing.

In this way, the decoding apparatus200performs the decoding operations.

As described above, according to Embodiment 2, the motion compensation filter coefficient referred to least frequently is identified with reference to the management table for reference history of filter coefficients201. In addition, the filter coefficient transfer control unit102writes the motion compensation filter coefficient to an area which is of the filter coefficient storage unit103and stores the motion compensation filter coefficient referred to least frequently so far. Accordingly, the motion compensation filter coefficient unlikely to be referred to is not held in the filter coefficient storage unit103. Thus, it is possible to hold the motion compensation filter coefficient highly likely to be referred to frequently in the filter coefficient storage unit103. As a result, it is possible to reduce the number of times of access to the memory109. For this reason, it is possible to reduce the memory bandwidth and the memory access latency related to the motion compensation filter coefficient.

In the above description, when the filter coefficient storage unit103does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, the motion compensation filter coefficient referred to least frequently is identified with reference to the management table for reference history of filter coefficients201. In the above description, the filter coefficient transfer control unit102reads out, from the memory, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, and writes the read-out motion compensation filter coefficient to an area which is of the filter coefficient storage unit103and stores the motion compensation filter coefficient referred to least frequently. However, the area to which the motion compensation filter coefficient is written is not limited thereto. For example, the motion compensation filter coefficient may be written to: an area that stores the motion compensation filter coefficient used least frequently recently (a predetermined past period); an area that stores the motion compensation filter coefficient used most frequently recently; or an area that stores the motion compensation filter coefficient referred to most frequently so far. Any area selection method is possible as long as the method enables reduction in the access to the memory109and reduction in the capacity of the filter coefficient storage unit103.

The above description is given of a case where the motion compensation filter coefficient ID is header information different from a motion vector, but the motion compensation filter coefficient ID is not limited thereto. For example, the motion vector may be the motion compensation filter coefficient ID, or information including the motion vector not only the header information different from the motion vector may be the motion compensation filter coefficient ID.

The above description is given of a case where the motion vector is set in the motion compensation unit107, but the decimal part of the motion vector may be set.

FIG. 9is a block diagram showing a structure of a decoding apparatus according to Embodiment 3 of the present invention. Each ofFIG. 10andFIG. 11is an example of information held in a management table for statistical information of filter coefficients. InFIG. 9, the same structural elements as those inFIG. 1are assigned with the same reference signs, and the same descriptions thereof are not repeated.

A decoding apparatus300as shown inFIG. 9includes a decoding unit101, a filter coefficient transfer control unit102, a filter coefficient storage unit103, a management table for storage states of filter coefficients104, a reference image transfer control unit105, a reference image storage unit106, a motion compensation unit107, an adder108, a memory109, a pre-decoding unit301, and a management table for statistical information of filter coefficients302. Unlike the decoding apparatus100according to Embodiment 1, the decoding apparatus300as shown inFIG. 9includes the management table for statistical information of filter coefficients302.

The pre-decoding unit301decodes, from a coded stream, at least one of the motion compensation filter coefficients included in a coded stream, prior to the decoding by the decoding unit101. More specifically, the pre-decoding unit301decodes at least one or all of the coded stream generated according to the standard of the video compression technique, prior to the decoding by the decoding unit101by at least one bit in the coded stream. Here, the pre-decoding unit301has a function of outputting at least the motion compensation filter coefficient ID.

It is to be noted that the decoding apparatus300may additionally include a CABAC decoding unit which decodes arithmetic codes such as CABAC (Context-based Adaptive Binary Arithmetic Coding), and the pre-decoding unit301may be included in the CABAC decoding unit. In addition, the pre-decoding unit301may be provided at a pre-stage of the decoding unit101. In other words, it is only necessary for the pre-decoding unit301to decode the motion compensation filter coefficient ID and output it to the management table for statistical information of filter coefficients302without decoding an image, prior to the decoding by the decoding unit101by at least one bit in the coded stream.

The management table for statistical information of filter coefficients302corresponds to a statistical information management unit in the CLAIMS of the present application. The management table for statistical information of filter coefficients302manages the use state of each of the motion compensation filter coefficients decoded by the pre-decoding unit301and to be decoded by the decoding unit101. More specifically, the management table for statistical information of filter coefficients302has a function of receiving, as an input, the motion compensation filter coefficient ID output by the pre-decoding unit301, and providing, as an output, the number of times of use of the motion compensation filter indicated by each of the motion compensation filter coefficient ID included in the coded stream to be decoded by the decoding unit101.

The management table for statistical information of filter coefficients302holds, for example, information as shown inFIG. 10. More specifically, the management table for statistical information of filter coefficients302holds, as the filter coefficient ID, the motion compensation filter coefficient included in the header information of the coded stream decoded by the pre-decoding unit301. In addition, the management table for statistical information of filter coefficients302holds, as the number of times of future reference to motion compensation filter coefficients in the stream, the number of times of future reference to motion compensation filter coefficients up to a reference to the motion compensation filter coefficient that is identified by the motion compensation filter coefficient ID and is referred to in the decoding by the decoding unit101. In addition, the management table for statistical information of filter coefficients302holds, for example, information as shown inFIG. 11. More specifically, the management table for statistical information of filter coefficients302holds, as the filter coefficient ID, the motion compensation filter coefficient ID included in the header information of the coded stream decoded by the pre-decoding unit301. In addition, the management table for statistical information of filter coefficients302may hold, as information indicating at which block (how many times ahead) in the stream the motion compensation filter coefficient identified by the motion compensation filter coefficient ID is to be referred to in the decoding by the decoding unit101.

The decoding apparatus300is structured as described above. More specifically, by managing the occurrence frequency of the motion compensation filter coefficient ID to be used for decoding by the decoding unit101using the management table for statistical information of filter coefficients302, it is possible to check, in advance, the occurrence probability of the motion compensation filter coefficient ID to be used for the decoding by the decoding unit101. Accordingly, it is possible to store the motion compensation filter coefficient having a high occurrence probability into the filter coefficient storage unit103, and to discard the motion compensation filter coefficient having a low occurrence probability from the filter coefficient storage unit103. This makes it possible to enable reduction in the number of times of access to the memory109, and to enable reduction in the memory bandwidth for the filter coefficient storage unit103.

Here, taking an example, a description is given of the difference in the effect of the information indicating the next reference order of the motion compensation filter coefficient for a block in the stream as shown inFIG. 10and the information indicating the number of times of reference to be made to the motion compensation filter coefficient in the stream as shown inFIG. 11.

Each ofFIGS. 12A and 12BandFIGS. 13A and 13Bis an illustration or a table showing how motion compensation filter coefficients in the filter coefficient storage unit are updated based on the management table for statistical information of filter coefficients. Here,FIGS. 13A and 13Bare consecutive in time toFIGS. 12A and 12B, respectively. Compared toFIGS. 12A and 12B,FIGS. 13A and 13Bshows how a macroblock which is next to the macroblock decoded inFIGS. 12A and 12Bis decoded by the decoding unit101. Here, the pre-decoding unit301performs decoding of the coded stream, prior to the decoding by the decoding unit101by two macroblocks. In addition, the filter coefficient storage unit103is assumed to hold only two motion compensation filter coefficients.

As shown inFIG. 12A, the pre-decoding unit301performs decoding of the coded stream prior to the decoding by the decoding unit101by, for example, three macroblocks, and the information as shown inFIG. 12Bis held in the management table for statistical information of filter coefficients302. For example, the management table for statistical information of filter coefficients302holds the pieces of filter coefficients ID (0, 1, and 2) decoded so far in the decoding of the coded stream by the pre-decoding unit301, and holds the pieces of information (∞, 2, and 1) each indicating at which block (how many times ahead) in the stream the corresponding one of the filter coefficient is referred to. Here, inFIG. 12B, ∞ shows that there is no filter coefficient ID 0 at the macroblock decoded by the pre-decoding unit301prior to the macroblock decoded by the decoding unit101by two blocks. On the other hand, the filter coefficient ID 1 is referred to by the decoding unit101in the future decoding (of the block to be decoded next). Likewise, the filter coefficient ID 2 is referred to by the decoding unit101in the future decoding (of the block to be decoded next to the next).

For this, the filter coefficient transfer control unit102changes the area in which the motion compensation filter coefficient corresponding to the filter coefficient ID 0 is stored from among the motion compensation filter coefficients corresponding to the pieces of filter coefficient ID (0, 1) held in the filter coefficient storage unit103, with reference to the management table for statistical information of filter coefficients302and the management table for storage states of filter coefficients104. In other words, the filter coefficient transfer control unit102changes the motion compensation filter coefficient corresponding to the filter coefficient ID 0 held in the filter coefficient storage unit103to the motion compensation filter coefficient corresponding to the filter coefficient ID 2.

The case as shown inFIGS. 13A and 13Bis similar to the above case. Specifically, the filter coefficient transfer control unit102determines, as the motion compensation filter coefficient that should be changed, one of the motion compensation filter coefficients corresponding to the pieces of filter coefficient ID (2, 1) held in the filter coefficient storage unit103, with reference to the management table for statistical information of filter coefficients302and the management table for storage states of filter coefficients104. Here, it is possible to determine to change the area that stores the motion compensation filter coefficient corresponding to the filter coefficient ID 2 with reference to the management table for statistical information of filter coefficients302. In addition, the filter coefficient transfer control unit102can find out that the filter coefficient storage unit103holds the motion compensation filter coefficient corresponding to the filter coefficient ID 1 with reference to the management table for storage states of filter coefficients104. The filter coefficient transfer control unit102determines that there is no need to make any modification for the filter coefficient storage unit103, and thus does nothing for the filter coefficient storage unit103.

In this way, it is possible to reduce the number of times of access to the memory109by checking, in advance, the occurrence probability of the motion compensation filter coefficient ID to be used for the decoding by the decoding unit101.

Next, a description is given of decoding operations performed by the decoding apparatus300configured as mentioned above.FIG. 14is a flowchart of the decoding operations performed by the decoding apparatus according to Embodiment 3 of the present invention.

As shown inFIG. 14, in the decoding apparatus300, the decoding unit101decodes coded header information in an input coded stream (S301), and outputs, as decoded header information, at least motion compensation filter coefficients. Next, the filter coefficient transfer control unit102writes, to the memory109, all the motion compensation filter coefficients decoded, in Step S301, by the decoding unit101(S302). Next, the decoding unit101determines whether a current image to be decoded has a picture type of P, B or I (S303). When the decoding unit101determines that the current image to be decoded has a picture type of P or B (P or B in S303), the pre-decoding unit301determines that motion compensation is required for the current image, performs decoding of the coded stream prior to the decoding by the decoding unit101, and updates the management table for statistical information of filter coefficients302(S305). More specifically, prior to the decoding by the decoding unit101, the pre-decoding unit301performs decoding of the coded stream, and writes, in the management table for statistical information of filter coefficients302, the occurrence frequency of the motion compensation filter coefficient ID in the part that is of the coded stream and is not yet decoded by the decoding unit101.

Next, the decoding unit101decodes the header information in the coded stream and the prediction residual signal, and outputs, as the header information, at least the motion compensation filter coefficient ID, the motion vector information, and the prediction residual signal (S306).

Here, when the current image has a picture type of I (the case of I in S303), the decoding unit101proceeds to S316, performs intra-picture decoding, and proceeds to S315.

Next, the filter coefficient transfer control unit102checks whether or not the filter coefficient storage unit103holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID decoded by the decoding unit101in S306, with reference to the management table for storage states of filter coefficients104(S307).

When the filter coefficient storage unit103does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of No in S307), the filter coefficient transfer control unit102reads out, from the memory109, the motion compensation filter coefficients indicated by the motion compensation filter coefficient ID. In addition, the filter coefficient transfer control unit102writes the motion compensation filter coefficient to, for example, an area which is of the filter coefficient storage unit103and stores the motion compensation filter coefficient to be referred to least frequently (S308). Here, the filter coefficient transfer control unit102identifies the motion compensation filter coefficient to be referred to least frequently in the decoding by the decoding unit101, with reference to the management table for statistical information of filter coefficients302.

Next, the filter coefficient transfer control unit102updates the management table for storage states of filter coefficients104(S309). More specifically, the filter coefficient transfer control unit102updates, to “held”, the storage state which is of the read out motion compensation filter coefficient and is indicated in the management table for storage states of filter coefficients104, and updates the storage state of the erased motion compensation filter coefficient to “not held”.

On the other hand, when the filter coefficient storage unit103holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of Yes in S307), the filter coefficient transfer control unit102proceeds to S310without reading out the motion compensation filter coefficient from the memory109(without transferring it to the memory109).

The processes from S310to S315are the same as the processes from S209to S214, and thus the same descriptions thereof are not repeated.

In this way, the decoding apparatus300performs the decoding operations.

As described above, according to Embodiment 3, the pre-decoding unit301performs decoding of the stream prior to the decoding by the decoding unit101, and the occurrence frequency of the motion compensation filter coefficient ID to be used for the decoding by the decoding unit101is managed using the management table for statistical information of filter coefficients302. In this way, it is possible to check, in advance, the occurrence probability of the motion compensation filter coefficient ID used for the decoding by the decoding unit101. Accordingly, by storing the motion compensation filter coefficient having a high occurrence probability in the filter, coefficient storage unit103, it is possible to reduce the number of times of access to the memory109and reduce the memory bandwidth and memory access latency related to the motion compensation filter coefficient.

It is to be noted that, when the filter coefficient storage unit103does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, the motion compensation filter coefficient to be referred to least frequently is identified with reference to the management table for statistical information of filter coefficients302. In the above description, the filter coefficient transfer control unit102reads out, from the memory109, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, and writes the read-out motion compensation filter coefficient to an area which is of the filter coefficient storage unit103and stores the motion compensation filter coefficient to be referred to least frequently. However, the area to which the motion compensation filter coefficient is written is not limited thereto. For example, the motion compensation filter coefficient is written to an area that stores the motion compensation filter coefficient that is not to be used for a while or that is to be referred to not frequently for a predetermined future period. Any area selection method is possible as long as the method enables reduction in the access to the memory109and reduction in the capacity of the filter coefficient storage unit103in the same manner.

The above description is given of a case where the motion compensation filter coefficient ID is header information different from a motion vector, but the motion compensation filter coefficient ID is not limited thereto. For example, the motion vector may be the motion compensation filter coefficient ID. Alternatively, information including the motion vector not only the header information different from the motion vector may be the motion compensation filter coefficient ID.

The above description is given of a case where the motion vector is set in the motion compensation unit107, but the decimal part of the motion vector may be set.

In the above description, the decoding apparatus300includes the decoding unit101, the filter coefficient transfer control unit102, the filter coefficient storage unit103, the management table for storage states of filter coefficients104, the reference image transfer control unit105, the reference image storage unit106, the motion compensation unit107, the adder108, the memory109, the pre-decoding unit301, and the management table for statistical information of filter coefficients302. However, the structure of the decoding apparatus300is not limited thereto. As shown inFIG. 15, it is only necessary that the decoding apparatus300includes a decoding apparatus unit350as its essential element. More specifically, it is only necessary for the decoding apparatus300to include the decoding apparatus unit350including the decoding unit101, the filter coefficient transfer control unit102, the filter coefficient storage unit103, the memory109, and the pre-decoding unit301.FIG. 15is a block diagram showing essential structural elements of a decoding apparatus according to Embodiment 3 of the is present invention.

More specifically, the decoding apparatus unit350may include: a decoding unit101which decodes, from the coded stream, a plurality of motion compensation filter coefficients for use in the motion-compensation decoding; a pre-decoding unit301which decodes, from the coded stream, a plurality of motion compensation filter coefficients, prior to the decoding by the decoding unit101; a memory109for holding the motion compensation filter coefficients decoded by the pre-decoding unit301; a filter coefficient storage unit103for holding at least one motion compensation filter coefficient required for the motion compensation from among the motion compensation filter coefficients decoded by the decoding unit101; and a filter coefficient transfer control unit102which stores, in the filter coefficient storage unit103, the at least one motion compensation filter coefficient, based on the motion compensation filter coefficients decoded by the pre-decoding unit301.

The decoding apparatus unit350including at least the decoding apparatus unit10as its essential element performs pre-reading and pre-analysis of the stream using the motion compensation filter coefficients decoded by the pre-decoding unit301, and stores at least one motion compensation filter coefficient for use in the motion-compensation decoding into the filter coefficient storage unit103. Accordingly, the following advantageous effect is provided: when the at least one motion compensation filter coefficient is held in the filter coefficient storage unit103, it is possible to reduce the number of times of direct access to the memory109by using the motion compensation filter coefficient held in the filter coefficient storage unit103.

FIG. 16is a block diagram showing a structure of a decoding apparatus according to Embodiment 4 of the present invention. InFIG. 15, the same structural elements as those inFIG. 1are assigned with the same reference signs, and the same descriptions thereof are not repeated.

A decoding apparatus400as shown inFIG. 16includes a decoding unit101, a filter coefficient transfer control unit102, a filter coefficient storage unit103, a reference image transfer control unit105, a reference image storage unit106, a motion compensation unit107, an adder108, a memory109, a reversible coding unit401, and a reversible decoding unit402. Here, unlike the decoding apparatus100according to Embodiment 1, the decoding apparatus400as shown inFIG. 16includes the reversible coding unit401and the reversible decoding unit402, and does not include the management table for storage states of filter coefficients104.

The reversible coding unit401reversibly codes the motion compensation filter coefficients included in the coded stream. More specifically, the reversible coding unit401has a function of reversibly coding the motion compensation filter coefficients decoded by the decoding unit101, and writing the reversibly-coded motion compensation filter coefficients to the memory109.

The reversible decoding unit402reversibly decodes the at least one motion compensation filter coefficient for use in the decoding from among the plurality of motion compensation filter coefficients reversibly coded by the reversible coding unit401, and writes the reversibly decoded motion compensation filter coefficient to the filter coefficient storage unit103via the filter coefficient transfer control unit102. More specifically, the reversible decoding unit402has a function of reading out the motion compensation filter coefficients stored in the memory109, reversibly decoding the read-out motion compensation filter coefficients, and writing the reversibly-decoded motion compensation filter coefficients to the filter coefficient storage unit103via the filter coefficient transfer control unit102. It is to be noted that, the reversible decoding unit402may write the motion compensation filter coefficients to the filter coefficient storage unit103without the control by the filter coefficient transfer control unit102.

Next, a description is given of decoding operations performed by the decoding apparatus400configured as mentioned above.FIG. 17is a flowchart of the decoding operations performed by the decoding apparatus according to Embodiment 4 of the present invention.

As shown inFIG. 17, in the decoding apparatus400, the decoding unit101decodes the header information in an input coded stream (S401), and outputs, as header information, at least one motion compensation filter coefficient. Next, the reversible coding unit401reversibly codes the motion compensation filter coefficient decoded by the decoding unit101(S402). Next, the filter coefficient transfer control unit102writes, to the memory109, the motion compensation filter coefficient reversibly coded by the reversible coding unit401(S403).

Next, the decoding unit101determines whether a current image to be decoded has a picture type of P, B or I (S404). When the current image has a picture type of P or B (the case of P or B in S404), the decoding unit101determines that motion compensation is required, decodes the header information in the coded stream and a prediction residual signal (S405), and outputs, as header information, at least the motion compensation filter coefficient ID, the motion vector information, and the prediction residual signal. On the other hand, when the current image has a picture type of I (the case of I in S404), the decoding unit101determines that motion compensation is not required, proceeds to S414and performs intra-picture decoding, and then proceeds to S413.

Next, the filter coefficient transfer control unit102reads out, from the memory109, the reversibly-coded motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (S406). Next, the reversible decoding unit402reversibly decodes the reversibly-coded motion compensation filter coefficient, and writes the reversibly-decoded motion compensation filter coefficient to the filter coefficient storage unit103(S407).

The processes from S408to S413are the same as the processes from S108to S112, and thus the same descriptions thereof are not repeated.

In this way, the decoding apparatus400performs the decoding operations.

As described above, according to Embodiment 4, the reversible coding unit401compresses the motion compensation filter coefficient, and thus it is possible to reduce the data size of the motion compensation filter coefficient to be stored in the memory109. In this way, it is possible to reduce the memory capacity related to the motion compensation filter coefficient.

It is to be noted that the decoding apparatus400as shown inFIG. 16may include at least the aforementioned management table for storage states of filter coefficients104. Alternatively, the decoding apparatus400may additionally include one of the management table for reference history of filter coefficients201and the management table for statistical information of filter coefficients302. The latter option is preferable because it is possible to reduce the memory bandwidth and the memory access latency as described earlier, in addition to reduction in the memory capacity related to the motion compensation filter coefficient.

In addition, in the above, the decoding unit101decodes the motion compensation filter coefficient once, the reversible coding unit401re-codes the motion compensation filter coefficient and then stores the coefficient to the memory109, and the reversible decoding unit402decodes the coefficient. However, this case is an example. For example, it is also good that the motion compensation filter coefficient part of the coded stream is stored into the memory109without decoding of the motion compensation filter coefficients by the decoding unit101, and that the reversible decoding unit402decodes these coefficients.

In addition, the coding algorithm for use in the reversible coding unit401may be any coding scheme as long as the coding scheme makes the output size of the motion compensation filter coefficient smaller than the input size thereof.

In addition, although the above description is given of a case where the motion compensation filter coefficient ID is header information different from a motion vector, the motion compensation filter coefficient ID is not limited thereto. For example, the motion vector may be the motion compensation filter coefficient ID. Alternatively, information including the motion vector not only the header information different from the motion vector may be the motion compensation filter coefficient ID.

The above description is given of a case where the motion vector is set in the motion compensation unit107, but the decimal part of the motion vector may be set.

FIG. 18is a block diagram showing a structure of a coding apparatus according to Embodiment 5 of the present invention.

A coding apparatus500as shown inFIG. 18is a coding apparatus which performs motion-compensation coding using a current image to be coded and a reference image. The coding apparatus500includes a filter coefficient generation, unit501, a filter coefficient transfer control unit502, a filter coefficient storage unit503, a management table for storage states of filter coefficients504, a reference image transfer control unit505, a reference image storage unit506, a motion estimation unit507, a subtractor508, a memory509, and a coding unit510.

The filter coefficient generation unit501corresponds to a generation unit in the CLAIMS of the present application. The filter coefficient generation unit501generates a plurality of motion compensation filter coefficients for use in the motion-compensation coding. More specifically, the filter coefficient generation unit501has a function of receiving a current image to be coded as an input or receiving a current image to be coded and a reference image as inputs, and outputting motion compensation filter coefficients for motion compensation using variable coefficients. Typically, the filter coefficient generation unit501generates filter coefficients (motion compensation filter coefficients) for use in the motion compensation using the variable coefficients to be included in the header information of, for example, a GOP or a unit of pictures in the sequence in the coded stream to be coded by the coding unit510. Here, the motion compensation filter coefficients may be generated in units of the GOP or the sequence, not only in the unit of a picture, or may be generated in units of a slice or a macroblock. Alternatively, these units may be arbitrarily combined.

The filter coefficient transfer control unit502writes, to the memory509, the plurality of motion compensation filter coefficients generated by the filter coefficient generation unit501, and transfers the required motion compensation filter coefficient from the memory509to the filter coefficient storage unit503, only when the filter coefficient storage unit503does not hold the required motion compensation filter coefficient for the motion compensation. More specifically, the filter coefficient transfer control unit502has a function of writing, to the memory509, the motion compensation filter coefficients generated by the filter coefficient generation unit501. In addition, the filter coefficient transfer control unit502has a function of referring to the information in the management table for storage states of filter coefficients504, based on the information indicating the motion compensation filter coefficient for use in the motion compensation determined by the motion estimation unit507(the latter information is the motion compensation filter coefficient ID). The filter coefficient transfer control unit502checks whether or not the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID is held in the filter coefficient storage unit503, based on the information in the management table for storage states of filter coefficients504.

For example, when the filter coefficient transfer control unit502finds out that the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID is not held in the filter coefficient storage unit503, the filter coefficient transfer control unit102reads out, from the memory509, the motion compensation filter coefficient indicted by the motion compensation filter coefficient ID, and writes the read-out motion compensation filter coefficient in the filter coefficient storage unit503(transfers the coefficient thereto). On the other hand, when the filter coefficient transfer control unit502finds out that the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID is held (stored) in the filter coefficient storage unit503, the filter coefficient transfer control unit102does not read out, from the memory509, the motion compensation filter coefficient (does not transfer the coefficient thereto).

It is to be noted that the filter coefficient transfer control unit502may issue a write instruction to the filter coefficient generation unit501so that the filter coefficient generation unit501writes the motion compensation filter coefficient to the memory509. Likewise, the filter coefficient transfer control unit502may issue a read instruction to the filter coefficient storage unit503so that the filter coefficient storage unit503reads out, from the memory509, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, and holds the read out motion compensation filter coefficient.

The filter coefficient storage unit is for holding the required motion compensation filter coefficients for use in the motion compensation among the plurality of motion compensation filter coefficients held in the memory509. More specifically, the filter coefficient storage unit503has a function of holding at least two kinds of motion compensation filter coefficients transferred from the memory509, and a function of setting, in the motion estimation unit507, the motion compensation filter coefficient indicated by the motion compensation filter coefficients ID.

The management table for storage states of filter coefficients504manages information indicating whether or not the filter coefficient storage unit503holds the motion compensation filter coefficient identified by the motion compensation filter coefficient ID. More specifically, the management table for storage states of filter coefficients504has a function of receiving, as an input, the motion compensation filter coefficient ID, and outputting, to the filter coefficient transfer control unit502, information indicating whether or not the filter coefficient storage unit503holds the motion compensation filter coefficient identified by the motion compensation filter coefficient ID. For example, as shown inFIG. 3mentioned earlier, the management table for storage states of filter coefficients504holds, as a table (information), the storage states indicating whether or not the motion compensation filter coefficient identified by the motion compensation filter coefficient ID is stored in the filter coefficient storage unit503.

The reference image transfer control unit505has a function of reading out, from the memory509, the required reference pixels for use in motion estimation, based on the position information of a search area which is within a reference image and is to be used in the motion estimation, and writing the reference pixels to the reference image storage unit506.

It is to be noted that the reference image transfer control unit505may issue a read instruction to the reference image storage unit506based on the position information of the search area in the reference image and used for the motion estimation, so that the reference image storage unit506reads out the reference pixels required for the motion estimation and holds the read-out reference pixels.

The reference image storage unit506has a function of holding the reference images transferred from the memory509.

The motion estimation unit507generates a prediction image from the current image to be coded and the reference image by performing motion compensation using the motion compensation filter coefficient which is held in the filter coefficient storage unit503and required for the motion compensation. More specifically, the motion estimation unit507has a function of receiving, as inputs, the current image to be coded and the reference image, and determining the motion vector and the motion compensation filter coefficient ID indicating the type of the motion compensation filter coefficient for use in the motion compensation, and outputting the determined motion vector and motion compensation filter coefficient to the coding unit510. Furthermore, the motion estimation unit507has a function of outputting the determined motion compensation filter coefficient ID to the filter coefficient transfer control unit502. In addition, the motion estimation unit507has a function of obtaining the reference pixels indicated, as being required for the motion compensation, by the motion vector held in the reference image storage unit506and the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID held in the filter coefficient storage unit503, generating a prediction image by performing motion compensation using these pieces of information, and outputting the generated prediction image to the subtractor508.

Here, a method performed by the motion estimation unit507to determine the motion compensation filter coefficient ID is explained taking an example.FIG. 19is an example of information held in a management table for storage states of filter coefficients.FIG. 20is illustrations for explaining a method performed by the motion estimation unit507to determine the motion compensation filter coefficient ID.

As shown inFIG. 19, it is assumed here that the management table for storage states of filter coefficients holds pieces of motion compensation filter coefficient ID ranging from 0 to 100 and the storage states of the corresponding motion compensation filter coefficients in the filter coefficient storage unit503. In addition, it is assumed that the filter coefficient storage unit503stores motion compensation filter coefficients corresponding to pieces of motion compensation filter coefficient ID ranging from 70 to 79. Furthermore, it is assumed that the filter coefficient storage unit503can hold only twenty motion compensation filter coefficients. Accordingly, as shown in (a) inFIG. 20, the filter coefficient storage unit503stores the ten motion compensation filter coefficients corresponding to the motion compensation filter coefficient ID ranging from 70 to 79, and has an area available for holding ten more motion compensation filter coefficients.

In this case, the motion estimation unit507determines the motion compensation filter coefficient ID indicating the type of the motion compensation filter coefficient used for the motion compensation in the manner indicated below.

First, as shown in (b) inFIG. 20, the motion estimation unit507causes the filter coefficient transfer control unit502to write the motion compensation filter coefficients corresponding to the pieces of motion compensation filter coefficient ID ranging from 0 to 9 into the filter coefficient storage unit503, and checks whether or not the pieces of written motion compensation filter coefficient ID are the pieces of motion compensation filter coefficient ID for use in the motion compensation.

Next, as shown in (c) inFIG. 20, the motion estimation unit507causes the filter coefficient transfer control unit502to write the motion compensation filter coefficients corresponding to the pieces of motion compensation filter coefficient ID ranging from 10 to 19 into the filter coefficient storage unit503, and checks whether or not the pieces of written motion compensation filter coefficient ID are the pieces of motion compensation filter coefficient ID for use in the motion compensation.

In this way, the motion estimation unit507causes the filter coefficient transfer control unit502to write the motion compensation filter coefficients corresponding to the pieces of motion compensation filter coefficient ID ranging from 0 to 69 and 80 to 100 into the filter coefficient storage unit503, and checks whether or not the pieces of written motion compensation filter coefficient ID are the pieces of motion compensation filter coefficient ID for use in the motion compensation.

On the other hand, the motion compensation filter coefficients corresponding to the pieces of motion compensation filter coefficients ID ranging from 70 to 79 are stored in the filter coefficient storage unit503in advance. Accordingly, the filter coefficient transfer control unit502does not transfer the motion compensation filter coefficients corresponding to the pieces of motion compensation filter coefficient ID ranging from 70 to 79. In this way, it is possible to reduce the memory bandwidth and the memory access latency related to the motion compensation filter coefficients.

The above-described method performed by the motion estimation unit507to determine the pieces of motion compensation filter coefficient ID is a mere example, and other methods are possible as a matter of course.

The subtractor508has a function of performing subtraction between the input current image to be coded and the prediction error signal output by the motion estimation unit507, and outputting, as a prediction error signal, the current image and the prediction error signal to the coding unit510.

The memory509has a function of holding the motion compensation filter coefficients and reference, pictures that are referred to by the motion estimation unit507. It is to be noted that the memory509may hold only the motion compensation filter coefficients. In such a case, it is only necessary that the decoding apparatus500additionally includes a frame memory for holding the reference images that are referred to by the motion estimation unit507.

The coding unit510has a function of receiving, as inputs, at least the prediction error signal output by the subtractor508, the motion vector output by the motion estimation unit507, and the motion compensation filter coefficient ID, coding these inputs according to the standard of the video compression technique, and outputting the coded stream.

The decoding apparatus500is structured as described above.

Next, a description is given of coding operations performed by the coding apparatus500.FIG. 21is a flowchart of the decoding operations performed by the decoding apparatus according to Embodiment 5 of the present invention.

As shown inFIG. 21, first in the coding apparatus500, the filter coefficient generation unit501receives a current image to be coded, generates motion compensation filter coefficients for use in the following coding, and outputs the generated motion compensation filter coefficients (S501). Next, the filter coefficient transfer control unit502writes, to memory509, all of motion compensation filter coefficients generated by the filter coefficient generation unit501in S501.

Next, the coding apparatus500performs, on the current image (blocks) both of intra-picture prediction in S503and inter-picture prediction in S504to S511. As for the intra-picture prediction, a conventional scheme is applicable, and thus no description thereof is given here. The following describes how the inter-picture prediction in S504to S511is performed.

In S504, the reference image transfer control unit505reads out, from the memory509, the reference pixels required for motion estimation, based on the position information of a search area which is within a reference image and is to be used in the motion estimation, and writes the reference pixels to the reference image storage unit506.

Next, the motion estimation unit507receives, as inputs, the current image to be coded and the reference image, outputs, as a motion vector, information at the pixel position that is in the reference image and has a high correlation with the current image to be coded (S505), identifies a motion compensation filter coefficient that yields a smaller prediction error, and outputs motion compensation filter coefficient ID indicating the identified motion compensation filter coefficient.

Next, the filter coefficient transfer control unit502checks whether or not the filter coefficient storage unit503holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID decoded in S506, with reference to the management table for storage states of filter coefficients504(S507).

Next, when the filter coefficient storage unit503does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of No in S507), the filter coefficient transfer control unit502reads out, from the memory509, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID. Next, the filter coefficient transfer control unit502writes the motion compensation filter coefficient in an area which is of the filter coefficient storage unit503and holds the motion compensation filter coefficient read out from the memory509at earliest time (S508). Next, the filter coefficient transfer control unit502updates the management table for storage states of filter coefficients504(S509). More specifically, the filter coefficient transfer control unit502updates, to “held”, the storage state which is of the read out motion compensation filter coefficient and is indicated in the management table for storage states of filter coefficients504, and updates the storage state of the erased motion compensation filter coefficient to “not held”.

On the other hand, when the filter coefficient storage unit503holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient IDs (the case of Yes in S507), the filter coefficient transfer control unit502proceeds to S510without reading out the motion compensation filter coefficient from the memory509(without transferring it to the memory109).

Next, the motion estimation unit507reads out the motion compensation filter coefficient for use in the motion compensation from the filter coefficient storage unit503, based on the determined motion vector and the identified motion compensation filter coefficient ID, and sets the motion compensation filter coefficient (S510). Next, the motion estimation unit507reads out the reference image held in the reference image storage unit506, performs motion compensation on the reference image to generate a prediction image (S511), and outputs the generated prediction image to the subtractor508.

In this way, the coding apparatus500performs intra-picture prediction of the current image (blocks) to be coded in S503, and performs inter-picture prediction thereof in S504to S511. Next, the coding unit510compares the coding efficiency in the case of the inter-picture prediction coding and the coding efficiency in the case of the intra-picture prediction coding, and thereby determines one of the coding modes which provides a higher coding efficiency. The coding unit510determines the coding mode which provides the higher coding efficiency as the coding mode (macroblock type) of the current image (blocks) to be coded (S512).

Next, the subtractor508performs subtraction according to the coding mode of the current image (blocks) to be coded determined in S512. More specifically, when the coding mode of the current image (blocks) to be coded is determined to be inter-picture prediction (S512), the subtractor508performs subtraction between the prediction image output by the motion estimation unit507in S511and the current image (blocks) to be coded (S513), and outputs a prediction residual (prediction error) signal. On the other hand, when the coding mode of the current image (blocks) to be coded is determined to be intra-picture prediction (S512), the subtractor508performs subtraction between the prediction image output in S503and the current image (blocks) to be coded (S513) to obtain a prediction residual (error) signal, and outputs the prediction residual (error) signal.

Next, the coding unit510receives, as inputs, at least (i) the prediction residual (error) signal output by the subtractor508and (ii) the motion vector and the motion compensation filter coefficient ID output by the motion estimation unit507, codes the prediction residual (error) signal, the motion vector, and the motion compensation filter coefficient ID according to the standard of the video compression technique, and outputs the resulting coded stream (S514).

Next, the coding unit510determines whether or not all of the motion compensation blocks that need to be motion-compensation using the motion compensation filter coefficients are already coded (S515). When the answer is positive (in the case of Y in S515), the coding unit510completes the coding. On the other hand, when there remains any motion compensation block that is not yet decoded (N in S515), the coding unit510returns to both S503and S504, and repeats the following processing.

As described above, the coding apparatus500performs the coding operations.

According to Embodiment 5, when it is found out, with reference to the management table for storage states of filter coefficients504, that the filter coefficient storage unit503stores the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, the motion compensation filter coefficient is not read out from the memory509(is not transferred). For this reason, it is possible to reduce the number of times of access to the memory509. For this reason, it is possible to reduce the memory bandwidth and the memory access latency related to the motion compensation filter coefficient.

In the above case, when the filter coefficient storage unit503does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, the filter coefficient transfer control unit502reads out from the memory509(transfers) the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID. The above description is given of a case of writing the motion compensation filter coefficient to an area which is of the filter coefficient storage unit503and stores the motion compensation filter coefficient read out from the frame memory509at the earliest time. However, the area to which the motion compensation filter coefficient is written is not limited thereto. For example, the motion compensation filter coefficient may be written to an area which stores the motion compensation filter coefficient read out from the memory509most recently. Alternatively, the motion compensation filter coefficient may be written to an area which stores another filter coefficient and is selected at random. Any other area selection method is possible as long as the method enables reduction in the access to the memory509and reduction in the capacity of the filter coefficient storage unit103.

The above description is given of a case where the motion compensation filter coefficient ID is header information different from a motion vector, but the motion compensation filter coefficient ID is not limited thereto. For example, the motion vector may be the motion compensation filter coefficient ID. Alternatively, not only the header information different from the motion vector but also the motion vector may be the motion compensation filter coefficient ID.

In addition, the coding apparatus500includes the filter coefficient generation unit501, the filter coefficient transfer control unit502, the filter coefficient storage unit503, the management table for storage states of filter coefficients504, the reference image transfer control unit505, the reference image storage unit506, the motion estimation unit507, the subtractor508, the memory509, and the coding unit510. However, the structure of the coding apparatus500is not limited to the above structure. The coding apparatus500may include, as essential elements, the filter coefficient generation unit501, the filter coefficient transfer control unit502, the filter coefficient storage unit503, the motion estimation unit507, and the memory509.

More specifically, the coding apparatus500may be the moving image coding apparatus which performs motion-compensation coding using a current image to be coded and a reference image, and may include: a filter coefficient generation unit501which generates a plurality of motion compensation filter coefficients for use in the motion-compensation coding; a memory509for holding the motion compensation filter coefficients generated by the filter coefficient generation unit501; a filter coefficient storage unit503for holding at least one motion compensation filter coefficient required for the motion compensation, from among the motion compensation filter coefficients held in the memory509; a motion estimation unit507which generates a prediction image by performing motion compensation between the current image to be coded and the reference image, using the required motion compensation filter coefficients held in the filter coefficient storage unit503; and a filter coefficient transfer control unit502which writes, to the memory509, the motion compensation filter coefficients generated by the filter coefficient generation unit501, and transfers the required motion compensation filter coefficient from the memory509to the filter coefficient storage unit503, only when the filter coefficient storage unit503does not hold the required motion compensation filter coefficient.

With these essential elements, the coding apparatus500is capable of skipping reading of the motion compensation filter coefficient from the memory509when the filter coefficient storage unit503holds the required motion compensation filter coefficient. For this reason, it is possible to reduce the number of times of access to the memory509. For this reason, it is possible to reduce the memory bandwidth and the memory access latency related to the motion compensation filter coefficient.

FIG. 22is a block diagram showing a structure of a coding apparatus according to Embodiment 6 of the present invention. InFIG. 22, the same structural elements as those inFIG. 18are assigned with the same reference signs, and the same descriptions thereof are not repeated.

A coding apparatus600as shown inFIG. 22includes a filter coefficient generation unit501, a filter coefficient transfer control unit502, a filter coefficient storage unit503, a management table for storage states of filter coefficients504, a reference image transfer control unit505, a reference image storage unit506, a motion estimation unit507, a subtractor508, a memory509, a coding unit510, and a management table for reference history of filter coefficients601. Unlike the coding apparatus500according to Embodiment 5, the coding apparatus600as shown inFIG. 22includes the management table for reference history of filter coefficients601.

The management table for reference history of filter coefficients601manages use history which is of each of the motion compensation filter coefficients for use in motion compensation and indicates the number of times of reference to the motion compensation filter coefficient from the start of the motion compensation. More specifically, the management table for reference history of filter coefficients601has a function of receiving, as an input, the motion compensation filter coefficient ID, and outputs the number of times of reference to the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID from the start of the coding. It is to be noted that the information held in the management table for reference history of filter coefficients601is the same as the contents as shown inFIG. 7AandFIG. 7B, and thus the description thereof is not repeated here.

Next, a description is given of coding operations performed by the coding apparatus600configured as mentioned above.FIG. 23is a flowchart of the decoding operations performed by the decoding apparatus according to Embodiment 6 of the present invention.

As shown inFIG. 23, first in the coding apparatus600, the filter coefficient generation unit501receives a current image to be coded, and generates and outputs motion compensation filter coefficients for use in the following coding (S601). The processes of S602is the same as the process of S502, and thus the same description thereof is not repeated.

Next, the coding apparatus600performs, on the current image (blocks) to be coded, both of intra-picture prediction in S603and inter-picture prediction in S604to S612. As for the intra-picture prediction, a conventional scheme is applicable as in S503, and thus no description thereof is given here. The following describes how the inter-picture prediction in S604to S612is performed. Here, the processes of S604to S606are the same as the processes of S504to S506, and thus the same descriptions thereof are not repeated.

Next, when the filter coefficient storage unit503does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of No in S607), the filter coefficient transfer control unit502reads out, in S607, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID from the memory509. In addition, the filter coefficient transfer control unit502writes the motion compensation filter coefficient to an area which is of the filter coefficient storage unit503and stores the motion compensation filter coefficient referred to least frequently so far (S608). Here, the filter coefficient transfer control unit502identifies the motion compensation filter coefficient referred to least frequently with reference to the management table for reference history of filter coefficients601.

Next, the filter coefficient transfer control unit502updates the management table for reference history of filter coefficients601(S609). More specifically, the filter coefficient transfer control unit502increments the number of times of reference to the motion compensation filter coefficient in the management table for reference history of filter coefficients601.

Next, the filter coefficient transfer control unit502updates the management table for storage states of filter coefficients504(S610). More specifically, the filter coefficient transfer control unit502updates, to “held”, the storage state which is of the read out motion compensation filter coefficient and is indicated in the management table for storage states of filter coefficients504, and updates the storage state of the erased motion compensation filter coefficient to “not held”.

On the other hand, when the filter coefficient storage unit503holds the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID (the case of Yes in S607), the filter coefficient transfer control unit502proceeds to S611without reading out the motion compensation filter coefficient from the memory509(without transferring it).

Next, the motion estimation unit507reads out the motion compensation filter coefficient for use in the motion compensation from the filter coefficient storage unit503, based on the determined motion vector and the identified motion compensation filter coefficient ID, and sets the motion compensation filter coefficient (S611). The process of S611is the same as the process of S510, and thus the same description thereof is not repeated.

In this way, the coding apparatus600performs intra-picture prediction of the current image (blocks) to be coded in S603, and performs inter-picture prediction thereof in S604to S612. Next, the coding unit510determines, based on the performed intra-picture prediction and inter-picture prediction, which one of the coding modes provides a higher coding efficiency from among the coding efficiency in the case of performing the inter-picture coding and the coding efficiency in the case of performing the intra-picture coding. The coding unit510determines the coding mode determined as providing the higher coding efficiency as the coding mode (macroblock type) of the current image (blocks) to be coded.

The following processes of S614to S616are the same as the processes of S513to S515, and thus the same descriptions thereof are not repeated.

As described above, the coding apparatus600performs the coding operations.

As described above, according to Embodiment 6, the motion compensation filter coefficient referred to least frequently is identified with reference to the management table for reference history of filter coefficients601. In addition, the filter coefficient transfer control unit502writes the motion compensation filter coefficients to an area which is of the filter coefficient storage unit503and stores the motion compensation filter coefficients referred to least frequently so far. Accordingly, the motion compensation filter coefficients unlikely to be referred to are not held in the filter coefficient storage unit503. Thus, it is possible to hold the motion compensation filter coefficients highly likely to be referred to frequently in the filter coefficient storage unit503. As a result, it is possible to reduce the number of times of access to the memory509. In this way, it is possible to reduce the memory bandwidth and the memory access latency related to the motion compensation filter coefficients.

It is to be noted that, when the filter coefficient storage unit503does not hold the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, the motion compensation filter coefficient referred to least frequently is identified with reference to the management table for reference history of filter coefficients601. In the above description, the filter coefficient transfer control unit502reads out, from the memory509, the motion compensation filter coefficient indicated by the motion compensation filter coefficient ID, and writes the read-out motion compensation filter coefficient to an area which is of the filter coefficient storage unit503and stores the motion compensation filter coefficient referred to least frequently. However, the area to which the motion compensation filter coefficient is written is not limited thereto. For example, the motion compensation filter coefficient may be written to: an area that stores the motion compensation filter coefficient used least frequently recently (a predetermined past period); an area that stores the motion compensation filter coefficient used most frequently recently; or an area that stores the motion compensation filter coefficient referred to most frequently so far. Any area selection method is possible as long as the method enables reduction in the access to the memory109and reduction in the capacity of the filter coefficient storage unit103.

The above description is given of a case where the motion compensation filter coefficient ID is header information different from a motion vector, but the motion compensation filter coefficient ID is not limited thereto. For example, the motion vector may be the motion compensation filter coefficient ID. Alternatively, not only the header information different from the motion vector but also the motion vector may be the motion compensation filter coefficient ID.

FIG. 24is a block diagram showing a structure of a coding apparatus according to Embodiment 7 of the present invention. InFIG. 24, the same structural elements as those inFIG. 18are assigned with the same reference signs, and the same descriptions thereof are not repeated.

A coding apparatus700as shown inFIG. 24includes a filter coefficient generation unit501, a filter coefficient transfer control unit502, a filter coefficient storage unit503, a reference image transfer control unit505, a reference image storage unit506, a motion estimation unit507, a subtractor508, a memory509, a coding unit510, a reversible coding unit701, and a reversible decoding unit702. Unlike the coding apparatus500according to Embodiment 5, the coding apparatus700as shown inFIG. 24includes the reversible coding unit701and the reversible decoding unit702, and does not include the management table for storage states of filter coefficients504.

The reversible coding unit701reversibly codes the motion compensation filter coefficients included in the coded stream. More specifically, the reversible coding unit701has a function of reversibly coding the motion compensation filter coefficients generated by the filter coefficient generation unit501, and writing the reversibly-coded motion compensation filter coefficients to the memory509.

The reversible decoding unit702reversibly decodes at least one motion compensation filter coefficient required for the decoding from among the motion compensation filter coefficients reversibly coded by the reversible coding unit701, and writes the reversibly decoded motion compensation filter coefficient to the filter coefficient storage unit503via the filter coefficient transfer control unit502. More specifically, the reversible decoding unit702has a function of reading out the motion compensation filter coefficients stored in the memory509, reversibly decoding the read-out motion compensation filter coefficients, and writing the reversibly-decoded motion compensation filter coefficients to the filter coefficient storage unit503via the filter coefficient transfer control unit502. It is to be noted that, the reversible decoding unit702may write the at least one motion compensation filter coefficient to the filter coefficient storage unit503without the control by the filter coefficient transfer control unit502.

Next, a description is given of coding operations performed by the coding apparatus700configured as mentioned above.FIG. 25is a flowchart of the decoding operations performed by the decoding apparatus according to Embodiment 7 of the present invention.

As shown inFIG. 25, first in the coding apparatus700, the filter coefficient generation unit501receives a current image to be coded, and generates and outputs the at least one motion compensation filter coefficient for use in the following coding (S701).

Next, the reversible coding unit701reversibly codes the motion compensation filter coefficient generated by the filter coefficient generation unit501(S702).

Next, the filter coefficient transfer control unit502writes, to the memory509, the motion compensation filter coefficient reversibly coded, in S702, by the reversible coding unit701(S703).

Next, the coding apparatus700performs, on the current image (blocks) to be coded both of intra-picture prediction in S704and inter-picture prediction in S705to S711. As for the intra-picture prediction, a conventional scheme is applicable as in S503, and thus no description thereof is given here. The following describes how the inter-picture prediction in S705to S711is performed. Here, the processes of S705to S707are the same as the processes of S504to S506, and thus the same descriptions thereof are not repeated.

Next, the filter coefficient transfer control unit502reads out, in S707, the reversibly-coded motion compensation filter coefficient indicated by the motion compensation filter coefficient ID from the memory509(S708). Next, the reversible decoding unit702reversibly decodes the reversibly-coded motion compensation filter coefficient, and writes the reversibly-decoded motion compensation filter coefficient to the filter coefficient storage unit503(S709).

Next, the motion estimation unit507reads out the motion compensation filter coefficient for use in the motion compensation from the filter coefficient storage unit503, based on the identified motion compensation filter coefficient ID and the determined motion vector, and sets the motion compensation filter coefficient (S710). Next, the motion estimation unit507reads out the reference image held in the reference image storage unit506, performs motion compensation on the reference image to generate a prediction image (S711), and outputs the generated prediction image to the subtractor508.

In this way, the coding apparatus700performs intra-picture prediction of the current image (blocks) to be coded in S704, and performs inter-picture prediction thereof in S705to S711. Next, the coding unit510determines, based on the performed intra-picture prediction and inter-picture prediction, which one of the coding modes provides a higher coding efficiency from among the coding efficiency in the case of performing the inter-picture coding and the coding efficiency in the case of performing the intra-picture coding. The coding unit510determines the coding mode determined as providing the higher coding efficiency as the coding mode (macroblock type) of the current image (blocks) to be coded.

Here, the processes of S713to S715are the same as the processes of S513to S515, and thus the same descriptions thereof are not repeated.

As described above: the coding apparatus700performs the coding operations.

As described above, according to Embodiment 7, the reversible coding unit701compresses the motion compensation filter coefficient, and thus it is possible to reduce the data size of the motion compensation filter coefficients to be stored in the memory509. In this way, it is possible to reduce the memory capacity related to the motion compensation filter coefficient.

It is to be noted that the decoding apparatus700as shown inFIG. 23may include at least the aforementioned management table for storage states of filter coefficients504, and may additionally include the management table for reference history of filter coefficients601. The latter option is preferable because it is possible to reduce the memory bandwidth and the memory access latency as described earlier, in addition to reduction in the memory capacity related to the motion compensation filter coefficients.

In addition, in the above, the decoding unit101decodes the motion compensation filter coefficients once, the reversible coding unit401re-codes the motion compensation filter coefficients and then stores these coefficients to the memory109, and the reversible decoding unit402decodes these coefficients. However, this case is an example. For example, the coding unit510may store, in the memory509, the motion compensation filter coefficient part coded into the coded stream, and the reversible decoding unit702may decode the motion compensation filter coefficient part.

In addition, the coding algorithm for use in the reversible coding unit701may be any coding scheme as long as the coding scheme makes the output size of the motion compensation filter coefficients smaller than the input size thereof.

In addition, although the above description is given of a case where the motion compensation filter coefficient ID is header information different from a motion vector, the motion compensation filter coefficient ID is not limited thereto. For example, the motion vector may be the motion compensation filter coefficient ID.

Alternatively, not only the header information different from the motion vector but also the motion vector may be the motion compensation filter coefficient ID.

According to the present invention, it is possible to implement an image decoding apparatus, image coding apparatus, image decoding circuit, and image decoding method for enabling reduction in the memory bandwidth and reduction in the memory access latency for filter coefficients used to perform inter-picture prediction with motion compensation using variable coefficients.

In Embodiment 1 to 7, each of the memory109, the memory509, the filter coefficient storage unit103, and the filter coefficient storage unit503is typically configured in the form of a DDR. However, it is to be noted that each element need not to be always configured in the form of the DDR, and may be configured in the form of an SRAM or a flip-flop. In short, any recordable devices are possible. Preferably, each of the memory109and the memory509be configured using a low-speed storage device, and each of the filter coefficient storage unit103and the filter coefficient storage unit503be configured using a high-speed storage device.

The moving image coding apparatus, moving image decoding apparatus, moving image coding method and/or moving image decoding method as described in the above embodiments are applicable as applications.

For example, the processing described in each of the embodiments can be simply implemented by an independent computer system, by recording, in a recording medium, a program for implementing the moving image coding method and/or the moving image decoding method as described in the above embodiments. Here, the recording medium may be any recording medium as long as the program can be recorded, such as a magnetic disk, an optical disk, a magnetic optical disk, an IC card, and a semiconductor memory.

Hereinafter, applications to the moving picture coding methods and moving picture decoding methods described in the embodiments and systems using the same will be described.

FIG. 26is a diagram showing an overall configuration of a content providing system ex100for achieving content distribution services. In the content providing system ex100as shown inFIG. 26, the area for providing communication services is divided into cells having a desired size, and each of base stations ex107to ex110which are fixed wireless stations is placed in a corresponding one of the cells.

In the content providing system ex100, devices such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cellular phone ex114, and a game machine ex115are connected to the Internet ex101via a telephone network ex104as well as the base stations ex107to ex110.

Here, the configuration of the content providing system ex100is not limited to the configuration shown inFIG. 26, and a combination in which any of the elements are combined is acceptable. In addition, each of the devices in the content providing system ex100may be directly connected to the telephone network ex104, rather than via the base stations ex107to ex110which are the fixed wireless stations. Furthermore, the devices may be interconnected to each other via a short distance wireless communication and others.

For example, the camera ex113, such as a digital video camera, is capable of capturing moving pictures. In addition, the camera ex116, such as a digital video camera, is capable of capturing still pictures and moving pictures.

The cellular phone ex114may be the one that meets any of the standards such as Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Wideband-Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access (HSPA). Alternatively, the cellular phone ex114may be a Personal Handyphone System (PHS).

In the content providing system ex100, a streaming server ex103is connected to the camera ex113and others via the telephone network ex104and the base station ex109, which enables distribution of a live show and others.

More specifically, in such a live distribution, a content (for example, video of a music live show) captured by the user using the camera ex113is subjected to the coding as described in the above embodiments, and the coded content is transmitted to the streaming server ex103. On the other hand, the streaming server ex103carries out stream distribution of the received content data to the clients upon their requests. The clients include the computer ex111, the PDA ex112, the camera ex113, the cellular phone ex114, and the game machine ex115that are capable of decoding the above-mentioned coded data. Each of the devices that have received the distributed data decodes and reproduces the coded data.

The captured data may be coded by the camera ex113or the streaming server ex103that transmits the data, or the coding processes may be shared between the camera ex113and the streaming server ex103. Similarly, the distributed data may be decoded by the clients or the streaming server ex103, or the decoding processes may be shared between the clients and the streaming server ex103.

Furthermore, the data of the still pictures and moving pictures captured by not only the camera ex113but also the camera ex116may be transmitted to the streaming server ex103through the computer ex111. The coding processes may be performed by the camera ex116, the computer ex111, or the streaming server ex103, or shared among them.

These coding and decoding processes are performed by an LSI ex500generally included in each of the computer ex111and each of the devices. Here, the LSI ex500may be configured of a single chip or plural chips. It is to be noted that software for coding and decoding moving pictures may be integrated into some type of a recording medium (such as a CD-ROM, a flexible disk, a hard disk) that is readable by the computer ex111and others, and the coding and decoding processes may be performed using the software. Furthermore, when the cellular phone ex114is equipped with a camera, the moving picture data obtained by the camera may be transmitted. The video data is data coded by the LSI ex500included in the cellular phone ex114.

Furthermore, the streaming server ex103may be composed of servers and computers, and may divide data into data portions, and process, record, and distribute the data portions.

As described above, the clients can receive and reproduce the coded data in the content providing system ex100.

In other words, the clients can receive and decode information transmitted by the user, and reproduce the decoded data in real time in the content providing system ex100, so that the user who does not have any particular right and equipment can implement personal broadcasting.

It is to be noted that the present invention is not limited to the example where at least one of the moving picture coding apparatuses and one of the moving picture decoding apparatuses in the respective embodiments are incorporated into the content providing system ex100. As shown inFIG. 27, one of the moving picture coding apparatuses and one of the moving picture decoding apparatuses may be incorporated into a digital broadcasting system ex200. The following descriptions are given taking an example of the digital broadcasting system ex200.FIG. 27is a diagram showing an overall configuration of the digital broadcasting system ex200.

More specifically, a broadcast station ex201communicates or transmits a bitstream of video information via radio waves to a broadcast satellite ex202. The bitstream is a coded bitstream obtained by the moving picture coding method according to each of the embodiments.

Upon receipt of the bitstream, the broadcast satellite ex202generates radio waves for broadcasting.

The antenna ex204is a home-use antenna having a function of receiving satellite broadcasts and receives the radio waves for broadcasting from the broadcast satellite ex202.

A device that is a television (receiver) ex300, a set top box (STB) ex217, or the like decodes and reproduces a bit stream included in the radio waves for broadcasting received from the antenna ex204.

A reader/recorder ex218is capable of reading and decoding a coded bit stream recorded on a recording medium ex215such as a DVD and a BD. In addition, the reader/recorder ex218is capable of coding and writing a video signal onto the recording medium ex215. Here, the reader/recorder ex218mounts one of the moving image decoding apparatus and/or one of the moving image coding apparatus as described in the embodiments. Here, the reproduced video signals reproduced by the reader/recorder ex218are displayed on a monitor ex219, and can be reproduced by another device or system, using the recording medium ex215on which the coded bit stream is recorded.

It is to be noted that the set top box ex217may be connected to one of the cable ex203for cable television and the antenna ex204for satellite/terrestrial wave broadcasting, may mount the moving image decoding apparatus inside the device itself, and may display the video signals on the monitor ex219. In addition, the moving picture decoding apparatus may be incorporated not in the set top box ex217but in the television ex300.

FIG. 28is a block diagram showing an example of a structure of a television ex300.

The television ex300uses one of the moving image decoding method and one of the moving image coding method as described in the embodiments. The television ex300includes: a tuner ex301that obtains or provides a bitstream of video information from and through the antenna ex204or the cable ex203, etc. that receives a broadcast; a modulation/demodulation unit ex302that demodulates the received coded data or modulates data into coded data to be supplied outside; and a multiplexing/demultiplexing unit ex303that demultiplexes the modulated data into video data and audio data, or multiplexes the coded video data and audio data into data. The television ex300further includes: a signal processing unit ex306including an audio signal processing unit ex304and a video signal processing unit ex305that decode audio data and video data and code audio data and video data, respectively; a speaker ex307that provides the decoded audio signal; and an output unit ex309including a display unit ex308that displays the decoded video signal, such as a display. Furthermore, the television ex300includes an interface unit ex317including an operation input unit ex312that receives an input of a user operation. Furthermore, the television ex300includes a control unit ex310that integrally controls all the constituent elements of the television ex300, and a power supply circuit unit ex311that supplies power to each of the elements.

It is to be noted that the operation input unit ex312, the interface unit ex317may include: a bridge ex313that is connected to an external device, such as the reader/recorder ex218; a slot unit ex314for enabling attachment of the recording medium ex216, such as an SD card; a driver ex315to be connected to an external recording medium, such as a hard disk; and a modem ex316to be connected to a telephone network. Here, the recording medium ex216can electrically record information using a non-volatile/volatile semiconductor memory element for storage. The constituent elements of the television ex300are connected to each other through a synchronous bus.

First, a description is given of a configuration in which the television ex300decodes data obtained from outside through the antenna ex204and others and reproduces the decoded data.

In the television ex300, upon receipt of a user operation from a remote controller ex220or the like, the multiplexing/demultiplexing unit ex303demultiplexes the video data and audio data demodulated by the modulation/demodulation unit ex302, under control of the control unit ex310including a CPU. Furthermore, in the television ex300, the audio signal processing unit ex304decodes the demultiplexed audio data and the video signal processing unit ex305decodes the demultiplexed video data, using the decoding method described in each of the embodiments. The output unit ex309outputs each of the decoded video signal and audio signal. When the output unit ex309outputs the video signal and the audio signal, the signals may be temporarily stored in buffers ex318and ex319, or the like so that the signals are reproduced in synchronization with each other.

It is to be noted that the television ex300may read a coded bitstream not through a broadcast or the like but from the recording media ex215and ex216, such as a magnetic disk, an optical disk, and a SD card.

Next, a description is given of a configuration in which the television ex300codes an audio signal and a video signal, and transmits the data outside or writes the data on a recording medium.

In the television ex300, upon receipt of a user operation from the remote controller ex220or the like, the audio signal processing unit ex304codes an audio signal, and the video signal processing unit ex305codes a video signal using the coding method corresponding to the moving picture coding method as described in each of the embodiments, under control of the control unit ex310. The multiplexing/demultiplexing unit ex303multiplexes the coded video signal and audio signal, and outputs the resulting signal outside. When the multiplexing/demultiplexing unit ex303multiplexes the video signal and the audio signal, the signals may be temporarily stored in buffers ex320and ex321, or the like so that the signals are reproduced in synchronization with each other. Here, the buffers ex318to ex321may be plural as illustrated, or at least one buffer may be shared in the television ex300. Furthermore, data may be stored in a buffer other than the buffers ex318to ex321so that the system overflow and underflow may be avoided between the modulation/demodulation unit ex302and the multiplexing/demultiplexing unit ex303, for example.

Furthermore, the television ex300may include an element for receiving an AV input from a microphone or a camera in addition to the element for obtaining audio and video data from a broadcast or a recording medium, and may code the obtained data.

Although the television ex300can code, multiplex, and provide outside data in the description, it may be not capable of coding, multiplexing, and providing outside data but capable of only one of receiving, decoding, and providing outside data.

Furthermore, when the reader/recorder ex218reads or writes a coded bit stream from or in a recording medium, one of the television ex300and the reader/recorder ex218may decode or code the coded bit stream, and the television ex300and the reader/recorder ex218may share the decoding or coding.

As an example,FIG. 29illustrates a configuration of an information reproducing/recording unit ex400when data is read or written from or in an optical disk.FIG. 29is a block diagram showing an example of a structure of an information reproducing and recording unit that reads and writes information from and on a recording medium that is an optical disk.

The optical head ex401irradiates a laser spot on a recording surface of the recording medium ex215that is an optical disk to write information, and detects reflected light from the recording surface of the recording medium ex215to read the information.

The modulation recording unit ex402electrically drives a semiconductor laser included in the optical head ex401, and modulates the laser light according, to recorded data.

The reproduction demodulating unit ex403amplifies a reproduction signal obtained by electrically detecting the reflected light from the recording surface using a photo detector included in the optical head ex401, and demodulates the reproduction signal by separating a signal component recorded on the recording medium ex215to reproduce the necessary information.

The buffer ex404temporarily holds the information to be recorded on the recording medium ex215and the information reproduced from the recording medium ex215.

A disk motor ex405rotates the recording medium ex215.

A servo control unit ex406moves the optical head ex401to a predetermined information track while controlling the rotation drive of the disk motor ex405so as to follow the laser spot.

The system control unit ex407controls overall the information reproducing/recording unit ex400. The system control unit ex407implements the reading and writing processes by (i) generating and adding new information as necessary, based on various information stored in the buffer ex404, and (ii) recording and reproducing information through the optical head ex401while causing the modulation recording unit ex402, the reproduction demodulating unit ex403, and the servo control unit ex406to operate in a coordinated manner. The system control unit ex407includes, for example, a microprocessor, and executes processing by causing a computer to execute a program for read and write.

The optical head ex401irradiates a laser spot in the above description. However, it is to be noted that the optical head ex401may perform high-density recording using near field light.

FIG. 30shows an example of a structure of a recording medium that is an optical disk.FIG. 30shows a schematic view of the recording medium ex215that is the optical disk.

On the recording surface of the recording medium ex215, guide grooves are spirally formed, and an information track ex230records, in advance, address information indicating an absolute position on the disk according to change in a shape of the guide grooves. The address information includes information for determining positions of recording blocks ex231that are a unit of recording data. A device that records and reproduces data reproduces the information track ex230and reads the address information so as to determine the positions of the recording blocks. Furthermore, the recording medium ex215includes a data recording area ex233, an inner circumference area ex232, and an outer circumference area ex234. The data recording area ex233is an area for use in recording the user data. The inner circumference area ex232and the outer circumference area ex234that are inside and outside of the data recording area ex233, respectively are for specific use except for recording the user data.

The information reproducing/recording unit400reads and writes coded audio data, coded video data, or coded data obtained by multiplexing the coded audio data and the coded video data, from and on the data recording area ex233of the recording medium ex215.

Although an optical disk having a single layer, such as a DVD and a BD is described as an example in the description, it is to be noted that the optical disk is not limited thereto, and may be an optical disk having a multilayer structure and capable of recording on a part other than the surface. Furthermore, the optical disk may have a structure for multidimensional recording/reproduction, such as recording of information using light of colors with different wavelengths in the same portion of the optical disk and recording information having different layers from various angles.

Furthermore, the car ex210having the antenna ex205can receive data from the satellite ex202and others, and reproduce video on the display device that is, for example, the car navigation system ex211set in the car ex210, in a digital broadcasting system ex200. Here, a configuration of the car navigation system ex211will be a configuration, for example, including a GPS receiving unit from the configuration illustrated inFIG. 28. The same will be true for the configuration of the computer ex111, the cellular phone ex114, and others. Furthermore, similarly to the television ex300, a terminal such as the cellular phone ex114may have three types of implementation configurations including not only (i) a transmitting and receiving terminal including both a coding apparatus and a decoding apparatus, but also (ii) a transmitting terminal including only a coding apparatus and (iii) a receiving terminal including only a decoding apparatus.

As such, the moving picture coding method and moving picture decoding method in each of the embodiments can be used in any of the apparatuses, devices and systems described. Thus, the advantageous effects described in the embodiments can be obtained.

Furthermore, the present invention is not limited to the above-described embodiments, and various modifications and revisions are possible without departing from the scope of the present invention.

Each of the moving picture coding method and apparatus and moving picture decoding method and apparatus as shown in the embodiments is typically implemented as an LSI. As an example,FIG. 31shows a structure of a single-chip LSI ex500. Here,FIG. 31is a block diagram showing an example of a structure of an integrated circuit for performing the image coding method and the image decoding method according to each of the embodiments.

The LSI ex500includes elements ex502to ex509to be described below, and the elements are connected to each other through a bus ex510. The power supply circuit unit ex505is activated by supplying each of the elements with power when power is on.

For example, when coding is performed, the LSI ex500receives an input of an AV signal from a microphone ex117, a camera ex113, and others through an AV I/O ex509, under control of the control unit ex501including the CPU ex502, the memory controller ex503, the stream controller ex504. The received AV signal is temporarily stored in a memory ex511such as an SDRAM outside the LSI ex500. The stored data is divided into data portions according to the processing amount and speed as necessary. Then, the data portions are transmitted to a signal processing unit ex507, under control of the control unit ex501. The signal processing unit ex507codes an audio signal and/or a video signal. Here, the coding of the video signal is the coding as described in the above embodiments. Furthermore, depending on cases, the signal processing unit ex507multiplexes the coded audio data and the coded video data, and a stream I/O ex504provides the multiplexed data outside. The provided bit stream is transmitted to a base station ex107, or written into a recording medium ex215. Here, it is good to save the data into the buffer ex508for synchronization at the time of multiplexing the data.

FIG. 32is a block diagram for simply explaining the coding performed here. In other words,FIG. 32is a block diagram showing the moving image coding according to each of the embodiments and performed by the integrated circuit. As shown inFIG. 32, the prediction error signal that is a difference between an input signal and a prediction signal is transformed by the transform unit ex601, and then is quantized by the quantization unit ex602. The quantized coefficients are entropy coded by the entropy coding unit ex606to generate a coded signal, and the coded signal is output. As described above taking an example of the television ex300with reference toFIG. 28, this output may be saved in the buffer ex508or a memory ex511in order to be multiplexed with the coded audio data. The inverse quantization unit ex604, the inverse transform unit ex605, and the prediction unit ex608function as a delay unit which enables comparison between a current signal and a prediction signal generated from the signal preceding the current signal.

It is good to perform adjustment for preventing an overflow and/or an underflow on the LSI ex500. Examples of such adjustment include saving quantized coefficients into a buffer, for example, the buffer ex508or the memory ex511provided inside the LSI ex500. Other than the above adjustment for the quantized coefficients, it is good to divide the data into data portions and process the data portions in parallel according to the processing amount and processing speed, and to adjust the processing while saving, as necessary, the data that is currently being processed in a recording unit such as an internal memory or an external storage.

The above-described processing is performed under control of the control unit ex501.

For example, in the case of decoding, the LSI ex500saves, in the memory ex511etc., the coded data obtained from the base station ex107through the stream I/O ex506or read out from the recording medium ex215, under control of the control unit ex501. Under control of the control unit ex501, the stored data is divided into data portions according to the processing amount and processing speed as necessary, is transmitted to the signal processing unit ex507, and then is decoded by the signal processing unit ex507into audio data and/or video data. Here, the decoding of the video signal is the decoding as described in the above embodiments. Furthermore, depending on cases, it is good to save, in the buffer ex508, both the decoded audio signal and the decoded video signal so that these signals can be reproduced in synchronization with each other. The decoded output signals are provided from an output unit that is the cellular phone ex114, the game machine ex115, the television ex300, or the like, through the memory ex511etc. as necessary.

FIG. 33is a block diagram for simply explaining the decoding performed here.FIG. 33is a block diagram showing the moving image decoding according to each of the embodiments and performed by the integrated circuit.

As shown inFIG. 33, the input coded signal is entropy-decoded by the entropy decoding unit ex701. The quantized coefficients obtained through the entropy decoding are inversely quantized by the inverse quantization unit ex703, and then inversely transformed by the inverse transform unit ex704. The inverse transform here means transform in decoding, and is not always limited to the inverse of the transform in the coding. To the output, a prediction signal is added. Then, the output including the prediction signal is output as a decoded signal to outside. The memory ex511stores decoded signals, and functions as a delay unit which enables reference in the decoding of the succeeding coded signals. The prediction unit ex705generates a prediction signal based on the decoded signal stored in the memory ex511. As the description of output to outside given above taking an example of the television ex300with reference toFIG. 28, the decoded signals may be saved in the buffer ex508or in the external memory ex511so that the output decoded signal can be displayed in synchronization with the decoded audio signals. It is good to divide the quantized coefficients into predetermined units of processing, and processes the units of quantized coefficients in parallel, so as to prevent an overflow and/or underflow in the processing. In this case, for example, the quantized coefficients may be stored in the buffer ex508or the memory ex511. The above-described processing is performed under control of the control unit ex501.

Although the above memory ex511is described as a device outside the LSI ex500, the memory ex511may be configured inside the LSI ex500. The number of buffers ex508is not limited to one, and a plurality of buffers may be provided. All of the elements of LSI ex500may be integrated into a single chip or each of the elements may be configured as a chip.

The name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Moreover, ways to achieve integration are not limited to the LSI, and special circuit or general purpose processor and so forth can also achieve the integration. Field Programmable Gate Array (FPGA) that can be programmed after manufacturing LSI or a reconfigurable processor that allows re-configuration of the connection or configuration of LSI can be used for the same purpose.

Furthermore, when a circuit integration technology for replacing LSIs with new circuits appears in the future with advancement in semiconductor technology and derivative other technologies, the circuit integration technology may be naturally used to integrate functional blocks. Application of biotechnology is one such possibility.

As described above, the moving image coding apparatus, the moving image decoding apparatus, the moving image coding method and/or the moving image decoding method as described in the above embodiments are applicable as applications.

It is to be noted that each of the functional blocks making up the decoding apparatus100as shown inFIG. 1in Embodiment 1 is typically implemented in the form of an LSI that is an integrated circuit. Each of the functional blocks may be implemented as a separate chip such as a decoding circuit and an external memory. Alternatively, at least one or all of the functional blocks may be integrated into a single chip. In other words, these functional blocks may be implemented as an integrated system on a single LSI.

Likewise, each of the functional blocks making up the decoding apparatus200as shown inFIG. 6in Embodiment 2 is typically implemented in the form of an LSI that is an integrated circuit. Each of the functional blocks may be implemented as a separate chip such as a decoding circuit and an external memory. Alternatively, at least one or all of the functional blocks may be integrated into a single chip. In other words, these functional blocks may be implemented as an integrated system on a single LSI.

Likewise, each of the functional blocks making up the decoding apparatus300as shown inFIG. 9in Embodiment 3 is typically implemented in the form of an LSI that is an integrated circuit. Each of the functional blocks may be implemented as a separate chip such as a decoding circuit and an external memory. Alternatively, at least one or all of the functional blocks may be integrated into a single chip. In other words, these functional blocks may be implemented as an integrated system on a single LSI.

Likewise, each of the functional blocks making up the decoding apparatus400as shown inFIG. 16in Embodiment 4 is typically implemented in the form of an LSI that is an integrated circuit. Each of the functional blocks may be implemented as a separate chip such as a decoding circuit and an external memory. Alternatively, at least one or all of the functional blocks may be integrated into a single chip. In other words, these functional blocks may be implemented as an integrated system on a single LSI.

Likewise, each of the functional blocks making up the decoding apparatus500as shown inFIG. 18in Embodiment 5 is typically implemented in the form of an LSI that is an integrated circuit. Each of the functional blocks may be implemented as a separate chip such as a coding circuit and an external memory. Alternatively, at least one or all of the functional blocks may be integrated into a single chip. In other words, these functional blocks may be implemented as an integrated system on a single LSI.

Likewise, each of the functional blocks making up the decoding apparatus600as shown inFIG. 22in Embodiment 6 is typically implemented in the form of an LSI that is an integrated circuit. Each of the functional blocks may be implemented as a separate chip such as a coding circuit and an external memory. Alternatively, at least one or all of the functional blocks may be integrated into a single chip. In other words, these functional blocks may be implemented as an integrated system on a single LSI.

Likewise, each of the functional blocks making up the decoding apparatus700as shown inFIG. 24in Embodiment 7 is typically implemented in the form of an LSI that is an integrated circuit. Each of the functional blocks may be implemented as a separate chip such as a coding circuit and an external memory. Alternatively, at least one or all of the functional blocks may be integrated into a single chip. In other words, these functional blocks may be implemented as an integrated system on a single LSI.

In addition, although reference images and filter coefficients for motion compensation are stored in the memory109or the memory509in Embodiments 1 to 7, the reference images and filter coefficients for motion compensation may not be always stored in the same memory.

As described above, in Embodiments 1 to 7, each of the memory109and the memory509is typically implemented in the form of a DDR. However, each memory is not necessarily implemented in the form of a DDR, and may be implemented in the form of an SRAM or a flip-flop. In short, any recordable devices are possible.

Up to this point, the moving image decoding apparatus, the moving image coding apparatus, the moving image decoding circuit, and the moving image decoding method according to the present invention have been described based on the embodiments. However, the present invention is not limited to these embodiments. Those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments, and also other embodiments are obtainable by arbitrarily combining the structural elements in the embodiments without materially departing from the scope of the present invention. Accordingly, all such modifications and other embodiments are intended to be included within the scope of the present invention.

INDUSTRIAL APPLICABILITY

The present invention is applicable to a moving image decoding apparatus, a moving image coding apparatus, a moving image decoding circuit, and a moving image decoding method. The present invention is particularly useful in apparatuses which decode and/or display pictures that make up a video. Examples of such apparatuses include a cellular telephone, a DVD device, a BD device, a personal computer, a television telephone, a set top box, a digital television receiver, an automobile, a security system.

REFERENCE SIGNS LIST