Patent Description:
As a video encoding scheme intended for high-efficiency transmission and accumulation of video information, an encoding scheme of the ISO/IEC <NUM>-<NUM> Advanced Video Coding (AVC) standard is described in Non Patent Literature (NPL) <NUM>. Moreover, NPL <NUM> proposes improvement in compression efficiency of video encoding, by extending (increasing) a pixel bit length of an input image upon video encoding to enhance operation precision of intra prediction and motion-compensated prediction (inter-frame prediction).

Patent Literature (PTL) <NUM> proposes switching between entropy encoding and non-compression encoding (PCM encoding) per predetermined encoded unit, to guarantee a fixed processing time for a video encoding device or a video decoding device. <CIT> discloses a video encoding device with transform means for transforming an input image; entropy encoding means for entropy-encoding output data of the transform means; non-compression encoding means for non-compression-encoding input data; and multiplexed data selection means for selecting output data of the entropy encoding means or output data of the non-compression encoding means.

<CIT> discloses an image encoding and decoding system with a pixel bit depth increase converter.

<FIG> is a block diagram showing a video encoding device obtained by simply combining the technique described in NPL <NUM> and the technique described in PTL <NUM>. Hereafter, the video encoding device shown in <FIG> is referred to as a typical video encoding device.

A structure and an operation of the typical video encoding device that receives input of each frame of digitized video and outputs a bitstream are described below, with reference to <FIG>.

The video encoding device shown in <FIG> includes a pixel bit length increasing unit <NUM>, a transformer/quantizer <NUM>, an entropy encoder <NUM>, an inverse transformer/inverse quantizer <NUM>, a buffer <NUM>, a predictor <NUM>, a PCM encoder <NUM>, a PCM decoder <NUM>, a multiplexed data selector <NUM>, a multiplexer <NUM>, a switch <NUM>, and a switch <NUM>.

The video encoding device shown in <FIG> divides each frame into blocks of <NUM> × <NUM> pixel size called macroblocks (MBs), and encodes each MB sequentially from top left of the frame. In AVC described in NPL <NUM>, each MB is further divided into blocks of <NUM> × <NUM> pixel size, and each block of <NUM> × <NUM> pixel size is encoded.

<FIG> is an explanatory diagram showing an example of block division in the case where the frame has a spatial resolution of QCIF (Quarter Common Intermediate Format). The following describes an operation of each component by focusing only on pixel values of luminance, for simplicity's sake.

The pixel bit length increasing unit <NUM> increases a pixel bit length of the block-divided input video, based on pixel bit length increase information set from outside. Let bit_depth_luma be the pixel bit length of the input video, and increased_bit_depth_luma be the pixel bit length increase information (increased pixel bit length). The pixel bit length increasing unit <NUM> shifts each pixel value of the input video to the left by increased_bit_depth_luma bits. As a result, the output data of the pixel bit length increasing unit <NUM> has a pixel bit length of bit_depth_luma + increased_bit_depth_luma bits.

A prediction signal supplied from the predictor <NUM> is subtracted from the image increased in pixel bit length which is output from the pixel bit length increasing unit <NUM>, and the resulting image is input to the transformer/quantizer <NUM>. There are two types of prediction signal, namely, an intra prediction signal and an inter-frame prediction signal. Each of the prediction signals is described below.

The intra prediction signal is a prediction signal created based on an image of a reconstructed picture that has the same display time as a current picture and is stored in the buffer <NUM>. Referring to <NUM>. <NUM> Intra_4×<NUM> prediction process for luma sample, <NUM>. <NUM> Intra_8×<NUM> prediction process for luma samples, and <NUM>. <NUM> Intra_16×<NUM> prediction process for luma samples in NPL <NUM>, intra prediction modes of three block sizes, i.e. Intra_4×<NUM>, Intra_8×<NUM>, and Intra_16×<NUM>, are available for intra prediction.

As can be understood from <FIG>, Intra_4×<NUM> and Intra_8×<NUM> are respectively intra prediction of <NUM> × <NUM> block size and <NUM> × <NUM> block size. Each circle (o) in <FIG> indicates a reference pixel used for intra prediction, i.e. a pixel of the reconstructed picture having the same display time as the current picture.

In intra prediction of Intra_4×<NUM>, reconstructed peripheral pixels are directly set as reference pixels, and used for padding (extrapolation) in nine directions shown in <FIG> to form the prediction signal. In intra prediction of Intra_8×<NUM>, pixels obtained by smoothing peripheral pixels of the image of the reconstructed picture by low-pass filters (<NUM>/<NUM>, <NUM>/<NUM>, <NUM>/<NUM>) shown under the right arrow in <FIG> are set as reference signals, and used for extrapolation in the nine directions shown in <FIG> to form the prediction signal.

As shown in <FIG>, Intra_16×<NUM> is intra prediction of <NUM> × <NUM> block size. Each circle (o) in <FIG> indicates a reference pixel used for intra prediction, i.e. a pixel of the reconstructed picture having the same display time as the current picture, as in the example shown in <FIG>. In intra prediction of Intra_16×<NUM>, peripheral pixels of the reconstructed image are directly set as reference pixels, and used for extrapolation in four directions shown in <FIG> to form the prediction signal.

Hereafter, an MB encoded using the intra prediction signal is referred to as an intra MB, a block size of intra prediction is referred to as an intra prediction mode, and a direction of extrapolation is referred to as an intra prediction direction.

The inter-frame prediction signal is a prediction signal created from an image of a reconstructed picture that has a different display time from the current picture and is stored in the buffer <NUM>. Hereafter, an MB encoded using the inter-frame prediction signal is referred to as an inter MB. A block size of the inter MB can be selected from, for example, <NUM> × <NUM>, <NUM> × <NUM>, <NUM> × <NUM>, <NUM> × <NUM>, <NUM> × <NUM>, <NUM> × <NUM>, and <NUM> × <NUM>.

<FIG> is an explanatory diagram showing an example of inter-frame prediction using <NUM> × <NUM> block size as an example. A motion vector MV = (mvx, mvy) shown in <FIG> is one of prediction parameters of inter-frame prediction, which indicates the amount of translation of an inter-frame prediction block (inter-frame prediction signal) of a reference picture relative to a block to be encoded. In AVC, the prediction parameters of inter-frame prediction include not only an inter-frame prediction direction representing a direction of the reference picture of the inter-frame prediction signal relative to a picture to be encoded of the block to be encoded, but also a reference picture index for identifying the reference picture used for inter-frame prediction of the block to be encoded. This is because, in AVC, a plurality of reference pictures stored in the buffer <NUM> can be used for inter-frame prediction.

Inter-frame prediction is described in more detail in <NUM> Inter prediction process in NPL <NUM>.

Hereafter, an MB encoded using the inter-frame prediction signal is referred to as an inter MB, a block size of inter-frame prediction is referred to as an inter prediction mode, and a direction of inter-frame prediction is referred to as an inter prediction direction.

A picture encoded including only intra MBs is called an I picture. A picture encoded including not only intra MBs but also inter MBs is called a P picture. A picture encoded including inter MBs that use not only one reference picture but two reference pictures simultaneously for inter-frame prediction is called a B picture. In the B picture, inter-frame prediction in which the direction of the reference picture of the inter-frame prediction signal relative to the picture to be encoded of the block to be encoded is to the past is called forward prediction, inter-frame prediction in which the direction of the reference picture of the inter-frame prediction signal relative to the picture to be encoded of the block to be encoded is to the future is called backward prediction, and inter-frame prediction involving both the past and the future is called bidirectional prediction.

The transformer/quantizer <NUM> frequency-transforms the image increased in pixel bit length from which the prediction signal has been subtracted (prediction error image).

The transformer/quantizer <NUM> further quantizes the frequency-transformed prediction error image (frequency transform coefficient), with a quantization step width Qs according to the increased pixel bit length increased_bit_depth_luma of the pixel bit length increasing unit <NUM>. Let Qsluma be a normal quantization step width. Then, Qs = Qsluma * <NUM>increased_bit_depth_luma, as an example. Hereafter, the quantized frequency transform coefficient is referred to as a transform quantization value.

The entropy encoder <NUM> entropy-encodes prediction parameters and the transform quantization value. The prediction parameters are information related to MB prediction, such as intra MB/inter MB, intra prediction mode, intra prediction direction, inter MB block size, and motion vector mentioned above.

The inverse transformer/inverse quantizer <NUM> inverse-quantizes the transform quantization value, with the quantization step width according to the increased pixel bit length increased_bit_depth_luma of the pixel bit length increasing unit <NUM>. The inverse transformer/inverse quantizer <NUM> further inverse-frequency-transforms the frequency transform coefficient obtained by the inverse quantization. The prediction signal is added to the reconstructed prediction error image obtained by the inverse frequency transform, and the resulting image is supplied to the switch <NUM>.

The multiplexed data selector <NUM> monitors the amount of input data per predetermined encoded unit (e.g. macroblock) to the entropy encoder <NUM>. In the case where the entropy encoder <NUM> is capable of entropy-encoding the input data within a processing time corresponding to the predetermined encoded unit, the multiplexed data selector <NUM> controls the switch <NUM> to select the output data of the entropy encoder <NUM>. As a result, the output data of the entropy encoder <NUM> is supplied to the multiplexer <NUM> via the switch <NUM>. The multiplexed data selector <NUM> further controls the switch <NUM> to select the output data of the inverse transformer/inverse quantizer <NUM>. As a result, the output data of the inverse transformer/inverse quantizer <NUM> is supplied to the buffer <NUM> via the switch <NUM>.

In the case where the entropy encoder <NUM> is not capable of entropy-encoding the input data within the processing time, the multiplexed data selector <NUM> controls the switch <NUM> to select the output data of the PCM encoder <NUM> obtained by PCM-encoding the output data of the pixel bit length increasing unit <NUM>. As a result, the output data of the PCM encoder <NUM> is supplied to the multiplexer <NUM> via the switch <NUM>. The multiplexed data selector <NUM> further controls the switch <NUM> to select the output data of the PCM decoder <NUM> obtained by PCM-decoding the output data of the PCM encoder <NUM>. As a result, the output data of the PCM decoder <NUM> is supplied to the buffer <NUM> via the switch <NUM>.

The buffer <NUM> stores the reconstructed image supplied via the switch <NUM>. The reconstructed image per frame is referred to as a reconstructed picture.

The multiplexer <NUM> multiplexes the pixel bit length increase information with the output data of the entropy encoder <NUM> and the output data of the PCM encoder <NUM>, and outputs the multiplexing result.

Based on the operation described above, the typical video encoding device creates the bitstream.

In the case of using the typical technique described above, it is possible to both enhance operation precision of intra prediction or inter-frame prediction by pixel bit length extension and guarantee a fixed processing time for a video encoding device or a video decoding device.

However, in the typical technique described above, the image increased in pixel bit length is PCM-encoded, which causes a problem that output data of PCM encoding increases by the pixel bit length increase amount despite a lack of PSNR (Peak Signal to Noise Ratio) improvement. For example, in the case where bit_depth_luma is <NUM> bits and increased_bit_depth_luma is <NUM> bits, the output data of PCM encoding is <NUM> bits, which is twice as large as the <NUM>-bit input image.

In view of this, the present invention has an object of suppressing increase of output data of PCM encoding, in video encoding based on pixel bit length increase and PCM encoding. This object is achieved with the features of the claims. The claims are directed to decoding only.

According to the present invention, through the claimed decoding device, method and recording medium it is possible to suppress increase of output data of PCM encoding, in video encoding based on pixel bit length increase and PCM encoding, which also provide the complementary effects when decoding.

A video encoding device in this reference includes: means for making a pixel bit length of an image corresponding to output data of entropy encoding and a pixel bit length of an image corresponding to output data of PCM encoding different from each other; means for increasing a pixel bit length of a decoded image of PCM decoding based on pixel bit length increase information; and means for multiplexing the pixel bit length increase information in a bitstream.

As shown in <FIG>, the video encoding device in this reference includes a pixel bit length increasing unit <NUM> for increasing a pixel bit length of a decoded image of the PCM decoder <NUM> based on pixel bit length increase information, in addition to the pixel bit length increasing unit <NUM>, the transformer/quantizer <NUM>, the entropy encoder <NUM>, the inverse transformer/inverse quantizer <NUM>, the buffer <NUM>, the predictor <NUM>, the PCM encoder <NUM>, the PCM decoder <NUM>, the multiplexed data selector <NUM>, the multiplexer <NUM>, the switch <NUM>, and the switch <NUM> included in the typical video encoding device shown in <FIG>.

When comparing <FIG> and <FIG>, it can be understood that the video encoding device in this reference supplies an input image before pixel bit length increase to the PCM encoder <NUM>, in order to make a pixel bit length of an image corresponding to output data of entropy encoding and a pixel bit length of an image corresponding to output data of PCM encoding different from each other. The image corresponding to the output data of entropy encoding is an image of input video increased in pixel bit length which is supplied to the transformer/quantizer <NUM>, and a reconstructed image of the image of the input video increased in pixel bit length which is supplied from the inverse transformer/inverse quantizer <NUM>. The image corresponding to the output data of PCM encoding is an image of input video not increased in pixel bit length which is supplied to the PCM encoder <NUM>, and a PCM-decoded image of the input video not increased in pixel bit length which is supplied from the PCM decoder <NUM>.

The pixel bit length increasing unit <NUM> increases a pixel bit length of block-divided input video, based on pixel bit length increase information set from outside.

Let bit_depth_luma be a pixel bit length of luminance of the input video, and increased_bit_depth_luma be pixel bit length increase information of luminance (increased pixel bit length). The pixel bit length increasing unit <NUM> shifts each pixel value of luminance of the input video to the left by increased_bit_depth_luma bits. As a result, the output data of the pixel bit length increasing unit <NUM> has a pixel bit length of bit_depth_luma + increased_bit_depth_luma bits. Likewise, for color difference (Cb and Cr components), let bit_depth_chroma be a pixel bit length of color difference of the input video, and increased_bit_depth_chroma be pixel bit length increase information of color difference. The pixel bit length increasing unit <NUM> shifts each pixel value of color difference of the input video to the left by increased_bit_depth_luma bits.

A prediction signal supplied from the predictor <NUM> is subtracted from the image increased in pixel bit length which is output from the pixel bit length increasing unit <NUM>, and the resulting image is input to the transformer/quantizer <NUM>. The transformer/quantizer <NUM> frequency-transforms the image increased in pixel bit length from which the prediction signal has been subtracted (prediction error image).

The transformer/quantizer <NUM> further quantizes the frequency-transformed prediction error image (frequency transform coefficient), with a quantization step width Qs according to the increased pixel bit lengths increased_bit_depth_luma and increased_bit_depth_chroma of the pixel bit length increasing unit <NUM>. Let Qsluma be a normal quantization step width of luminance. Then, Qs = Qsluma * <NUM>increased_bit_depth_luma, as an example. Hereafter, the quantized frequency transform coefficient is referred to as a transform quantization value.

The entropy encoder <NUM> entropy-encodes prediction parameters supplied from the predictor <NUM> and the transform quantization value supplied from the transformer/quantizer <NUM>. The prediction parameters are information related to macroblock prediction, such as intra MB/inter MB, intra prediction mode, intra prediction direction, inter MB block size, and motion vector.

The inverse transformer/inverse quantizer <NUM> inverse-quantizes the transform quantization value, with the quantization step width according to the increased pixel bit lengths increased_bit_depth_luma and increased_bit_depth_chroma of the pixel bit length increasing unit <NUM>. The inverse transformer/inverse quantizer <NUM> further inverse-frequency-transforms the frequency transform coefficient obtained by the inverse quantization. The prediction signal is added to the reconstructed prediction error image obtained by the inverse frequency transform, and the resulting image is supplied to the switch <NUM>.

The PCM encoder <NUM> PCM-encodes the input image before the increase of the pixel bit length. Output data pcm_sample_luma[i] of luminance of the PCM encoder <NUM> has the pixel bit length bit_depth_luma of luminance of the input video. Here, i (<NUM> ≤ i ≤ <NUM>) is an index in raster scan order within the macroblock. Likewise, output data pcm_sample_chroma[i] (i: <NUM> ≤ i ≤ <NUM>) of color difference of the PCM encoder <NUM> has the pixel bit length bit_depth_chroma of color difference of the input video.

The PCM decoder <NUM> PCM-decodes pcm_sample_luma[i] and pcm_sample_chroma[i]. Hereafter, PCM decoding is also referred to as PCM data reading.

The pixel bit length increasing unit <NUM> shifts PCM-data-read pcm_sample_luma[i] to the left by increased_bit_depth_luma bits. As a result, a reconstructed image obtained via the PCM decoder <NUM> has bit_depth_luma + increased_bit_depth_luma bits, and is supplied to the switch <NUM>. Likewise, PCM-data-read pcm_sample_chroma[i] is shifted to the left by increased_bit_depth_chroma bits, and supplied to the switch <NUM>.

In the case where the entropy encoder <NUM> is not capable of entropy-encoding the input data within the processing time, the multiplexed data selector <NUM> first causes the entropy encoder <NUM> to encode and output information indicating that the macroblock is an intra MB of PCM. In detail, when complying with <NUM>. <NUM> Macroblock layer syntax in NPL <NUM>, mb_type is entropy-encoded and output as I_PCM.

Following this, the output bit of the entropy encoder <NUM> is byte-aligned. In detail, when complying with <NUM>. <NUM> Macroblock layer syntax in NPL <NUM>, the entropy encoder <NUM> supplies a predetermined number of pcm_alignment_zero_bit to the multiplexer <NUM>. Moreover, the entropy encoder <NUM> initializes an encoding engine, for subsequent encoding.

An example of encoding engine initialization is described in <NUM>. <NUM> Initialization process for the arithmetic encoding engine (informative) in NPL <NUM>.

The multiplexed data selector <NUM> further controls the switch <NUM> to select the output data of the PCM encoder <NUM>. As a result, the output data of the PCM encoder <NUM> is supplied to the multiplexer <NUM> via the switch <NUM>.

Lastly, the multiplexed data selector <NUM> controls the switch <NUM> to select the output data of the pixel bit length increasing unit <NUM>. As a result, the output data of the pixel bit length increasing unit <NUM> is supplied to the buffer <NUM> via the switch <NUM>. Here, the pixel bit length increasing unit <NUM> increases the number of bits by shifting, to the left by increased_bit_depth_luma bits, the output data pcm_sample_luma[i] of the PCM decoder <NUM> obtained by reading the output data pcm_sample_luma[i] of the PCM encoder <NUM>. Likewise, the pixel bit length increasing unit <NUM> increases the number of bits by shifting, to the left by increased_bit_depth_chroma bits, the output data pcm_sample_chroma[i] of the PCM decoder <NUM> obtained by reading the output data pcm_sample_chroma[i] of the PCM encoder <NUM>.

The multiplexer <NUM> multiplexes the pixel bit length increase information with the output data of the entropy encoder <NUM> and the output data of the PCM encoder <NUM>, and outputs the multiplexing result. When complying with Specification of syntax functions, categories, and descriptors in NPL <NUM>, the pixel bit length increase information (increased_bit_depth_luma and increased_bit_depth_chroma) may be multiplexed following bit_depth_luma_minus8 and bit_depth_chroma_minus8 of sequence parameters, as in the list shown in <FIG>. Here, bit_depth_luma_minus8 is a value obtained by subtracting <NUM> from the pixel bit length bit_depth_luma of luminance of the input video, bit_depth_chroma_minus8 is a value obtained by subtracting <NUM> from the pixel bit length bit_depth_chroma of color difference of the input video, increased_bit_depth_luma is the increased pixel bit length of luminance, and increased_bit_depth_chroma is the increased pixel bit length of color difference.

The expressions ("C" and "Descriptor") in the list shown in <FIG> are, for example, in compliance with <NUM> Specification of syntax functions, categories, and descriptors in NPL <NUM>.

Based on the operation described above, the video encoding device in this reference creates the bitstream.

Operations of the entropy encoder <NUM>, the PCM encoder <NUM>, the PCM decoder <NUM>, and the pixel bit length increasing unit <NUM> in the case of not being capable of entropy-encoding within the processing time are described below with reference to the flowchart in <FIG>.

As shown in <FIG>, in step S101, the entropy encoder <NUM> entropy-encodes mb_type as I_PCM and supplies it to the multiplexer <NUM>, in order to guarantee a fixed processing time for a video encoding device or a video decoding device.

In step S102, the entropy encoder <NUM> supplies pcm_alignment_zero_bit to the multiplexer <NUM>, to byte-align the output bit.

In step S103, the entropy encoder <NUM> initializes the encoding engine for subsequent entropy encoding.

In step S104, the PCM encoder <NUM> PCM-encodes the input image before the increase of the pixel bit length and supplies it to the multiplexer <NUM>, so as not to increase output data of PCM encoding.

In step S105, the PCM decoder <NUM> PCM-decodes (PCM-data-reads) the PCM encoding result pcm_sample_luma[i] and pcm_sample_chroma[i].

In step S106, the pixel bit length increasing unit <NUM> shifts pcm_sample_luma[i] and pcm_sample_chroma[i] PCM-data-read by the PCM decoder <NUM> to the left respectively by increased_bit_depth_luma bits and increased_bit_depth_chroma bits, in order to enhance operation precision of subsequent intra prediction and inter-frame prediction.

Thus, in the case of not being capable of entropy-encoding within the processing time corresponding to the predetermined encoded unit, the entropy encoder <NUM> and the PCM encoder <NUM> operate as described above.

In the video encoding device in this reference, the input image before the increase of the pixel bit length is supplied to the PCM encoder <NUM>, in order to make the pixel bit length of the image corresponding to the output data of entropy encoding and the pixel bit length of the image corresponding to the output data of PCM encoding different from each other. Such a structure enables suppression of increase of output data of PCM encoding, in video encoding based on pixel bit length increase and non-compression encoding.

Moreover, the video encoding device in this reference includes the pixel bit length increasing unit <NUM> for increasing the pixel bit length of the decoded image of PCM decoding based on the pixel bit length increase information. The pixel bit length increasing unit <NUM> can suppress reduction of operation precision of intra prediction and inter-frame prediction caused by making the pixel bit lengths different from each other.

Furthermore, in the video encoding device in this reference, the multiplexer <NUM> multiplexes the pixel bit length increase information in the bitstream so that the pixel bit length of the decoded image of PCM decoding is equally increased in video decoding. Such a structure contributes to enhanced interoperability of the video encoding device and the video decoding device. That is, the video encoding device and the video decoding device co-operate with each other, with it being possible to suppress increase of PCM encoding in the system and also suppress reduction of operation precision of intra prediction and inter-frame prediction.

A video decoding device in this exemplary embodiment decodes a bitstream in which a pixel bit length of an image corresponding to input data of entropy decoding means and a pixel bit length of an image corresponding to input data of PCM decoding means are different from each other. The image corresponding to the input data of the entropy decoding means is a reconstructed image of an image of input video increased in pixel bit length which is supplied from an inverse transformer/inverse quantizer <NUM> described later. The image corresponding to the input data of the PCM decoding means is a PCM-decoded image of input video not increased in pixel bit length which is supplied from a PCM decoder <NUM> described later.

As shown in <FIG>, the video decoding device in this exemplary embodiment includes a de-multiplexer <NUM>, a decoding controller <NUM>, the PCM decoder <NUM>, an entropy decoder <NUM>, a pixel bit length increasing unit <NUM>, the inverse transformer/inverse quantizer <NUM>, a predictor <NUM>, a buffer <NUM>, a pixel bit length decreasing unit <NUM>, a switch <NUM>, and a switch <NUM>.

The de-multiplexer <NUM> de-multiplexes an input bitstream, to extract pixel bit length increase information and an entropy-encoded or PCM-encoded video bitstream. When complying with Specification of syntax functions, categories, and descriptors in NPL <NUM>, the pixel bit length increase information (increased_bit_depth_luma and increased_bit_depth_chroma) following bit_depth_luma_minus8 and bit_depth_chroma_minus8 of the sequence parameters as in the list shown in <FIG> is extracted.

The entropy decoder <NUM> entropy-decodes the video bitstream. In the case where mb_type of a macroblock is not I_PCM (PCM encoding), the entropy decoder <NUM> entropy-decodes prediction parameters and a transform quantization value of the macroblock, and supplies them to the inverse transformer/inverse quantizer <NUM> and the predictor <NUM>.

The inverse transformer/inverse quantizer <NUM> inverse-quantizes the transform quantization value of luminance and color difference, with a quantization step width according to the pixel bit length increase information increased_bit_depth_luma and increased_bit_depth_chroma extracted by the de-multiplexing. The inverse transformer/inverse quantizer <NUM> further inverse-frequency-transforms the frequency transform coefficient obtained by the inverse quantization.

The predictor <NUM> creates a prediction signal using an image of a reconstructed picture stored in the buffer <NUM>, based on the entropy-decoded prediction parameters.

The prediction signal supplied from the predictor <NUM> is added to the reconstructed prediction error image obtained by the inverse frequency transform by the inverse transformer/inverse quantizer <NUM>, and the resulting image is supplied to the switch <NUM>.

The decoding controller <NUM> changes the switch <NUM> so that the reconstructed prediction error image to which the prediction signal has been added is supplied to the buffer <NUM> as the reconstructed image.

In the case where mb_type of the macroblock is PCM encoding, the decoding controller <NUM> causes the de-multiplexer <NUM> to byte-align the video bitstream which is in the middle of entropy decoding. When complying with <NUM>. <NUM> Macroblock layer syntax in NPL <NUM>, the decoding controller <NUM> causes the de-multiplexer <NUM> to read pcm_alignment_zero_bit until the video bitstream is byte-aligned.

The decoding controller <NUM> then causes the entropy decoder <NUM> to initialize a decoding engine. An example of decoding engine initialization is described in <NUM>. <NUM> Initialization process for the arithmetic decoding engine in NPL <NUM>.

Following this, the decoding controller <NUM> changes the switch <NUM> so that the byte-aligned video bitstream is supplied to the PCM decoder <NUM>.

The PCM decoder <NUM> PCM-decodes (PCM-data-reads) PCM-encoded luminance data pcm_sample_luma[i] and color difference data pcm_sample_chroma[i] from the byte-aligned video bitstream.

The pixel bit length increasing unit <NUM> shifts PCM-data-read pcm_sample_luma[i] and pcm_sample_chroma[i] to the left, respectively according to the pixel bit length increase information increased_bit_depth_luma and increased_bit_depth_chroma extracted by the de-multiplexing. When complying with the description of <NUM>. <NUM> Sample construction process for I_PCM macroblocks in NPL <NUM>, a PCM-decoded luminance image S'L and a PCM-decoded color difference image S'Cb and S'Cr are computed according to Equation (<NUM>-<NUM>') and Equation (<NUM>-<NUM>') below. <IMG>
<IMG>.

The decoding controller <NUM> changes the switch <NUM> so that the PCM-decoded image increased in pixel bit length is supplied to the buffer <NUM> as the reconstructed image. The decoding controller <NUM> changes the switch <NUM> so that the output data of the de-multiplexer <NUM> is supplied to the entropy decoder <NUM>, for decoding of a next macroblock.

The pixel bit length decreasing unit <NUM> decreases the pixel bit length of the reconstructed picture stored in the buffer <NUM> according to the pixel bit length increase information increased_bit_depth_luma and increased_bit_depth_chroma extracted by the de-multiplexing, and outputs the result.

Based on the operation described above, the video decoding device in this exemplary embodiment creates the decoded image.

Operations of the decoding controller <NUM>, the entropy decoder <NUM>, the PCM decoder <NUM>, and the pixel bit length increasing unit <NUM> in the case where mb_type of the macroblock is PCM encoding, which are features of the present invention, are described below with reference to the flowchart in <FIG>.

In step S201, the de-multiplexer <NUM> reads pcm_alignment_zero_bit so as to byte-align the video bitstream which is in the middle of entropy decoding.

In step S202, the entropy decoder <NUM> initializes the decoding engine for subsequent entropy decoding.

In step S203, the PCM decoder <NUM> PCM-decodes (PCM-data-reads) the PCM encoding result pcm_sample_luma[i] and pcm_sample_chroma[i].

In step S204, the pixel bit length increasing unit <NUM> shifts PCM-data-read pcm_sample_luma[i] and pcm_sample_chroma[i] to the left respectively by increased_bit_depth_luma bits and increased_bit_depth_chroma bits, in order to enhance operation precision of subsequent intra prediction and inter-frame prediction.

Thus, in the case where mb_type of the macroblock is PCM encoding, the decoding controller <NUM>, the entropy decoder <NUM>, the PCM decoder <NUM>, and the pixel bit length increasing unit <NUM> operate as described above.

The video decoding device in this exemplary embodiment includes the pixel bit length increasing unit <NUM> for increasing the pixel bit length of the decoded image of PCM decoding based on the pixel bit length increase information extracted by the de-multiplexing. The pixel bit length increasing unit <NUM> can suppress reduction of operation precision of intra prediction and inter-frame prediction caused by making the pixel bit lengths of the images corresponding to the inputs of the entropy decoding means and the PCM decoding means different from each other. Moreover, the reconstructed image same as in video decoding can be obtained, which contributes to enhanced interoperability of the video encoding device and the video decoding device. That is, the video encoding device and the video decoding device co-operate with each other, with it being possible to suppress increase of PCM encoding in the system and also suppress reduction of operation precision of intra prediction and inter-frame prediction.

The video encoding device in Reference <NUM> shown in <FIG> is a video encoding device that supplies the input image before the increase of the pixel bit length to the PCM encoder <NUM>, in order to make the pixel bit length of the image corresponding to the output data of entropy encoding and the pixel bit length of the image corresponding to the output data of PCM encoding different from each other.

<FIG> is a block diagram showing a video encoding device of another structure (not forming part of the claimed invention) for achieving the same advantageous effects as the video encoding device shown in <FIG>.

When compared with the video encoding device shown in <FIG>, the video encoding device shown in <FIG> additionally includes a pixel bit length decreasing unit <NUM>. That is, the video encoding device shown in <FIG> has a structure in which the pixel bit length decreasing unit <NUM> that receives the image increased in pixel bit length supplies, to the PCM encoder <NUM>, the image decreased in pixel bit length based on the pixel bit length increase information. As in Reference <NUM>, the video encoding device shown in <FIG> can suppress increase of output data of PCM encoding, and also suppress reduction of operation precision of intra prediction and inter-frame prediction caused by making the pixel bit lengths different from each other.

In the above explanation, the pixel of the reconstructed picture is a pixel increased in pixel bit length. For size reduction of the buffer for storing the reconstructed picture, however, a configuration in which the above-mentioned pixel bit length increasing unit and pixel bit length decreasing unit are used for input/output of the buffer is also conceivable. In such configuration, too, the suppression of increase of output data of PCM encoding and the suppression of reduction of operation precision of intra prediction caused by making the pixel bit lengths different from each other can both be achieved.

In the above explanation, the PCM decoder and the pixel bit length increasing unit are independent functional blocks. As can be easily understood from Equation (<NUM>-<NUM>') and Equation (<NUM>-<NUM>'), however, the PCM decoder and the pixel bit length increasing unit may be integrated as one functional block.

In the above explanation, the video encoding device multiplexes increased_bit_depth_luma and increased_bit_depth_chroma in the bitstream following bit_depth_luma_minus8 and bit_depth_chroma_minus8, in order to explicitly signal the pixel bit length increase information to the video decoding device (see <FIG>). Alternatively, the video encoding device may multiplex, as the pixel bit length increase information, pixel bit length information after the increase of the pixel bit length in the bitstream, in order to implicitly signal the pixel bit length increase information to the video decoding device (it is assumed here that the original pixel bit length of the input video is, for example, <NUM> bits in the video encoding device and the video decoding device).

In this case, the video encoding device multiplexes pixel bit length increase information (internal_bit_depth_luma_minus8 and internal_bit_depth_chroma_minus8) shown in <FIG> in the sequence parameters, instead of bit_depth_luma_minus8 and bit_depth_chroma_minus8 of the sequence parameters. Here, internal_bit_depth_luma_minus8 is the value of increased_bit_depth_luma, and internal_bit_depth_chroma_minus8 is the value of increased_bit_depth_chroma.

In the case of multiplexing the pixel bit length increase information shown in <FIG> in the sequence parameters, the PCM encoder <NUM> PCM-encodes the input image before the increase of the pixel bit length. That is, the PCM encoder <NUM> PCM-encodes <NUM>-bit pcm_sample_luma[i] and pcm_sample_chroma[i]. The PCM decoder <NUM> PCM-decodes <NUM>-bit pcm_sample_luma[i] and pcm_sample_chroma[i]. The pixel bit length increasing unit <NUM> shifts PCM-decoded pcm_sample_luma[i] and pcm_sample_chroma[i] to the left respectively by increased_bit_depth_luma bits and increased_bit_depth_chroma bits.

A video decoding device corresponding to the case of multiplexing the pixel bit length increase information shown in <FIG> in the sequence parameters de-multiplexes the pixel bit length increase information (internal_bit_depth_luma_minus8 and internal_bit_depth_chroma_minus8) from the sequence parameters, and computes increased_bit_depth_luma and increased_bit_depth_chroma as follows. <MAT> <MAT>.

By the above-mentioned computation, the video decoding device can de-multiplex the pixel bit length increase information implicitly signaled by the video encoding device.

In the above-mentioned case where the video encoding device implicitly signals the pixel bit length increase information to the video decoding device, there is a problem that PCM encoding cannot be performed due to non-distortion when the original pixel bit length of the input video is longer than <NUM> bits. For example, quantization distortion occurs with <NUM>-bit pcm_sample_luma[i] and pcm_sample_chroma[i] when the original pixel bit length of the input video is <NUM> bits.

To support PCM encoding without quantization distortion when the original pixel bit length of the input video is N bits (N > <NUM>), pcm_sample_bit_depth_is_internal_bit_depth_flag which is a flag indicating whether or not the bit length of PCM is the pixel bit length after the pixel bit length increase may be added to the sequence parameters as shown in <FIG>.

In the case where pcm_sample_bit_depth_is_internal_bit_depth_flag is <NUM>, the PCM encoder <NUM> PCM-encodes the input image before the increase of the pixel bit length. That is, the PCM encoder <NUM> PCM-encodes <NUM>-bit pcm_sample_luma[i] and pcm_sample_chroma[i]. The PCM decoder <NUM> PCM-decodes <NUM>-bit pcm_sample_luma[i] and pcm_sample_chroma[i]. The pixel bit length increasing unit <NUM> shifts PCM-decoded pcm_sample_luma[i] and pcm_sample_chroma[i] to the left respectively by increased_bit_depth_luma (= internal_bit_depth_luma_minus8) bits and increased_bit_depth_chroma (= internal_bit_depth_chroma_minus8) bits.

In the case where pcm_sample_bit_depth_is_internal_bit_depth_flag is <NUM>, the PCM encoder <NUM> PCM-encodes the image increased in pixel bit length. That is, the PCM encoder <NUM> PCM-encodes pcm_sample_luma[i] of N bits (internal_bit_depth_luma_minus8 + <NUM> bits) and pcm_sample_chroma[i] of N bits (internal_bit_depth_chroma_minus8 + <NUM> bits). The PCM decoder <NUM> PCM-decodes pcm_sample_luma[i] of N bits and pcm_sample_chroma[i] of N bits. The pixel bit length increasing unit <NUM> shifts PCM-decoded pcm_sample_luma[i] and pcm_sample_chroma[i] to the left by <NUM> bit (i.e. does not shift PCM-decoded pcm_sample_luma[i] and pcm_sample_chroma[i] to the left).

To support PCM encoding without quantization distortion when the original pixel bit length of the input video is N bits (N > <NUM>), pcm_sample_bit_depth_luma_minus8 and pcm_sample_bit_depth_chroma_minus8 which are respectively the bit lengths of PCM of luminance and color difference may be added to the sequence parameters instead of pcm_sample_bit_depth_is_internal_bit_depth_flag, as shown in <FIG>.

In the case of adding pcm_sample_bit_depth_luma_minus8 and pcm_sample_bit_depth_chroma_minus8 to the sequence parameters, the PCM encoder <NUM> PCM-encodes pcm_sample_luma[i] of pcm_sample_bit_depth_luma_minus8 + <NUM> bits and pcm_sample_croma[i] of pcm_sample_bit_depth_chroma_minus8 + <NUM> bits. In the case of adding pcm_sample_bit_depth_luma_minus8 and pcm_sample_bit_depth_chroma_minus8 to the sequence parameters, the PCM decoder <NUM> PCM-decodes pcm_sample_luma[i] of pcm_sample_bit_depth_luma_minus8 + <NUM> bits and pcm_sample_croma[i] of pcm_sample_bit_depth_chroma_minus8 + <NUM> bits. The pixel bit length increasing unit <NUM> shifts PCM-decoded pcm_sample_luma[i] and pcm_sample_chroma[i] to the left respectively by increased_bit_depth_luma bits and increased_bit_depth_chroma bits. Here, increased_bit_depth_luma and increased_bit_depth_chroma are computed as follows. <MAT> <MAT>.

It is clear from the above-mentioned computation that the video encoding device implicitly signals the pixel bit length increase information to the video decoding device in the case where increased_bit_depth_luma is more than <NUM> and also internal_bit_depth_luma_minus8 + <NUM> is less than N, and equally the video encoding device implicitly signals the pixel bit length increase information to the video decoding device in the case where internal_bit_depth_chroma_minus8 + <NUM> is less than N.

The above-explained configurations may be realized by hardware, or may be realized by a computer program.

An information processing system shown in <FIG> includes a processor <NUM>, a program memory <NUM>, a storage medium <NUM> for storing video data, and a storage medium <NUM> for storing a bitstream. The storage medium <NUM> and the storage medium <NUM> may be separate storage media, or may be a storage area composed of the same storage medium. As a storage medium, a magnetic storage medium such as a hard disk is applicable.

In the information processing system shown in <FIG>, a program for realizing the functions of the blocks (except the block of the buffer) shown in each of <FIG>, <FIG>, and <FIG> is stored in the program memory <NUM>. The processor <NUM> realizes the functions of the video encoding device or the video decoding device shown in <FIG>, <FIG>, or <FIG>, by executing processing according to the program stored in the program memory <NUM>.

<FIG> is a block diagram showing a main part of a video encoding device. As shown in <FIG>, the video encoding device includes: pixel bit length increasing means <NUM> (e.g. the pixel bit length increasing unit <NUM> shown in <FIG>) for increasing a pixel bit length of an input image based on pixel bit length increase information; transform means <NUM> (e.g. the transformer/quantizer <NUM> shown in <FIG>) for transforming output data of the pixel bit length increasing means <NUM>; entropy encoding means <NUM> (e.g. the entropy encoder <NUM> shown in <FIG>) for entropy-encoding output data of the transform means <NUM>; non-compression encoding means <NUM> (e.g. the PCM encoder <NUM>) for non-compression-encoding input data; multiplexed data selection means <NUM> (e.g. the switch <NUM>) for selecting output data of the entropy encoding means <NUM> or output data of the non-compression encoding means <NUM>; and multiplexing means <NUM> (e.g. the multiplexer <NUM>) for multiplexing the pixel bit length increase information in a bitstream, wherein a pixel bit length of an image corresponding to the output data of the entropy encoding means <NUM> and a pixel bit length of an image corresponding to the output data of the non-compression encoding means <NUM> are different from each other.

To make the pixel bit lengths different from each other, the video encoding device includes, as an example, means for supplying the input image before the increase of the pixel bit length to the non-compression encoding means <NUM>. In such a case, the input image not increased in pixel bit length is non-compression-encoded (e.g. PCM-encoded).

<FIG> is a block diagram showing a main part of another video encoding device. As shown in <FIG>, in addition to the structure shown in <FIG>, the other video encoding device includes pixel bit length decreasing means <NUM> (e.g. the pixel bit length decreasing unit <NUM> shown in <FIG>) for decreasing a pixel bit length based on the pixel bit length increase information, wherein the input data of the non-compression encoding means <NUM> is output data of the pixel bit length decreasing means <NUM>.

<FIG> is a block diagram showing a main part of another video encoding device. As shown in <FIG>, in addition to the structure shown in <FIG>, the other video encoding device includes: prediction means <NUM> (e.g. the predictor <NUM> shown in <FIG>) for predicting an image; inverse transform means <NUM> (e.g. the inverse transformer/inverse quantizer <NUM> shown in <FIG>) for inverse-transforming the output data of the transform means <NUM>; and non-compression decoding means <NUM> (e.g. the PCM decoder <NUM> shown in <FIG>) for decoding the output data of the non-compression encoding means <NUM>, wherein the non-compression decoding means <NUM> increases a pixel bit length of a decoded image obtained by non-compression decoding, based on at least the pixel bit length increase information.

<FIG> is a block diagram showing a main part of a video decoding device according to the present invention. As shown in <FIG>, the video decoding device according to the present invention includes: de-multiplexing means <NUM> (e.g. the de-multiplexer <NUM> shown in <FIG>) for de-multiplexing a bitstream including at least pixel bit length increase information; entropy decoding means <NUM> (e.g. the entropy decoder <NUM> shown in <FIG>) for entropy-decoding transformed data of an image included in the bitstream; inverse transform means <NUM> (e.g. the inverse transformer/inverse quantizer <NUM> shown in <FIG>) for inverse-transforming the entropy-decoded transformed data of the image; non-compression decoding means <NUM> (e.g. the PCM decoder <NUM> shown in <FIG>) for non-compression-decoding non-compression-encoded data of an image included in the bitstream; and decoding control means <NUM> (e.g. the decoding controller <NUM> shown in <FIG>) for controlling the entropy decoding means <NUM> and the non-compression decoding means <NUM>, wherein a pixel bit length of an image corresponding to input data of the entropy decoding means <NUM> and a pixel bit length of an image corresponding to input data of the non-compression decoding means <NUM> are different from each other.

<FIG> is a block diagram showing a main part of another video decoding device according to the present invention. As shown in <FIG>, in addition to the structure shown in <FIG>, the video decoding device according to the present invention includes prediction means <NUM> (e.g. the predictor <NUM> shown in <FIG>) for predicting an image.

As described above, means is provided for making a pixel bit length of an image corresponding to output data of entropy encoding and a pixel bit length of an image corresponding to output data of non-compression encoding different from each other, in video encoding based on pixel bit length increase and non-compression encoding. Thus, a problem can be solved that output data of PCM encoding increases by the pixel bit length increase amount, while both enhancing operation precision of intra prediction and inter-frame prediction by pixel bit length extension and guaranteeing a fixed processing time for a video encoding device or a video decoding device.

Though the present invention has been described with reference to the above exemplary embodiments and examples, the present invention is not limited to the above exemplary embodiments and examples. Various changes understandable by those skilled in the art within the scope of the present claims can be made to the structures and details of the present invention.

Claim 1:
A video decoding device comprising:
a de-multiplexer (<NUM>) configured to de-multiplex a bitstream including pixel bit length increase information;
an entropy decoding unit (<NUM>) configured to entropy-decode transformed data of an image in the bitstream;
a non-compression decoding unit (<NUM>) configured to non-compression-decode non-compression-encoded data of an image in the bitstream; and
a decoding control unit (<NUM>) configured to control the entropy decoding unit (<NUM>) and the non-compression decoding unit (<NUM>),
wherein a pixel bit length of an image corresponding to input data of the non-compression decoding unit (<NUM>) is less than a pixel bit length of an image corresponding to input data of the entropy decoding unit (<NUM>), and
wherein the video decoding device further comprises a pixel bit length increasing unit (<NUM>) configured to increase the pixel bit length of an image corresponding to input data of the non-compression decoding unit (<NUM>) so as to be equal to the pixel bit length of an image corresponding to input data of the entropy decoding unit (<NUM>), by shifting non-compression decoded data to the left according to the pixel bit length increase information.