Source: https://patents.google.com/patent/JP4659005B2/en
Timestamp: 2019-12-15 15:10:35
Document Index: 230592960

Matched Legal Cases: ['art 102', 'art 103', 'art 104', 'art 105', 'art 106', 'art 107', 'art 108', 'art 109']

JP4659005B2 - Moving picture encoding method, decoding method, encoding apparatus, decoding apparatus based on texture synthesis, program thereof, and recording medium thereof - Google Patents
Moving picture encoding method, decoding method, encoding apparatus, decoding apparatus based on texture synthesis, program thereof, and recording medium thereof Download PDF
JP4659005B2
JP4659005B2 JP2007212682A JP2007212682A JP4659005B2 JP 4659005 B2 JP4659005 B2 JP 4659005B2 JP 2007212682 A JP2007212682 A JP 2007212682A JP 2007212682 A JP2007212682 A JP 2007212682A JP 4659005 B2 JP4659005 B2 JP 4659005B2
JP2007212682A
JP2009049626A (en
和也 早瀬
勇 本吉
眞也 西田
誠之 高村
2007-08-17 Application filed by 日本電信電話株式会社 filed Critical 日本電信電話株式会社
2007-08-17 Priority to JP2007212682A priority Critical patent/JP4659005B2/en
2009-03-05 Publication of JP2009049626A publication Critical patent/JP2009049626A/en
2011-03-30 Publication of JP4659005B2 publication Critical patent/JP4659005B2/en
The present invention relates to a high-efficiency image signal encoding method and decoding method, and in particular, a moving image code based on texture synthesis that enables two advantages of maintaining subjective image quality and maintaining waveform fidelity to be used according to image characteristics. The present invention relates to the conversion and decoding method.
In AVC and JSVC (see Non-Patent Document 1), a prediction signal is generated by intra-frame prediction, inter-frame prediction, and inter-layer prediction, and the prediction signal and the residual signal of the original signal are to be encoded. These conventional methods aim to accurately reproduce the original signal waveform. However, in these prediction processes, it is impossible to predict high-frequency components above a certain level due to the structure. For this reason, the prediction performance deteriorates for an image containing many high-frequency components. For example, such a decline in prediction performance is noticeable in scenes such as water surfaces where a rigid body movement model is not established in the case of inter-frame prediction, and in the case of intra-frame prediction and inter-layer prediction, in a fine-pattern texture area. It becomes.
On the other hand, a technique called texture synthesis is being studied. In texture synthesis, the difference is that the feature value of the texture of the original signal is to be reproduced rather than pursuing accurate waveform reproduction of the original signal. As a texture synthesis method, for example, there is a method proposed in Non-Patent Document 2. In this method, a histogram of coefficients obtained as an output of a directional filter bank called “Steerable pyramid” is used as an input of texture synthesis processing as a texture feature amount. A signal generated by texture synthesis is not guaranteed to be similar to the original signal from the viewpoint of waveform reproduction, but subjectively, a signal having an image quality close to that of the original signal can be obtained. For this reason, application of texture synthesis to moving picture coding is expected.
Non-Patent Document 3 describes H.264. JM, which is H.264 reference software, is described.
H. Schwarz, and D. Marpe, and T. Wiegand, Overview of the Scalable H.264 / MPEG4-AVC Extension, IEEE International Conference on Image Processing (ICIP'06), Atlanta, GA, USA, October 2006. D. Heeger, and J. Bergen, Pyramid-Based Texture Analysis / Synthesis. Proc ACM SIGGRAPH 95, 1995. KPLim and G. Sullivan and T. Wiegand, Text Description of Joint Model Reference Encoding Methods and Decoding Concealment Methods. Joint Video Team (JVT) of ISO / IEC MPEG and ITU-T VCEG, JVT-R95, Jan., 2006.
In the following, we consider a video coding method that divides a frame into blocks and adaptively switches between inter prediction, intra prediction, inter-layer prediction (in the case of scalable coding), and texture synthesis for each block. Here, a block generated by texture synthesis is called a “texture synthesis block”, and a block that obtains a decoded signal by encoding a prediction residual for inter prediction, intra prediction, and inter-layer prediction is called a “waveform reproduction block”.
When texture synthesis is applied to video coding, the texture synthesis block and the waveform reproduction block are mixed in the same frame, and the decoded signal of the texture synthesis block degrades the decoded signal of the waveform reproduction block. . An example is shown using FIG.
FIG. 1 shows a case where a texture synthesis block and a waveform reproduction block are mixed in a reference block for inter-frame prediction. The shaded area indicated by (1) in the t-1 frame is a part of the texture synthesis block, and the white area indicated by (2) in the t-1 frame is a part of the waveform reproduction block. It is. In the shaded area shown in (1), the correlation of the waveform level is low with respect to the block to be predicted in the t-th frame, and the possibility that the prediction residual can be reduced by inter-frame prediction is low. For this reason, when the interframe prediction mechanism in the conventional AVC, JSVC, or the like is used, the prediction residual increases conversely, and the coding efficiency may decrease.
Another example is shown using FIG. In FIG. 2, the shaded area indicated by (3) in the t-1 frame is a part of the texture synthesis block, and the white areas indicated by (1) and (2) in the t-1 frame are , Part of the waveform reproduction block. In block-based coding schemes such as AVC and JSVC, deblocking filters are applied to block boundaries in order to suppress block distortion between adjacent blocks. That is, the signal at the block boundary of the waveform reproduction block is linearly weighted with the signal of the texture synthesis block. By such filter processing, there is a possibility that the reproducibility of the decoded signal is deteriorated (PSNR is lowered) at the block boundary portion of the waveform reproduction block.
The present invention has been made in view of such circumstances, and in the video coding based on texture synthesis, the PSNR of the waveform reproduction block is lowered by mixing the texture synthesis block and the waveform reproduction block in the same frame. It aims at establishing the adaptive process for avoiding.
[Reference method of texture synthesis block]
When a texture synthesis block and a waveform reproduction block coexist in a reference area in inter-frame prediction, the following adaptive processing is performed according to the block type. When referring to an area included in the texture synthesis block, the reference signal is a decoded signal of the pixel in the predicted block of the encoding target frame. That is, no prediction residual is added to the prediction signal. On the other hand, when referring to an area included in the waveform reproduction block, difference information between the reference signal and the original signal is encoded as a prediction residual signal. Since the texture synthesis block and the waveform reproduction block in the reference block can be identified by the encoded information in the reference frame, it is not necessary to add new additional information for this identification.
[Deblocking method of waveform reproduction block adjacent to texture synthesis block]
When the texture synthesis block and the waveform reproduction block are adjacent, the signal after deblocking filter processing is used only as the decoded signal of the corresponding frame for the block boundary of the waveform reproduction block adjacent to the texture synthesis block. As a reference signal, a signal before deblocking filter processing is used.
According to the present invention, in the moving image coding based on texture synthesis, it is possible to avoid the PSNR of the waveform reproduction block from being lowered by mixing the texture synthesis block and the waveform reproduction block in the same frame. As a result, the advantages of maintaining subjective image quality based on texture synthesis and waveform fidelity based on reproduction of prediction residuals can be used according to the characteristics of the image.
[Flowchart (Reference method of texture synthesis block)]
An embodiment of the [texture synthesis block reference method] of the present invention will be described in accordance with the flow of the encoding process shown in FIG.
Step S101: For all frames that can serve as reference frames for the current frame to be encoded, mode information (information representing the distinction between texture synthesis block and waveform reproduction block) of each block in the same frame Even after the decoding process, it is held in the register together with the spatial position information in the frame of each block.
Step S102: The signal of the block in the frame (referred to as the block signal) and the signal in the frame (can be a plurality of frames) that are candidates for the reference frame are input, and the motion estimation process between the frames is performed. A process of calculating a motion vector indicating the association and information for designating a reference frame (referred to as a reference frame index) is performed and written to a register. In addition, when dividing the block into sub-blocks and performing inter-frame motion estimation processing for each sub-block, a motion vector and a reference frame index are calculated for each sub-block, along with information indicating block division, Write to register. The specific estimation method of these parameters shall be given from the outside. For example, H.M. The estimation method used by JM, which is H.264 reference software [see Non-Patent Document 3], and JSVM, which is JSVC reference software, is used.
Step S103: The mode information output in step S101, the motion vector output in step S102, and the reference frame index are read as inputs, and the mode information included in the reference block is written to the register.
Step S104: Using the mode information read in Step S103 as an input, it is determined whether or not a texture synthesis block is included in the reference block. If included, the process proceeds to Step S105, otherwise. , The process proceeds to step S109.
Step S105: Using the mode information output in Step S103 as an input, for each pixel position in the reference block, a binary value in which 1 is assigned to the pixel position belonging to the waveform reproduction block and 0 is assigned to the pixel position belonging to the texture synthesis block Is written in a register as a pixel class map.
Step S106: The pixel class map, the block signal, and the reference block signal (referred to as reference block signal) output in step S105 are input, and the block signal and reference are referenced for the pixel at the position where the pixel class map is 1. A process of generating a prediction residual signal as a difference signal from the block signal is performed, and the difference signal is written to the register as a prediction residual signal.
Step S107: Using the pixel class map output in step S105, the block signal, and the reference block signal (referred to as a reference block signal) as inputs, a prediction residual signal is output for a pixel at a position where the pixel class map is 0. Padding processing to set to zero is performed, and the result is written to the register as a prediction residual signal.
Step S108: Using the prediction residual signal output in step S106 or S107 as input, orthogonal transformation, quantization, and entropy coding are performed on the prediction residual signal to obtain encoded information.
The feature of the present invention is that only the pixels belonging to the waveform reproduction block in the reference block are to be encoded as the prediction residual signal substantially.
Step S109: A process of generating a difference signal between both blocks is performed with the encoding target block and the reference block as inputs, and the difference signal is written to the register as a prediction residual signal.
Step S110: Using the prediction residual signal output in step S109 as input, orthogonal transformation, quantization and entropy coding are performed on the prediction residual signal to obtain encoded information.
Step S111: Steps S102 to S110 are performed for all blocks in the frame.
[Flow chart (Deblocking method of waveform reproduction block adjacent to texture synthesis block)]
An embodiment of [deblocking method of waveform reproduction block adjacent to texture synthesis block] of the present invention will be described with reference to the drawing (FIG. 4).
Step S201: The mode information of each block in the frame (information indicating whether the texture synthesis block or the waveform reproduction block) is stored in the register together with the spatial position information in the frame with respect to the block.
Step S202: Using the coding information of the block as an input, a decoding process is performed, a local decoded image of the block is generated, and written to a register.
Step S203: The local decoded image of the block output in step S202 is read as input and written to the reference frame buffer. Here, the reference frame buffer is a frame buffer that stores a decoded signal that becomes a reference signal when a subsequent frame refers to the frame.
Step S204: The local decoded image of the block output in step S202 is read as input and written to the display frame buffer. Here, the display frame buffer is a frame buffer that stores a decoded image to be displayed on the display device.
Step S205: Using the signal in the display frame buffer as an input, deblocking processing is performed on the signal of the block, and the signal after deblocking is overwritten at the position of the block in the display frame buffer. Specific deblocking processing is given from the outside. For example, H.M. A deblocking filter specified by H.264 or JSVC is used.
Step S206: Using the mode information output in step S201 as input, it is determined whether or not the block is a waveform reproduction block. If the block is a waveform reproduction block, the process proceeds to step S208; The process proceeds to S207.
Step S207: Using the signal in the reference frame buffer as an input, deblocking processing is performed on the signal of the block, and the signal after deblocking is overwritten at the position of the block in the reference frame buffer. If it is desired to reduce the amount of computation, this process can be skipped.
Step S208: Using the mode information output in Step S201 as an input, it is determined whether or not the mode of the adjacent block on the left side of the block is a texture synthesis block. If the mode is a texture synthesis block, the process of Step S209 is performed. Is skipped, otherwise the process proceeds to step S209.
Step S209: Using the signal in the reference frame buffer as an input, deblocking processing is performed on the block boundary at the left end of the block, and the signal after deblocking processing is overwritten on the block boundary position in the reference frame buffer.
Step S210: Using the mode information output in step S201 as input, determine whether or not the mode of the adjacent block above the block is a texture synthesis block. If the mode is a texture synthesis block, the process of step S211 Is skipped, otherwise the process proceeds to step S211.
Step S211: The signal in the reference frame buffer is input, deblocking processing is performed on the block boundary at the upper end of the block, and the signal after deblocking is overwritten on the block boundary position in the reference frame buffer.
Step S212: Steps S202 to S211 are performed for all blocks in the frame.
[Encoding device (reference method for texture synthesis block)]
FIG. 5 shows a block diagram of an encoding apparatus according to an embodiment of the [texture synthesis block reference method] of the present invention.
Note that this apparatus is used in a prediction processing unit (311 in FIG. 7) in the video encoding device and a prediction processing unit (413 and 428 in FIG. 8) in the scalable video encoding device.
Reference frame storage unit 101: Stores a reference frame signal used to generate a prediction signal when encoding an encoding target frame signal.
Frame storage unit 102: Stores the encoding target frame signal.
Motion vector estimation processing unit 103: The signal of the block of the frame read from the frame storage unit 102 and the signal of a frame (can be a plurality of frames) to be a reference frame candidate read from the reference frame storage unit 101 The motion estimation process between the frames is performed, the motion vector indicating the correspondence between the frames, the process of estimating the information (reference frame index) specifying the reference frame is performed, and the estimation result is written in the estimated motion vector storage unit 104. In addition, when dividing the block into sub-blocks and performing motion estimation between frames for each sub-block, the motion vector and reference frame index are estimated for each sub-block, along with information indicating the block division, Write to the estimated motion vector storage unit 104. The specific estimation method shall be given from the outside. For example, H.M. The estimation method used by JSVM, which is the reference software of JM and JSVC, which is the reference software of H.264, is used.
Mode information storage unit 105: For all frames that can serve as reference frames for the current frame, mode information (information representing the distinction between texture synthesis blocks and waveform reproduction blocks) of each block in the same frame. Stored together with spatial position information in the frame.
Texture synthesis block determination unit 106: The mode information read from the mode information storage unit 105 is input, and it is determined whether or not a texture synthesis block is included in the reference block. The process proceeds to the generation processing unit 107. At this time, a control signal that reads the data in the reference frame storage unit 101 and the frame storage unit 102 as an input to the prediction residual signal generation processing unit 107 is sent to the prediction residual signal generation processing unit 107. On the other hand, when the texture synthesis block is not included, the process proceeds to the processing of the pixel class map generation processing unit 110. At this time, a control signal for reading the data in the reference frame storage unit 101 and the frame storage unit 102 as an input to the pixel class map generation processing unit 110 is sent to the pixel class map generation processing unit 110.
Prediction residual signal generation processing unit 107: Performs processing to generate a difference signal between both signals by using the block signal read from the frame storage unit 102 and the reference block signal read from the reference frame storage unit 101 as inputs. The signal is written in the prediction residual signal storage unit 108 as a prediction residual signal.
Prediction residual signal encoding unit 109: The prediction residual signal read from the prediction residual signal storage unit 108 is input, and orthogonal transformation, quantization, and entropy encoding are performed on the prediction residual signal to perform encoding. Write to the stream storage unit 115.
Pixel class map generation processing unit 110: With the mode information read from the mode information storage unit 105 as an input, for each pixel position in the encoding target block, 1 for the pixel position belonging to the waveform reproduction block, and 1 for the texture synthesis block A binary two-dimensional matrix in which 0 is assigned to the pixel position is written in the pixel class map storage unit 111 as a pixel class map.
Prediction residual signal generation processing unit 112: a pixel class map read from the pixel class map storage unit 111, the block signal read from the frame storage unit 102, and the reference block signal read from the reference frame storage unit 101 as inputs. A process for generating a prediction residual signal as a difference signal between the original signal and the reference signal is performed for a pixel at a position where the class map is 1, and a prediction is performed for a pixel at a position where the pixel class map is 0 A padding process for setting the residual to zero is performed, and the prediction residual signal is written to the prediction residual signal storage unit 113.
Prediction residual signal encoding unit 114: The prediction residual signal read from the prediction residual signal storage unit 113 is input, and orthogonal transformation, quantization, and entropy encoding are performed on the prediction residual signal to perform encoding. Information is written to the encoded stream storage unit 115.
The above processing is performed for all blocks in the frame.
[Decoding device (deblocking method of waveform reproduction block adjacent to texture synthesis block)]
An embodiment of [deblocking method of waveform reproduction block adjacent to texture synthesis block] of the present invention will be described with reference to FIG. FIG. 6 is a block diagram illustrating a part of the encoding device or the decoding device.
The present apparatus is used in the decoding apparatus, and also includes a deblocking processing unit (309 in FIG. 7) in the video encoding device and a deblocking processing unit (410 and 426 in FIG. 8) in the scalable video encoding device. ).
Mode information storage unit 201: The mode information (information indicating whether the texture synthesis block or the waveform reproduction block) of each block in the frame is stored in a register together with the spatial position information in the frame with respect to the block.
Decoding processing unit 202: Inputs the encoding information of the block, performs decoding processing, generates a local decoded signal of the block, and writes it to the frame storage unit 203.
Deblocking processing unit 204: The local decoding signal of the block read from the frame storage unit 203 is input, deblocking processing is performed, and the signal after deblocking processing is written to the display frame storage unit 205.
Reference frame storage unit 206: Writes the local decoded signal of the block read from the frame storage unit 203.
Waveform reproduction block determination unit 207: The mode information output from the mode information storage unit 201 is input to determine whether the block is a waveform reproduction block. If the block is a waveform reproduction block, the deblocking processing unit 208 Otherwise, the process proceeds to the process of the left adjacent block determination unit 210 and the upper adjacent block determination unit 212.
Deblocking processing unit 208: With the signal read from the reference frame storage unit 206 serving as a reference frame buffer as an input, the signal of the block is subjected to deblocking processing, and the signal after deblocking processing is referred to as the reference frame storage unit 209. Export to Note that the reference frame storage unit 206 and the reference frame storage unit 209 are memories on the same physical memory, and this process overwrites the block boundary portion of the block in the reference frame storage unit 206 with the pixel value after the deblocking process. It corresponds to what you did. Further, specific deblocking processing is given from the outside. For example, H.M. A deblocking filter specified by H.264 or JSVC is used. If it is desired to reduce the amount of computation, this process can be skipped.
Left adjacent block determination unit 210: The mode information output from the mode information storage unit 201 is input, and a determination process is performed to determine whether the mode of the adjacent block on the left side of the block is a texture synthesis block. If not, the process proceeds to the deblocking processing unit 211.
Deblocking processing unit 211: Using the signal in the reference frame buffer read from the reference frame storage unit 206 as an input, deblocking processing is performed on the block boundary at the left end of the block, and the signal after deblocking processing is used as a reference frame. Write to the storage unit 209.
Upper adjacent block determination unit 212: Using the mode information output from the mode information storage unit 201 as an input, a determination process is performed to determine whether the mode of the adjacent block above the block is a texture synthesis block. If not, the process proceeds to the deblocking processing unit 213.
Deblocking processing unit 213: Using the signal in the reference frame buffer read from the reference frame storage unit 206 as an input, deblocking processing is performed on the block boundary at the upper end of the block, and the signal after deblocking processing is used as a reference frame. Write to the storage unit 209.
FIG. 7 is a block diagram showing an example of a moving picture coding apparatus to which the present invention is applied. The texture synthesis block reference method described in FIGS. 3 and 5 is used in the prediction processing unit 311 in FIG. Further, the deblocking method of the waveform reproduction block adjacent to the texture synthesis block described in FIGS. 4 and 6 is used in the deblocking processing unit 309 in FIG. The basic functions and operations of the video encoding device shown in FIG. 7 are the same as those of the well-known video encoding device except for the prediction processing unit 311 and the deblocking processing unit 309.
Hierarchy Separator 301: Inputs an encoding target frame of an input image, separates it into hierarchies having different spatial resolutions, and sets signals of each layer as encoding targets.
Transform unit 302: The difference signal between the output of the hierarchical separator 301 and the prediction signal is input, for example, a transform process such as a discrete cosine transform process is performed, and the calculated transform coefficient is written to the transform coefficient storage unit 303.
Quantization unit 304: Inputs the transform coefficient read from the transform coefficient storage unit 303, performs quantization processing, and writes the quantized value to the quantized value storage unit 305.
Inverse quantization unit 306: The quantization value read from the quantization value storage unit 305 is input, an inverse quantization process is performed, and the result is written to the inverse quantization value storage unit 307.
Inverse transform unit 308: Using the transform coefficient read from the inverse quantized value storage unit 307 as an input, performs an inverse transform process, and sends it to the deblocking processor 309.
Deblocking processing unit 309: executes the deblocking processing described with reference to FIGS. 4 and 6, and writes the result to the local decoded signal storage unit 310.
Prediction processing unit 311: Prediction processing using the addition value of the local decoded image read from the local decoded signal storage unit 310 and the output of the delay unit 313 as input, and using the texture synthesis block reference method described with reference to FIGS. 3 and 5 Is performed, and is written to the predicted signal storage unit 412.
Delay unit 313: Delays the prediction signal read from the prediction signal storage unit 312 by one frame.
The entropy encoding unit 314 receives the quantized value read from the quantized value storage unit 305 as input, performs entropy encoding processing, and outputs the encoded result as an encoded stream.
FIG. 8 is a block diagram showing an example of a scalable video encoding apparatus to which the present invention is applied. In FIG. 8, 40 is a base layer encoding unit, and 41 is an enhancement layer encoding unit. The texture synthesis block reference method described in FIGS. 3 and 5 is used in the prediction processing unit 413 and the prediction processing unit 428 in FIG. Further, the deblocking method of the waveform reproduction block adjacent to the texture synthesis block described in FIGS. 4 and 6 is used in the deblocking processing unit 410 and the deblocking processing unit 426 in FIG. The basic functions and operations of the scalable video encoding device shown in FIG. 8 are the same as those of the well-known scalable video encoding device except for the prediction processing units 413 and 428 and the deblocking processing units 410 and 426. It is the same.
Hierarchy Separator 401: Input an encoding target frame of an input image as an input, and separate it into hierarchies having different spatial resolutions. The transmitted and extended layer signals are written to the extended layer signal storage unit 402.
Transform unit 403: The difference signal between the base layer signal and the prediction signal, which is the output of the layer separator 401, is input, performs a transform process such as a discrete cosine transform process, and the calculated transform coefficient is a transform coefficient storage unit. Write to 404.
Quantization unit 405: Inputs the transform coefficient read from the transform coefficient storage unit 404, performs quantization processing, and writes the quantized value to the quantized value storage unit 406.
Inverse quantization unit 407: The quantized value read from the quantized value storage unit 406 is input, an inverse quantization process is performed, and the result is written to the inverse quantized value storage unit 408.
Inverse transform unit 409: Using the transform coefficient read from the inverse quantized value storage unit 408 as an input, performs an inverse transform process, and sends it to the deblocking processor 410.
Deblocking processing unit 410: The deblocking processing described in FIGS. 4 and 6 is executed, and the result is written to the local decoded signal storage unit 412.
Prediction processing unit 413: Prediction processing using the addition value of the local decoded image read from the local decoded signal storage unit 412 and the output of the delay unit 415 as an input, and using the texture synthesis block reference method described in FIG. 3 and FIG. Is performed, and is written in the predicted signal storage unit 414. In addition, the result of the prediction process is sent to the inter-layer prediction processing unit 418.
Delay unit 415: Delays the prediction signal read from the prediction signal storage unit 414 by one frame.
The entropy encoding unit 416 receives the quantized value read from the quantized value storage unit 406 as input, performs entropy encoding processing, and writes the encoded result to the encoded stream storage unit 417.
Inter-layer prediction processing unit 418: Inter-layer prediction processing is performed using a signal obtained by up-sampling the decoded signal of the base layer.
The processing from the conversion unit 419 to the encoded stream storage unit 432 in the enhancement layer encoding unit 41 is the same as the processing from the conversion unit 403 to the encoded stream storage unit 417 in the base layer encoding unit 40.
Multiplexer 433: Performs a process of multiplexing the encoded stream read from the encoded stream storage unit 417 and the encoded stream storage unit 432, and outputs the result as a scalable encoding result.
[Flowchart (reference method of texture synthesis block in decoding process)]
An embodiment of the [texture synthesis block reference method] in the decoding process will be described in accordance with the flow of the decoding process shown in FIG.
Step S301: First, mode information of the current block to be decoded (information indicating whether a texture synthesis block or a waveform reproduction block) is decoded and held in a register.
Step S302: The mode information decoded in step S301 is input to determine whether or not the current block is a texture synthesis block. If the current block is a texture synthesis block, the process proceeds to step S303. , The process proceeds to step S305.
Step S303: If the current block is a texture synthesis block, 0 is written in the pixel class map corresponding to the pixel position of the block in the frame.
Step S304: A decoded signal is obtained by texture synthesis processing as described in Non-Patent Document 2. Thereafter, the process proceeds to step S315. It should be noted that the parameter indicating the texture feature quantity required for the texture synthesis process is given from the encoded stream.
Step S305: If the current block is not a texture synthesis block, 1 is written in the pixel class map corresponding to the pixel position of the block in the frame.
Step S306: Decode the prediction residual signal of the current block.
Step S307: It is determined whether or not the current block is an inter block. If the current block is an inter block, the process proceeds to step S308. Otherwise, the process proceeds to step S314.
Step S308: The motion vector and the reference frame index are decoded.
Step S309: The reference signal of the reference frame indicated by the reference frame index decoded in step S308 is read from the reference frame buffer.
Step S310: The pixel class map at the reference block position is read.
Step S311: A process for determining whether or not a zero value is included in the pixel class map in the reference block is performed. If a zero value is included, the process proceeds to step S312; otherwise, the process proceeds to step S313. Move.
Step S312: The prediction residual signal and the reference signal are added to the pixel position where the pixel class map is 1.
Step S313: If a zero value is not included in the pixel class map, the prediction residual signal of the current block and the reference signal are added.
Step S314: If the current block is an intra block, H.264 performs a decoding process.
Step S315 :: Steps S301 to S314 are performed for all blocks in the frame.
[Decoding device (reference method for texture synthesis block)]
An embodiment of [reference method of texture synthesis block] in the decoding device will be described with reference to the block diagram of the decoding device shown in FIG.
Encoded stream storage unit 501: Input an encoded stream encoded by the encoding apparatus shown in FIG. 5 and store it.
Mode information decoding unit 502: Decodes mode information of the current block to be decoded (information indicating whether a texture synthesis block or a waveform reproduction block) and stores it in the mode information storage unit 503.
Texture synthesis block determination unit 504: Reads mode information stored in the mode information storage unit 503, performs a determination process as to whether or not the current block is a texture synthesis block, and if it is a texture synthesis block, a texture synthesis processing unit If not, the control is switched so that the process of the inter block determination unit 508 is executed. Also, information on whether or not the current block is a texture synthesis block is passed to the pixel class map generation processing unit 505.
Pixel class map generation processing unit 505: When the current block is a texture synthesis block, 0 is written to the pixel class map in the pixel class map storage unit 506 corresponding to the pixel position of the block of the frame, 1 is written in the pixel class map in the pixel class map storage unit 506 corresponding to the pixel position of the block in the frame.
Texture synthesis processing unit 507: Obtains a decoded signal obtained by synthesizing a texture by texture synthesis processing as described in Non-Patent Document 2, and writes it to the decoded signal storage unit 515.
Inter block determination unit 508: Performs determination processing whether or not the current block is an inter block. If the current block is not an inter block, the process proceeds to the intra decoding processing unit 514. If the current block is an inter block, the prediction residual is determined. The processing moves to the signal decoding unit 509 and the estimated motion vector decoding unit 511.
Prediction residual signal decoding unit 509: Decodes the prediction residual signal of the current block and stores it in the prediction residual signal storage unit 510.
Estimated motion vector decoding unit 511: The motion vector and the reference frame index are decoded and stored in the estimated motion vector storage unit 512.
Adaptive addition processing unit 513: Reads the reference signal of the reference frame indicated by the reference frame index in the estimated motion vector storage unit 512 from the reference frame storage unit 516 and also reads the pixel class map at the reference block position from the pixel class map storage unit 506 The prediction residual signal read from the prediction residual signal storage unit 510 and the reference signal read from the reference frame storage unit 516 are added to the pixel position where the pixel class map is 1, and the result is written to the decoded signal storage unit 515. . For the pixel position where the pixel class map is 0, the reference signal read from the reference frame storage unit 516 is written to the decoded signal storage unit 515.
Intra decoding processing unit 514: When the current block is an intra block, The decoding process specified by H.264 is performed, and the decoded signal is written in the decoded signal storage unit 515.
The description supplements the pixel class map used in the above-described texture synthesis block reference method and the waveform reproduction block deblocking method adjacent to the texture synthesis block.
The pixel class map is a table in which attributes of pixels in the frame (information indicating whether they belong to a waveform reproduction block or a texture synthesis block) are recorded. Generated when encoding and decoding each frame.
The coping method when the waveform reproduction block and the texture synthesis block are mixed in the reference block is as follows. The pixel class map for the reference block is read, and the decoding process is adaptively changed between the pixels belonging to the waveform reproduction block and the pixels belonging to the texture synthesis block for each pixel. Further, since the pixel class map for the reference frame can be generated by a common rule in the encoding device and the decoding device, additional information for the pixel class map is unnecessary as encoding information.
The video coding or video decoding processing including the above-mentioned texture synthesis block reference and the waveform blocking block deblocking processing adjacent to the texture synthesis block can also be realized by a computer and a software program. The program can be provided by being recorded on a computer-readable recording medium or can be provided through a network.
It is a figure explaining the example in case the area | region of a texture synthetic | combination block and a waveform reproduction block is mixed in a reference block. It is a figure explaining the example in case a texture synthetic | combination block and a waveform reproduction block adjoin. It is a figure which shows the flow of the encoding process of embodiment regarding the adaptive process in a reference block. It is a figure which shows the flow of the adaptive deblocking filter process of the waveform reproduction block adjacent to a texture synthetic | combination block. It is a block diagram which shows the encoding apparatus of embodiment regarding the adaptive process in a reference block. It is a figure which shows a part of encoding apparatus or decoding apparatus of embodiment regarding an adaptive deblocking filter process. It is a block diagram which shows the example of the moving image encoder to which this invention is applied. It is a block diagram which shows the example of the scalable moving image encoder to which this invention is applied. It is a figure which shows the flow of the decoding process of embodiment regarding the adaptive process in a reference block. It is a block diagram which shows the decoding apparatus of embodiment regarding the adaptive process in a reference block.
DESCRIPTION OF SYMBOLS 101 Reference frame memory | storage part 102 The said frame memory | storage part 103 Motion vector estimation process part 104 Estimated motion vector memory | storage part 105 Mode information memory | storage part 106 Texture synthetic | combination block determination part 107 Prediction residual signal generation process part 108 Prediction residual signal memory | storage part 109 Prediction Residual signal encoding unit 110 Pixel class map generation processing unit 111 Pixel class map storage unit 112 Prediction residual signal generation processing unit 113 Prediction residual signal storage unit 114 Prediction residual signal encoding unit 115 Encoded stream storage unit
A moving picture coding method for dividing a frame into blocks and coding a moving picture in which a waveform synthesis by texture synthesis and a waveform reproduction for a prediction residual are mixed for each block,
When a texture synthesis block related to waveform synthesis by the texture synthesis and a waveform reproduction block related to waveform reproduction for the prediction residual are mixed in a reference area referred to by an encoding target block in inter-frame prediction,
The pixel portion that refers to the region included in the waveform reproduction block encodes difference information between the reference signal and the original signal to be encoded as a prediction residual signal,
The pixel portion that refers to the region included in the texture synthesis block is encoded with the prediction residual signal set to 0 so that the reference signal becomes a decoded signal of the pixel in the predicted block of the encoding target frame. A video encoding method based on texture synthesis.
The video encoding method based on texture synthesis according to claim 1,
When the texture synthesis block and the waveform reproduction block are adjacent to each other, the signal after the deblocking filter processing is used only as a decoded signal of the frame with respect to the block boundary of the waveform reproduction block adjacent to the texture synthesis block. A video coding method based on texture synthesis, characterized in that a signal before deblocking filter processing is used as a reference signal for subsequent frames.
A video decoding method for decoding an encoded stream output by a video encoding method based on texture synthesis according to claim 1,
When a texture synthesis block related to waveform synthesis by the texture synthesis and a waveform reproduction block related to waveform reproduction for the prediction residual are mixed in a reference region referred to by a decoding target block in inter-frame prediction,
The pixel portion that refers to the region included in the waveform reproduction block has a signal obtained by adding the reference signal to the decoded prediction residual signal as a decoded signal of the pixel in the predicted block of the decoding target frame,
A moving picture decoding method based on texture synthesis, wherein a pixel portion that refers to an area included in the texture synthesis block uses a reference signal as a decoded signal of the pixel in a predicted block of a decoding target frame.
The video decoding method based on texture synthesis according to claim 3,
When the texture synthesis block and the waveform reproduction block are adjacent to each other, the signal after the deblocking filter processing is used only as a decoded signal of the frame with respect to the block boundary of the waveform reproduction block adjacent to the texture synthesis block. A video decoding method based on texture synthesis, wherein a signal before deblocking filter processing is used as a reference signal for a subsequent frame.
A video encoding device that divides a frame into blocks and encodes a video in which waveform synthesis by texture synthesis and waveform reproduction for the prediction residual are mixed for each block,
A pixel portion that refers to an area included in the waveform reproduction block, means for encoding difference information between a reference signal and an original signal to be encoded as a prediction residual signal;
The pixel portion that refers to the area included in the texture synthesis block has means for encoding the prediction residual signal as 0 so that the reference signal becomes a decoded signal of the pixel in the predicted block of the encoding target frame. A moving picture coding apparatus based on texture synthesis, comprising:
A video decoding device for decoding an encoded stream output by a video encoding device based on texture synthesis according to claim 5,
A pixel portion that refers to an area included in the waveform reproduction block, and a means that uses a signal obtained by adding a reference signal to a decoded prediction residual signal as a decoded signal of the pixel in the predicted block of the decoding target frame;
The pixel portion that refers to the region included in the texture synthesis block includes means for using the reference signal as a decoded signal of the pixel in the predicted block of the decoding target frame. apparatus.
The moving picture decoding apparatus based on texture synthesis according to claim 6,
Reference frame storage means for storing a reference frame referred to by a decoding target frame in inter-frame prediction;
Deblocking processing means for performing deblocking filter processing on the decoded signal;
When the decoded signal is a signal of a waveform reproduction block and the block is adjacent to the texture synthesis block, a deblocking filter by the deblocking processing means is applied to the block boundary of the waveform reproduction block adjacent to the texture synthesis block. Based on texture synthesis, the processed signal is used only as a decoded signal of the frame, and the signal stored in the reference frame storage means is provided with a decoding processing means for making a signal before deblocking filter processing. Video decoding device.
A moving picture coding program based on texture synthesis for causing a computer to execute the moving picture coding method based on texture synthesis according to claim 1 or 2.
A moving picture decoding program based on texture synthesis for causing a computer to execute the moving picture decoding method based on texture synthesis according to claim 3 or 4.
A computer-readable recording medium on which the moving picture encoding program based on texture synthesis according to claim 8 is recorded.
A computer-readable recording medium in which the moving picture decoding program based on texture synthesis according to claim 9 is recorded.
JP2007212682A 2007-08-17 2007-08-17 Moving picture encoding method, decoding method, encoding apparatus, decoding apparatus based on texture synthesis, program thereof, and recording medium thereof Active JP4659005B2 (en)
JP2007212682A JP4659005B2 (en) 2007-08-17 2007-08-17 Moving picture encoding method, decoding method, encoding apparatus, decoding apparatus based on texture synthesis, program thereof, and recording medium thereof
JP2009049626A JP2009049626A (en) 2009-03-05
JP4659005B2 true JP4659005B2 (en) 2011-03-30
ID=40501429
JP2007212682A Active JP4659005B2 (en) 2007-08-17 2007-08-17 Moving picture encoding method, decoding method, encoding apparatus, decoding apparatus based on texture synthesis, program thereof, and recording medium thereof
JP (1) JP4659005B2 (en)
JP2006519533A (en) * 2003-02-28 2006-08-24 フラウンホッファー−ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ Method and assembly for video coding where video coding includes texture analysis and texture synthesis, corresponding computer program and corresponding computer-readable recording medium
2007-08-17 JP JP2007212682A patent/JP4659005B2/en active Active
JP2009049626A (en) 2009-03-05
JP5008664B2 (en) 2012-08-22 Macroblock adaptive inter-layer texture prediction method and apparatus
KR100748194B1 (en) 2007-08-10 Improved interpolation of video compression frames
JP2010515310A (en) 2010-05-06 Method and apparatus for intraframe predictive encoding / decoding
ES2265531T3 (en) 2007-02-16 Procedure for predictive decodification of images.
JP3778798B2 (en) 2006-05-24 Efficient macroblock header coding for video compression