Source: https://patents.justia.com/patent/9794587
Timestamp: 2020-08-14 18:09:21
Document Index: 334996289

Matched Legal Cases: ['Application No. 201080066019', 'Application No. 2013', 'Application No. 2014', 'Application No. 2014', 'Application No. 2014', 'Application No. 10849496', 'Application No. 2013', 'Application No. 201310142052', 'Application No. 201080066017', 'Application No. 2012', 'Application No. 201410051546', 'Application No. 201410051029', 'Application No. 201410051514', 'Application No. 2016']

US Patent for Image encoding method and image decoding method Patent (Patent # 9,794,587 issued October 17, 2017) - Justia Patents Search
Justia Patents Motion VectorUS Patent for Image encoding method and image decoding method Patent (Patent # 9,794,587)
Feb 26, 2014 - KABUSHIKI KAISHA TOSHIBA
This application is a Continuation application of U.S. application Ser. No. 13/647,124, filed Oct. 8, 2012, which is a Continuation application of PCT Application No. PCT/JP2010/056400, filed Apr. 8, 2010, the entire contents of each of which are incorporated herein by reference.
Recently, a moving image coding method in which a coding efficiency is largely improved is recommended as ITU-T Rec. H.264 and ISO/IEC 14496-10 (hereinafter referred to as H.264) by ITU-T and ISO/IEC. In H.264, prediction processing, transform processing, and entropy coding processing are performed in rectangular block units (for example, a 16-by-16 pixel block unit and an 8-by-8 pixel block unit). In the prediction processing, motion compensation is performed to a rectangular block of an encoding target (hereinafter referred to as an encoding target block). In the motion compensation, a prediction in a temporal direction is performed by referring to an already-encoded frame (hereinafter referred to as a reference frame). In the motion compensation, it is necessary to encode and transmit motion information including a motion vector to a decoding side. The motion vector is information on a spatial shift between the encoding target block and a block referred to in the reference frame. In the case that the motion compensation is performed using a plurality of reference frames, it is necessary to encode a reference frame number in addition to the motion information. Therefore, a code amount related to the motion information and the reference frame number may increase.
FIG. 7A is a view illustrating an example of a size of a motion compensation block used in the inter prediction processing.
FIG. 10 is a view illustrating an example of the available block that is selected from the motion reference blocks in FIG. 8 by the method in FIG. 9.
The image encoder 100 encodes the input image signal 10 in units of pixel blocks (for example, a macroblock, a sub-block, and one pixel) into which the original image is divided. Therefore, the input image signal 10 is sequentially input to the image encoder 100 in units of pixel blocks into which the original image is divided. In the present embodiment, the processing unit for encoding is set to the macroblock, the pixel block (macroblock) that is of an encoding target corresponding to the input image signal 10 is simply referred to as an encoding target block. An image frame including the encoding target block, namely, the image frame of the encoding target is referred to as an encoding target frame.
The encoding processing may be performed to each pixel block in the encoding target frame in any order. In the present embodiment, for the sake of convenience, it is assumed that, as illustrated in FIG. 3, the encoding processing is performed from the upper-left pixel block of the encoding target frame toward the lower-right pixel block, namely, in a raster-scan order.
The inverse-quantization/inverse-transform module 105 inversely quantizes the transform coefficient information 13 generated in Step S503, and the inverse transform processing is performed to the inversely-quantized transform coefficient information 13 to generate a decoded prediction error signal 15 (Step S505). The decoded prediction error signal 15 is added to the reference image signal 17 used in Step S501 to create a locally-decoded image signal 16 (Step S506), and the locally-decoded image signal 16 is stored as the reference image signal in the frame memory 107 (Step S507).
In the inter prediction, the block size suitable for the encoding target block can be selected from a plurality of motion compensation blocks. That is, the encoding target block is divided into small pixel blocks, and the motion compensation may be performed in each small pixel block. FIGS. 7A to 7C illustrate the size of the motion compensation block in units of macroblocks, and FIG. 7D illustrates the size of the motion compensation block in units of sub-blocks (the pixel block that is less than or equal to the 8-by-8-pixel block). As illustrated in FIG. 7A, in the case that the encoding target block has the 64×64 pixels, the 64-by-64-pixel block, the 64-by-32-pixel block, the 32-by-64-pixel block, or the 32-by-32-pixel block can be selected as the motion compensation block. As illustrated in FIG. 7B, in the case that the encoding target block has 32×32 pixels, the 32-by-32-pixel block, the 32-by-16-pixel block, the 16-by-32-pixel block, or the 16-by-16-pixel block can be selected as the motion compensation block. As illustrated in FIG. 7C, in the case that the encoding target block has 16×16 pixels, the motion compensation block can be set to the 16-by-16-pixel block, the 16-by-8-pixel block, the 8-by-16-pixel block, or the 8-by-8-pixel block. As illustrated in FIG. 7D, in the case that the encoding target block has the 8×8 pixels, the 8-by-8-pixel block, the 8-by-4-pixel block, the 4-by-8-pixel block, or the 4-by-4-pixel block can be selected as the motion compensation block.
The motion reference block is selected from the already-encoded regions (blocks) in the encoding target frame and in the reference frame according to the method decided by both the image encoding apparatus in FIG. 1 and an image decoding apparatus. FIG. 8A illustrates an example of dispositions of the motion reference blocks that are selected according to the position of the encoding target block. In the example in FIG. 8A, nine motion reference blocks A to D and TA to TE are selected from the already-encoded regions in the encoding target frame and the already-encoded regions in the reference frame. Specifically, four blocks A, B, C, and D that are adjacent to a left, a top, an upper right, and an upper left of the encoding target block are selected as the motion reference block from the encoding target frame, and the block TA in the same position as the encoding target block and four pixel blocks TB, TC, TD, and TE that are adjacent to a right, a bottom, the left, and the top of the block TA are selected as the motion reference block from the reference frame. In the present embodiment, the motion reference block selected from the encoding target frame is referred to as a spatial-direction motion reference block, and the motion reference block selected from the reference frame is referred to as a temporal-direction motion reference block. A symbol p added to each motion reference block in FIG. 8A indicates an index of the motion reference block. The index is numbered in the order of the temporal-direction motion reference block and the order of the spatial-direction motion reference block. Alternatively, the index may be numbered in any order unless the indexes are overlapped with each other. For example, the temporal-direction and spatial-direction motion reference blocks may be numbered in a random order.
The spatial-direction motion reference block is not limited to the example in FIG. 8A. For example, as illustrated in FIG. 8B, the spatial-direction motion reference blocks may be blocks (for example, macroblocks or a sub-macroblocks) to which pixels a, b, c, and d adjacent to the encoding target block belong. In this case, a relative position (dx,dy) of each of the pixels a, b, c, and d is set with respect to an upper-left pixel e in the encoding target block as illustrated in FIG. 8C. In the examples in FIGS. 8A and 8B, it is assumed that the macroblock is an N-by-N-pixel block.
In each of the cases, when the numbers and the positions of the spatial-direction and temporal-direction motion reference blocks are previously decided between the encoding apparatus and decoding apparatus, the numbers and the positions of the motion reference block may be set in any manner. It is not always necessary that the size of the motion reference block be identical to that of the encoding target block. For example, as illustrated in FIG. 8D, the motion reference block may be larger than or smaller than the encoding target block. The motion reference block is not limited to the square shape, and the motion reference block may be formed into any shape, such as a rectangular shape. The motion reference block may be set to any size.
As illustrated in FIG. 9, the motion reference block having an index p of zero is selected (S800). In FIG. 9, it is assumed that the motion reference block is sequentially processed from the index p of 0 to an index p of M−1 (where M indicates the number of motion reference blocks). It is assumed that availability determination processing is ended to the motion reference blocks having indexes p of 0 to p−1, and that the motion reference block that is of an availability determination processing target has an index of p.
The available-block acquiring module 109 determines whether the motion reference block p has the motion information 18, namely, whether at least one motion vector is allocated to the motion reference block p (S801). When the motion reference block p does not have the motion vector, namely, when the temporal-direction motion reference block p is a block in an I-slice that does not have the motion information or when the intra prediction encoding is performed to all the small pixel blocks in the temporal-direction motion reference block p, the flow goes to Step S805. In Step S805, the available-block acquiring module 109 determines that the motion reference block p is an unavailable block.
When determining that the motion reference block p is the available block or the unavailable block, the available-block acquiring module 109 determines whether the availability determination is made for all the motion reference blocks (S806). When a motion reference block for which the availability determination is not made yet exists, for example, in the case of p<M−1, the flow goes to Step S807. Then the available-block acquiring module 109 increments the index p by 1 (Step S807), and performs Steps S801 to S806 again. When the availability determination is made for all the motion reference blocks in Step S806, the availability determination processing is ended.
FIG. 12C illustrates an example in which one of the motion reference block p and the available block q is the spatial-direction block A while the other is the temporal-direction block TB. In FIG. 12C, the temporal-direction block TB is divided into small blocks, and the small blocks have the pieces of motion information 18. In the example in FIG. 12C, the determination that the pieces of motion information 18 are identical to each other is made, when all the blocks having the pieces of motion information 18 have the identical motion information 18, and when the pieces of motion information 18 on the blocks are identical to the motion information 18 on the spatial-direction block A. At this point, it is not necessary that the sizes of the blocks A and TB be equal to each other.
FIG. 12F illustrates still another example in which both the motion reference block p and the available block q are the temporal-direction blocks. In FIG. 12F, the temporal-direction block TE is divided into small blocks, and the small blocks having the pieces of motion information 18 exist in the block TE. The determination that the pieces of motion information 18 on the blocks TD and TE are identical to each other is made, when all the small blocks having the pieces of motion information 18 have the identical motion information 18 in the block TE, and when the small blocks having the pieces of motion information 18 are identical to the motion information 18 possessed by the block TD.
The temporal-direction-motion-information acquiring module 111 may evaluate an average value or a representative value of the motion vectors included in the motion information 19 possessed by each small pixel block, and output the average value or the representative value of the motion vectors as the motion information 18B.
Based on the pieces of motion information 18A and 18B output from the spatial-direction-motion-information acquiring module 110 and the temporal-direction-motion-information acquiring module 111, the motion information selector switch 112 in FIG. 13 properly selects one available block as the selection block, and outputs the motion information 18 (or the group of the pieces of motion information 18) corresponding to the selection block to the motion compensator 113. The motion information selector switch 112 also outputs the selection block information 31 on the selection block. The selection block information 31 includes the index p or the motion reference block name, and is simply referred to as selection information. The selection block information 31 is not limited to the index p and the motion reference block name, and any information may be used as the selection block information 31 as long as the position of the selection block can be specified.
The motion compensator 113 derives the position of the pixel block, in which the reference image signal 17 is taken out as the predicted image signal, based on the reference motion information (or the motion information group) that is possessed by the selection block selected by the motion information selector 118. In the case that the motion information group is input to the motion compensator 113, the motion compensator 113 acquires the predicted image signal 11 from the reference image signal 17 by dividing the pixel block taken out as the predicted image signal by the reference image signal 17 into small pixel blocks (for example, 4-by-4-pixel blocks) and applying the corresponding motion information to each small pixel block. For example, as illustrated in FIG. 4A, the position of the block in which the predicted image signal 11 is acquired is shifted from the small block in the spatial direction according to the motion vector 18a included in the motion information 18.
The motion compensation processing identical to that of H.264 can be used as the motion compensation processing performed to the encoding target block. An interpolation technique of the ¼ pixel accuracy will specifically be described by way of example. In the interpolation of the ¼ pixel accuracy, the motion vector points out an integral pixel position in the case that each component of the motion vector is a multiple of 4. In other cases, the motion vector points out a predicted position corresponding to an interpolation position of fractional accuracy.
where x and y indicate indexes in vertical and horizontal directions of a beginning position (for example, an upper-left top) of the prediction target block, and x_pos and y_pos indicate the corresponding predicted position of the reference image signal 17. (mv_x,mv_y) indicates the motion vector having the ¼ pixel accuracy. A predicted pixel is generated with respect to the determined pixel position through processing of compensating or interpolating the corresponding pixel position of the reference image signal 17. FIG. 15 illustrates an example of the generation of the predicted pixel in H.264. In FIG. 15, a square (a square drawn by oblique lines) indicated by a capital-letter alphabet expresses the pixel in the integral position, and a hatched square expresses the interpolation pixel in the ½ pixel position. A white square expresses the interpolation pixel corresponding to the ¼ pixel position. For example, in FIG. 15, the processing of interpolating ½ pixels corresponding to the positions of the alphabets b and h is calculated by the following mathematical formula (3).
The letters (for example, b, h, and C1) indicated in the mathematical formulae (3) and (4) indicate the value of the pixel to which the same letters are provided in FIG. 16. “>>” indicates a right shift calculation, and “>>5” corresponds to a division by 32. That is, the interpolation pixel in the ½ pixel position is calculated with a six-tap FIR (Finite Impulse Response) filter (tap coefficient: (1, −5, 20, 20, −5, 1)/32).
The scaling processing will specifically be described with reference to FIG. 21. In FIG. 21, a symbol tc indicates a time distance (POC (number indicating display order) distance) between the encoding target frame and the motion reference frame, and is calculated by the following mathematical formula (5). In FIG. 21, a symbol tr[i] indicates a time distance between the motion reference frame and a frame i referred to by the selection block, and is calculated by the following mathematical formula (6).
where curPOC is the POC (Picture Order Count) of the encoding target frame, colPOC is the POC of the motion reference frame, and refPOC is the POC of the frame i referred to by the selection block. Clip(min, max, target) is a clip function. The clip function Clip(min, max, target) outputs min in the case that the target is smaller than min, outputs max in the case that the target is larger than max, and outputs the target in other cases. DiffPicOrderCnt(x,y) is a function that calculates a difference between the POCs.
MV_x=(MVr_x×tc+Abs(tr[i]/2))/tr[i]
MV_y=(MVr_y×tc+Abs(tr[i]/2))/tr[i] (7)
In the scaling in each encoding target block, using the following mathematical formula (9), the motion vector MV can be calculated by the multiplication, addition, and the shift calculation.
Each of the parts includes a further detailed syntax. The high-level syntax 901 includes sequence-level and picture-level syntaxes, such as a sequence-parameter-set syntax 902 and a picture-parameter-set syntax 903. The slice-level syntax 904 includes a slice header syntax 905 and a slice data syntax 906. The macroblock-level syntax 907 includes a macroblock-layer syntax 908 and a macroblock prediction syntax 909.
FIG. 23A illustrates the syntax in the case that the selection block information is encoded after an mb_type. The stds_idx is encoded in the case that a mode indicated by the mb_type is a predetermined size or a mode (TARGET_MODE), and in the case that the available_block_num is larger than 1. For example, in the case that the motion information on the selection block is available, and in the case that the block size is 64×64 pixels, 32×32 pixels, or 16×16 pixels, or in the case of the direct mode, the stds_idx is encoded.
A syntax element that is not defined herein can be inserted in a line space of the table in FIGS. 23A and 23B, and a description related to another conditional branching may be included in the line space. Alternatively, the syntax table may be divided or integrated into a plurality of tables. It is not always necessary to use an identical term, and the term may arbitrarily be changed according to an application mode. Each syntax element described in the macroblock-layer syntax may be changed so as to be clearly described in a macroblock data syntax.
The information on the mb_type can be reduced using the information on the stds_idx. FIG. 24A illustrates the mb_type in the B-slice of H.264 and a code table corresponding to the mb_type. In FIG. 24A, N is a value, such as 16, 32, and 64, which indicates the size of the encoding target block, and M is half the value of N. Accordingly, in the case that the mb_type is 4 to 21, the encoding target block is the rectangular block. In FIG. 24A, L0, L1, and Bi indicate a unidirectional prediction (only a List0 direction), a unidirectional prediction (only a List1 direction), and a bidirectional prediction, respectively. In the case that the encoding target block is the rectangular block, the mb_type includes information indicating which prediction, L0, L1, or Bi is performed to each of the two rectangular blocks in the encoding target block. A symbol B_Sub means that the above processing is performed to each of the four pixel blocks into which the macroblock is divided. For example, in the case that the encoding target block is the 64-by-64-pixel macroblock, the encoding target block is encoded while the mb_type is further allocated to each of the four 32-by-32-pixel blocks into which the macroblock is divided.
FIGS. 30A and 30B illustrate examples of the macroblock layer syntax of the second embodiment. An available_block_num in FIG. 30A indicates the number of available blocks. In the case that the available_block_num is larger than 1, the selection block encoder 216 encodes the selection block information 31. A stds_flag is a flag indicating whether the motion information on the selection block is used as the motion information on the encoding target block in the motion compensation prediction, namely, a flag indicating which of the first predictor 101 and the second predictor 202 is selected by the prediction method selector switch 203. In the case that the number of available blocks is larger than 1 while the stds_flag is 1, the motion information possessed by the selection block is used in the motion compensation prediction. In the case that the stds_flag is 0, while the motion information possessed by the selection block is not used, like H.264 the motion information 18 is directly encoded or the predicted difference value is encoded. An stds_idx indicates the selection block information, and the code table corresponding to the number of available blocks is described above.
As described above, the image encoding apparatus of the second embodiment selectively switches between the first predictor 101 of the first embodiment and the second predictor 202 in which the prediction method, such as H.264, is used such that the encoding cost is reduced, and performs compression encoding of the input image signal. Accordingly, in the image encoding apparatus of the second embodiment, the encoding efficiency is improved compared with the image encoding apparatus of the first embodiment.
In the encoded sequence decoder 301, the decoding is performed in each frame or field by a syntax analysis based on the syntax. Specifically, the encoded sequence decoder 301 sequentially performs variable length decoding of an encoded sequence of each syntax, and decodes decoding parameters related to the decoding target block. The decoding parameters include transform coefficient information 33, selection block information 61, and the pieces of prediction information, such as the block size information and the prediction mode information.
The prediction error signal 34 restored by the inverse-quantization/inverse-transform module 302 is input to the adder 303. The adder 303 generates a decoded image signal 36 by adding the prediction error signal 34 and a predicted image signal 35 generated by the predictor 305. The generated decoded image signal 36 is output from the image decoder 300, and temporarily stored in the output buffer 308. Then the decoded image signal 36 is output in output timing managed by the decoding controller 350. The decoded image signal 36 is also stored as a reference image signal 37 in the frame memory 304. The reference image signal 37 is sequentially read in each frame or field from the frame memory 304 and input to the predictor 305.
The spatial-direction motion reference block is not limited to the example in FIG. 8A. For example, as illustrated in FIG. 8B, blocks to which pixels a, b, c, and d adjacent to the decoding target block belong may be selected as the spatial-direction motion reference blocks. In this case, a relative position (dx,dy) of each of the pixels a, b, c, and d is set with respect to the upper-left pixel in the decoding target block as illustrated in FIG. 8C.
As illustrated in FIG. 8D, all blocks A1 to A4, B1, B2, C, and D adjacent to the decoding target block may be selected as the spatial-direction motion reference blocks. In FIG. 8D, there are eight spatial-direction motion reference blocks.
Some of the temporal-direction motion reference blocks TA to TE may be overlapped as illustrated in FIG. 8E, or the temporal-direction motion reference blocks TA to TE may be separated as illustrated in FIG. 8F. The temporal-direction motion reference block is not necessarily located in and around the collocate position, and the temporal-direction motion reference block may be disposed at any position in the motion reference frame. For example, the reference block pointed out by the motion vector included in the motion information may be selected as a center (for example, the block TA) of the motion reference block using the motion information of the already-decoded block adjacent to the decoding target block. It is not always necessary that the temporal-direction reference blocks be disposed at equal intervals.
An operation of the available-block acquiring module 307 will be described with reference to the flowchart in FIG. 9. The available-block acquiring module 307 determines whether the motion reference block (index p) has the motion information (Step S801). That is, in Step S801, the available-block acquiring module 307 determines whether at least one of the small pixel block in the motion reference block p has the motion information. When the motion reference block p does not have the motion information, namely, when the temporal-direction motion reference block is the block in the I-slice that does not have the motion information or when the intra prediction decoding is performed to all the small pixel blocks in the temporal-direction motion reference block, the flow goes to Step S805. In Step S805, the available-block acquiring module 307 determines that the motion reference block p is the unavailable block.
FIG. 12B illustrates an example in which one of the motion reference block p and the available block q is the spatial-direction block A while the other is the temporal-direction block TB. In FIG. 12B, one block having the motion information exists in the temporal-direction block TB. The determination that the pieces of motion information 38 are identical to each other is made when the motion information 38 on the temporal-direction block TB is identical to the motion information 38 on the spatial-direction block A. At this point, it is not necessary that the sizes of the blocks A and TB be equal to each other.
FIG. 12F illustrates still another example in which both the motion reference block p and the available block q are the temporal-direction blocks. In FIG. 12F, the temporal-direction block TE is divided into small blocks, and the small blocks having the pieces of motion information 38 exist in the block TE. The determination that the pieces of motion information 38 on the blocks TD and TE are identical to each other is made, when all the small blocks having the pieces of motion information 38 have the identical motion information 38 in the block TE, and when the small blocks having the pieces of motion information 38 are identical to the motion information 38 possessed by the block TD.
FIG. 32 is a block diagram illustrating the encoded sequence decoder 301 in detail. As illustrated in FIG. 32, the encoded sequence decoder 301 includes a separator 320 that separates the encoded data 80 in units of syntaxes, a parameter decoder 322 that decodes the transform coefficient, a transform coefficient decoder 323 that decodes the selection block information, and a parameter decoder 321 that decodes parameters related to the predicted block size and the quantization.
The motion information selector switch 312 selects one of the motion information 38A from the spatial-direction-motion-information acquiring module 310 and the motion information (or the motion information group) 38B from the temporal-direction-motion-information acquiring module 311 according to the selection block information 61, and obtains the motion information 38. The selected motion information 38 is transmitted to the motion compensator 313 and the motion information memory 306. According to the selected motion information 38, the motion compensator 313 performs the same motion compensation prediction as the motion compensator 113 of the first embodiment to generate the predicted image signal 35.
FIG. 23B illustrates the syntax in the case that the selection block information is decoded before the mb_type. In the case that the available_block_num is larger than 1, the stds_idx is decoded. In the case that the available_block_num is 0, because the conventional motion compensation typified by H.264 is performed, the mb_type is decoded.
As described above, the image decoding apparatus of the third embodiment decodes the image that is encoded by the image encoding apparatus of the first embodiment. Accordingly, in the image decoding of the third embodiment, a high-quality decoded image can be reproduced from a relatively small amount of encoded data.
FIG. 34 schematically illustrates an image decoding apparatus according to a fourth embodiment. As illustrated in FIG. 34, the image decoding apparatus includes an image decoder 400, decoding controller 350, and an output buffer 308. The image decoding apparatus of the fourth embodiment corresponds to the image encoding apparatus of the second embodiment. A component and an operation, which are different from those of the third embodiment, are mainly described in the fourth embodiment. As illustrated in FIG. 34, the image decoder 400 of the fourth embodiment differs from the image decoder 300 of the third embodiment in an encoded sequence decoder 401 and a predictor 405.
FIG. 36 is a block diagram illustrating the predictor 405 in detail. The predictor 405 in FIG. 34 includes a first predictor 305, a second predictor 410, and a prediction method selector switch 411. Using the motion information 40 decoded by the encoded sequence decoder 401 and a reference image signal 37, the second predictor 410 performs the same motion compensation prediction as the motion compensator 313 in FIG. 33, and generates predicted image signal 35B. The first predictor 305 is identical to the predictor 305 of the third embodiment, and generates the predicted image signal 35B. Based on the prediction switching information 62, the prediction method selector switch 411 selects one of the predicted image signal 35B from the second predictor 410 and the predicted image signal 35A from the first predictor 305, and outputs the selected predicted image signal as a predicted image signal 35 of the predictor 405. At the same time, the prediction method selector switch 411 transmits the motion information, which is used in the selected one of the first predictor 305 and the second predictor 410, as motion information 38 to a motion information memory 306.
FIGS. 30A and 30B illustrate examples of the macroblock layer syntax of the fourth embodiment. An available_block_num in FIG. 30A indicates the number of available blocks. In the case that the available block_num is larger than 1, the selection block decoder 423 decodes the selection block information in the encoded data 80C. A stds_flag is a flag indicating whether the motion information on the selection block is used as the motion information on the decoding target block in the motion compensation prediction, namely, a flag indicating which of the first predictor 305 and the second predictor 410 is selected by the prediction method selector switch 411. In the case that the number of available blocks is larger than 1 while the stds_flag is 1, the motion information possessed by the selection block is used in the motion compensation prediction. In the case that the stds_flag is 0, while the motion information possessed by the selection block is not used, like H.264 the motion information 18 is directly encoded or the predicted difference value is decoded. An stds_idx indicates the selection block information, and the code table corresponding to the number of available blocks is described above.
FIG. 30A illustrates the syntax in the case that the selection block information is decoded after an mb_type. The stds_flag and the stds_idx are decoded only in the case that a mode indicated by the mb_type is a predetermined size or a mode. For example, the stds_flag and the stds_idx are decoded in the case that the block size is 64×64 pixels, 32×32 pixels, or 16×16 pixels, or in the case of the direct mode.
(2) In the first to fourth embodiments, a luminance signal and a color-difference signal are not distinguished from each other, but a comprehensive description is made about a color signal component. The luminance signal may be different from the color-difference signal in the prediction processing, or the luminance signal may de identical to the color-difference signal in the prediction processing. In the case that different pieces of prediction processing are used, the prediction method selected for the color-difference signal is encoded and decoded by the same method as the luminance signal.
circuitry configured to: determine, based on input encoded data, which one of a first prediction method and a second prediction method to use to determine motion information for generating a predicted image of a target block using inter prediction; when determined to use the first prediction method, determine whether a plurality of candidate blocks, which is positioned in a predetermined positional relationship with respect to the target block, is available in an order defined according to the positional relationship; decode the input encoded data to obtain identification information; select a selection block from one or more candidate blocks that are determined to be available, in accordance with the identification information; and generate the predicted image of the target block based on motion information of the selection block; and when determined to use the second prediction method, decode the input encoded data to obtain embedded motion information; and generate the predicted image of the target block based on the embedded motion information,
wherein the plurality of candidate blocks includes a plurality of blocks in a first frame to which the target block belongs and a plurality of blocks in a second frame different from the first frame, and
wherein, when determining whether the plurality of candidate blocks is available, the circuitry is further configured to: determine that a current candidate block is available if the current candidate block includes motion information and no candidate block among the plurality of candidate blocks has been determined to be available; determine that the current candidate block is available if the current candidate block includes the motion information and the motion information of the current candidate block is different from motion information of each candidate block among the plurality of candidate blocks that has been determined to be available; and determine that the current candidate block is not available if the current candidate block includes the motion information and the motion information of the current candidate block is the same as motion information of a candidate block among the plurality of candidate blocks that has been determined to be available.
2. The image decoding apparatus of claim 1, wherein
the plurality of candidate blocks includes a block adjacent to an upper left of the target block and a block adjacent to a top of the target block, and
the block adjacent to the top of the target block precedes the block adjacent to the upper left of the target block in the order defined according to the positional relationship.
3. The image decoding apparatus of claim 1, wherein the identification information has a data length corresponding to a number of candidate blocks that are determined to be available and a position of the selection block.
circuitry configured to: determine, based on input encoded data, which one of a first prediction method and a second prediction method to use to determine motion information for generating a predicted image of a target block using inter prediction; when determined to use the first prediction method, determine whether a plurality of motion information items of a plurality of candidate blocks, which is positioned in a predetermined positional relationship with respect to the target block, is available in an order defined according to the positional relationship; decode the input encoded data to obtain identification information; select a selection block from one or more candidate blocks that are deter mined to be available, in accordance with the identification information; and generate the predicted image of the target block based on a motion information item of the selection block; and when determined to use the second prediction method, decode the input encoded data to obtain an embedded motion information item; and generate the predicted image of the target block based on the embedded motion information item,
wherein, when determining whether the plurality of motion information items of the plurality of candidate blocks is available, the circuitry is further configured to: determine that a motion information item of a current candidate block is available if the current candidate block includes the motion information item and no motion information item of the plurality of candidate blocks has been determined as available; determine that the motion information item of the current candidate block is available if the current candidate block includes the motion information item and the motion information item of the current candidate block is different from each motion information item of a corresponding candidate block among the plurality of candidate blocks that has been determined to be available; and determine that the motion information item of the current candidate block is not available if the current candidate block includes the motion information item and the motion information item of the current candidate block is the same as a motion information item of the plurality of candidate blocks that has been determined to be available.
5. The image decoding apparatus of claim 4, wherein
6. The image decoding apparatus of claim 4, wherein the identification information has a data length corresponding to a number of candidate blocks that are determined to be available and a position of the selection block.
determining, based on input encoded data, which one of a first prediction method and a second prediction method to use to determine motion information for generating a predicted image of a target block using inter prediction;
when determined to use the first prediction method, determining whether a plurality of candidate blocks, which is positioned in a predetermined positional relationship with respect to the target block, is available in an order defined according to the positional relationship; decoding the input encoded data to obtain identification information; selecting a selection block from one or more candidate blocks that are determined to be available, in accordance with the identification information; and generating the predicted image of the target block based on motion information of the selection block; and
when determined to use the second prediction method, decoding the input encoded data to obtain embedded motion information; and generating the predicted image of the target block based on the embedded motion information,
wherein determining whether the plurality of candidate blocks is available comprises: determining that a current candidate block is available if the current candidate block includes motion information and no candidate block among the plurality of candidate blocks has been determined as available; determining that the current candidate block is available if the current candidate block includes the motion information and the motion information of the current candidate block is different from motion information of each candidate block among the plurality of candidate blocks that has been determined to be available; and determining that the current candidate block is not available if the current candidate block includes the motion information and the motion information of the current candidate block is the same as motion information of a candidate block among the plurality of candidate blocks that has been determined to be available.
8. The image decoding method of claim 7, wherein
9. The image decoding method of claim 7, wherein the identification information has a data length corresponding to a number of candidate blocks that are determined to be available and a position of the selection block.
10. A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method comprising:
determining, based on input encoded data, which one of a first prediction method and a second prediction method to use to determine motion information for generating a predicted image of a target block using inter prediction:
11. An image decoding method comprising:
when determined to use the first prediction method, determining whether a plurality of motion information items of a plurality of candidate blocks, which is positioned in a predetermined positional relationship with respect to the target block, is available in an order defined according to the positional relationship; decoding the input encoded data to obtain identification information; selecting a selection block from one or more candidate blocks that are determined to be available, in accordance with the identification information; and generating a predicted image of the target block based on a motion information item of the selection block; and
when determined to use the second prediction method, decoding the input encoded data to obtain an embedded motion information item; and generating the predicted image of the target block based on the embedded motion information item,
wherein determining whether the plurality of motion information items of the plurality of candidate blocks is available comprises: determining that a motion information item of a current candidate block is available if the current candidate block includes the motion information item and no motion information item of the plurality of candidate blocks has been determined as available; determining that the motion information item of the current candidate block is available if the current candidate block includes the motion information item and the motion information item of the current candidate block is different from each motion information item of a corresponding candidate block among the plurality of candidate blocks that has been determined to be available; and determining that the motion information item of the current candidate block is not available if the current candidate block includes the motion information item and the motion information item of the current candidate block is the same as a motion information item of the plurality of candidate blocks that has been determined to be available.
12. The image decoding method of claim 11, wherein
13. The image decoding method of claim 11, wherein the identification information has a data length corresponding to a number of candidate blocks that are determined to be available and a position of the selection block.
20040013308 January 22, 2004 Jeon et al.
20040013309 January 22, 2004 Choi et al.
20040223548 November 11, 2004 Kato
20050117646 June 2, 2005 Joch et al.
20060188020 August 24, 2006 Wang
20070019726 January 25, 2007 Cha et al.
20090022228 January 22, 2009 Wang et al.
20090245376 October 1, 2009 Choi et al.
20090304084 December 10, 2009 Hallapuro
20100027655 February 4, 2010 Matsuo et al.
20100195723 August 5, 2010 Ikai et al.
20100239002 September 23, 2010 Park et al.
20110129016 June 2, 2011 Sekiguchi et al.
20110211640 September 1, 2011 Kim et al.
0 579 319 January 1994 EP
2149262 February 2010 EP
6-168330 June 1994 JP
8-18976 January 1996 JP
2000-50279 February 2000 JP
2004-23458 January 2004 JP
Combined Office Action and Search Report issued Oct. 10, 2014 in Chinese Patent Application No. 201080066019.6 (with English translation).
Office Action issued Jan. 28, 2014 in Japanese Patent Application No. 2013-116884 (with English language translation).
Office Action issued Feb. 25, 2014 in Japanese Patent Application No. 2014-010560 with English language translation.
Office Action issued Feb. 25, 2014 in Japanese Patent Application No. 2014-010561 with English language translation.
Office Action issued Feb. 25, 2014 in Japanese Patent Application No. 2014-010562 with English language translation.
Extended European Search Report issued Jun. 17, 2014 in Patent Application No. 10849496.4.
Guillaume Laroche, et al., “RD optimized coding for motion vector predictor selection”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, No. 9, XP011231739, Sep. 2008, pp. 1247-1257.
“Video coding using extended block sizes”, Qualcomm Inc., International Telecommunications Union, Telecommunications Standardization Sector, COM 16-C 123-E, XP030003764, Jan. 2009, pp. 1-4.
Sung Deuk Kim, et al., “An efficient motion vector coding scheme based on minimum bitrate prediction”, IEEE Transactions on Image Processing, vol. 8, No. 8, XP011026355, Aug. 1999, pp. 1117-1120.
Japanese Office Action issued Jul. 29, 2014, in Japan Patent Application No. 2013-186629 (with English translation).
Combined Chinese Office Action and Search Report issued Sep. 6, 2015 in Patent Application No. 201310142052.8 (with English language translation).
Combined Chinese Office Action and Search Report issued Sep. 23, 2014 in Patent Application No. 201080066017.7 (with English language translation).
Japanese Office Action Issued Apr. 2, 2013 in Patent Application No. 2012-509253 (with English translation).
English translation of the International Preliminary Report on Patentability issued Nov. 29, 2012, in PCT/JP2010/056400, filed Apr. 8, 2010.
Written Opinion of the International Searching Authority issued Jul. 13, 2010, in PCT/JP2010/056400, filed Apr. 8, 2010 (submitting English-language translation only, previously filed Oct. 8, 2012).
ITU-T Q-6/SG16 Document, VCEG-AC06, Joel Jung, Jul. 17, 2006, “Competition-Based Scheme for Motion Vector Selection and Coding”.
International Search Report issued on Jul. 13, 2010 for PCT/JP2010/056400 filed on Apr. 8, 2010 (with English translation).
International Written Opinion issued on Jul. 13, 2010 for PCT/JP2010/056400 filed on Apr. 8, 2010.
Takeshi, Chujoh, Description of video coding technology proposal by Toshiba, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-A117rl, Apr. 15, 2010, pp. 4-6.
ITU-T Rec.H.264 (Mar. 2005), Chap. 8.4.1 “Derivation process for motion vector components and reference indices”.
Kemal Ugur, et al., “Appendix to Description of Video Coding Technology Proposal by Tandberg Nokia Ericsson”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, No. JCTVC-A119, Apr. 2010, pp. 1-55.
Jungsun Kim, et al., “Encoding Complexity Reduction for Intra Prediction by Disabling NxN Partition”, LG Electronics, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, No. JCTVC-C218, Oct. 2010, pp. 1-5.
Combined Chinese Office Action and Search Report issued on Jun. 24, 2016 in Patent Application No. 201410051546.X (with unedited computer generated English translation and English translation of categories of cited documents).
Combined Chinese Office Action and Search Report issued on Jun. 28, 2016 in Patent Application No. 201410051029.2 (with unedited computer generated English translation and English translation of categories of cited documents).
Combined Chinese Office Action and Search Report issued on Jul. 22, 2016 in Patent Application No. 201410051514.X (with unedited computer generated English translation and English translation of categories of cited documents).
Office Action mailed Sep. 7, 2016 in U.S. Appl. No. 14/190,779.
U.S. Office Action dated Jan. 17, 2017, issued in U.S. Appl. No. 15/350,265.
Office Action issued Jan. 31, 2017 in Japanese Patent Application No. 2016-028133.
“Series H: Audiovisual and Multimedia Systems: Infrastructure of audiovisual services—Coding of moving video: High efficiency video coding” ITU-T Telecommunication Standardization Sector of ITU, H.265, Apr. 2013, 26 pages.
Takeshi Chujoh, et al., “Description of video coding technology proposal by Toshiba” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Apr. 2010, 6 pages.
Office Action mailed Mar. 10, 2017 on co-pending U.S. Appl. No. 13/647,140.
Detlev Marpe, et al., “Context-Based Adaptive Binary Arithmetic Coding in the H.264/AVC Video Compression Standard” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, No. 7, Jul. 2003, pp. 620-636.
U.S. Office Action, dated Aug. 9, 2017 in U.S. Appl. No. 13/647,140.
Patent Publication Number: 20140177724
Application Number: 14/190,909
International Classification: H04N 7/12 (20060101); H04N 19/513 (20140101); H04N 19/105 (20140101); H04N 19/52 (20140101); H04N 19/139 (20140101); H04N 19/159 (20140101); H04N 19/176 (20140101); H04N 19/119 (20140101); H04N 19/137 (20140101); H04N 19/182 (20140101); H04N 19/124 (20140101); H04N 19/15 (20140101); H04N 19/61 (20140101); H04N 19/70 (20140101); H04N 19/96 (20140101);