Method and apparatus for motion vector sign prediction in video coding

A method and apparatus of motion vector reconstruction based on one MVP (motion vector predictor) and one MVD (motion vector difference) are disclosed. According to this method, input data associated with a video sequence including a current block in a current picture are received. Candidate reconstructed MV (motion vector) positions corresponding to all candidate sign pairs of the MVD are derived, where each candidate reconstructed MV position is derived as a vector sum of the MVP and the first magnitude and the second magnitude of the MVD with one candidate sign pair of the MVD. Template matching between the template for the current block and a corresponding template for each candidate reconstructed MV position is evaluated. A target candidate MV position among all candidate reconstructed MV positions is selected. The current block is encoded or decoded using a target candidate MV corresponding to the target candidate MV position.

TECHNICAL FIELD

The present invention relates to motion vector prediction for video coding. In particular, the present invention relates to motion vector sign derivation or prediction to improve coding efficiency of motion vector coding.

BACKGROUND

High-Efficiency Video Coding (HEVC) is a new international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on the hybrid block-based motion-compensated DCT-like transform coding architecture. The basic unit for compression, termed as coding unit (CU), is a 2N×2N square block, and each CU can be recursively split into four smaller CUs until the predefined minimum size is reached. Each CU contains one or multiple prediction units (PUs).

To achieve the best coding efficiency of hybrid coding architecture in HEVC, there are two kinds of prediction modes for each PU, which are Intra prediction and Inter prediction. For Intra prediction modes, the spatial neighbouring reconstructed pixels can be used to generate the directional predictions. There are up to 35 directions in HEVC. For Inter prediction modes, the temporal reconstructed reference frames can be used to generate motion compensated predictions. There are three different modes, including Skip, Merge and Inter Advanced Motion Vector Prediction (AMVP) modes.

When a PU is coded in Inter AMVP mode, motion-compensated prediction is performed with transmitted motion vector differences (MVDs) that can be used together with Motion Vector Predictors (MVPs) for deriving motion vectors (MVs). To decide MVP in Inter AMVP mode, the advanced motion vector prediction (AMVP) scheme is used to select a motion vector predictor among an AMVP candidate set including two spatial MVPs and one temporal MVP. Therefore, in AMVP mode, an MVP index for MVP and the corresponding MVDs are required to be encoded and transmitted. In addition, the Inter prediction direction to specify the prediction directions among bi-prediction, and uni-prediction related to list 0 (L0) and list 1 (L1) along with the reference frame index for each list should also be encoded and transmitted.

When a PU is coded in either Skip or Merge mode, no motion information is transmitted except for the Merge index of the selected candidate. That is because the Skip and Merge modes utilize motion inference methods (i.e., MV=MVP+MVD, where MVD is zero) to obtain the motion information from spatial neighbouring blocks (spatial candidates) or a temporal block (temporal candidate) located in a co-located picture where the co-located picture is the first reference picture in list 0 or list 1, which is signalled in the slice header. In the case of a Skip PU, the residual signal is also omitted. To decide the Merge index for the Skip and Merge modes, the Merge scheme is used to select a motion vector predictor among a Merge candidate set containing four spatial MVPs and one temporal MVP.

SUMMARY

A method and apparatus of motion vector reconstruction in a video coding system based on one MVP (motion vector predictor) and one MVD (motion vector difference) are disclosed. According to this method, input data associated with a video sequence including a current block in a current picture are received. An MVP from the input data for the current block is determined. Also, first magnitude and second magnitude of an MVD associated with the current block are determined. Candidate reconstructed MV (motion vector) positions corresponding to all candidate sign pairs of the MVD are derived, where each candidate reconstructed MV position is derived as a vector sum of the MVP and the first magnitude and the second magnitude of the MVD with one candidate sign pair of the MVD. A template for the current block is determined, where the template is formed from neighbouring reconstructed pixels of the current block. Template matching between the template for the current block and a corresponding template for each candidate reconstructed MV position is evaluated. A target candidate MV position among all candidate reconstructed MV positions is selected based on one or more performance criteria including best template matching among all candidate reconstructed MV positions. The current block is encoded or decoded using a target candidate MV corresponding to the target candidate MV position.

The template matching is evaluated based on a pre-determined criterion comprising SAD (sum of absolution differences) or SSD (sum of squared differences). The performance criteria further include other decoder side information comprising statistics of previous selected sign pair or previous MVP. In one embodiment, the template matching between the template for the current block and the corresponding template for each candidate reconstructed MV position is evaluated based on nearby integer pixel positions of the corresponding template if the MVP or the MVD has fractional-pel precision. In this case, if two corresponding templates for two candidate reconstructed MV positions fall into a same location, locations of said two corresponding templates are modified to become separate. Furthermore, the locations of the two corresponding templates can be modified to be on a left side and a right side of the MVP, or on an upper side and a lower side of the MVP.

The method may comprise pruning one or more candidate reconstructed MV positions based on distance between one MVP and one candidate reconstructed MV position, where if a specific candidate reconstructed MV position is closer to another MVP in a candidate MVP set than the MVP, the specific candidate reconstructed MV position is pruned and not evaluated for the template matching.

Whether to encode or decode the current block using the target candidate MV can be turned on or off explicitly by signalling a control flag. Alternatively, whether to encode or decode the current block using the target candidate MV can also be turned on or off implicitly. For example, whether to encode or decode the current block using the target candidate MV can be turned on or off based on a result of an exclusive OR operation applied to the first magnitude and the second magnitude of the MVD.

The selected sign pair corresponding to the target candidate MV position can be used as a predictor for encoding or decoding signs of the MVD. A prediction hit condition corresponding to the signs of the MVD being equal to the selected sign pair can be coded using a single bit or the shortest codeword. Other prediction conditions corresponding to the signs of the MVD being not equal to the selected sign pair can be coded using two or more bits, where the other prediction conditions include a horizontal flip condition, a vertical flip condition and a diagonal flip condition. The bins generated for prediction conditions associated with blocks in the current picture can be context coded.

In another embodiment, if any of the first magnitude and the second magnitude of the MVD is zero, a first single sign from the current block can be combined with a second single sign from another block or another prediction list to form a joint sign pair, and a prediction hit condition corresponding to both the first single sign and the second single sign being correctly predicted can be coded using a single bit or the shortest codeword. Other prediction conditions corresponding to at least one of the joint sign pair being not correctly predicted can be coded using two or more bits, where the other prediction conditions include a first single sign hit and a second single sign miss condition, a first single sign miss and a second single sign hit condition and both the first single sign miss and the second single sign miss condition.

Aspects of the disclosure further provide a non-transitory computer readable medium storing program instructions for causing a processing circuit of an apparatus to perform a motion vector reconstruction method based on one MVP (motion vector predictor) and one MVD (motion vector difference). In some embodiment, input data associated with a video sequence including a current block in a current picture are received. An MVP from the input data for the current block is determined. Also, first magnitude and second magnitude of an MVD associated with the current block are determined. Candidate reconstructed MV (motion vector) positions corresponding to all candidate sign pairs of the MVD are derived, where each candidate reconstructed MV position is derived as a vector sum of the MVP and the first magnitude and the second magnitude of the MVD with one candidate sign pair of the MVD. A template for the current block is determined, where the template is formed from neighbouring reconstructed pixels of the current block. Template matching between the template for the current block and a corresponding template for each candidate reconstructed MV position is evaluated. A target candidate MV position among all candidate reconstructed MV positions is selected based on one or more performance criteria including best template matching among all candidate reconstructed MV positions. The current block is encoded or decoded using a target candidate MV corresponding to the target candidate MV position.

DETAILED DESCRIPTION

When a current block (or a prediction unit, PU) is coded in Inter AMVP mode, an MVP index and two MVDs for X and Y components are coded to represent the MVs. According to HEVC, for each MVD, magnitude and sign are coded separately using CABAC (Context-Adaptive Binary Arithmetic Coding) entropy coding. Due to different statistics, the magnitude part is always coded using context bin and the sign part is coded using bypass bin (i.e., with equal probability). In this invention, a scheme to code (i.e., encode/decode) the MVD signs more efficiently is disclosed.

When MVP is used, MVs are reconstructed using MVP and MVDs at the decoder side. With the signs of MVDs un-knowing, there are four possibilities for each MV, as shown inFIG. 1. As shown inFIG. 1, absMvdHor and absMvdVer are the absolute values (i.e., magnitudes) of the X and Y components of the MVD110respectively. As shown in this example, if the signs of both X and Y components of the MVD are positive, then the MV can be derived using MVP plus MVD (i.e., MV=MVP+MVD) as indicated inFIG. 1. If the decoder can infer the correct sign position, there is no need to code the signs in the bitstream. In order to perform the decoding process correctly, the same sign derivation process will be performed at the encoder side as well. InFIG. 1, the circle labelled as MVP indicates the position pointed to by the MVP. The four circles labelled with different sign pairs (i.e., (+, +), (+, −), (−, +), (−, −)) correspond to four possible sign positions pointed to by the reconstructed MV. In this disclosure, the MVD may also refer to a vector with MVD X component and MVD Y component for convenience.

In one embodiment, template matching method is used for the decoder to derive the correct sign position. An example of deriving the correct sign position is shown inFIG. 2, where the MV of a current block (214) is to be decoded. The neighbouring reconstructed pixels of current block214are used to form a template (210). The MVP can be derived at the decoder side according to MVP index. At the encoder side, the MVP will be selected from a candidate list and the index of the selected MVP is signalled in the bitstream. When the absMvdHor and absMvdVer are decoded at the decoder, the four possible positions of the reconstructed MV can be determined as shown by the four circles labelled with different sign pairs. The corresponding templates at each possible reconstructed MV position (also referred as “candidate reconstructed MV position” in this disclosure) can be formed. InFIG. 2, the corresponding template (220) for possible position corresponding to (−,+) is shown. Similarly, the templates for three other possible positions can be determined. The corresponding templates are formed with the same number of pixels, shape and distance as the template for the current block in order to perform the template matching. For example, template210is matched with template220. After the matching results for all four possible positions are determined, the best match position based on a predetermined criterion (e.g. SAD or SSD) can be inferred to be the final MVD sign position. Any other information available at the decoder side can also be used to help the sign deriving process and to improve the correctness. For example, the previous statistics of the sign position, MVP position etc. can be used.

In another embodiment, the templates for the four possible sign positions can be derived without using the exact MV precision. For example, if the MVs are coded using quarter-pixel precision, the template positions can be derived using the nearest integer position. However, the two integer-based template positions associated with the “−” and “+” signs in the X direction or the Y direction may fall into the same position, which may occur particularly for small absMvdHor or absMvdVer. If any two of these integer-based templates fall into a same position, constraints or rules can be applied to make sure the two templates are apart from each other. For example, if two templates fall into the same position, one should be located at the left hand side of the MVP and the other should be located at the right hand side of the MVP. Alternatively, one should be located at the upper side of the MVP and the other should be located at the lower side of the MVP.

In another embodiment, the available information at the decoder can be used to improve the correctness of the sign prediction derivation process.FIG. 3illustrates an example, where there are a total of two MVPs (e.g. MVP1and MVP2) and MVP1is used for the current block by the encoder. In other words, MVP2is in the candidate MVP set, but not selected by the encoder as the MVP for the current block. At the decoder side, one or more of the four possible MVD sign positions can be pruned according to the corresponding distances from a sign position to these two MVPs. As shown in this example, if the (−,+) position is used to reconstruct the MV, the resulted MV position will be closer to MVP2position instead of MVP1position. Therefore, the sign (−,+) position can be pruned since the encoder would favour a shorter distance (i.e., smaller MVD magnitude) between a sign position and the MVP position. If the (−,+) position is the correct position, the encoder would have selected MVP2instead of MVP1. By pruning some of the sign positions, it can reduce the possibility of selecting a wrong sign position and also reduce the required computations for evaluating the template matching.

In another embodiment, the sign derivation process can be signalled using one explicit flag to turn on and off.

In another embodiment, the sign derivation process can be determined using implicit method to turn on and off. For example, to turn on the process when the XOR (i.e., exclusive OR operation) between the X component of the MVD and the Y component of the MVD results in an even number (i.e., “0”), and to turn off otherwise.

In another embodiment, the derived sign position can be used a predictor. The true MVD sign can be coded using residues. Table 1 illustrates an example of a codeword table to code the sign residues using the sign predictors. According to the codeword table, if the predictor inferred by the derived sign process matches the true MVD sign position, only one bin or bin (i.e., “0” in Table 1) needs to be coded. On the other hand, the original method will require two bins or bits (one for each sign) for the MVDs. If the predictor is wrong and is the true MVD sign position matches the diagonal sign position of the derived sign position (referred as Diagonal flipping for this condition), more bins or bits (three in this example) are needed. However, no matter how many bins or bits are used in this sign residue coding, coding sign will no longer use bypass bin with equal probability. Instead, it can use context coding with more efficient adaptive probability update. Table 1 also include horizontal flip prediction condition and vertical flip prediction condition. The horizontal flip prediction condition refers to the condition that the true MVD sign position matches the other sign position in the horizontal direction of the derived sign position. The vertical flip prediction condition refers to the condition that the true MVD sign position matches the other sign position in the vertical direction of the derived sign position. In the example as shown in Table 1, the predictor matching the true MVD sign position (i.e., a prediction hit condition corresponding to the signs of the MVD being equal to the selected sign pair) is coded using shortest codeword.

In another embodiment, when any one of the X component and Y component of the MVD is zero, there exist only one MVD sign and two possible sign positions for the non-zero components. In that case, only one code word is needed to indicate if the prediction is hit or not (i.e., miss) in Table 1. The codeword table in Table 1 is still applicable. However, more efficient variation may be applied. Two single sign residues that may come from different prediction lists or different blocks can be combined into one and then use the codeword table in Table 2. In this situation, the code word table will represent different semantics as shown in Table 2.

TABLE 2Code wordBoth hit0The first hit and the second miss10The first miss and the second hit110Both miss111

In another embodiment, at the encoder side, the bits resulted from the sign residues can be taken into consideration during the mode decision in order to choose the best modes. Moreover, during the motion estimation for searching the MVs, it can also consider the sign residue bits for every MV position. In other words, the MV not only generates the better predictors for the current block, but also results in less coded bin for the MVD sign residues.

FIG. 4illustrates an exemplary flowchart of video coding system incorporate an embodiment of the present invention, where the signs of motion vector differences (MVDs) are derived or predicted using template matching. The steps shown in the flowchart, as well as other flowcharts in this disclosure, may be implemented as program codes executable on one or more processors (e.g., one or more CPUs) at the encoder side and/or the decoder side. The steps shown in the flowchart may also be implemented based on hardware such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to this method, input data associated with a video sequence including a current block in a current picture are received in step410. At the encoder side, the input data may correspond to pixel data associated with a video sequence. At the decoder side, the input data may correspond to video bitstream or compressed data for the video sequence. An MVP from the input data for the current block is determined in step420. As is known in video coding, the encoder may selects the MVP from a candidate MVP set and signal the MVP index associated with the selected MVP. The decoder can selected the MVP from the candidate MVP set according to the MVP index. The first magnitude and second magnitude of an MVD associated with the current block are determined in step430. The encoder can obtain the MVD from the difference between the current MV and the MVP. The magnitudes of the MVD are signalled in the bitstream. The decoder may recover the magnitudes of the MVD from the bitstream. Candidate reconstructed MV (motion vector) positions corresponding to all candidate sign pairs of the MVD are derived in step440, where each candidate reconstructed MV position is derived as a vector sum of the MVP and the first magnitude and the second magnitude of the MVD with one candidate sign pair of the MVD. This same step is performed at the encoder as well as the decoder. A template for the current block is determined in step450, where the template is formed from neighbouring reconstructed pixels of the current block. Template matching between the template for the current block and a corresponding template for each candidate reconstructed MV position is then performed in step460. A target candidate MV position is selected among all candidate reconstructed MV positions based on one or more performance criteria including best template matching among all candidate reconstructed MV positions in step470. Encoding or decoding is then applied to the current block using a target candidate MV corresponding to the target candidate MV position in step480.

The flowchart shown is intended to illustrate an example of video coding according to the present invention. A person skilled in the art may modify each step, re-arranges the steps, split a step, or combine steps to practice the present invention without departing from the spirit of the present invention. In the disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. A skilled person may practice the present invention by substituting the syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be one or more circuit circuits integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention. Embodiment of the present invention as described above may be implemented in a video encoder and a video decoder. The components of the video encoder and video decoder may be implemented by hardware components, one or more processors configured to execute program instructions stored in a memory, or a combination of hardware and processor. For example, a processor executes program instructions to control receiving of input data associated with a video sequence including a current block in a current picture. The processor is equipped with a single or multiple processing cores. In some examples, the processor executes program instructions to perform functions in some components in the encoder and the decoder, and the memory electrically coupled with the processor is used to store the program instructions, information corresponding to the reconstructed images of blocks, and/or intermediate data during the encoding or decoding process. The memory in some embodiment includes a non-transitory computer readable medium, such as a semiconductor or solid-state memory, a random access memory (RAM), a read-only memory (ROM), a hard disk, an optical disk, or other suitable storage medium. The memory may also be a combination of two or more of the non-transitory computer readable medium listed above.