Patent Description:
Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a series of pictures, each picture having a spatial dimension of, for example, <NUM> x <NUM> luminance samples and associated chrominance samples. The series of pictures can have a fixed or variable picture rate (informally also known as frame rate), of, for example <NUM> pictures per second or <NUM>. Uncompressed video has significant bitrate requirements. For example, 1080p60 <NUM>:<NUM>:<NUM> video at <NUM> bit per sample (1920x1080 luminance sample resolution at <NUM> frame rate) requires close to <NUM> Gbit/s bandwidth. An hour of such video requires more than <NUM> GBytes of storage space.

In some video compression techniques, an MV applicable to a certain area of sample data can be predicted from other MVs, for example from those related to another area of sample data spatially adjacent to the area under reconstruction, and preceding that MV in decoding order. Doing so can substantially reduce the amount of data required for coding the MV, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example, because when coding an input video signal derived from a camera (known as natural video) there is a statistical likelihood that areas larger than the area to which a single MV is applicable move in a similar direction and, therefore, can in some cases be predicted using a similar motion vector derived from MVs of neighboring area. That results in the MV found for a given area to be similar or the same as the MV predicted from the surrounding MVs, and that in turn can be represented, after entropy coding, in a smaller number of bits than what would be used if coding the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (namely: the MVs) derived from the original signal (namely: the sample stream). In other cases, MV prediction itself can be lossy, for example because of rounding errors when calculating a predictor from several surrounding MVs.

Various MV prediction mechanisms are described in <NPL>). Out of the many MV prediction mechanisms that H. <NUM> offers, described here is a technique henceforth referred to as "spatial merge".

<NPL>; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11) discloses a video coding specification including a flag for use to specify a resolution of a motion vector difference.

<NPL>; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG. <NUM>) discloses generation of a motion vector predictor candidate list.

<NPL>, discloses an adaptive motion vector resolution scheme for enhanced video coding.

Referring to <FIG>, a current block (<NUM>) comprises samples that have been found by the encoder during the motion search process to be predictable from a previous block of the same size that has been spatially shifted. Instead of coding that MV directly, the MV can be derived from metadata associated with one or more reference pictures, for example from the most recent (in decoding order) reference picture, using the MV associated with either one of five surrounding samples, denoted A0, A1, and B0, B1, B2 (<NUM> through <NUM>, respectively). <NUM>, the MV prediction can use predictors from the same reference picture that the neighboring block is using.

The invention is set out in the set of appended claims. Aspects of the disclosure provide methods and apparatuses for video encoding/decoding. In some examples, an apparatus for video decoding includes processing circuitry. The processing circuitry decodes prediction information of a current block from a coded video bitstream. The prediction information is indicative of an inter prediction mode that performs inter prediction based on a motion vector predictor selected from a list of candidates and a motion vector difference. Then, the processing circuitry decodes precision information of the motion vector difference and derives the motion vector predictor from a subset of the list of candidates when the precision information is indicative of a specific precision. Then, the processing circuitry reconstructs a sample of the current block according to the motion vector predictor and the motion vector difference.

In some embodiments, the processing circuitry decodes a first index with a first specific value indicative of the specific precision according to an adaptive motion vector resolution (AMVR). In an embodiment, the processing circuitry infers a flag that is indicative of the subset of the list of candidates based on the first index being the first specific value. In another embodiment, the processing circuitry decodes a second index that is indicative of the motion vector predictor in the subset of the list of candidates. The second index is coded in a first number of bits that is shorter than a second number of bits for coding the list of candidates.

In some embodiments, the processing circuitry derives the motion vector predictor from the subset of the list of candidates that excludes history based motion vector predictor candidates from the list of candidates. In an example, the processing circuitry determines an exclusion of the history based motion vector predictor candidates from the list of candidates based on an adaptive motion vector resolution (AMVR) index.

In some examples, the processing circuitry decodes, from the coded video bitstream, a flag that is indicative of the subset of the list of candidates when the first index is a second value that is different from the first specific value.

According to an aspect of the disclosure, the processing circuitry derives the motion vector predictor from the subset of the list of candidates when the precision information is indicative of a predetermined default precision value.

In some examples, the processing circuitry determines the subset of the list of candidates based on the precision information of the motion vector difference.

Aspects of the disclosure also provide a non-transitory computer-readable medium storing instructions which when executed by a computer for video decoding cause the computer to perform the method for video decoding.

A streaming system may include a capture subsystem (<NUM>), that can include a video source (<NUM>), for example a digital camera, creating for example a stream of video pictures (<NUM>) that are uncompressed. In an example, the stream of video pictures (<NUM>) includes samples that are taken by the digital camera. The stream of video pictures (<NUM>), depicted as a bold line to emphasize a high data volume when compared to encoded video data (<NUM>) (or coded video bitstreams), can be processed by an electronic device (<NUM>) that includes a video encoder (<NUM>) coupled to the video source (<NUM>). The video encoder (<NUM>) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (<NUM>) (or encoded video bitstream (<NUM>)), depicted as a thin line to emphasize the lower data volume when compared to the stream of video pictures (<NUM>), can be stored on a streaming server (<NUM>) for future use. One or more streaming client subsystems, such as client subsystems (<NUM>) and (<NUM>) in <FIG> can access the streaming server (<NUM>) to retrieve copies (<NUM>) and (<NUM>) of the encoded video data (<NUM>). A client subsystem (<NUM>) can include a video decoder (<NUM>), for example, in an electronic device (<NUM>). The video decoder (<NUM>) decodes the incoming copy (<NUM>) of the encoded video data and creates an outgoing stream of video pictures (<NUM>) that can be rendered on a display (<NUM>) (e.g., display screen) or other rendering device (not depicted). In some streaming systems, the encoded video data (<NUM>), (<NUM>), and (<NUM>) (e.g., video bitstreams) can be encoded according to certain video coding/compression standards. Examples of those standards include ITU-T Recommendation H. In an example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

Aspects of the disclosure provide techniques for motion vector prediction (MVP) derivation restriction based on adaptive motion vector resolution (AMVR).

According to some aspects of the disclosure, the techniques of AMVR based MVP derivation restriction can be used in various coding technologies beyond High Efficiency Video Coding (HEVC), such as Versatile Video Coding (VVC), Audio Video coding Standard (AVS) of China, and the like. In some embodiments, when an AMVR index has certain values, MVP derivation can be restricted to a subset of MVP candidates. Then, in some examples, signaling flags can be saved and/or the number of signaling bits can be reduced to improve coding efficiency.

According to some aspects of the disclosure, MVP candidates can include candidates from various sources, such as spatial candidate from spatial neighbor blocks, temporal candidate from collocated blocks; history-based MVP candidates; and the like. In some examples, the spatial candidate(s) and the temporal candidate(s) are referred to as traditional MVP candidates. In an example, a list that includes the spatial candidates and the temporal candidates can be referred to as traditional MVP candidate list, and a list that includes the HMVP candidates is referred to as HMVP candidate list.

For spatial candidate derivation, according to an aspect of the disclosure, the derivation of spatial merge candidates in VVC is similar to that in HEVC. For example, a maximum of four merge candidates are selected among candidates located in the positions A0-A1 and B0-B2 depicted in <FIG>. The order of derivation is A1, B1, B0, A0 and B2. Position B2 is considered only when any CU of position A1, B1, B0, A0 is not available (e.g. belonging to another slice or tile) or is intra coded. After candidate at position A1 is added, the addition of the remaining candidates is subject to a redundancy check which ensures that candidates with same motion information are excluded from the list so that coding efficiency is improved. To reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check.

<FIG> shows a diagram illustrating redundancy check pairs for some embodiments. In an embodiment, only pairs linked with an arrow in <FIG> are considered in redundancy check and a candidate is only added to the list if the corresponding candidate used for redundancy check has not the same motion information.

For temporal candidate derivation, according to an aspect of the disclosure, only one candidate is added to the list. Particularly, in the derivation of the temporal merge candidate, a scaled motion vector is derived based on a co-located CU belonging to the collocated reference picture. The reference picture list to be used for derivation of the co-located CU is explicitly signaled in the slice header in an example.

<FIG> shows an example for temporal candidate derivation. In the <FIG> example, a sequence of pictures are shown, the sequence of pictures includes a current picture having a current CU, a collocated picture having a col-located CU of the current CU, a reference picture of the current picture and a reference picture of the col-located picture. In an example, a picture order count (POC) distance (e.g., difference of POCs) between the reference picture of the current picture and the current picture is denoted as tb, and the POC distance between the reference picture of the col-located picture and the col-located picture is denoted as td. The scaled motion vector for temporal merge candidate is shown by <NUM> in <FIG>, which is scaled from the motion vector <NUM> of the co-located CU using the POC distances, tb and td (e.g., ratio of tb over td). The reference picture index of temporal merge candidate is set equal to zero in an example.

According to an aspect of the disclosure, a HMVP candidate is defined as the motion information of a previously coded block. In some embodiments, a table with multiple HMVP candidates is maintained during the encoding/decoding process. For example, the table is emptied at a beginning of a new slice. When there is an inter-coded block, such as in merge mode, in skip mode and the like, the associated motion information is added to the last entry of the table as a new HMVP candidate.

During operation in an example using the HMVP technique, a table with HMVP candidates is loaded before decoding a block. Then, the block is decoded with the HMVP candidates in the table in an example. Further, the table is updated with decoded motion information of the block. The updated table can be loaded to decode subsequent blocks.

In some examples, HMVP technique uses first-in-first-out (FIFO) rule to maintain the candidate list, for example based on a table. In an example, a table size S is set to be a constant value, such as <NUM>, which indicates that up to, for example <NUM> HMVP candidates, may be added into the table. In an embodiment, the table is implemented according to the FIFO rule. Further, at a time to insert a new motion candidate, referred to as a new HMVP candidate, into the table, a constrained FIFO rule is utilized. In some embodiments, the table is implemented using a buffer.

<FIG> shows an example of using the constrained FIFO rule to insert a new motion candidate. According to the constrained FIFO rule, a redundancy check is applied to determine whether the table includes an identical HMVP candidate to the new HMVP candidate. When an identical HMVP candidate, such as HMVP2 shown in <FIG>, is found, the identical HMVP candidate is removed from the table and all the HMVP candidates afterwards are moved forward by <NUM> position, and the new HMVP candidate is added at the end (the latest position) of the table.

In some embodiments, HMVP candidates are used in the merge candidate list construction process. In an example, the latest several HMVP candidates in the table are checked in order (from the latest to older ones) and inserted to the candidate list after the temporal motion vector prediction (TMVP) candidate.

According to an aspect of the disclosure, adaptive motion vector resolution (AMVR) can be used in video codec, such as video codec according to VVC and AVS3. In some examples, fixed resolution can be used. For example, in HEVC, motion vector differences (MVDs) (between the motion vector and predicted motion vector of a CU) are signalled in the unit of quarter-luma-sample when use_integer_mv_flag is equal to <NUM> in the slice header. AMVR allows MVD of the CU to be coded in different resolutions, such as quarter-luma-sample, integer-luma-sample or four-luma-sample. In an example, such as in VVC, the CU-level MVD resolution indication is conditionally signalled in coded video bitstream if the current CU has at least one non-zero MVD component. If all MVD components (that is, both horizontal and vertical MVDs for reference list L0 and reference list L1) are zero, quarter-luma-sample MVD resolution is inferred in an example.

In some implementation examples, at the encoder side, the encoder determines the motion vector resolution for the current CU using rate distortion check. In some examples, rate distortion check can be performed for different resolutions, such as a precision of a full pixel size sample (also known as "an integer-luma-sample precision"), a precision of a half of a luma sample (also known as "half-luma-sample precision"), a precision of a quarter of a luma sample (also known as "quarter-luman-sample precision"), a precision of four-pixel size sample (also known as "four-luma-sample precision"), etc.. To avoid always performing CU-level rate distortion check three times for MVD resolutions, in some examples (e.g.,VTM-<NUM>), the rate distortion (RD) check of four-luma-sample MVD resolution is only invoked conditionally. For example, the RD cost of quarter-luma-sample MVD precision is computed first. Then, the RD cost of integer-luma-sample MVD precision is compared to RD cost of quarter-luma-sample MVD precision to decide whether further RD cost check of four-luma-sample MVD precision is needed. When the RD cost for quarter-luma-sample MVD precision is much smaller than the RD cost of the integer-luma-sample MVD precision, the RD check of four-luma-sample MVD precision can be skipped.

In some inter coding implementation examples, such as in VTM and AVS3, AMVR and HMVP are both used. In some examples, AMVR and HMVP are used together, thus the best MVP can be searched in the traditional special/temporal MVP candidate list and/or HMVP candidate list based on different motion vector resolutions that are used. Then, an AMVR index and an MVP index with the best rate distortion cost are transmitted in the coded video bitstream from the encoder side to the decoder side in an example. In an example, AMVR index with value "<NUM>" corresponds to the quarter-luma-sample precision, AMVR index with value "<NUM>" corresponds to the integer-luma-sample precision, and AMVR index with value "<NUM>" corresponds to four-luma-sample precision. It is note that other suitable corresponding relationship of AMVR indexes to precisions can be used.

In some embodiments, such as in AVS3, to save the encoding time, a look up table for combining AMVR index and HMVP index is proposed. Specifically, when AMVR index is specified, and the encoder decides to choose the MVP from the HMVP candidate list instead of the traditional MVP candidate list, the MVP is chosen from a specified value in HMVP candidate list according to the lookup table. Otherwise, if the encoder chooses the MVP from the traditional MVP candidate list, the MVP can only be derived by the traditional methods according to the spatial and temporal candidates (e.g., traditional candidates). In some examples, the bitstream includes the AMVR index and a flag indicating whether the MVP is derived from HMVP candidates or not. For example, when the flag is true (e.g., binary <NUM>), the MVP is derived from the HMVP candidates, and when the flag is false (e.g., binary <NUM>), the MVP is not derived from the HMVP candidates. The flag is referred to as HMVP flag in the present disclosure, but can be given other suitable name.

Regardless of the implementation, the spatial and temporal MVP candidate list (traditional MVP candidate list) and HMVP candidate list and any other suitable list that can increase the number of MVP candidates can be gathered and combined, and can be seen as one set of MVP candidates, and the combined list of MVP candidates is referred to as a combined MVP candidate list in the present disclosure.

Aspects of the disclosure provide techniques and methods for motion vector prediction derivation restriction based on AMVR index to improve coding efficiency. The proposed methods may be used separately or combined in any order. Further, each of the methods (or embodiments), encoder, and decoder may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program that is stored in a non-transitory computer-readable medium. In the following description, the term block may be interpreted as a prediction block, a coding block, or a coding unit, i.e. CU.

According to an aspect of the disclosure, the MVP derivation is restricted based on AMVR index adaptively. In some examples, the MVP derivation is restricted to a subset of the combined MVP candidate list, and then the signaling of the derived MVP candidate can be skipped or with less bits. For some cases in which the MVP derivation is not restricted, the signaling is not changed.

In an embodiment, whether to signal the HMVP flag or not is decided based on the value of AMVR index. In some examples, when the AMVR index is some specified values, MVP can be derived from all candidates in the combined MVP candidate list, and the HMVP flag can be signaled for indicating whether the MVP is derived from HMVP candidates or other additional candidate list. When the AMVR index is not the specified values, in some examples, MVP is derived from a subset of the combined MVP candidate list, such as the subset for traditional MVP candidates. Then, in an example, the HMVP flag can be inferred based on the value of the AMVR index, thus can be skipped without signaling.

In an example in AVS3, when the AMVR index is larger than a value (e.g., <NUM>), the HMVP flag for indicating whether the MVP is derived from HMVP is signaled. Then, when AMVR index is <NUM>, MVP is derived from the candidates excluding HMVP. Thus, the HMVP flag is inferred to be zero when the AMVR index is <NUM> in an example, thus signaling of HMVP flag can be skipped.

In another embodiment, the MVP index can be coded by different schemes, and the coding scheme for MVP index is based on the value of AMVR index. In some examples, when the AMVR index is some specified values, a first MVP index coding scheme is used, the first MVP index coding scheme is used to indicate a selected MVP from the combined MVP candidate list. When the AMVR index is not the specified values, in some examples, the HMVP candidate list is excluded from the MVP derivation, and the MVP derivation is performed on a subset of the combined MVP candidate list, such as the traditional MVP candidate list. The traditional MVP candidate list has a smaller number of candidates than the combined MVP candidate list. Thus, a coding scheme with a smaller number of bits, such as a more efficient entropy coding, can be used to indicate the selected MVP from the subset.

In an example, when AMVR index larger than a value(e.g., <NUM>), the MVP index coding does not change, such as a default coding scheme for indicating a selected MVP from the combined MVP candidate list. When the AMVP index is <NUM>, which implicitly indicates that the selected MVP is not derived from HMVP candidate list, then the index range for the selected MVP is smaller than the combined MVP candidate list, thus the MVP index can use smaller number of bits in fix length coding in an example.

It is noted that, other suitable algorithm, besides the HMVP, that can increase the number of candidates in a MVP candidate list can be used. The MVP candidate list before the increase of candidates is referred to as traditional MVP candidate list, and the MVP candidate list after the increase of candidates is referred to as combined MVP candidate list for ease of description. Similar techniques for deciding the MVP index coding based on the value of AMVR index can be used. For example, when the AMVR index is some specified values, the MVP can only be derived from a subset of the MVP candidate list, such as the traditional MVP candidate list, then MVP index can use smaller number of bits or more efficient entropy coding method. Otherwise, the MVP index coding and the derivation process do not change and are the same as the ones used for indicating one candidate from the combined MVP candidate list.

<FIG> shows a flow chart outlining a process (<NUM>) according to an embodiment of the disclosure. The process (<NUM>) can be used in the reconstruction of a block, so to generate a prediction block for the block under reconstruction. In various embodiments, the process (<NUM>) are executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>) and (<NUM>), the processing circuitry that performs functions of the video encoder (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), the processing circuitry that performs functions of the video encoder (<NUM>), and the like. In some embodiments, the process (<NUM>) is implemented in software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>). The process starts at (S1101) and proceeds to (S1110).

At (S1110), prediction information of a current block is decoded from a coded video bitstream. The prediction information is indicative of an inter prediction mode that performs an inter prediction based on a motion vector predictor selected from a list of candidates and a motion vector difference.

At (S1120), precision information of the motion vector difference is decoded. In an example, an AMVR index is decoded from the video bitstream. The AMVR index has a value corresponding to a precision.

At (S1130), the motion vector prediction is derived from a subset of the list of candidates when the precision information is indicative of a specific precision. In an example, when the AMVR index is <NUM>, a HMVP flag can be inferred to be <NUM>. When HMVP flag is zero, HMVP candidates are excluded from the list of candidates to form a subset of the list of candidates. Then, an MVP index can be decoded. The MVP index can be coded in the video bitstream in a smaller number of bits due to the exclusion of some candidates (e.g., HMVP candidates) from the list of candidates.

At (S1140), samples of the current block are reconstructed according to the motion vector predictor and the motion vector difference. Then, the process proceeds to (S1199), and terminates.

Computer system (<NUM>) can also include an interface to one or more communication networks. Networks can for example be wireless, wireline, optical. Networks can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks to include GSM, <NUM>, <NUM>, <NUM>, LTE and the like, TV wireline or wireless wide area digital networks to include cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial to include CANBus, and so forth. Certain networks commonly require external network interface adapters that attached to certain general purpose data ports or peripheral buses (<NUM>) (such as, for example USB ports of the computer system (<NUM>)); others are commonly integrated into the core of the computer system (<NUM>) by attachment to a system bus as described below (for example Ethernet interface into a PC computer system or cellular network interface into a smartphone computer system). Using any of these networks, computer system (<NUM>) can communicate with other entities. Such communication can be uni-directional, receive only (for example, broadcast TV), uni-directional send-only (for example CANbus to certain CANbus devices), or bidirectional, for example to other computer systems using local or wide area digital networks. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

Claim 1:
A method for video decoding in a decoder, comprising:
decoding prediction information of a current block from a coded video bitstream, the prediction information being indicative of an inter prediction mode that performs inter prediction based on a motion vector predictor selected from a list of candidates and a motion vector difference;
decoding precision information of the motion vector difference;
deriving the motion vector predictor from a subset of the list of candidates when the precision information is indicative of a specific precision; and
reconstructing a sample of the current block according to the motion vector predictor and the motion vector difference.