Patent Description:
Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a series of pictures, each picture having a spatial dimension of, for example, <NUM> x <NUM> luminance samples and associated chrominance samples. The series of pictures can have a fixed or variable picture rate (informally also known as frame rate), of, for example <NUM> pictures per second or <NUM>. Uncompressed video has significant bitrate requirements. For example, 1080p60 <NUM>:<NUM>:<NUM> video at <NUM> bit per sample (1920x1080 luminance sample resolution at <NUM> frame rate) requires close to <NUM> Gbit/s bandwidth. An hour of such video requires more than <NUM> GBytes of storage space.

In some video compression techniques, an MV applicable to a certain area of sample data can be predicted from other MVs, for example from those related to another area of sample data spatially adjacent to the area under reconstruction, and preceding that MV in decoding order. Doing so can substantially reduce the amount of data required for coding the MV, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example, because when coding an input video signal derived from a camera (known as natural video) there is a statistical likelihood that areas larger than the area to which a single MV is applicable move in a similar direction and, therefore, can in some cases be predicted using a similar motion vector derived from MVs of neighboring area. That results in the MV found for a given area to be similar or the same as the MV predicted from the surrounding MVs, and that in turn can be represented, after entropy coding, in a smaller number of bits than what would be used if coding the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (namely: the MVs) derived from the original signal (namely: the sample stream). In other cases, MV prediction itself can be lossy, for example because of rounding errors when calculating a predictor from several surrounding MVs.

Various MV prediction mechanisms are described in H. <NUM>/HEVC (ITU-T Rec. <NUM>, "High Efficiency Video Coding", December <NUM>). Out of the many MV prediction mechanisms that H. <NUM> offers, described here is a technique henceforth referred to as "spatial merge".

Referring to <FIG>, a current block (<NUM>) comprises samples that have been found by the encoder during the motion search process to be predictable from a previous block of the same size that has been spatially shifted. Instead of coding that MV directly, the MV can be derived from metadata associated with one or more reference pictures, for example from the most recent (in decoding order) reference picture, using the MV associated with either one of five surrounding samples, denoted A0, A1, and B0, B1, B2 (<NUM> through <NUM>, respectively). <NUM>, the MV prediction can use predictors from the same reference picture that the neighboring block is using.

The invention is defined by the claims annexed hereto. The invention is characterised over <CIT> (<NUM>-<NUM>-<NUM>), which discloses how the regular merge mode is checked (mergeFlag) and, only if FALSE, the so called refined merge mode checked (the mergeRefinedFlag).

A streaming system may include a capture subsystem (<NUM>), that can include a video source (<NUM>), for example a digital camera, creating for example a stream of video pictures (<NUM>) that are uncompressed. In an example, the stream of video pictures (<NUM>) includes samples that are taken by the digital camera. The stream of video pictures (<NUM>), depicted as a bold line to emphasize a high data volume when compared to encoded video data (<NUM>) (or coded video bitstreams), can be processed by an electronic device (<NUM>) that includes a video encoder (<NUM>) coupled to the video source (<NUM>). The video encoder (<NUM>) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (<NUM>) (or encoded video bitstream (<NUM>)), depicted as a thin line to emphasize the lower data volume when compared to the stream of video pictures (<NUM>), can be stored on a streaming server (<NUM>) for future use. One or more streaming client subsystems, such as client subsystems (<NUM>) and (<NUM>) in <FIG> can access the streaming server (<NUM>) to retrieve copies (<NUM>) and (<NUM>) of the encoded video data (<NUM>). A client subsystem (<NUM>) can include a video decoder (<NUM>), for example, in an electronic device (<NUM>). The video decoder (<NUM>) decodes the incoming copy (<NUM>) of the encoded video data and creates an outgoing stream of video pictures (<NUM>) that can be rendered on a display (<NUM>) (e.g., display screen) or other rendering device (not depicted). In some streaming systems, the encoded video data (<NUM>), (<NUM>), and (<NUM>) (e.g., video bitstreams) can be encoded according to certain video coding/compression standards. Examples of those standards include ITU-T Recommendation H. In an example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

In various unclaimed embodiments, the video encoder (<NUM>) also includes a residue decoder (<NUM>).

<FIG> shows a diagram of a video decoder (<NUM>) according to another unclaimed embodiment of the disclosure.

In an unclaimed embodiment, the video encoders (<NUM>), (<NUM>), and (<NUM>), and the video decoders (<NUM>), (<NUM>), and (<NUM>) can be implemented using one or more processors that execute software instructions.

<FIG> is a schematic illustration of a current block (<NUM>) and a motion vector (<NUM>) expressed according to a merge with motion vector difference (MMVD) mode in accordance with an embodiment. A predictor of the current block (<NUM>) can be generated by inter-picture prediction based on the motion vector (<NUM>).

A motion vector of a block can be expressed by the MMVD mode (or also referred to as an ultimate motion vector expression, UMVE) with simplified signaling. For example, the motion vector (<NUM>) for a current block (<NUM>) can be expressed using a base motion vector (<NUM>), a distance of adjustment (<NUM>), and a direction of adjustment (<NUM>). The distance of adjustment (<NUM>) and the direction of adjustment (<NUM>) can be referred to as motion vector difference information for the current block, and together define a difference vector (<NUM>). The motion vector (<NUM>) can be derived as a combination of the base motion vector (<NUM>) and the difference vector (<NUM>).

The base motion vector (<NUM>) can be selected from a merge candidate list, where the selection information can be indicated by a syntax element (e.g., "base_mv_idx" in VVC), which corresponds to a base candidate index indicating a selected candidate in the merge candidate list for deriving the base motion vector. In some examples, only a portion of the merge candidate list is applicable for selection to determine the base motion vector according to MMVD. In at least one unclaimed embodiment, the base candidate index can range from <NUM> to a maximum number defined by a maximum MMVD index (e.g., defined as "max_mmvd_base_idx - <NUM>" in VVC). The maximum MMVD index can be a value provided in an applicable video coding standard or signaled in a coded video bitstream.

The distance of adjustment (<NUM>) can be indicated by a syntax element (e.g., "distance_idx" in VVC), which corresponds to a distance between the motion vector (<NUM>) and the base motion vector (<NUM>), or corresponds to a magnitude of the difference vector (<NUM>). In some unclaimed embodiments, the direction of adjustment (<NUM>) can be indicated by a syntax element (e.g., "direction _idx" in VVC), which corresponds to a direction from the motion vector (<NUM>) to the base motion vector (<NUM>), or corresponds to a direction of the difference vector (<NUM>).

In some unclaimed embodiments, to indicate whether a motion vector for a current block is generated according to the MMVD mode, a flag (e.g., "mmvd_flag" in VVC) can be signaled. In some examples, mmvd_flag can be set to <NUM> when the motion vector for the current block is to be generated according to the MMVD mode; and mmvd_flag can be set to <NUM> when the motion vector for the current block is not to be generated according to the MMVD mode. In some examples, when mmvd_flag applicable to the current block is not present, mmvd_flag can be inferred to have a value corresponding to the value for indicating that the motion vector for the current block is not to be generated according to the MMVD mode.

In some examples regarding decoding a current block, when mmvd_flag[x0][y0] is set to <NUM>, a motion vector for the current block is to be generated according to the MMVD mode, where (x0, y0) correspond to the location of the top-left luma sample of the current block relative to the top-left luma sample of a current picture that includes the current block. In some examples regarding decoding the current block, after parsing mmvd_flag[x0][y0] and determining that mmvd_flag[x0][y0] is set to <NUM>, the syntax elements base_mv_idx[x0][y0], distance _idx[x0] [y0], and direction_idx[x0][y0] can be further parsed to obtain applicable motion vector difference information.

A motion vector of a block can be determined according to a sub-block merge prediction mode. In some examples, the sub-block merge prediction mode can be a sub-block affine merge prediction mode or a sub-block based alternative temporal motion vector prediction (ATMVP) mode.

<FIG> is a schematic illustration of a current block (<NUM>) and spatial neighboring blocks denoted A0, A1, A2, B0, B1, B2, and B3 (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>, respectively) and a temporal neighboring block denoted C0 (<NUM>) for the current block according to a sub-block affine merge prediction mode in accordance with an embodiment. In some examples, spatial neighboring blocks A0, A1, A2, B0, B1, B2, and B3 and the current block (<NUM>) belong to a same picture. The temporal neighboring block C0 can belong to a reference picture and correspond to a position outside the current block (<NUM>) and adjacent to a lower-right corner of the current block (<NUM>).

The current block (<NUM>) can be divided into sub-blocks (<NUM>, <NUM>, <NUM>, and <NUM>). The motion information of each sub-block (<NUM>, <NUM>, <NUM>, and <NUM>) inside the current block (<NUM>) can be determined according to a sub-block affine merge prediction mode described by a <NUM>-parameter or a simplified <NUM>-parameter affine model. In some examples, an affine model has <NUM> parameters (e.g., the <NUM>-parameter affine model) to describe the motion vector of a sub-block. In an example, the <NUM> parameters of a sub-block can be represented by three motion vectors (also referred to as three control point motion vectors) at three different locations of the current block (e.g., control points CP0, CP1, and CP2 at upper-left, upper-right, and lower-left corners of the current block (<NUM>) in <FIG>). In another example, a simplified affine model (e.g., a <NUM>-parameter affine model) uses four parameters to describe the motion information of a sub-block, which can be represented by two motion vectors (also referred to as two control point motion vectors) at two different locations of the block (e.g., control points CP0 and CP1 at upper-left and upper-right corners in <FIG>).

<FIG> is a schematic illustration of a current block (<NUM>) in a current picture (<NUM>) and a reference block (<NUM>) in a reference picture (<NUM>) according to a sub-block based alternative temporal motion vector prediction (ATMVP) mode in accordance with one embodiment. According to the ATMVP mode, the motion vectors of sub-blocks within a block are determined by first identifying a corresponding reference block in a reference picture, and then splitting the current block into sub-blocks and obtaining the motion vectors as well as the reference indices of the sub-blocks in the current block from corresponding motion information of the reference block.

In some examples, the reference picture (<NUM>) and a reference vector (<NUM>) for a current block (<NUM>) can be identified based on decoded information, such as motion information of one or more spatial neighboring blocks of the current block (<NUM>) or other applicable approaches. After determining the reference picture (<NUM>) and the reference vector (<NUM>), the reference block (<NUM>) can be identified based on a position of the current block (<NUM>) and the determined the reference vector (<NUM>).

In <FIG>, the reference block (<NUM>) can be further divided into <NUM> sub-blocks with reference motion information MRa through MRd. The motion information can include motion vectors and corresponding reference indices. The current block (<NUM>) can be further divided into <NUM> sub-blocks, and the motion information MVa through MVd for the sub-blocks in the current block (<NUM>) can be derived from the reference motion information MRa through MRd, with temporal scaling in some examples. In some embodiments, the reference block (<NUM>) and the current block (<NUM>) can be divided into more or less than <NUM> sub-blocks, and the reference block (<NUM>) and the current block (<NUM>) do not have to have the same number of sub-blocks or the same sub-block partitioning structure.

In some embodiments, to indicate whether a motion vector for a current block is generated according to a sub-block merge prediction mode, such as the sub-block affine merge prediction mode or the ATMVP mode, a flag (e.g., "sub_block_flag" in VVC) can be signaled. In addition, a syntax element (e.g., "merge _idx" in VVC) can be further signaled to indicate which candidate within a sub-block merge candidate list is selected when the flag (e.g., "sub_block_flag" in VVC) indicates that the sub-block merge prediction mode is used.

In some examples regarding decoding a current block, when sub_block_flag[x0] [y0] is set to <NUM>, a motion vector for the current block is to be generated according to the sub-block merge prediction mode, where (x0, y0) correspond to the location of the top-left luma sample of the current block relative to the top-left luma sample of a current picture that includes the current block. In some examples regarding decoding the current block, after parsing sub_block_flag[x0][y0] and determining that sub_block_flag[x0][y0] is set to <NUM>, the syntax element merge_idx[x0][y0] can be further parsed to obtain applicable candidate selection information.

<FIG> is a schematic illustration of a current block (<NUM>) that is reconstructed according to at least an inter predictor (<NUM>) and an intra predictor (<NUM>) for the current block (<NUM>) according to a multi-hypothesis prediction for Intra (MHIntra) mode in accordance with one embodiment.

According to the MHIntra mode, the current block (<NUM>) can be reconstructed by combining a final predictor (<NUM>) and residual samples (<NUM>) of the current block (<NUM>). The final predictor (<NUM>) can be generated based on at least one prediction method, such as based on a combination of the inter predictor (<NUM>) generated by inter prediction and the intra predictor (<NUM>) generated by intra prediction. The residual samples (<NUM>) represent the difference between the current block (<NUM>) and the final predictor (<NUM>) and can be obtained by decoding a received video bitstream.

In some unclaimed embodiments regarding generating the inter predictor (<NUM>), the applicable motion information and reference indices can be explicitly signaled or derived according to a suitable motion information prediction method.

In some unclaimed embodiments regarding generating the intra predictor (<NUM>), for a luma component of the current block, the applicable intra prediction mode can be selected from an intra candidate list, and the intra candidate list can be derived from <NUM> intra prediction modes (e.g., a Direct Current mode, a Planar mode, a Horizontal mode, and a Vertical mode). The selection information for determining the applicable intra prediction mode can be indicated by an intra mode index. In some unclaimed embodiments, for a chroma component of the current block, a Derived mode based on the selected intra prediction mode for the corresponding luma component can be applied without extra signaling.

In order to indicate whether a current block is to be decoded according to the MHIntra mode, a flag (e.g., "MHIntra_flag" in VVC) can be signaled. To further indicate which intra mode is used for generating the intra predictor (<NUM>) when the flag (e.g., "MHIntra_flag") indicates that the MHIntra mode is used, a syntax element (e.g., "MHIntra_mode_idx" in VVC) can be signaled.

In some unclaimed embodiments regarding decoding a current block, when MHIntra_flag[x0][y0] is set to <NUM>, the current block will be decoded according to the MHIntra mode, where (x0, y0) specifies the location of the top-left luma sample of the current block relative to the top-left luma sample of the picture. In some examples regarding decoding the current block, after parsing MHIntra_flag[x0][y0] and determining that MHIntra_flag[x0][y0] is set to <NUM>, the syntax element MHIntra_mode_idx[x0][y0] can be further parsed to obtain applicable intra prediction mode. Also, after parsing MHIntra_flag[x0][y0] and determining that MHIntra_ flag[x0][y0] is set to <NUM>, to further indicate the motion information for generating the inter predictor (<NUM>) of the current block (<NUM>), the motion information can be explicitly signaled or derived based on information signaled by one or more additional syntax elements.

<FIG> is a schematic illustration of two splitting examples for a current block (1110A or 1110B) according to a triangle prediction unit mode in accordance with one embodiment. In some embodiments according to the triangle prediction unit mode, a current block (1110A or 1110B) can be split into two triangular prediction units. In some examples, each triangular prediction unit in the current block (1110A or 1110B) can be inter-predicted based on respective uni-prediction motion vector and reference frame index. After predictors for the triangular prediction units are generated, an adaptive weighting process can be applied to the diagonal edge between the two triangular prediction units to derive a final predictor for the current block (1110A or 1110B).

In Example <NUM> shown in <FIG>, the current block (1110A) can be divided into two triangular prediction units (<NUM> and <NUM>) along a diagonal extending from a top-left corner to a bottom-right corner of the current block (1110A). In some examples, such as Example <NUM> as shown in <FIG>, the current block (1110B) can be divided into two triangular prediction units (<NUM> and <NUM>) along a diagonal extending from a top-right corner to a bottom-left corner of the current block (1110B).

In some unclaimed embodiments, to indicate whether a current block is to be decoded according to the triangle prediction unit mode, a flag (e.g., "merge_triangle_flag" in VVC) can be signaled. To further indicate the splitting method and the indices of the candidates used by the two triangular prediction units, one or more syntax elements (e.g., "merge_triangle_idx," "merge_triangle_idx0" and/or "merge_triangle_idx1" in VVC) can be signaled.

In some unclaimed embodiments regarding decoding a current block, when merge_triangle_flag[x0][y0] is set to <NUM>, the current block will be decoded according to the triangle prediction unit mode, where (x0, y0) specifies the location of the top-left luma sample of the current block relative to the top-left luma sample of the picture. In some examples regarding decoding the current block, after parsing merge_triangle_flag[x0][y0] and determining that merge _triangle_flag[x0][y0] is set to <NUM>, the syntax element merge_triangle_idx[x0][y0] can be parsed to obtain applicable division direction and the syntax elements merge_triangle_idx0[x0] [y0] and merge _triangle_idx1[x0][y0] can be further parsed to obtain applicable candidate selection information for different triangular prediction units.

All the coding modes described with references to <FIG> are available for video coding. In some examples, which mode is used to decode a particular block can be determined based on various flags, such as flags mmvd_flag, sub_block_flag, MHIntra_flag, and merge_triangle_flag. In some examples, a portion or all of the aforementioned flags can be signaled according to an order of mmvd _flag, sub_block_flag, MHIntra _flag, and merge _triangle_flag, and not all of these flags need to be signaled, provided the signaled portion of the flags can unambiguously specify the applicable mode used to decode the particular block.

These flags can be conditionally signaled as illustrated in Table I, where the syntax elements and the corresponding conditions for signaling the respective syntax elements are listed. In Table I, a flag being "False" represents the coding mode represented by such flag is not used for a current block, and a flag being "True" represents the coding mode represented by such flag is used for a current block.

In some unclaimed embodiments according to Table I, when flags mmvd _flag, sub_block_flag, MHIntra _flag, and merge _triangle_flag are all set to false, a regular merge mode is used where a syntax element merge_idx for regular merge mode is signaled.

The regular merge mode may be a most-frequently used mode in a particular picture. Therefore, the flags or syntax elements may be reconfigured so it does not need to signal all flags mmvd_flag, sub_block_flag, MHIntra _flag, and merge _triangle_flag every time the regular merge mode is used. Also, to improve the coding efficiency, the syntax element merge _idx may be configured to represent various parameters for different modes. For example, the syntax element merge _idx may indicate candidate selection information for the regular merge mode and may indicate base vector selection information for the MMVD mode.

In addition, the flags and/or syntax elements may be reconfigured as follows. A block may correspond to a prediction block, a coding block, or a coding unit, and the term width and height denote the width and height of a block.

In some embodiments, a new syntax element (denoted as "merge_list_idc" in this disclosure) can be introduced in place of some or all of flags mmvd_flag, sub_block_flag, MHIntra_flag, and merge_triangle_flag. For example, merge_list_idc can be set to <NUM> (or False) to indicate that the regular merge mode is used, and merge_list_idc can be set to <NUM> (or True) to indicate that the regular merge mode is not used. When merge_list_idc is True, which modes among all other modes may be signaled or derived based on one or more additional flags or syntax elements, which is not limited to the modes, flags, or syntax elements described herein and can be simplified or expanded.

In one example, merge_list_idc can be used to signal whether the regular merge mode is used, a value of sub_block_flag, and a value of merge_triangle_flag. The flags mmvd_flag and MHIntra_flag can be conditionally signaled or inferred according to the value of merge_list_idc. In some examples, merge_list_idc can be binarized according to the usage frequency of the signaled flags (corresponding to the usage frequency of the respective modes), where the most frequently used flag has the smallest length code and the least frequently used flag has the longest length code. For example, various values of merge_list_idc after binarization can represent the following settings: "<NUM>" means the regular merge mode is used, "<NUM>" means the sub_block_flag is True (i.e., the sub-block merge prediction mode is used), and "<NUM>" means the merge_triangle_flag is True (i.e., the triangle prediction unit mode is used).

In another embodiment, merge_list_idc can be used to signal whether the regular merge mode is used, a value of mmvd _flag, a value of sub_block_flag, and a value of merge_triangle_flag. The MHIntra_flag can be inferred according to the value of merge_list_idc. In some examples, merge_list_idc can also be binarized according to the usage frequency of the signaled flags (corresponding to the usage frequency of the respective modes), where the most frequently used flag has the smallest length code and the least frequently used flag has the longest length code. For example, various values of merge_list_idc after binarization can represent the following settings: "<NUM>" means the regular merge mode is used, "<NUM>" means the mmvd_flag is True (i.e., the MMVD mode is used), "<NUM>" means the sub_block_flag is True (i.e., the sub-block merge prediction mode is used), and "<NUM>" means the merge_triangle_flag is True (i.e., the triangle prediction unit mode is used).

In another embodiment, merge_list_idc can be used to signal whether the regular merge mode is used, a value of mmvd _flag, a value of sub_block_flag, and a value of MHIntra_flag. If more merge modes are added to merge_list_idc, simple extension may be applied.

In some examples, for binarization of merge_list_idc, truncated unary code may be used. In some embodiments, first one or two bins are variable length coded while the remaining bins are fixed length coded. Basically, the order of indicating the use of various modes in merge_list_idc may be arranged based on the usage frequency of these merge modes. In some examples, it is necessary to place the most-frequently used mode as the first (hence may correspond to a smallest length code). In one embodiment, the merge modes are ordered based on the absolute difference between the usage frequency and <NUM>%. For example, the mode which has the largest absolute difference can be placed as the first one.

In one embodiment, the binarization of merge_list_idc can be performed based on Table II.

In another embodiment, the binarization of merge_list_idc can be performed based on Table III.

In another embodiment, the binarization of merge_list_idc can be performed based on Table IV.

In another embodiment, the binarization of merge_list_idc can be performed based on Table V.

The merge_list_idc may be signaled in a high-level syntax structure in a bitstream, such as in Sequence Parameter Set (SPS), Picture Parameter Set (PPS), or slice header, or tile group header, or tile header, or a suitable header that is associated with a picture or a fraction of a picture.

The merge_list_idc can be coded with context model. In one example, the merge_list_idc from the spatial neighboring blocks that are above or left to a current block are used to derive the context model for coding the merge_list_idc of the current block. In one example, width and height of the current block can be used to derive the context model for coding the merge_list_idc of the current block. In some embodiments, each bin of merge_list_idc can be context coded, where different bins may use different contexts. In one example, the bin indicating regular merge mode can be coded according to spatial neighbors as context. In another example, the bin indicating sub-block merge prediction mode can be coded according to one single context which is not dependent on spatial neighbors.

In some embodiments, the MMVD related syntax elements that indicate corresponding MMVD related information, such as mmvd_flag and/or motion vector difference information, can be signaled after signaling merge_list_idc or a dedicated flag (denoted "regular_merge_flag" in this disclosure) indicating whether the regular merge mode is used, and after the syntax element merge _idx is signaled. In one example, the syntax element merge _idx is signaled after regular_merge _flag/merge_list_idc and before mmvd _flag, sub_block_flag, MHIntra _flag, and merge _triangle_flag, if any, signaled for a current block.

In another example, the regular_merge_flag/merge_list_idc is firstly signaled to explicitly indicate whether the regular merge mode is used or not, and then the syntax element merge _idx is signaled. In some embodiments, information carried by "base_mv_idx" described above can be signaled by "merge _idx. " After signaling regular_merge_flag/merge_list_idc and merge _idx, the MMVD related syntax elements can be conditionally signaled. In one example, if regular merge_flag/merge_list_idc indicates that the regular merge mode or the MMVD mode can be used and a candidate index indicated by merge _idx is smaller than max_mmvd_base_idx, it is possible that MMVD mode could be used and the mmvd_flag is further signaled; otherwise, the MMVD mode can be ruled out and thus the mmvd_flag is inferred to be False and not signaled. If the mmvd_flag is True, the syntax elements distance _idx and direction _idx are signaled.

In some embodiments, the regular merge mode and MMVD mode can be categorized as block based merge modes, and sub-block merge prediction mode (including ATMVP and affine modes), triangle merge prediction mode, and MHIntra mode can be categorized as sub-block based merge modes. Accordingly, merge_list_idc can be used to signal whether the mode for coding the current block is one of the block based merge modes or the sub-block based merge modes.

For example, when merge _flag is True indicating that a merge mode is used, a flag merge_list_idc can be signaled to indicate whether the current block is coded according to one of the block based merge modes or the sub-block based merge modes. In some examples, merge _idx is signaled after merge_list_idc but before all other merge related flags; or it can be signaled after all other merge related flags.

When one of the block based merge modes is used, depending on the merge_idx value (if signaled before all other merge related flags), mmvd_flag may be signaled. In such examples, information carried by "base_mv_idx" described above can be signaled by "merge _idx" instead. For example, if merge_list_idc indicates that the regular merge mode or the MMVD mode can be used and a candidate index indicated by merge _idx is smaller than max_mmvd_base_idx, it is possible that MMVD mode could be used and the mmvd_flag is signaled; otherwise, the MMVD mode can be ruled out and thus the mmvd_flag is inferred to be False and not signaled. If the mmvd_flag is True, MMVD mode related information will be further signaled.

When one of the sub-block merge modes is used, depending on the merge _idx value, (if signaled before all other merge related flags), sub_block_merge_flag (corresponding to affine merge mode and ATMVP mode) may be signaled. In one example, when a sub-block merge mode is used, and sub_block_flag is False, depending on the conditions to enable MHIntra mode, MHIntra_flag may be signaled. If MHIntra_flag is True, MHIntra mode related information will be further signaled. In one example, when sub-block merge mode is used, and sub_block_flag and MHIntra_flag are False, depending on the conditions to enable triangle prediction unit mode, merge_triangle_flag may be signaled. If merge _triangle_flag is True, triangle prediction unit mode related information will be further signaled.

The modes and orders of different modes in a category described above are non-limiting examples and can be changed or varied. For example, triangle prediction unit mode may be considered or checked before MHIntra mode and used as conditions for signaling the flags corresponding to other modes.

In at least one embodiment, a syntax design according to various examples described herein can be configured as follows:
<IMG>
<IMG>.

In some embodiments, a new syntax element regular_merge _flag can be introduced to explicitly indicate whether the regular merge mode is used or not. If regular_merge _flag is set to True, the regular merge mode is used.

The syntax element regular_merge _flag is coded with context model. In one example, the regular_merge _flag of the current block is coded with a fixed context model. For example, the context model with index <NUM> is always used for coding the regular_merge _flag.

In another example, the regular_merge _flag from the spatial neighboring blocks above and left to the current block are used to derive the context model for coding the regular merge_flag of the current block. In one example, if the regular_merge _flag from the spatial neighboring blocks above and left to the current block are all False, the context model with index <NUM> can be used for coding the regular_merge _flag for the current coding block. In one example, if the regular merge_flag from the spatial neighboring blocks above and left to the current block are all True, the context model with index <NUM> can be used for coding the regular_merge _flag for the current coding block. In some examples, if the regular_merge _flag from the spatial neighboring blocks above and left to the current block are not all False and are not all True, the context model with index <NUM> can be used for coding the regular_merge _flag for the current coding block.

In another example, if the regular_merge _flag from the spatial neighboring blocks above and left to the current block are all False, the context model with index <NUM> can be used for coding the regular_merge _flag for the current coding block; otherwise, the context model with index <NUM> can be used for coding the regular_merge _flag for the current coding block.

<FIG> shows a flow chart outlining a decoding process (1200A) according to an unclaimed embodiment of the disclosure. The process (1200A) can be used to determine a mode for identifying and applying motion information to a block (i.e., a current block) of a picture. In some unclaimed embodiments, one or more operations are performed before or after process (1200A), and some of the operations illustrated in <FIG> may be reordered or omitted.

In various embodiments, the process (1200A) is executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>), and (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), (<NUM>), or (<NUM>), and the like. In some unclaimed embodiments, the process (1200A) is implemented by software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (1200A). The process starts at (S1201) and proceeds to (S1210).

At (S1210), a first syntax element is obtained from a coded video bitstream. The first syntax element is associated with a plurality of modes for identifying and applying motion information to the current block. A first value of the first syntax element can indicate a subset of the plurality of modes for the current block. In some embodiments, the first syntax element corresponds to syntax element "merge_list_idc" described above.

The plurality of modes includes a regular merge mode corresponding to deriving motion information for the current block according to a selected motion information candidate from a list of motion information candidates, a sub-block merge prediction mode corresponding to splitting the current block into plural rectangular sub-blocks and deriving respective motion information for the plural rectangular sub-blocks, and a triangle prediction unit mode corresponding to splitting the current block into two triangular sub-partitions and deriving respective motion information for the triangular sub-partitions. The plurality of modes can further include a merge with motion vector difference (MMVD) mode corresponding to deriving the motion information for the current block according to the selected motion information candidate from the list of motion information candidates and motion vector difference information. The plurality of modes further includes a multi-hypothesis prediction for intra (MHIntra) mode corresponding to generating a final predictor for the current block by combining an intra predictor and an inter predictor.

The subset of the plurality of modes for the current block indicated by the value of the first syntax element can include a single mode or one of a plurality of subsets of modes. In one example, the value of the first syntax element can indicate whether the motion information of the current block is to be derived according to the regular merge mode. In one example, the value of the first syntax element can indicate that the motion information of the current block is to be derived according to the regular merge mode or the merge with motion vector difference mode. In one example, the value of the first syntax element can indicate that the motion information of the current block is to be derived according to the sub-block merge prediction mode, the triangle prediction unit mode, or the MHIntra mode.

The first syntax element is set to one of different bins that represent respective ones of the plurality of modes. The plurality of can be arranged according to a predetermined order consistent with an order of decreasing usage frequency of the modes. In some examples, the order can be arranged consistent, in whole or in part, with any one of the examples illustrated in Tables II-V. For any two modes of the plurality of modes, a code length for coding one of the two modes that is arranged in front of the other one of the two modes in the order is not greater than a code length for coding the other one of the two modes. For example, an order of the regular merge mode, the MMVD mode, the sub-block merge prediction mode, the MHIntra mode, and the triangle prediction unit mode, as illustrated in Table II.

In some embodiments, a most-frequently used mode of the plurality of modes in a picture that includes the current block is arranged first in the order and represented by a bin of the bins that has a least number of bits. In at least one example, the most-frequently used mode is the regular merge mode.

In some embodiments, if the subset of the plurality of modes for the current block corresponds to more than one mode, the applicable mode within the subset of the plurality of modes can be derived or determined according to one or more additional syntax elements or flags provided after the first syntax element. Additional non-limiting examples of determining the applicable mode from the subset of the plurality of modes will be further illustrated with reference to <FIG>.

At (S1270), a predictor of the current block is generated according to the subset of the plurality of modes indicated by the first value of the first syntax element. In some embodiments, the predictor of the current block can be generated according to the applicable mode within the subset of the plurality of modes by at least inter prediction.

At (S1280), reconstructed samples of the current block are generated (e.g., for output) based on at least the predictor from (S1270).

After (S1280), the process proceeds to (S1299) and terminates.

<FIG> shows a flow chart outlining another decoding process (1200B) according to an embodiment of the disclosure. The process (1200B) includes (S1201), (S1210), (S1220), (S1270), (S1280), and (S1299), and can be inserted between (s1220) and (S1270) in process (1200A) in <FIG>. Therefore, (S1201), (S1210), (S1220), (S1270), (S1280), and (S1299) are not shown or further illustrated in <FIG>.

The process (1200B) can be used to determine a mode for identifying and applying motion information to a block (i.e., a current block) of a picture. In some embodiments, one or more operations are performed before or after process (1200B), and some of the operations illustrated in <FIG> may be reordered or omitted.

In various embodiments, the process (1200B) is executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>), and (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), (<NUM>), or (<NUM>), and the like. In some unclaimed embodiments, the process (1200B) is implemented by software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (1200B). The process starts at (S1201) and proceeds to (S1210) and (S1220), and then to (S1230).

At (S1230), a second syntax element that is provided after the first syntax element from the coded video bitstream is obtained. A second value of the second syntax element indicates an index for selecting a motion information candidate from a list of motion information candidates. In some embodiments, the second syntax element corresponds to syntax element "merge _idx" described above.

At (S1240), it is determined whether the subset of the plurality of modes includes the regular merge mode and the MMVD mode. If the subset includes the regular merge mode and the MMVD mode, the process proceeds to (S1245); otherwise, the process proceeds to (S1255).

In some embodiments, whether the subset of the plurality of modes includes the regular merge mode and the MMVD mode can be determined according to the first value of the first syntax element. In some embodiments, when the first value of the first syntax element indicates that the mode to be used to generate the predictor of the current block is a block based merge mode, the subset of the plurality of modes includes at least one of the regular merge mode and the MMVD mode. In some embodiments, when the first value of the first syntax element indicates that the mode to be used to generate the predictor of the current block is a sub-block based merge mode, the subset of the plurality of modes includes at least one of the sub-block merge prediction mode, the triangle prediction unit mode, and the MHIntra mode.

In some embodiments, at (S1240), whether the subset of the plurality of modes includes the regular merge mode and the MMVD mode can be further determined according to yet another syntax element that is provided after the first syntax element from the coded video bitstream. A value of the other syntax element explicitly indicates whether the mode for generating the predictor of the current block is a regular merge mode, when the first value of the first syntax element indicates that the subset of the plurality of modes includes at least the regular merge mode. In some embodiments, the yet another syntax element corresponds to syntax element "regular_merge_flag" described above.

At (S1245), whether to obtain a third syntax element is determined, where a third value of the third syntax element indicates whether to generate the predictor of the current block according to the regular merge mode or the MMVD mode. In some embodiments, when the index indicated by the second value of the second syntax element is smaller than a maximum number of motion information candidates for the MMVD mode, the third syntax element is obtained from the coded video bitstream and the process proceeds to (S <NUM>); otherwise, the third syntax element does not need to be obtained, the applicable mode for the current block can be determined to be the regular merge mode, and the process proceeds to (S <NUM>). In some embodiments, the third syntax element corresponds to syntax element "mmvd_flag" described above.

At (S1250), based on the obtained third syntax element, whether the applicable mode for the current block is the regular merge mode or the MMVD mode can be determined. If it is indicated by the value of the third syntax element that the MMVD mode is used, the process proceeds to (S <NUM>); otherwise, the applicable mode can be determined to be the regular merge mode and the process proceeds to (S1255).

At (S1255), the applicable mode for the current block has been determined as the regular merge mode, or to be determined based on one or more additional flags as one of other possible modes, such as one of the sub-block merge prediction mode, the triangle prediction unit mode, and/or the MHIntra mode. In some embodiments, the one or more additional flags correspond to one or more of "sub_block_flag," "MHIntra_flag," and "merge_triangle_flag" described above.

At (S <NUM>), one or more fourth syntax elements that are provided after the third syntax element from the coded video bitstream can be obtained when the third value of the third syntax element indicates that the mode for generating the predictor of the current block is the MMVD mode. In some embodiments, the one or more fourth syntax elements indicate the motion vector difference information for the MMVD mode. In some embodiments, the one or more fourth syntax elements correspond to one or more of "base_mv_idx," "distance _idx," and "direction _idx" described above. In some embodiments, information carried by "base_mv_idx" can be signaled by "merge _idx," and "base _mv_idx" can be omitted.

After (S1255) or (S1260), the process proceeds to (S1270), followed by (S1280) and to (S1299) and terminates.

<FIG> shows a flow chart outlining an encoding process (<NUM>) according to an embodiment of the disclosure. The process (<NUM>) can be used to encode a block (i.e., a current block) of a picture coded according to one of a plurality of modes for identifying and applying motion information to the current block. In some embodiments, one or more operations are performed before or after process (<NUM>), and some of the operations illustrated in <FIG> may be reordered or omitted.

In various embodiments, the process (<NUM>) is executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>), and (<NUM>), the processing circuitry that performs functions of the video encoder (<NUM>), (<NUM>), or (<NUM>), and the like. In some unclaimed embodiments, the process (<NUM>) is implemented by software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>). The process starts at (S1301) and proceeds to (S1310).

At (S1310), one of a plurality of modes for identifying and applying motion information to a current block is determined as an applicable mode for the current block.

In some embodiments, the plurality of modes includes a regular merge mode corresponding to deriving motion information for the current block according to a selected motion information candidate from a list of motion information candidates, a sub-block merge prediction mode corresponding to splitting the current block into plural rectangular sub-blocks and deriving respective motion information for the plural rectangular sub-blocks, and a triangle prediction unit mode corresponding to splitting the current block into triangular sub-partitions and deriving respective motion information for the triangular sub-partitions. In some embodiments, the plurality of modes further includes an MMVD mode corresponding to deriving the motion information for the current block according to the selected motion information candidate from the list of motion information candidates and motion vector difference information. In some embodiments, the plurality of modes further includes an MHIntra mode corresponding to generating a final predictor for the current block by combining an intra predictor and an inter predictor.

At (S1320), a predictor of the current block is generated according to the determined applicable mode from (S1310). At (S1330), residual samples of the current block are generated according to the predictor. In some embodiments, the residual samples correspond to a difference between original samples of the current block and the predictor.

At (S1340), a syntax element associated with the plurality of modes is generated. A value of the syntax element indicates a subset of the plurality of modes that include the determined mode for the current block. In some embodiments, the first syntax element corresponds to syntax element "merge_list_idc" described above.

In one example, the value of the syntax element can indicate whether the motion information of the current block is to be signaled according to the regular merge mode. In one example, the value of the syntax element can indicate that the motion information of the current block is to be signaled according to the regular merge mode or the MMVD mode. In one example, the value of the syntax element can indicate that the motion information of the current block is to be signaled according to the sub-block merge prediction mode, the triangle prediction unit mode, or the MHIntra mode.

In some embodiments, the syntax element is set to one of different bins that represent respective ones of the plurality of modes. The plurality of can be arranged according to a predetermined order consistent with an order of decreasing usage frequency of the modes. In some examples, the order can be arranged consistent, in whole or in part, with any one of the examples illustrated in Tables II-V. In some embodiments, for any two modes of the plurality of modes, a code length for coding one of the two modes that is arranged in front of the other one of the two modes in the order is not greater than a code length for coding the other one of the two modes. For example, an order of the regular merge mode, the MMVD mode, the sub-block merge prediction mode, the MHIntra mode, and the triangle prediction unit mode, as illustrated in Table II.

In some embodiments, one or more additional flags, such as flags mmvd_flag, sub_block_flag, MHIntra _flag, and merge _triangle_flag, can be conditionally signaled for, together with the syntax element, indicating the determined applicable mode.

At (S1380), a coded video bitstream for output can be generated based on at least the syntax element from (S1340) and the residual samples of the current block from (S1330).

After (S1380), the process proceeds to (S1399) and terminates.

Computer system (<NUM>) can also include an interface to one or more communication networks. Networks can for example be wireless, wireline, optical. Networks can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks to include GSM, <NUM>, <NUM>, <NUM>, LTE and the like, TV wireline or wireless wide area digital networks to include cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial to include CANBus, and so forth. Certain networks commonly require external network interface adapters that attached to certain general purpose data ports or peripheral buses (<NUM>) (such as, for example USB ports of the computer system (<NUM>)); others are commonly integrated into the core of the computer system (<NUM>) by attachment to a system bus as described below (for example Ethernet interface into a PC computer system or cellular network interface into a smartphone computer system). Using any of these networks, computer system (<NUM>) can communicate with other entities. Such communication can be uni-directional, receive only (for example, broadcast TV), uni-directional send-only (for example CANbus to certain CANbus devices), or bi-directional, for example to other computer systems using local or wide area digital networks. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

Claim 1:
A method for video decoding in a decoder, comprising:
obtaining a first syntax element from a coded video bitstream, the first syntax element comprises merge list information for identifying at least one merge mode from among a plurality of merge modes for a current block, and a first value of the first syntax element indicating a subset of the plurality of merge modes for the current block;
generating a predictor of the current block according to the subset of the plurality of modes indicated by the first value of the first syntax element; and
generating reconstructed samples of the current block based on the predictor,
wherein the plurality of merge modes includes:
a regular merge mode corresponding to deriving motion information for the current block according to a selected motion information candidate from a list of motion information candidates;
a sub-block merge prediction mode corresponding to splitting the current block into plural rectangular sub-blocks and deriving respective motion information for the plural rectangular sub-blocks; and
a triangle prediction unit mode corresponding to splitting the current block into triangular sub-partitions and deriving respective motion information for the triangular sub-partitions.