Reference picture list and collocated picture signaling in video coding

A video decoder may be configured to receive, in response to receiving a first syntax element indicating that reference picture list information is included in a picture header syntax structure, a second syntax element in the picture header syntax structure indicating whether a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list or a second reference picture list; receive a slice of the video data that refers to the picture header syntax structure; and in response to the slice being a P slice, set a value for a third syntax element associated with the slice to a first value for the third syntax element, with the first value for the third syntax element indicating that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list.

TECHNICAL FIELD

BACKGROUND

SUMMARY

The techniques of this disclosure relate to inter prediction and, more particularly, to the signaling of reference pictures used in inter prediction. As discussed in more detail below, the reference picture list signaling used in VVC Draft 8 may have some inefficiencies and other potential problems. For instance, the signaling techniques of VVC Draft 8 allow for PH-level signaling indicating that a collocated picture for temporal motion vector prediction is to be derived from reference picture list 1, even though the PH may have associated P slices that only use reference picture list 0. According to the techniques of this disclosure, a video decoder may be configured to in response to the slice being a P slice, set a value for a syntax element associated with the slice to a value for the syntax element that indicates that the collocated picture used for temporal motion vector prediction is to be derived from the reference picture list 0. As described in more detail below, the video decoder may be configured to infer, without actually receiving an instance of the syntax element, the value that indicates that the collocated picture used for temporal motion vector prediction is to be derived from the reference picture list 0, thus effectively overriding the PH-level syntax for P slices. Thus, the techniques of this disclosure may advantageously enable a video decoder to parse and decode such a bitstream without unduly restricting the manner in which a video encoder may encode the bitstream.

According to one example of this disclosure, a method of decoding video data includes receiving a first syntax element; in response to the first syntax element indicating that reference picture list information is included in a picture header syntax structure, receiving a second syntax element in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; receiving a slice of the video data that refers to the picture header syntax structure; and in response to the slice being a P slice, setting a value for a third syntax element associated with the slice to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list.

According to another example of this disclosure, a device for decoding video data includes a memory configured to store video data; and one or more processors implemented in circuitry and configured to: receive a first syntax element; in response to the first syntax element indicating that reference picture list information is included in a picture header syntax structure, receive a second syntax element in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; receive a slice of the video data that refers to the picture header syntax structure; in response to the slice being a P slice, set a value for a third syntax element associated with the slice to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list.

According to another example of this disclosure, a computer-readable storage medium stores instructions that when executed by one or more processors cause the one or more processors to: receive a first syntax element; in response to the first syntax element indicating that reference picture list information is included in a picture header syntax structure, receive a second syntax element in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; receive a slice of the video data that refers to the picture header syntax structure; in response to the slice being a P slice, set a value for a third syntax element associated with the slice to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list.

According to another example of this disclosure, an apparatus for decoding video data includes means for receiving a first syntax element; means for receiving a second syntax element in a picture header syntax structure in response to the first syntax element indicating that reference picture list information is included in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; means for receiving a slice of the video data that refers to the picture header syntax structure; and means for setting a value for a third syntax element associated with the slice to a first value for the third syntax element in response to the slice being a P slice, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list.

According to another example of this disclosure, a method for encoding video data includes, in response to determining that reference picture list information is included in a picture header syntax structure, generating a first syntax element indicating that the reference picture list information is included in the picture header syntax structure; generating a second syntax element for inclusion in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; determining that a slice of the video data that refers to the picture header syntax structure is a P slice; in response to the slice being a P slice, determining that a value for a third syntax element associated with the slice is equal to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list; and outputting a bitstream of encoded video data that includes the first syntax element and the picture header syntax structure.

According to another example of this disclosure, a device for encoding video data includes a memory configured to store video data; and one or more processors implemented in circuitry and configured to: in response to determining that reference picture list information is included in a picture header syntax structure, generate a first syntax element indicating that the reference picture list information is included in the picture header syntax structure; generate a second syntax element for inclusion in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; determine that a slice of the video data that refers to the picture header syntax structure is a P slice; in response to the slice being a P slice, determine that a value for a third syntax element associated with the slice is equal to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list; and output a bitstream of encoded video data that includes the first syntax element and the picture header syntax structure.

According to another example of this disclosure, a computer-readable storage medium stores instructions that when executed by one or more processors cause the one or more processors to: in response to determining that reference picture list information is included in a picture header syntax structure, generate a first syntax element indicating that the reference picture list information is included in the picture header syntax structure; generate a second syntax element for inclusion in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; determine that a slice of the video data that refers to the picture header syntax structure is a P slice; in response to the slice being a P slice, determine that a value for a third syntax element associated with the slice is equal to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list; and output a bitstream of encoded video data that includes the first syntax element and the picture header syntax structure.

According to another example of this disclosure, an apparatus for encoding video data includes, means for generating a first syntax element indicating that reference picture list information is included in the picture header syntax structure in response to determining that reference picture list information is included in the picture header syntax structure; means for generating a second syntax element for inclusion in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; means for determining that a slice of the video data that refers to the picture header syntax structure is a P slice; means for determining that a value for a third syntax element associated with the slice is equal to a first value for the third syntax element in response to the slice being a P slice, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list; and means for outputting a bitstream of encoded video data that includes the first syntax element and the picture header syntax structure.

DETAILED DESCRIPTION

Video coding (e.g., video encoding and/or video decoding) typically involves predicting a block of video data from either an already coded block of video data in the same picture (e.g., intra prediction) or an already coded block of video data in a different picture (e.g., inter prediction). In some instances, the video encoder also calculates residual data by comparing the prediction block to the original block. Thus, the residual data represents a difference between the prediction block and the original block. To reduce the number of bits needed to signal the residual data, the video encoder transforms and quantizes the residual data and signals the transformed and quantized residual data in the encoded bitstream. The compression achieved by the transform and quantization processes may be lossy, meaning that transform and quantization processes may introduce distortion into the decoded video data.

A video decoder decodes and adds the residual data to the prediction block to produce a reconstructed video block that matches the original video block more closely than the prediction block alone. Due to the loss introduced by the transforming and quantizing of the residual data, the first reconstructed block may have distortion or artifacts. One common type of artifact or distortion is referred to as blockiness, where the boundaries of the blocks used to code the video data are visible.

To further improve the quality of decoded video, a video decoder can perform one or more filtering operations on the reconstructed video blocks. Examples of these filtering operations include deblocking filtering, sample adaptive offset (SAO) filtering, and adaptive loop filtering (ALF). Parameters for these filtering operations may either be determined by a video encoder and explicitly signaled in the encoded video bitstream or may be implicitly determined by a video decoder without needing the parameters to be explicitly signaled in the encoded video bitstream.

The techniques of this disclosure relate to inter prediction and, more particularly, to the signaling of reference pictures used in inter prediction. Inter prediction in ITU-T H.266, also referred to as Versatile Video Coding (VVC), utilizes reference picture lists, which generally refer to the list of reference pictures that is used for inter prediction of a P or B slice. Inter prediction in VVC utilizes two reference picture lists, with reference picture list 0 referring to the reference picture list used for inter prediction of a P or the first reference picture list used for inter prediction of a B slice, and reference picture list 1 referring to the second reference picture list used for inter prediction of a B slice. For the decoding process of a predictive (P) slice, only reference picture list 0 is used for inter prediction. For the decoding process of a bi-predictive (B) slice, both reference picture list 0 and reference picture list 1 are used for inter prediction. For decoding the slice data of an intra (I) slice, no reference picture list is used for for inter prediction.

A P slice is a slice that is decoded using intra prediction or using inter prediction with at most one motion vector and reference index to predict the sample values of each block. A B slice is a slice that is decoded using intra prediction or using inter prediction with at most two motion vectors and reference indices to predict the sample values of each block. An I slice is a slice that is decoded using intra prediction only.

Information for generating and maintaining reference picture lists is signalled in various syntax structures, including sequence parameter set (SPS) syntax structures, picture parameter set (PPS) syntax structures, picture header (PH) syntax structures, and slice header (SH) syntax structures. In VVC, an SPS refers to a syntax structure containing syntax elements that apply to zero or more entire coded layer video sequences (CLVSs) as determined by the content of a syntax element found in the PPS referred to by a syntax element found in each PH, and a PPS refers to a syntax structure containing syntax elements that apply to zero or more entire coded pictures as determined by a syntax element found in each PH. In VVC, a PH refers to a syntax structure containing syntax elements that apply to all slices of a coded picture, and a SH refers to a part of a coded slice containing the data elements pertaining to all tiles or coding tree unit (CTU) rows within a tile represented in the slice.

As discussed in more detail below, the reference picture list signaling used in VVC Draft 8 may have some inefficiencies and other potential problems. For instance, the signaling techniques of VVC Draft 8 allow for PH-level signaling indicating that a collocated picture for temporal motion vector prediction is to be derived from reference picture list 1, even though the PH may have associated P slices that only use reference picture list 0. According to the techniques of this disclosure, a video decoder may be configured to in response to the slice being a P slice, set a value for a syntax element associated with the slice to a value for the syntax element that indicates that the collocated picture used for temporal motion vector prediction is to be derived from the reference picture list 0. As described in more detail below, the video decoder may be configured to infer, without actually receiving an instance of the syntax element, the value that indicates that the collocated picture used for temporal motion vector prediction is to be derived from the reference picture list 0, thus effectively overriding the PH-level syntax for P slices. Thus, the techniques of this disclosure may advantageously enable a video decoder to parse and decode such a bitstream without unduly restricting the manner in which a video encoder may encode the bitstream.

In some examples, source device102may output encoded video data to file server114or another intermediate storage device that may store the encoded video data generated by source device102. Destination device116may access stored video data from file server114via streaming or download. File server114may be any type of server device capable of storing encoded video data and transmitting that encoded video data to the destination device116. File server114may represent a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device116may access encoded video data from file server114through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both that is suitable for accessing encoded video data stored on file server114. File server114and input interface122may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

Although not shown inFIG. 1, in some examples, video encoder200and video decoder300may each be integrated with an audio encoder and/or audio decoder, and may include appropriate MUX-DEMUX units, or other hardware and/or software, to handle multiplexed streams including both audio and video in a common data stream. If applicable, MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

Video encoder200and video decoder300may operate according to a video coding standard, such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC) or extensions thereto, such as the multi-view and/or scalable video coding extensions. Alternatively, video encoder200and video decoder300may operate according to other proprietary or industry standards, such as VVC. A recent draft of the VVC standard is described in Bross, et al. “Versatile Video Coding (Draft 8),” Joint Video Experts Team (WET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 17thMeeting: Brussels, BE, 7-17 Jan. 2020, WET-Q2001-v13 (hereinafter “VVC Draft 8”). The techniques of this disclosure, however, are not limited to any particular coding standard.

In some examples, a tile may be partitioned into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a tile may not be referred to as a tile.

The bricks in a picture may also be arranged in a slice. A slice may be an integer number of bricks of a picture that may be exclusively contained in a single network abstraction layer (NAL) unit. In some examples, a slice includes either a number of complete tiles or only a consecutive sequence of complete bricks of one tile.

Video encoder200may further generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, to video decoder300, e.g., in a PH, a block header, a SH, or other syntax data, such as a sequence parameter set (SPS), picture parameter set (PPS), or video parameter set (VPS). Video decoder300may likewise decode such syntax data to determine how to decode corresponding video data.

FIGS. 2A and 2Bare conceptual diagrams illustrating an example quadtree binary tree (QTBT) structure130, and a corresponding coding tree unit (CTU)132. The solid lines represent quadtree splitting, and dotted lines indicate binary tree splitting. In each split (e.g., non-leaf) node of the binary tree, one flag is signaled to indicate which splitting type (e.g., horizontal or vertical) is used, where 0 indicates horizontal splitting and 1 indicates vertical splitting in this example. For the quadtree splitting, there is no need to indicate the splitting type, because quadtree nodes split a block horizontally and vertically into 4 sub-blocks with equal size. Accordingly, video encoder200may encode, and video decoder300may decode, syntax elements (such as splitting information) for a region tree level of QTBT structure130(i.e., the solid lines) and syntax elements (such as splitting information) for a prediction tree level of QTBT structure130(i.e., the dashed lines). Video encoder200may encode, and video decoder300may decode, video data, such as prediction and transform data, for CUs represented by terminal leaf nodes of QTBT structure130.

In general, CTU132ofFIG. 2Bmay be associated with parameters defining sizes of blocks corresponding to nodes of QTBT structure130at the first and second levels. These parameters may include a CTU size (representing a size of CTU132in samples), a minimum quadtree size (MinQTSize, representing a minimum allowed quadtree leaf node size), a maximum binary tree size (MaxBTSize, representing a maximum allowed binary tree root node size), a maximum binary tree depth (MaxBTDepth, representing a maximum allowed binary tree depth), and a minimum binary tree size (MinBTSize, representing the minimum allowed binary tree leaf node size).

The root node of a QTBT structure corresponding to a CTU may have four child nodes at the first level of the QTBT structure, each of which may be partitioned according to quadtree partitioning. That is, nodes of the first level are either leaf nodes (having no child nodes) or have four child nodes. The example of QTBT structure130represents such nodes as including the parent node and child nodes having solid lines for branches. If nodes of the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), then the nodes can be further partitioned by respective binary trees. The binary tree splitting of one node can be iterated until the nodes resulting from the split reach the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). The example of QTBT structure130represents such nodes as having dashed lines for branches. The binary tree leaf node is referred to as a coding unit (CU), which is used for prediction (e.g., intra-picture or inter-picture prediction) and transform, without any further partitioning. As discussed above, CUs may also be referred to as “video blocks” or “blocks.”

In one example of the QTBT partitioning structure, the CTU size is set as 128×128 (luma samples and two corresponding 64×64 chroma samples), the MinQTSize is set as 16×16, the MaxBTSize is set as 64×64, the MinBTSize (for both width and height) is set as 4, and the MaxBTDepth is set as 4. The quadtree partitioning is applied to the CTU first to generate quad-tree leaf nodes. The quadtree leaf nodes may have a size from 16×16 (i.e., the MinQTSize) to 128×128 (i.e., the CTU size). If the quadtree leaf node is 128×128, the leaf quadtree node will not be further split by the binary tree, because the size exceeds the MaxBTSize (i.e., 64×64, in this example). Otherwise, the quadtree leaf node will be further partitioned by the binary tree. Therefore, the quadtree leaf node is also the root node for the binary tree and has the binary tree depth as 0. When the binary tree depth reaches MaxBTDepth (4, in this example), no further splitting is permitted. The binary tree node having a width equal to MinBTSize (4, in this example) implies that no further vertical splitting is permitted. Similarly, a binary tree node having a height equal to MinBTSize implies that no further horizontal splitting is permitted for that binary tree node. As noted above, leaf nodes of the binary tree are referred to as CUs and are further processed according to prediction and transform without further partitioning.

In VVC draft 8, a reference picture list (RPL) can be signaled in a PH or an SH. The enabling of temporal motion vector prediction (TMVP), however, is signaled only in a PH. A number of active reference pictures (the picture used for inter prediction) is signaled only in a SH.

An RPL is a list of reference pictures that is used for inter prediction of a P or B slice. Two reference picture lists, reference picture list 0 and reference picture list 1, are generated for each slice of a non-IDR picture. The set of unique pictures referred to by all entries in the two reference picture lists associated with a picture includes all reference pictures that may be used for inter prediction of the associated picture or any picture following the associated picture in decoding order. For the decoding process of a P slice, only reference picture list 0 is used for inter prediction. For the decoding process of a B slice, both reference picture list 0 and reference picture list 1 are used for inter prediction. For decoding the slice data of an I slice, no reference picture list is used for inter prediction. Reference picture list 0 is the reference picture list used for inter prediction of a P or the first reference picture list used for inter prediction of a B slice. Reference picture list 1 is the second reference picture list used for inter prediction of a B slice.

A PH is a syntax structure containing syntax elements that apply to all slices of a coded picture. An SH is a part of a coded slice containing the data elements pertaining to all tiles or CTU rows within a tile represented in the slice. A slice is an integer number of complete tiles or an integer number of consecutive complete CTU rows within a tile of a picture that are exclusively contained in a single NAL unit.

In VVC draft 8, the following syntax table is present:

rpl_info_in_ph_flag equal to 1 specifies that reference picture list information is present in the PH syntax structure and not present in slice headers referring to the PPS that do not contain a PH syntax structure. rpl_info_in_ph_flag equal to 0 specifies that reference picture list information is not present in the PH syntax structure and may be present in slice headers referring to the PPS that do not contain a PH syntax structure.

Picture Header

Slice Header

slice_collocated_ref_idx specifies the reference index of the collocated picture used for temporal motion vector prediction.

When slice_type is equal to P or when slice_type is equal to B and collocated_from_l0_flag is equal to 1, slice_collocated_ref_idx refers to an entry in reference picture list 0, and the value of slice_collocated_ref_idx shall be in the range of 0 to NumRefldxActive[0]−1, inclusive.

When slice_type is equal to B and slice_collocated_from_l0_flag is equal to 0, slice_collocated_ref_idx refers to an entry in reference picture list 1, and the value of slice_collocated_ref_idx shall be in the range of 0 to NumRefldxActive[1]−1, inclusive.

When slice_collocated_ref_idx is not present, the following applies:If rpl_info_in_ph_flag is equal to 1, the value of slice_collocated_ref_idx is inferred to be equal to ph_collocated_ref_idx.Otherwise (rpl_info_in_ph_flag is equal to 0), the value of slice_collocated_ref_idx is inferred to be equal to 0.

It is a requirement of bitstream conformance that the picture referred to by slice_collocated_ref_idx shall be the same for all slices of a coded picture.

It is a requirement of bitstream conformance that the values of pic_width_in_luma_samples and pic_height_in_luma_samples of the reference picture referred to by slice_collocated_ref_idx shall be equal to the values of pic_width_in_luma_samples and pic_height_in_luma_samples, respectively, of the current picture, and RprConstraintsActive[slice_collocated_from_l0_flag? 0:1][slice_collocated_ref_idx] shall be equal to 0.

The above implementation of PH and SH semantics may have some potential problems. As mentioned earlier, a TMVP enable flag is signaled only in a PH regardless of whether an RPL is signaled in the PH or SH. Also, VVC draft 8 imposes a constraint where bitstream conformance requires that the picture referred to by slice_collocated_ref_idx shall be the same for all slices of a coded picture.

In the case when an RPL is signaled in a PH and TMVP is enabled, then the number of active reference pictures in a slice of the picture may be smaller than the number of reference pictures included in the RPL in the PH, and the collocated picture may not be included into the active reference pictures.

In another example of a potential problem, for some coding scenarios, TMVP may be enabled in a PH, but RPLs may be signaled per slice. The number of active reference pictures may vary from slice to slice of the same picture, and a collocated picture may not be included in the RPLs for some slices. This disclosure describes techniques for disabling TMVP for the slices that violate collocated picture constraints.

There may be an inefficiency in RPL signaling in a SH. Current implementations (e.g., VVC draft 8) permit an RPL to be signaled for I-slices even when an RPL is not present for IDR pictures, such as when sps_idr_rpl_present_flag is equal to 0. There may also be inefficiency in weighted prediction signaling in a PH, which may be signaled when inter related syntax is not present, such as when ph_inter_slice_allowed_flag is equal to 0 and an RPL is not present for IDR pictures, such as when sps_idr_rpl_present_flag is equal to 0. This disclosure describes techniques that potentially address the aforementioned problems. The various techniques described below may be used either independently or in any combination.

In some examples, to address the redundant signaling of RPLs in PHs, this disclosure describes techniques for signaling an RPL when inter prediction is allowed, such as when ph_inter_slice_allowed_flag is equal to 1, or when an RPL is present for an IDR picture, such as when sps_idr_rpl_present_flag is equal to 1.

In one example, this technique may be implemented with the following syntax changes relative to VVC draft 8. In the examples below and throughout the rest of this disclosure, text in between <ADD> and </ADD> represents text being added to VVC draft 8, and text in between <DEL> and </DEL> represents text being removed, i.e., deleted, from VVC draft 8.

Video decoder300may be configured to infer that when ph_intra_slice_allowed_flag is not present that a value for ph_intra_slice_allowed_flag is inferred to be equal to 1, meaning that when inter prediction is not allowed then intra prediction shall be allowed. Thus, the above condition may be simplified as follows:

Also, video encoder200may be configured to signal a weighted prediction table when inter prediction is allowed, for example when ph_inter_slice_allowed_flag is equal to 1, or when RPL is present for IDR picture, such as when sps_idr_rpl_present_flag is equal to 1.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

Video decoder300may be configured to infer that when ph_intra_slice_allowed_flag is not present that a value for ph_intra_slice_allowed_flag is to be equal to 1, meaning that when inter prediction is not allowed then intra prediction shall be allowed. Thus, the above condition may be simplified as follows:

The two techniques above use similar conditions and can be combined to signal ref_pic_lists( ) and pred_weight_table( ), which may be advantageous as the result of implementing such conditions requires fewer conditional checks.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In another example, in view of the inference described above for ph_intra_slice_allowed_flag, these techniques may be implemented as shown below relative to VVC draft 8:

According to techniques of this disclosure, to avoid any inconsistency between the active entries across slices of the same picture, video encoder200may be configured to signal the number of active reference picture entries in a PH. In one example, video encoder200may signal the number of active reference entries under the condition of RPL presence in the PH, for example rpl_info_in_ph_flag is equal to 1, and inter prediction is allowed, for example when ph_inter_slice_allowed_flag being equal to 1, e.g. there may be P- or B-slices present in a picture, or when an RPL is present for an IDR picture, for example when sps_idr_rpl_present_flag is equal to 1.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

Video decoder300may be configured to infer that when ph_intra_slice_allowed_flag is not present that a value for ph_intra_slice_allowed_flag is to be equal to 1, meaning that when inter prediction is not allowed that intra prediction shall be allowed. Thus, the above condition can be simplified as follows:

Additionally, video encoder200may be configured to conditionally signal, and video decoder300configured to conditionally parse, an override flag num_ref_idx_active_override_flag in a SH based on an override flag ph_num_ref_idx_active_override_flag value in a PH. In one example, video encoder200may be configured to only signal a SH override flag num_ref_idx_active_override_flag when a PH override flag ph_num_ref_idx_active_override_flag is equal to 0.

Video decoder300may be configured to derive the number of active reference pictures NumRefldxActive[i] in a slice from the number of active reference pictures signaled in PH if an override flag in a PH ph_num_ref_idx_active_override_flag is equal to 1, which is applied for RefPicList0 and RefPicList1, respectively.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

The variable NumRefldxActive[i] is derived as follows:

In the weighted prediction table signaling, there are two syntax elements num_l0_weights and num_l1_weights, which indicate the number of weights signaled in RefPicList0 and RefPicList1. When an override flag is signaled in a PH, the number of entries in an RPL is known. Therefore, the value of num_l0_weights and/or num_l1_weights may not be needed. Video encoder200may be configured to conditionally signal those syntax elements based on the presence of a PH override flag ph_num_ref_idx_active_override_flag, and video decoder300may be configured to infer that the number of weights is to be equal to ph_num_ref_idx_active_minus1 for RefPicList0 and RefPicList1, respectively.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

if( wp_info_in_ph_flag <ADD>&&!ph_num_ref_idx_active_override_flag</ADD> )num_l0_weightsue(v)if( wp_info_in_ph_flag <ADD> &&!ph_num_ref_idx_active_override_flag</ADD> )num_l1_weightsue(v)
num_l0_weights specifies the number of weights signalled for entries in reference picture list. The value of num_l0_weights shall be in the range of 0 to num_ref_entries[0][PicRplsIdx[0]], inclusive.

<ADD>The variable NumWeightsL0 is derived as follows:

<ADD>The variable NumWeightsL0 is derived as follows:

According to techniques of this disclosure, to address the problem where a collocated picture may not be present in the RPL when TMVP is enabled, video encoder200and video decoder300may be configured to operate according to a constraint that a collocated picture shall be present among active reference pictures.

In one example, the ph_collocated_ref_idx constraint is modified in a way that the range of the ph_collocated_ref_idx is from 0 to the smallest number of active reference pictures of any slice belonging to the same picture. In this case, it may not be possible to signal a collocated reference index in a PH to indicate a collocated picture which may not be present in some slices of the same picture.

In one example, this constraint may be implemented by modifying the semantic constraints of ph_collocated_ref_idx as follows:

When ph_collocated_from_l0_flag is equal to 1, ph_collocated_ref_idx refers to an entry in reference picture list 0, and the value of ph_collocated_ref_idx shall be in the range of 0<ADD>to the smallest value of NumRefldxActive[0]−1 of all slices in a picture, inclusive</ADD>.
When ph_collocated_from_l0_flag is equal to 0, ph_collocated_ref_idx refers to an entry in reference picture list 1, and the value of ph_collocated_ref_idx shall be in the range of 0<ADD>to the smallest value of NumRefldxActive[1]−1 of all slices in a picture, inclusive</ADD>.

In some examples, the semantics of ph_collocated_ref_idx may be kept unchanged as those semantics may be used during the parsing of ph_collocated_ref_idx, but the bitstream conformance constraints may be added to reflect that ph_collocated_ref_idx shall not exceed the value of num_ref_idx_active_minus1[0][PicRplsIdx[0]]−1 or num_ref_idx_active_minus1[1][PicRplsIdx[1]]−1 of any slice in the picture.

In one example, the bitstream conformance constraint can be expressed as follows:

It is a requirement of bitstream conformance that the following conditions are true:

When ph_collocated_from_l0_flag is equal to 1, ph_collocated_ref_idx refers to an entry in reference picture list 0, and the value of ph_collocated_ref_idx shall be in the range of 0 to <ADD>the smallest value of NumRefldxActive[0]−1 of all slices in a picture</ADD>, inclusive.When ph_collocated_from_l0_flag is equal to 0, ph_collocated_ref_idx refers to an entry in reference picture list 1, and the value of ph_collocated_ref_idx shall be in the range of 0 to <ADD>the smallest value of NumRefldxActive[1]−1 of all slices in a picture</ADD>, inclusive.

Thus, in some examples, video encoder200and video decoder300may operate according to the constraint for NumRefldxActive of a slice included into the picture, that if TMVP is enabled the number of active reference pictures NumRefldxActive shall be in a range of ph_collocated_ref_idx to num_ref_idx_active_minus1 for a reference picture list.

In one example, the constraint may be expressed as follows:

In yet another example, a constraint may be added to require a collocated picture be an active reference picture, i.e., the collocated picture must be present in the RPL.

In one example, the constraint may be expressed as follows:

<ADD>A collocated picture of a slice shall be an active entry. </ADD>

Optionally, a check whether TMVP is enabled may be added to the constraint, in one example as follows:

<ADD>A collocated picture of a slice if TMVP is enabled shall be an active entry. </ADD>

In VVC draft 8, a reference picture list from which collocated picture is derived can be signaled in a PH using ph_collocated_from_l0_flag. There may be a problem when this flag ph_collocated_from_l0_flag is equal to 0, i.e. collocated picture is derived from RefPicList1 and there are P and B slices present in the picture. In such a case, a collocated picture cannot be derived for a P slice because there is no RefPicList1 available for P slices.

To address this problem, video encoder200and video decoder300may be configured to operate according to a constraint that ph_collocated_from_l0_flag shall not be equal to 1 when there is a P slice in a picture. In one example, the constraint can be expressed as follows:

ph_collocated_from_l0_flag equal to 1 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0. ph_collocated_from_l0_flag equal to 0 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 1.
<ADD>When ph_temporal_mvp_enabled_flag is equal to 1, the value of ph_collocated_from_l0_flag shall be equal to 1 if there is at least one P slice in a picture. </ADD>

Thus, video encoder200and video decoder300may be configured to operate a according to a constraints that when a slice type is P and TMVP is enabled, then ph_collocated_from_l0_flag shall be equal to 1. In one example, the constraint can be expressed as follows:

slice_type specifies the coding type of the slice according to Table 9.

In some examples, when slice_collocated_from_l0_flag is not signaled due to the ph_collocated_from_l0_flag signaling in a PH, video decoder300may be configured to infer that the slice_collocated_from_l0_flag is equal to 1 for P slices. In one example, the constraint can be expressed as follows:

slice_collocated_from_l0_flag equal to 1 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 0. slice_collocated_from_l0_flag equal to 0 specifies that the collocated picture used for temporal motion vector prediction is derived from reference picture list 1.

When slice_type is equal to B or P, ph_temporal_mvp_enabled_flag is equal to 1, and slice_collocated_from_l0_flag is not present, the following applies:

<ADD>If slice_type is equal to P, the value of slice_collocated_from_l0_flag is inferred to be equal to 1.Otherwise, if rpl_info_in_ph_flag is equal to 1, slice_collocated_from_l0_flag is inferred to be equal to ph_collocated_from_l0_flag.

That is, for a P slice, video decoder300may be configured to set slice_collocated_from_l0_flag equal to 1, meaning prediction from l0, regardless of whether the value of ph_collocated_from_l0_flag is 0 or 1.

To address the problem of not having collocated picture in the RPL, the techniques of this disclosure include disabling the TMPV usage if a collocated picture is not present in the RPL but TMVP is enabled. In one example, video encoder200may be configured to signal a TMVP enable flag in a SH in addition to the TMVP enable flag in a PH. This allows the disabling of TMVP on a slice by slice basis, in one example, for the slices that violate collocated picture constraints if TMVP is enabled in a PH. In another example, video encoder200and video decoder300may be configured to not apply TMVP on a block basis if collocated picture is not present in the RPL even TMVP is enabled.

Video encoder200may be configured to conditionally signal a TMVP enable flag in a slice based on the TMVP enable flag in the PH. In one example, video encoder200may signal a TMVP enable flag in a slice if the TMVP enable flag in the PH is equal to 1. When the TMVP enable flag is not present SH, video decoder300may be configured to infer the value of the TMVP to be equal to the TMVP flag signaled in the PH.

In one example, the above techniques may be implemented as follows:

if( ph_temporal_mvp_enabled_flag )<ADD>slice_temporal_mvp_enabled_flagu(1)</ADD>
<ADD>slice_temporal_mvp_enabled_flag equal to 1 specifies that temporal motion vector predictors can be used for inter prediction. slice_temporal_mvp_enabled_flag is equal to 0 specifies that temporal motion vector predictors is disabled in a slice. When slice_temporal_mvp_enabled_flag is not present, it is inferred to be equal to ph_temporal_mvp_enabled_flag. </ADD>

In another example, the TMVP enable flag may be moved from PH to SH. If TMVP is disabled on a slice level, then video encoder200may be configured to not signal TMVP related information, such a slice_collocated_from_l0_flag and slice_collocated_ref_idx.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In another example, video encoder200may be configured to always signal a TMVP enable flag with an RPL, by for example always signaling a TMVP enable flag in a SH if an RPL is signaled in SH, and signal TMVP enable flag in a PH if an RPL is signaled in PH.

Additionally, video encoder200may be configured to signal the TMVP enable flag in a SH or a PH in a mutually exclusive manner, such that the TMVP enable flag cannot be signaled in both PH and SH.

When a collocated picture is indicated in a PH, for example by a ph_collocated_ref_idx, it is possible to indicate a collocated picture having a picture size different from the current picture size. To address this problem, this disclosure describes techniques for configuring video encoder200and video decoder300to operate according to a constraint that a collocated picture indicated in a PH shall have the same picture size as the current picture and no scaling of the collocated picture is applied or no different values of the scaling windows, which are used to derive the scaling ratio, of the reference collocated picture and the current picture are used.

In one example, the above techniques may be implemented as follows:

<ADD>It is a requirement of bitstream conformance that the values of pic_width_in_luma_samples and pic_height_in_luma_samples of the reference picture referred to by ph_collocated_ref_idx shall be equal to the values of pic_width_in_luma_samples and pic_height_in_luma_samples, respectively, of the current picture, and RprConstraintsActive[ph_collocated_from_l0_flag? 0:1][ph_collocated_ref_idx] shall be equal to 0. </ADD>

In VVC draft 8, an RPL can be signaled in an SPS, PH or SH. When signaled in a PH or SH, the RPL can be derived from the SPS. In the latter case, a flag rpl_sps_flag[i] is signaled to indicate that the RPL is derived from the SPS, and an RPL index is signaled rpl_idx[i] to indicate which RPL of SPS is to be used.

There can be two lists, RefPicList0 and RefPicList1, which utilize such RPL derivation. In the case of RefPicList1, an additional syntax element flag rpl1_idx_present_flag is signaled to indicate whether the RPL derivation from the SPS happens. If the RPL is not derived from SPS, then the RPL is explicitly signaled, which potentially introduces inconsistency into the design by having RefPicList0 and RefPicList1 signaling being treated in different manners.

In one example, video encoder200and video decoder300may code an rpl0_idx_present_flag, which indicates the presence of rpl_sps_flag[i] and rpl_idx[i] for RefPicList0, i.e. to provide the same signaling functionality as for RefPicList1.

In another example, rpl1_idx_present_flag may be replaced with another flag rpl1_present_flag, which indicates whether any RefPicList1 related syntax elements are present. In such case, this flag may be additionally used as B-slice presence in a picture.

In one example, such a flag may be signaled in PPS and implemented as follows:

<ADD>rpl1_present_flag equal to 0 specifies that RefPicList1 realted syntax elements are not present in the PH syntax structures or the slice headers for pictures referring to the PPS. rpl1_present_flag equal to 1 specifies that RefPicList1 realted syntax elements may be present in the PH syntax structures or the slice headers for pictures referring to the PPS. </ADD>
Then, video encoder200may be configured to conditionally signal, and video decoder300configured to conditionally parse, the RefPicList1 syntax elements based on this flag.

In one example, video encoder200may be configured to conditionally signal a collocated picture flag from RefPicList0 in a PH based on this flag, and if not present, then video decoder300may be configured to infer that the collocated picture flag indicates that a collocated picture is from RefPicList0. An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In another example, video encoder200may be configured to conditionally signal a zero MVD flag for RefPicList1 in a PH based on ph_collocated_from_l0_flag. An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In another example, video encoder200and video decoder300may be configured to operate according to a constraint where slice type is constrained that when rpl1_present_flag is equal to 0, slice may have only I-slice or P-slice types. For example, it may be expressed as follows:

<ADD>When rpl1_present_flag is equal to 0 the value of slice_type shall be equal to 1 or 2. </ADD>

In another example, video encoder200may be configured to conditionally signal, and video decoder300conditionally parse, a weighted prediction table for RefPicList1 based on rpl1_present_flag.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

The variable NumWeightsL0 is derived as follows:

In another example, video encoder200may be configured to conditionally signal RPL syntax elements for RefPicLIst1 based on rpl1_present_flag. An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In this case the condition checks in ref_pic_list signaling may be simplified.

In VVC draft 8, an RPL can be signaled for an I-slice of a non-IDR picture. However, such RPL is not used in I-slice decoding, and there is no way to avoid this redundant signaling. This disclosure describes techniques for configuring video encoder200to condition RPL signaling on a slice not being an I-slice type. This disclosure also describes techniques for configuring video encoder200signal an RPL for IDR pictures only when indicated that an RPL is present for such pictures, for example when sps_idr_rpl_present_flag is equal to 1.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In VVC draft 8, a reference picture override flag num_ref_idx_active_override_flag is signaled in the SH only when RPL is signaled in the PH. This potentially creates a problem when an RPL is signaled in the SH and some pictures should be signaled in RPL to be available for future reference, but the number of active reference picture cannot be changed because the override flag is not signaled in the SH. The only way to signal such pictures is to make the whole signaled RPL to be active, which requires extra overhead in reference index signaling per block, due to some picture included in the RPL not being used. Another problem is that num_ref_idx_active_override_flag is signaled for I-slices even when not needed.

According to techniques of this disclosure, video encoder200may be configured to signal an override flag num_ref_idx_active_override_flag in an SH even when an RPL is signaled in the SH. Additionally or alternatively, video encoder200may be configured to not signal (e.g., refrain from signaling) an override flag for I-slices of an IDR picture even when other signaling indicates that an RPL is present for such pictures. For example, when sps_idr_rpl_present_flag is equal to 1, as an RPL is not needed for I-slice decoding, there is no reference index signaling in I-slices, so there is not extra overhead.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In this example implementation, if the signaling indicates that RPL information is present for I-slices, when RPL information is signaled in a PH (e.g., when rpl_info_in_ph_flag is equal to 1) or RPL is present for IDR picture (e.g., sps_idr_rpl_present_flag is equal to 1), and the number of RPL entries is more than 1 in any reference picture list, the syntax element num_ref_idx_active_override_flag is not signaled. Or, in other words, if the signaling indicates that RPL is present in PH (rpl_info_in_ph_flag is equal to 1) or RPL is present for IDR picture (sps_idr_rpl_present_flag is equal to 1), and the number of RPL entries is more than 1 in any reference picture list, the num_ref_idx_active_override_flag is only signaled for P- or B-slice types.

In VVC draft 8, there is a flag gdr_or_irap_pic_flag in PH to indicate whether a picture is IRAP or GDR, which may be used to identify the starting point for decoding. However, the signaling of this flag is not constrained on the indication that there can be mixed NAL types in a picture. In such cases, it is not possible to have IRAP or GDR pictures.

According to techniques of this disclosure, video encoder200may be configured to conditionally signal gdr_or_irap_pic_flag based on an indication of the mixed NAL unit types mixed_nalu_types_in_pic_flag. Video decoder300may be configured to infer gdr_or_irap_pic_flag is equal to 0 when gdr_or_irap_pic_flag is not present in the bitstream.

An example implementation of these techniques is shown below as changes relative to VVC draft 8:

In some examples, video encoder200and video decoder300may be configured to operate according to a constraint where the semantics of gdr_or_irap_pic_flag are constrained such that gdr_or_irap_pic_flag shall be 0 when a mixed NAL unit type presence is indicated. An example implementation of these techniques is shown below as changes relative to VVC draft 8:

gdr_or_irap_pic_flag equal to 1 specifies that the current picture is a GDR or TRAP picture. gdr_or_irap_pic_flag equal to 0 specifies that the current picture may or may not be a GDR or IRAP picture. <ADD>gdr_irap_pic_flag shall be equal to 0 when mixed_nalu_types_in_pic_flag is equal to 1. </ADD>

In accordance with the techniques introduced above, video encoder200may be configured to determine that reference picture list information is included in a PH syntax structure and generate a first syntax element, such as rpl_info_in_ph_flag above to indicate that the reference picture list information is included in the PH syntax structure. Video encoder200may generate a second syntax element, such as ph_collocated_from_l0_flag above, for inclusion in the PH syntax structure, with a first value (e.g., 0) for the second syntax element indicating that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list (e.g., l0) and a second value (e.g., 1) for the second syntax element indicating that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list (e.g., l1). Video encoder200may determine that a slice of the video data that refers to the PH syntax structure is a P slice and in response to the slice being a P slice, determine that a value for a third syntax element associated with the slice, such as slice_collocated_from_l0_flag), is equal to a first value for the third syntax element, with the first value (e.g., 0) for the third syntax element indicating that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list (e.g., l0) and a second value (e.g., 1) for the third syntax element indicating that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list (e.g., l1). Video encoder200may output a bitstream of encoded video data that includes the first syntax element and the PH syntax structure.

In accordance with such a technique, video decoder300may be configured to receive a first syntax element, such as rpl_info_in_ph_flag above, indicating that reference picture list information is included in a PH syntax structure, and in response, receive a second syntax element, such as ph_collocated_from_l0_flag above in the PH syntax structure. For the second syntax element, a first value (e.g., 0) may indicate that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list (e.g., l0), and a second value (e.g., 1) may indicate that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list (e.g., l1). Video decoder300may receive a slice of the video data that refers to the PH syntax structure and in response to the slice being a P slice, set a value for a third syntax element associated with the slice, such as slice_collocated_from_l0_flag above, to a first value. The first value (e.g., 0) for the third syntax element may indicate that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list (e.g., l0), whereas a second value (e.g., 1) for the third syntax element would indicate that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list (e.g., l1).

Video decoder300may additionally be configured to receive, in response to receiving the first syntax element indicating that reference picture list information is included in the picture header syntax structure and in response to the slice being the P slice, an instance of a fourth syntax element, where a first value for the fourth syntax element indicates a fifth syntax element is included in the slice header and a second value for the fourth syntax element indicates the fifth syntax element is not included in the slice header. In response to the instance of the fourth syntax element being equal to the first value for the fourth syntax element, video decoder300may receive an instance of the fifth syntax element and determine a number of active reference pictures for the slice based on a value for the instance of the fifth syntax element.

FIG. 3is a block diagram illustrating an example video encoder200that may perform the techniques of this disclosure.FIG. 3is provided for purposes of explanation and should not be considered limiting of the techniques as broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes video encoder200according to the techniques of VVC (ITU-T H.266, under development), and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video encoding devices that are configured to other video coding standards.

In the example ofFIG. 3, video encoder200includes video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, decoded picture buffer (DPB)218, and entropy encoding unit220. Any or all of video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, DPB218, and entropy encoding unit220may be implemented in one or more processors or in processing circuitry. For instance, the units of video encoder200may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, of FPGA. Moreover, video encoder200may include additional or alternative processors or processing circuitry to perform these and other functions.

Video encoder200stores reconstructed blocks in DPB218. For instance, in examples where operations of filter unit216are not performed, reconstruction unit214may store reconstructed blocks to DPB218. In examples where operations of filter unit216are performed, filter unit216may store the filtered reconstructed blocks to DPB218. Motion estimation unit222and motion compensation unit224may retrieve a reference picture from DPB218, formed from the reconstructed (and potentially filtered) blocks, to inter-predict blocks of subsequently encoded pictures. In addition, intra-prediction unit226may use reconstructed blocks in DPB218of a current picture to intra-predict other blocks in the current picture.

Video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform the techniques of this disclosure, including the techniques of the claims below.

In the example ofFIG. 4, video decoder300includes coded picture buffer (CPB) memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and decoded picture buffer (DPB)314. Any or all of CPB memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and DPB314may be implemented in one or more processors or in processing circuitry. For instance, the units of video decoder300may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, of FPGA. Moreover, video decoder300may include additional or alternative processors or processing circuitry to perform these and other functions.

Video decoder300may store the reconstructed blocks in DPB314. For instance, in examples where operations of filter unit312are not performed, reconstruction unit310may store reconstructed blocks to DPB314. In examples where operations of filter unit312are performed, filter unit312may store the filtered reconstructed blocks to DPB314. As discussed above, DPB314may provide reference information, such as samples of a current picture for intra-prediction and previously decoded pictures for subsequent motion compensation, to prediction processing unit304. Moreover, video decoder300may output decoded pictures (e.g., decoded video) from DPB314for subsequent presentation on a display device, such as display device118ofFIG. 1.

In this manner, video decoder300represents an example of a video decoding device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform the techniques of this disclosure, including the techniques of the claims below.

FIG. 5is a flowchart illustrating an example method for encoding a current block. The current block may comprise a current CU. Although described with respect to video encoder200(FIGS. 1 and 3), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 5.

In this example, video encoder200initially predicts the current block (350). For example, video encoder200may form a prediction block for the current block. Video encoder200may then calculate a residual block for the current block (352). To calculate the residual block, video encoder200may calculate a difference between the original, unencoded block and the prediction block for the current block. Video encoder200may then transform the residual block and quantize transform coefficients of the residual block (354). Next, video encoder200may scan the quantized transform coefficients of the residual block (356). During the scan, or following the scan, video encoder200may entropy encode the transform coefficients (358). For example, video encoder200may encode the transform coefficients using CAVLC or CABAC. Video encoder200may then output the entropy encoded data of the block (360).

FIG. 6is a flowchart illustrating an example method for decoding a current block of video data. The current block may comprise a current CU. Although described with respect to video decoder300(FIGS. 1 and 4), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 6.

Video decoder300may receive entropy encoded data for the current block, such as entropy encoded prediction information and entropy encoded data for transform coefficients of a residual block corresponding to the current block (370). Video decoder300may entropy decode the entropy encoded data to determine prediction information for the current block and to reproduce transform coefficients of the residual block (372). Video decoder300may predict the current block (374), e.g., using an intra- or inter-prediction mode as indicated by the prediction information for the current block, to calculate a prediction block for the current block. Video decoder300may then inverse scan the reproduced transform coefficients (376), to create a block of quantized transform coefficients. Video decoder300may then inverse quantize and inverse transform the transform coefficients to produce a residual block (378). Video decoder300may ultimately decode the current block by combining the prediction block and the residual block (380).

FIG. 7is a flowchart illustrating an example method for encoding a current block. The current block may comprise a current CU. Although described with respect to video encoder200(FIGS. 1 and 3), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 7.

In the example ofFIG. 7, in response to determining that reference picture list information is included in a PH syntax structure, video encoder200generates a first syntax element, such as rpl_info_in_ph_flag above, indicating that the reference picture list information is included in the PH syntax structure (400).

Video encoder200generates a second syntax element, such as ph_collocated_from_l0_flag, for inclusion in the PH syntax structure, set to a value to indicate that a collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list (e.g., l1) of two reference picture lists (402). Video encoder200determines that a slice of the video data that refers to the PH syntax structure is a P slice (404). In response to the slice being a P slice, video encoder200determines that a value for a third syntax element associated with the slice, such as slice_collocated_from_l0_flag, is equal to a value indicating that the collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list (e.g., l0) of the two reference picture lists (406). Video encoder200outputs a bitstream of encoded video data that includes the first syntax element and the PH syntax structure (408).

FIG. 8is a flowchart illustrating an example method for decoding a current block of video data. The current block may comprise a current CU. Although described with respect to video decoder300(FIGS. 1 and 4), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 8.

In the example ofFIG. 8, in response to receiving a first syntax element, such as e.g., rpl_info_in_ph_flag, indicating that reference picture list information is included in a PH syntax structure, video decoder300receives a second syntax element, such as ph_collocated_from_l0_flag, in the PH syntax structure (410). Video decoder300determines based on a value of the second syntax element that a collocated picture used for temporal motion vector prediction is to be derived from a second reference picture list (e.g., l1) (412). Video decoder300receives a slice of the video data that refers to the PH syntax structure (414). In response to the slice being a P slice, video decoder300sets a value for a third syntax element associated with the slice, such as slice_collocated_from_l0_flag, to a value indicating that the collocated picture used for temporal motion vector prediction is to be derived from a second reference picture list (416).

Video decoder300may decode a slice of the video data based on the value for the third syntax element. For example, for a block of the slice, video decoder300may determine a temporal motion vector candidate for inclusion in a motion vector candidate list. To determine a temporal motion vector candidate for inclusion in a motion vector candidate list, video decoder300may identify the collocated picture used for the temporal motion vector prediction from the first reference picture list; identify a collocated block in the collocated picture; and derive the temporal motion vector candidate based on a motion vector used to decode the collocated block.

Video decoder300may output decoded video data that includes a decoded version of the slice. For example, video decoder300may output, for display or for storage, a decoded picture that includes the decoded slice.

The following clauses represent example implementations of the techniques and devices introduced above.

Clause 1: A method of decoding video data, the method comprising: receiving a first syntax element; in response to the first syntax element indicating that reference picture list information is included in a picture header syntax structure, receiving a second syntax element in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; receiving a slice of the video data that refers to the picture header syntax structure; and in response to the slice being a P slice, setting a value for a third syntax element associated with the slice to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list.

Clause 2: The method of clause 1, wherein a value for the second syntax element is equal to the second value for the second syntax element.

Clause 3: The method of clause 1 or 2, wherein setting the value for the third syntax element associated with the slice to the second value comprises inferring the value for the second flag to be the second value.

Clause 4: The method of any of clauses 1-3, wherein setting the value for the third syntax element associated with the slice to the second value comprises, setting the value for the third syntax element to be the second value without receiving an instance of the third syntax element in a slice header of the slice.

Clause 5: The method of any clauses 1-4, further comprising: for a block of the slice, determining a temporal motion vector candidate for inclusion in a motion vector candidate list, wherein determining the temporal motion vector candidate comprises: identifying the collocated picture used for the temporal motion vector prediction from the first reference picture list; identifying a collocated block in the collocated picture; and deriving the temporal motion vector candidate based on a motion vector used to decode the collocated block.

Clause 6: The method of any of clauses 1-5, further comprising: in response to receiving the first syntax element indicating that reference picture list information is included in the picture header syntax structure and in response to the slice being the P slice, receiving an instance of a fourth syntax element, wherein a first value for the fourth syntax element indicates a fifth syntax element is included in the slice header and a second value for the fourth syntax element indicates the fifth syntax element is not included in the slice header; in response to the instance of the fourth syntax element being equal to the first value for the fourth syntax element, receiving an instance of the fifth syntax element; and determining a number of active reference pictures for the slice based on a value for the instance of the fifth syntax element.

Clause 7: The method of clause 6, wherein the slice is a first slice, the instance of the fourth syntax element is a first instance of the fourth syntax element, the instance of the fifth syntax element is a first instance of the fifth syntax element, the method further comprising: receiving a second slice of the video data that refers to the picture header syntax structure; and in response to receiving the first syntax element indicating that reference picture list information is included in the picture header syntax structure and in response to the second slice being an I slice, receiving a slice header, without a second instance of the fourth syntax element, for the second slice.

Clause 8: A device for decoding video data, the device comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: receive a first syntax element; in response to the first syntax element indicating that reference picture list information is included in a picture header syntax structure, receive a second syntax element in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; receive a slice of the video data that refers to the picture header syntax structure; and in response to the slice being a P slice, set a value for a third syntax element associated with the slice to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list.

Clause 9: The device of clause 8, wherein a value for the second syntax element is equal to the second value for the second syntax element.

Clause 10: The device of clause 8 or 9, wherein to set the value for the third syntax element associated with the slice to the second value, the one or more processors are further configured to infer the value for the third syntax element to be the second value.

Clause 11: The device of any of clauses 8-10, wherein to set the value for the third syntax element associated with the slice to the second value comprises, the one or more processors are further configured to set the value for the third syntax element to be the second value without receiving an instance of the third syntax element in a slice header of the slice.

Clause 12: The device of any of clauses 8-11, wherein the one or more processors are further configured to: for a block of the slice, determine a temporal motion vector candidate for inclusion in a motion vector candidate list, wherein to determine the temporal motion vector candidate, the one or more processors are further configured to: identify the collocated picture used for the temporal motion vector prediction from the first reference picture list; identify a collocated block in the collocated picture; and derive the temporal motion vector candidate based on a motion vector used to decode the collocated block.

Clause 13: The device of any of clauses 8-12, wherein the one or more processors are further configured to: in response to receiving the first syntax element indicating that reference picture list information is included in the picture header syntax structure and in response to the slice being the P slice, receive an instance of a fourth syntax element, wherein a first value for the fourth syntax element indicates a fifth syntax element is included in the slice header and a second value for the fourth syntax element indicates the fifth syntax element is not included in the slice header; in response to the instance of the fourth syntax element being equal to the first value for the fourth syntax element, receive an instance of the fifth syntax element; and determine a number of active reference pictures for the slice based on a value for the instance of the fifth syntax element.

Clause 14: The device of clause 13, wherein the slice is a first slice, the instance of the fourth syntax element is a first instance of the fourth syntax element, the instance of the fifth syntax element is a first instance of the fifth syntax element, wherein the one or more processors are further configured to: receive a second slice of the video data that refers to the picture header syntax structure; in response to receiving the first syntax element indicating that reference picture list information is included in the picture header syntax structure and in response to the second slice being an I slice, receive a slice header, without a second instance of the fourth syntax element, for the second slice.

Clause 15: The device of any of clauses 8-14, wherein the device comprises a wireless communication device, further comprising a receiver configured to receive encoded video data.

Clause 16: The device of clause 15, wherein the wireless communication device comprises a telephone handset and wherein the receiver is configured to demodulate, according to a wireless communication standard, a signal comprising the encoded video data.

Clause 17: The device of any of clauses 8-16, further comprising: a display configured to display decoded video data.

Clause 18: The device of any of clauses 8-17, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

Clause 19: A method of encoding video data, the method comprising: in response to determining that reference picture list information is included in a picture header syntax structure, generating a first syntax element indicating that the reference picture list information is included in the picture header syntax structure; generating a second syntax element for inclusion in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; determining that a slice of the video data that refers to the picture header syntax structure is a P slice; in response to the slice being the P slice, determining that a value for a third syntax element associated with the slice is equal to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list; and outputting a bitstream of encoded video data that includes the first syntax element and the picture header syntax structure.

Clause 20: The method of clause 19, wherein a value for the second syntax element is equal to the second value for the second syntax element.

Clause 21: The method of clause 19 or 20, further comprising: generating, for inclusion in the bitstream of encoded video data, a slice header for the slice without including in the slice header an instance of the third syntax element.

Clause 22: The method of any of clauses 19-21, further comprising: for a block of the slice, determining a temporal motion vector candidate for inclusion in a motion vector candidate list, wherein determining the temporal motion vector candidate comprises: identifying the collocated picture used for the temporal motion vector prediction from the first reference picture list; identifying a collocated block in the collocated picture; and deriving the temporal motion vector candidate based on a motion vector used to decode the collocated block.

Clause 23: The method of any of clauses 19-22, further comprising: in response to determining that the reference picture list information is included in the picture header syntax structure and in response to determining that the slice is the P slice, generate an instance of a fourth syntax element, wherein a first value for the fourth syntax element indicates a fifth syntax element is included in the slice header and a second value for the fourth syntax element indicates the fifth syntax element is not included in the slice header; determining a number of active reference pictures for the slice; and in response to the instance of the fourth syntax element being equal to the first value for the fourth syntax element, generating, for inclusion in the bitstream of encoded video data, an instance of the fifth syntax element, wherein a value for the instance of the fifth syntax element indicates the number of active reference pictures for the slice.

Clause 24: The method of clause 23, wherein the slice is a first slice, the instance of the fourth syntax element is a first instance of the fourth syntax element, the instance of the fifth syntax element is a first instance of the fifth syntax element, the method further comprising: for a second slice of the video data that refers to the picture header syntax structure, in response to determining that reference picture list information is included in the picture header syntax structure and in response to the second slice being an I slice, generating, for inclusion in the bitstream of encoded video data, a slice header without a second instance of the fourth syntax element.

Clause 25: A device for encoding video data, the device comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: in response to determining that reference picture list information is included in a picture header syntax structure, generate a first syntax element indicating that the reference picture list information is included in the picture header syntax structure; generate a second syntax element for inclusion in the picture header syntax structure, wherein a first value for the second syntax element indicates that a collocated picture used for temporal motion vector prediction is to be derived from a first reference picture list and a second value for the second syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from a second reference picture list; determine that a slice of the video data that refers to the picture header syntax structure is a P slice; in response to the slice being the P slice, determine that a value for a third syntax element associated with the slice is equal to a first value for the third syntax element, wherein the first value for the third syntax element indicates that the collocated picture used for temporal motion vector prediction is to be derived from the first reference picture list and a second value for the third syntax element indicates that the collocated picture used for the temporal motion vector prediction is to be derived from the second reference picture list; and output a bitstream of encoded video data that includes the first syntax element and the picture header syntax structure.

Clause 26: The device of clause 25, wherein a value for the second syntax element is equal to the second value for the second syntax element.

Clause 27: The device of clause 25 or 26, further comprising: generating, for inclusion in the bitstream of encoded video data, a slice header for the slice without including in the slice header an instance of the third syntax element.

Clause 28: The device of any of clauses 25-27, further comprising: for a block of the slice, determining a temporal motion vector candidate for inclusion in a motion vector candidate list, wherein determining the temporal motion vector candidate comprises: identifying the collocated picture used for the temporal motion vector prediction from the first reference picture list; identifying a collocated block in the collocated picture; and deriving the temporal motion vector candidate based on a motion vector used to decode the collocated block.

Clause 29: The device of any of clauses 25-28, further comprising: in response to determining that the reference picture list information is included in the picture header syntax structure and in response to determining that the slice is a P slice, generate an instance of a fourth syntax element, wherein a first value for the fourth syntax element indicates a fifth syntax element is included in the slice header and a second value for the fourth syntax element indicates the fifth syntax element is not included in the slice header; determining a number of active reference pictures for the slice; and in response to the instance of the fourth syntax element being equal to the first value for the fourth syntax element, generating, for inclusion in the bitstream of encoded video data, an instance of the fifth syntax element, wherein a value for the instance of the fifth syntax element indicates the number of active reference pictures for the slice.

Clause 30: The device of clause 29, wherein the slice is a first slice, the instance of the fourth syntax element is a first instance of the fourth syntax element, the instance of the fifth syntax element is a first instance of the fifth syntax element, the method further comprising: for a second slice of the video data that refers to the picture header syntax structure, in response to determining that reference picture list information is included in the picture header syntax structure and in response to the second slice being an I slice, generating, for inclusion in the bitstream of encoded video data, a slice header without a second instance of the fourth syntax element.

Clause 31: The device of any of clauses 25-30, wherein the device comprises a wireless communication device, further comprising a transmitted configured to transmit encoded video data.

Clause 32: The device of clause 31, wherein the wireless communication device comprises a telephone handset and wherein the transmitter is configured to modulate, according to a wireless communication standard, a signal comprising the encoded video data.

Clause 33: The device of any of clauses 25-32, further comprising: a camera configured to capture the video data.

Clause 34: The device of any of clauses 25-33, wherein the device comprises one or more of a camera, a computer, or a mobile device.

Clause 35: A method of decoding video data, the method comprising: receiving a first flag; receiving a second flag; determining if a reference picture list is signaled in a syntax structure of the video data based on the first and the second flag.

Clause 36: The method of clause 35, wherein the first flag indicates whether or not inter slices are allowed for a picture.

Clause 37: The method of clause 35 or 36, wherein the second flag indicates if reference picture list syntax elements are present in slice headers of instantaneous decoder refresh pictures.

Clause 38: The method of any combination of clauses 35-37, wherein determining if the reference picture list is signaled in the data structure of the video data based on the first and the second flag comprises determining that the reference picture list is signaled in the data structure of the video data in response to at least one of the first flag or the second flag being true.

Clause 39: The method of any combination of clauses 35-37, further comprising: receiving a third flag indicating if reference picture list information is present in a picture header syntax structure or in slice headers.

Clause 40: The method of clause 39, wherein determining if the reference picture list is signaled in the data structure of the video data based on the first and the second flag comprises determining that the reference picture list is signaled in the data structure of the video data in response to (1) at least one of the first flag or the second flag being true and (2) the third flag being true.

Clause 41: The method of any combination of clauses 35-40, wherein the syntax structure comprises a picture header syntax structure.

Clause 42: A method of coding video data, the method comprising: receiving a first flag; receiving a second flag; determining if a weighted prediction table is signaled in a syntax structure of the video data based on the first and the second flag.

Clause 43: The method of clause 42, wherein the first flag indicates whether or not inter slices are allowed for a picture.

Clause 44: The method of clause 42 or 43, wherein the second flag indicates if reference picture list syntax elements are present in slice headers of instantaneous decoder refresh pictures.

Clause 45: The method of any combination of clauses 42-44, wherein determining if the weighted prediction table is signaled in the data structure of the video data based on the first and the second flag comprises determining that the weighted prediction table is signaled in the data structure of the video data in response to at least one of the first flag or the second flag being true.

Clause 46: The method of any combination of clauses 42-45, further comprising: receiving a third flag indicating if weighted prediction is applied.

Clause 47: The method of clause 46, wherein determining if the weighted prediction table is signaled in the data structure of the video data based on the first and the second flag comprises determining that the reference picture list is signaled in the data structure of the video data in response to (1) at least one of the first flag or the second flag being true, and (2) the third flag being true.

Clause 48: The method of any combination of clauses 42-47 wherein the syntax structure comprises a picture header syntax structure.

Clause 49: A device for coding video data, the device comprising one or more means for performing the method of any combination of clauses 35-48.

Clause 50: The device of clause 49, wherein the one or more means comprise one or more processors implemented in circuitry.

Clause 51: The device of any of clauses 49 and 50, further comprising a memory to store the video data.

Clause 52: The device of any of clauses 49-51, further comprising a display configured to display decoded video data.

Clause 53: The device of any of clauses 49-52, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

Clause 54: The device of any of clauses 49-53, wherein the device comprises a video decoder.

Clause 55: The device of any of clauses 49-54, wherein the device comprises a video encoder.

Clause 56: A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 35-48.