Coefficient coding with grouped bypass remaining levels for dependent quantization

The disclosure describes example techniques for determining a context used for encoding or decoding flags used to indicate a value of a coefficient. The techniques also relate to determining a quantization or dequantization factor to use for the coefficient. For determining the context and the quantization or dequantization factor, a video coder may determine values of flags used for encoding or decoding a previous coefficient and use the determined values of the flags for determining the context and the quantization or dequantization factor for the current coefficient.

TECHNICAL FIELD

BACKGROUND

SUMMARY

In general, this disclosure describes techniques for coefficient coding schemes that enable efficient parsing of coefficients and enable quantization selection. As described in more detail, a video coder (e.g., video encoder or video decoder) may determine a set of contexts for coding a significance flag (e.g., a flag that indicates whether an absolute value for a coefficient value is greater than zero) for a current coefficient based on the significance flag of a previous coefficient (e.g., an immediately previous coefficient). Based on the determined set of contexts, the video coder may determine a context, and code the current coefficient based on the determined context.

In some examples, the coefficient values may be the result of quantization. In one or more examples, the video coder may determine the quantization factor used to quantize (e.g., for encoding) or dequantize (e.g., for decoding) in a similar manner. For example, the quantization factor for quantizing or dequantizing a current coefficient may be based on the significance flag of the previous coefficient. In some examples, the quantization factor for the current coefficient may also be based on the quantization factor that was used when coding the previous coefficient. For instance, the quantization factor used when coding the previous coefficient and the significance flag of the previous coefficient may indicate the quantization factor that is to be used for the current coefficient.

In this way, the example techniques provide a practical application for implementing video coding. For instance, the example techniques provide a mechanism that selects a context to use in coding a coefficient value. As described in more detail, the example techniques may provide advantages such as allowing all context-coded values to be grouped together and coded one after the other and then values that are bypass-coded are coded. By allowing all context-coded values to be grouped together, the video coder may not expend overhead switching between context-coding and bypass-coding such as in cases where context-coding and bypass-coding are interleaved.

In one example, the disclosure describes a method of coding video data, the method comprising selecting, for coding a current coefficient of the video data, a context set from a plurality of context sets based on whether a previous coefficient has an absolute value greater than zero, determining a context from the selected context set, and coding a value for the current coefficient based on the determined context.

In one example, the disclosure describes a device for coding video data, the device comprising a memory configured to store contexts for a plurality of context sets and a video coder comprising at least one of fixed-function or programmable circuitry. The video coder is configured to select, for coding a current coefficient of the video data, a context set from the plurality of context sets stored in memory based on whether a previous coefficient has an absolute value greater than zero, determine a context from the selected context set, and code a value for the current coefficient based on the determined context.

In one example, the disclosure describes a computer-readable storage medium storing instructions thereon that when executed cause one or more processors of a device for coding video data to select, for coding a current coefficient of the video data, a context set from a plurality of context sets based on whether a previous coefficient has an absolute value greater than zero, determine context from the selected context set, and code a value for the current coefficient based on the determined context.

In one example, the disclosure describes a device for coding video data, the device comprising means for selecting, for coding a current coefficient of the video data, a context set from a plurality of context sets based on whether a previous coefficient has an absolute value greater than zero, means for determining context from the selected context set, and means for coding a value for the current coefficient based on the determined context.

DETAILED DESCRIPTION

In video encoding, a video encoder determines a prediction block for a current block being encoded. The video encoder determines a residual (e.g., difference) between the prediction block and the current block. In some examples, the video encoder transforms the residual values from sample or pixel domain to a transform domain (e.g., frequency domain). The values in the transform domain are referred to as coefficient values. In some examples, the coefficient values are quantized to generate quantized coefficient values. For ease, in this disclosure, from the perspective of video encoding, the term “coefficient values” may refer to quantized coefficient values or coefficient values prior to quantization or where quantization is skipped.

The video encoder signals information that a video decoder uses to determine the coefficient values. As one example, the video encoder generates a significance flag (also called greater than 0 or gt0 flag). The significance flag indicates whether the absolute value of the current coefficient is greater than 0. The video encoder generates a gt1 flag (that indicates whether the absolute value of the current coefficient is greater than 1), gt2 flag (that indicates whether the absolute value of the current coefficient is greater than 2), a remainder level value (that indicates the remainder of the value of the coefficient greater than two), a sign value (that indicates whether the coefficient is positive or negative), and in some examples, a parity value (that indicates whether the coefficient is even or odd). In some examples, there may be a gt3 flag, or more generally a gtX flag, and the remainder level value may be adjusted accordingly as a remainder of the value of the coefficient greater than X.

The video encoder encodes the various flags and remainder level values. For example, the gtX flags, including the significance flag, are context coded and the remainder level value is bypass coded. Examples of context coding and bypass coding is described in more detail below.

The video decoder receives the encoded flags and remainder level values and performs the inverse operation as that of the video encoder to reconstruct the coefficient values (e.g., quantized coefficient values or regular coefficient values where quantization is skipped). For example, the video decoder may decode the encoded context coded flag values and the bypass coded remainder level value. Based on the flags, the video decoder may determine which coefficients have coefficient values greater than 0, 1, or 2, and use the remainder level values to determine the coefficient values.

In examples where quantization is applied, the video decoder may perform inverse quantization to determine the final coefficient values. From the perspective of video decoding, the coefficient values may refer to the coefficient values before inverse quantization or may refer to the inverse quantized coefficient values.

In one or more examples, to perform the context-based coding of the significance flag of a current coefficient, the video encoder may determine a set of contexts based on a significance flag of a previous coefficient (e.g., a previously coded coefficient, such as the coefficient that is coded immediately prior to the current coefficient being coded). As one example, assume there is a first set of contexts and a second set of contexts. In one example, if the significance flag of the previous coefficient was a one, then the video encoder may determine the first set of contexts. If the significance flag of the previous coefficient was a zero, then the video encoder may determine the second set of contexts. The video encoder may determine a context in the determined set of contexts such as based on significance flags of below and right coefficients, and context encode the significance flag for the current coefficient based on the determined context. The video encoder may perform similar operations for the other flags such as gt1, gt2, and more generally gtX.

The video decoder may perform similar operations to determine the contexts. For example, if the significance flag of the previous coefficient (e.g., previously coded coefficient, where the previous coefficient is immediately previous coefficient in coding order) was a one, then the video decoder may determine the first set of contexts. If the significance flag of the previous coefficient was a zero, then the video decoder may determine the second set of contexts. The video decoder may determine a context in the determined set of contexts such as based on significance flags of below and right coefficients, and context decode the significance flag for the current coefficient based on the determined context. The video decoder may perform similar operations for the other flags such as gt1, gt2, and more generally gtX.

Accordingly, this disclosure describes example techniques for context-based coding of a flag for a current coefficient based on a corresponding flag for a previous coefficient (e.g., an immediately previous coefficient). For example, context for the significance flag for the current coefficient is based on the significance flag of the previous coefficient, context for the gt1 flag for the current coefficient is based on the gt1 flag of the previous coefficient, and so forth. For example, context for the gt2 flag for the current coefficient may be based on the gt2 flag of the previous coefficient. The significance flag of the previous coefficient indicates which one of a plurality of context sets to use (e.g., select between a first set of contexts and a second set of contexts). Then, the video encoder and video decoder may determine a context from the selected one of the plurality of context sets.

In some examples, the video encoder may be configured to select between a plurality of quantization factors to apply to the coefficients to generate the quantized coefficient values. Similarly, the video decoder may be configured to select between a plurality of dequantization factors to apply to the quantized coefficient values to generate dequantized coefficient values. In one or more examples described in this disclosure, the video encoder and the video decoder may perform similar operations as those described above to determine the context for determining the quantization or dequantization factors.

For example, video encoder may determine whether absolute value of a previous coefficient value is greater than zero (e.g., whether the previous coefficient is significant). For the video encoder, the significance flag may not be generated until entropy coding. For quantization, the video encoder may consider whether the previous coefficient was significant (e.g., whether a previous coefficient has an absolute value greater than zero). Based on whether the previous coefficient was significant, the video encoder may determine a quantization factor for the current coefficient (e.g., for quantizing the current coefficient). In some examples, the video encoder may further utilize the quantization factor of the previous coefficient in combination with whether the previous coefficient was significant to determine the quantization factor for the current coefficient.

Similarly, for dequantization, the video decoder may consider whether the previous coefficient was significant (e.g., whether a previous coefficient has an absolute value greater than zero). For the video decoder, the significance flag for the previous coefficient should be available. In any case, the significance flag is indicative of whether the absolute value of a coefficient is greater than zero. Based on whether the previous coefficient was significant, the video decoder may determine a dequantization factor for the current coefficient (e.g., for dequantizing the current coefficient). In some examples, the video decoder may further utilize the dequantization factor of the previous coefficient in combination with whether the previous coefficient was significant to determine the dequantization factor for the current coefficient.

The example techniques may provide various advantages. For example, M. Abrecht, et al., “Description of SDR, HDR, and 360 video coding technology proposal by Fraunhofer HHI” JVET-J0014, 2018 (hereinafter JVET-J0014) described using parity of the previous coefficient to determine a context set for coding a significance flag for the current coefficient. However, one limitation with these techniques in JVET-J0014 is that the parity information (e.g., whether the coefficient value is odd or even) is not known until the coefficient value is fully reconstructed.

Therefore, the video encoder may have needed to perform context-coding for certain flags (e.g., significance flag, gt1, gt2 flags) followed by bypass-coding for remainder level value of a previous coefficient before being able to perform context-coding for flags of the current coefficient. Stated another way, in JVET-J0014, because the video decoder needed the parity value of the previous coefficient to decode the significance flag for the current coefficient, the video decoder needed to completely reconstruct the previous coefficient (e.g., context-decode the significance, gt1, gt2 flags and bypass-decode the remainder level value) before starting the decoding of the current coefficient (e.g., before being able to context-decode the significance flag of the current coefficient).

Accordingly, in the JVET-J0014 techniques, the context-coding and the bypass-coding is interleaved. For instance, a plurality of coefficients forms a coefficient group. In JVET-J0014, all flags and the remainder level value of the first coefficient in the coefficient group are encoded or decoded, which means the video encoder and the video decoder started with context-coding (e.g., for the flags) and then switched to bypass-coding (e.g., for the remainder level value). Then, for the second coefficient, in JVET-J0014, the video encoder and the video decoder switched back from bypass-coding to context-coding for encoding or decoding the flags of the second coefficient before switching back to bypass-coding for encoding or decoding the remainder level value for the second coefficient, and so forth.

Interleaving between context-coding and bypass-coding may negatively impact video coding performance. For instance, the video encoder and video decoder may need to be initialized for performing context-coding each time the video encoder and video decoder start to perform context-coding. As one example, each time the video encoder and the video decoder switch from bypass-coding to context-coding, the video encoder and the video decoder may need to be initialized, which utilizes processing cycles. However, once initialized, the video encoder and video decoder can context-encode or decode multiple flags without needing to be initialized again. The initialization of the video encoder and video decoder may be needed when switching from bypass to context-encoding or decoding.

In accordance with one or more examples described in this disclosure, since a flag for a current coefficient is encoded or decoded based on a corresponding flag in the previous coefficient, the video encoder and video decoder may not need to wait until the previous coefficient is completely encoded or decoded before starting the encoding or decoding of the current coefficient. In other words, there may not be a need to interleave between context-coding and bypass-coding. For example, the video encoder and the video decoder may context-encode or decode the significance flag for each coefficient in the coefficient group, then context-encode or decode the gt1 flag for each coefficient in the coefficient group, and then context-encode or decode the gt2 flag for each coefficient in the coefficient group. Once all of the context-coding is complete, the video encoder and video decoder may bypass-encode or decode the remainder level values for each of the coefficients. In this manner, the number of times the video encoder and the video decoder are initialized for context-encoding or decoding is reduced as compared to the JVET-J0014 techniques, resulting in reduced processing cycles and faster encoding and decoding.

FIG. 1is a block diagram illustrating an example video encoding and decoding system100that may perform the techniques of this disclosure. The techniques of this disclosure are generally directed to coding (encoding and/or decoding) video data. In general, video data includes any data for processing a video. Thus, video data may include raw, uncoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

In some examples, source device102may output encoded video data to file server114or another intermediate storage device that may store the encoded video generated by source device102. Destination device116may access stored video data from file server114via streaming or download. File server114may be any type of server device capable of storing encoded video data and transmitting that encoded video data to the destination device116. File server114may represent a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device116may access encoded video data from file server114through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both that is suitable for accessing encoded video data stored on file server114. File server114and input interface122may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

Although not shown inFIG. 1, in some examples, video encoder200and video decoder300may each be integrated with an audio encoder and/or audio decoder, and may include appropriate MUX-DEMUX units, or other hardware and/or software, to handle multiplexed streams including both audio and video in a common data stream. If applicable, MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

Video encoder200and video decoder300may operate according to a video coding standard, such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC) or extensions thereto, such as the multi-view and/or scalable video coding extensions. Alternatively, video encoder200and video decoder300may operate according to other proprietary or industry standards, such as the Joint Exploration Test Model (JEM) or ITU-T H.266, also referred to as Versatile Video Coding (VVC). A recent draft of the VVC standard is described in Bross, et al. “Versatile Video Coding (Draft 5),” Joint Video Experts Team (WET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14thMeeting: Geneva, CH, 19-27 Mar. 2019, JVET-N1001-v3 (hereinafter “VVC Draft 5”). The techniques of this disclosure, however, are not limited to any particular coding standard.

In some examples, video encoder200and video decoder300may use a single QTBT structure to represent each of the luminance and chrominance components, while in other examples, video encoder200and video decoder300may use two or more QTBT structures, such as one QTBT structure for the luminance component and another QTBT structure for both chrominance components (or two QTBT structures for respective chrominance components).

In HEVC, coefficients (e.g., transform coefficients of a transform unit) are coded (e.g., encoded or decoded) in bitplane by bitplane order (partially). As described above, coefficients may be coded using the significance (or gt0) flag, gt1 flag, and so forth. Gt0, gt1, and so forth, each make up a bitplane of the bits that are used to indicate the coefficient value. In bitplane by bitplane order, first in a coefficient group, gt0 (e.g., greater than zero and also call significance) flags of all coefficients are coded, followed by gt1 flags (e.g., greater than one), and followed by greater than 2 (gt2) flags (e.g., greater than two). The rest of the levels are coded as absolute value of the remaining coefficient level in bypass mode. The gt0, gt1, gt2 flags are context coded (e.g., CABAC). In other words, in HEVC, in a coefficient group, a video coder (e.g., video encoder200or video decoder300) may context-code (e.g., encode or decode) the gt0 flags for all coefficients in the coefficient group, then context-code the gt1 flags for all coefficients in the coefficient group, and then context-code the gt2 flags for all coefficients in the coefficient group, and so on. Then, the video coder may bypass-code the remaining level values for the coefficients in the coefficient group.

One reason for using this approach is to group all bypass coded bins for simpler parsing. Also, not mixing the order of the bitplanes simplifies multi-bin decoding where contexts for sequentially decoded coefficients do not depend on the previous coefficient (breaks the dependency) and reduces the cases where speculative decoding needs to be performed (when the next bin to be decoded could belong to a different bitplane). This kind of design improves the arithmetic coding throughput. J. Sole, et. A1, “Transform Coefficient Coding in HEVC”, IEEE CSVT, December 2012 discusses transform coefficient coding in HEVC.

The coefficient coding scheme in JVET-J0014 breaks this scheme by requiring all bitplanes of a coefficient to be decoded before moving on to the next (e.g., gt0, gt1, gt2, remaining level values for a first coefficient need to be decoded before moving to the next coefficient). JVET-J0014 also interleaves the bypass and context coded bins, making multi-bin arithmetic coding difficult. For instance, as described above, interleaving between context-coding and bypass-coding can negatively impact processing time because the context-coding engine of video encoder200and video decoder300needs to be initialized each time when transitioning from bypass-coding to context-coding.

In this disclosure, an alternative scheme that separates the bypass bins from regular coded bins in a TU or coefficient group for dependent quantization is described. For instance, a video coder (e.g., video encoder200or video decoder300) may select, for coding a current coefficient (e.g., a significance flag of the current coefficient), a context set from a plurality of context sets based on whether a previous coefficient (e.g., an immediately previous coefficient in coding order) has an absolute value greater than zero. As one example, video encoder200and video decoder300may have determined a significance flag (e.g., gt0 flag) for the previous coefficient that indicates whether the absolute value of the previous coefficient is greater than zero. Video encoder200and video decoder300may utilize the significance flag of the previous coefficient, as one non-limiting example way in which to determine whether the previous coefficient has an absolute value greater than zero.

For instance, there may be a first context set and a second context set. The first context set and the second context set each include a plurality of contexts (e.g., probability values). If the previous coefficient was significant (e.g., the absolute value of the previous coefficient was greater than zero), then video encoder200and video decoder300may select the first context set, and if the previous coefficient was not significant (e.g., the absolute value of the previous coefficient was not greater than zero), the video encoder200and video decoder300may select the second context set.

Video encoder200and video decoder300may determine the context from the selected context set. As one example, video encoder200and video decoder300may determine context from the selected context set based on values of neighboring coefficients. For example, video encoder200and video decoder300may determine an offset value in the selected context set based on whether a bottom coefficient and a right coefficient were significant or not (e.g., had absolute values greater than 0). Based on the determination (e.g., neither was significant, bottom coefficient was significant but right coefficient was not, right coefficient was significant but bottom coefficient was not, and both were significant), video encoder200and video decoder300may determine the offset value in the selected context set, and determine the context based on the offset value. For instance, the offset value may be considered as an index in the selected context set.

Video encoder200and video decoder300may encode or decode a value for the current coefficient based on the determined context. As one example, video encoder200and video decoder300may utilize CABAC techniques to context-encode or decode the significance flag for the current coefficient based on the determined context.

In this manner, video encoder200and video decoder300may be able to encode or decode the significance flag for the current coefficient without needing to encode or decode the entire previous coefficient. For example, the significance flag of the previous coefficient is needed but the rest of the flags, parity information, and remainder level value of the previous coefficient are not needed for coding the significance flag of the current coefficient.

Accordingly, in some examples, the current coefficient is a current coefficient of a coding group that includes a plurality of coefficients. Video encoder200and video decoder300may encode or decode a significance flag of each respective coefficients in the plurality of coefficients that follow the current coefficient before coding subsequent flags used to indicate the value of the current coefficient. In other words, video encoder200and video decoder300may encode or decode all significance flags in the coefficient group followed by gt1 flags in the coefficient group, and so on.

Video encoder200and video decoder300may utilize similar techniques for encoding and decoding other flag values. For instance, in some examples, video encoder200and video decoder300may select, for coding a current coefficient (e.g., coding a gtX flag), a context set from a plurality of context sets based on whether a previous coefficient has an absolute value greater than X.

Stated another way, video encoder200and video decoder300may select, for coding a flag of a current coefficient, a context set from a plurality of context sets based on a corresponding flag of a previous coefficient. In this example, the flag of the current coefficient and the corresponding flag of the previous coefficient are the same type of coefficient (e.g., both are significance flags, both are gt1 flags, both are gt2 flags, and so forth). In some examples, there may not be a plurality of context sets for some flags. For instance, gt1 flag may only have one context set rather than a plurality of context sets. In such examples, video encoder200and video decoder300may utilize other techniques to determine the context used for encoding or decoding the gt1 flag.

FIGS. 2A and 2Bare conceptual diagram illustrating an example QTBT structure130, and a corresponding CTU132. The solid lines represent quadtree splitting, and dotted lines indicate binary tree splitting. In each split (i.e., non-leaf) node of the binary tree, one flag is signaled to indicate which splitting type (i.e., horizontal or vertical) is used, where 0 indicates horizontal splitting and 1 indicates vertical splitting in this example. For the quadtree splitting, there is no need to indicate the splitting type, since quadtree nodes split a block horizontally and vertically into 4 sub-blocks with equal size. Accordingly, video encoder200may encode, and video decoder300may decode, syntax elements (such as splitting information) for a region tree level of QTBT structure130(i.e., the solid lines) and syntax elements (such as splitting information) for a prediction tree level of QTBT structure130(i.e., the dashed lines). Video encoder200may encode, and video decoder300may decode, video data, such as prediction and transform data, for CUs represented by terminal leaf nodes of QTBT structure130.

In general, CTU132ofFIG. 2Bmay be associated with parameters defining sizes of blocks corresponding to nodes of QTBT structure130at the first and second levels. These parameters may include a CTU size (representing a size of CTU132in samples), a minimum quadtree size (MinQTSize, representing a minimum allowed quadtree leaf node size), a maximum binary tree size (MaxBTSize, representing a maximum allowed binary tree root node size), a maximum binary tree depth (MaxBTDepth, representing a maximum allowed binary tree depth), and a minimum binary tree size (MinBTSize, representing the minimum allowed binary tree leaf node size).

The root node of a QTBT structure corresponding to a CTU may have four child nodes at the first level of the QTBT structure, each of which may be partitioned according to quadtree partitioning. That is, nodes of the first level are either leaf nodes (having no child nodes) or have four child nodes. The example of QTBT structure130represents such nodes as including the parent node and child nodes having solid lines for branches. If nodes of the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), they can be further partitioned by respective binary trees. The binary tree splitting of one node can be iterated until the nodes resulting from the split reach the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). The example of QTBT structure130represents such nodes as having dashed lines for branches. The binary tree leaf node is referred to as a coding unit (CU), which is used for prediction (e.g., intra-picture or inter-picture prediction) and transform, without any further partitioning. As discussed above, CUs may also be referred to as “video blocks” or “blocks.”

In one example of the QTBT partitioning structure, the CTU size is set as 128×128 (luma samples and two corresponding 64×64 chroma samples), the MinQTSize is set as 16×16, the MaxBTSize is set as 64×64, the MinBTSize (for both width and height) is set as 4, and the MaxBTDepth is set as 4. The quadtree partitioning is applied to the CTU first to generate quad-tree leaf nodes. The quadtree leaf nodes may have a size from 16×16 (i.e., the MinQTSize) to 128×128 (i.e., the CTU size). If the leaf quadtree node is 128×128, then the node may not be further split by the binary tree, because the size exceeds the MaxBTSize (i.e., 64×64, in this example). Otherwise, the leaf quadtree node will be further partitioned by the binary tree. Therefore, the quadtree leaf node is also the root node for the binary tree and has the binary tree depth as 0. When the binary tree depth reaches MaxBTDepth (4, in this example), no further splitting is permitted. A binary tree node having width equal to MinBTSize (4, in this example) implies no further horizontal splitting is permitted. Similarly, a binary tree node having a height equal to MinBTSize implies no further vertical splitting is permitted for that binary tree node. As noted above, leaf nodes of the binary tree are referred to as CUs and are further processed according to prediction and transform without further partitioning.

As described above, one technical problem that may exist in the field of video coding is that JVET-J0014 requires all bitplanes of a coefficient to be decoded before moving on to the next. The JVET-J0014 techniques also interleave the bypass and context coded bins, making multi-bin arithmetic coding difficult.

For example, in dependent quantization algorithm of CE7.2.1 the coefficients are coded coefficient by coefficient. CE7.2.1 is a core experiment that is testing a part of JVET-J0014. CE7.2.1 is described in Schwarz et. al. “Description of Core Experiment 7 (CE 7): Quantization and coefficient coding” JVET of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10thmeeting: San Diego, US, 10-20 Apr. 2018 (hereinafter JVET-J1027) and JVET-K0027.

In the above examples, video encoder200and video decoder300are described as determining the context set from plurality of context sets for flags based on corresponding flags in the previous coefficient. In some examples, the quantization or dequantization factor that video encoder200and video decoder300apply may be based on the value of the previous coefficient. For example, there may be two quantization or dequantization factors represented by Q0 and Q1 and are also referred to as quantizers.

In some examples, whether video encoder200and video decoder300apply Q0 or Q1 to the current coefficient is based on whether the previous coefficient has an absolute value greater than zero. In addition, the quantization or dequantization factor used for the current coefficient may be based the quantization or dequantization factor used for the previous coefficient, as is explained in more detail with respect to a state diagram ofFIG. 4. Hence, in addition to example ways in which to determine the context for encoding or decoding, this disclosure describes example techniques for determining the quantization or dequantization factor that is applied.

In JVET-J0014, the dependent quantization state machine states depend on the parity of the coefficient levels. The coefficient significance and greater than 1 (gt1) flags coding contexts as well as the quantizers used for a coefficient are selected by the state of the previously coded coefficient. This scheme results in interleaved regular coded bins and bypass coded bins at coefficient level within a coefficient group. In the techniques described in this disclosure, a 4-state, two scalar quantizer Trellis Coded Quantization similar to the one described in JVET-J0014 is used, and described in more detail with respect toFIG. 4. First, the two scalar quantizers (e.g., quantization or dequantization factors Q0 and Q1) as shown inFIG. 3are used for performing the quantization.

FIG. 3illustrates the use of two scalar quantizers (e.g., two quantization and dequantization factors). The first quantizer Q0 maps the transform coefficient levels (the numbers below the points) to even integer multiples of the quantization step size Δ. The second quantizer Q1 maps the transform coefficient levels to odd integer multiples of the quantization step size Δ or to zero in JVET-J0014.

In JVET-J0014, transform coefficients are quantized in coding order and the quantizer (e.g., quantization or dequantization factor) that is used to perform the quantization is selected based on a state machine that is driven by its previous state and the parity of the level of the previously coded coefficient. The state machine and the example states are described in the example illustrated inFIG. 4.

For instance,FIG. 4illustrates a state machine diagram for coefficient significance-driven state machine. In particular, rather than relying upon the parity information, as is done in JVET-J0014, inFIG. 4, the significance information is used to drive the state machine. For instance, unlike the example ofFIG. 3, in which parity information is used to determine the quantization or dequantization factor, in one or more examples, the significance information is used to determine the quantization or dequantization factor.

InFIG. 4, transitioning from one state to another state is based on the current position in the state machine and the significance value. For example, assume that the current position in the state machine is “0.” In this example, if a coefficient is significant (e.g., has an absolute value greater than zero), the state changes to state 2. If a coefficient is not significant, the state remains at state 0.

Assume that the current position in the state machine is “1.” In this example, if a coefficient is significant, the state changes to state 0. If a coefficient is not significant, the state changes to state 2. Assume that the current position in the state machine is “2.” In this example, if a coefficient is significant, the state changes to state 3. If a coefficient is not significant, the state changes to state 1. Assume that the current position in the state machine is “3.” In this example, if a coefficient is significant, the state changes to state 1. If a coefficient is not significant, the state remains at state 3.

The example state diagram illustrated inFIG. 4may be used to understand the quantization or dequantization factor that is selected. For instance, as illustrated, Q0 is associated with states 0 and 1 and Q1 is associated with states 2 and 3. Accordingly, if for the current coefficient the state is 0 or 1, then video encoder200and video decoder300may quantize or dequantize using quantization or dequantization factor Q0. If for the current coefficient the state is 2 or 3, then video encoder200and video decoder300may quantize or dequantize using quantization or dequantization factor Q1.

For example, assume that the state after coding the previous coefficient is state 0. In this example, if the previous coefficient is significant, then the state becomes state 1, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q1. In this example, if the previous coefficient is not significant, then the state remains state 0, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q0.

Assume that the state after coding the previous coefficient is state 1. In this example, if the previous coefficient is significant, then the state becomes state 0, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q0. In this example, if the previous coefficient is not significant, then the state becomes state 2, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q1.

Assume that the state after coding the previous coefficient is state 2. In this example, if the previous coefficient is significant, then the state becomes state 3, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q1. In this example, if the previous coefficient is not significant, then the state becomes state 1, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q0.

Assume that the state after coding the previous coefficient is state 3. In this example, if the previous coefficient is significant, then the state becomes state 1, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q0. In this example, if the previous coefficient is not significant, then the state remains state 3, and the current coefficient is quantized or dequantized using the quantization or dequantization factor Q1.

It should be noted that the state diagram ofFIG. 4is provided for ease of illustration and should not be considered limiting. The state-machine may be different in that transitioning from one state to another may be based on different criteria (e.g., rather than transitioning from state 1 to state 0 when coefficient is significant, the state may transition from state 1 to state 3 when coefficient is significant). The other states may be changed accordingly as well.

In this manner, video encoder200and video decoder300may select at least one of a quantization or dequantization factor for the current coefficient based on whether the previous coefficient has an absolute value greater than zero and at least one of apply the selected quantization or dequantization factor for the current coefficient. As illustrated and described, in addition to whether the previous coefficient is significant, in some examples, the quantization or dequantization factor used for the previous coefficient (e.g., the state before the transition) may be used to select the quantization or dequantization factor. In other words, video encoder200and video decoder300may select at least one of the quantization or dequantization factor based on a quantization or dequantization factor used when quantizing or dequantizing the previous coefficient (e.g., whether the state was state 0, 1, 2, or 3) and whether the previous coefficient has an absolute value greater than zero (e.g., to indicate where the state is to transition to).

As described above, video encoder200and video decoder300may each utilize the example techniques described in this disclosure to determine the context to use for encoding or decoding the current coefficient and to determine the quantization or dequantization factor that is to be used (e.g., for quantizing a coefficient to generate a quantized coefficient for dequantizing a quantized coefficient to generate the coefficient). As described in this disclosure, for video encoder200, the coefficient may refer to the coefficient before or after quantizing. For video decoder300, the coefficient may refer to the quantized coefficient or the result of dequantizing.

In some examples, there may be correlation between the quantization or dequantization factor that is applied and the context. For example, quantization or dequantization factor Q0 may be correlated with the first context set of the plurality of context sets, and quantization or dequantization factor Q1 may be correlated with the second context set of the plurality of context sets.

In some examples, if video encoder200determined that a current coefficient is to be quantized using quantization factor Q0, then video encoder200may determine that the context for encoding the current coefficient is in the first context set. If video encoder200determined that a current coefficient is to be quantized using quantization factor Q1, then video encoder200may determine that the context for encoding the current coefficient is in the second context set. Similarly, if video decoder300determined that a current coefficient is decoded using a context in the first context set, then video decoder300may determine that the current coefficient is to be dequantized using dequantization factor Q0. If video decoder300determined that a current coefficient is decoded using a context in the second context set, then video decoder300may determine that the current coefficient is to be dequantized using dequantization factor Q1.

Accordingly, in implementation, once video encoder200determines the quantization factor, video encoder200may not need to perform additional operations to determine the context set since the context set can be determined based on the quantization factor that video encoder200selected. However, even in such examples, since the quantization factor is based on whether a previous coefficient has an absolute value greater than zero, video encoder200may be considered as selecting, for coding a current coefficient, a context set from a plurality of context sets based on whether a previous coefficient has an absolute value greater than zero.

Also, in implementation, once video decoder300determines the context set from a plurality of context sets, video decoder300may not need to perform additional operations to determine the dequantization factor since the dequantization factor can be determined based on the context that video decoder300selected. However, even in such examples, since the context set is based on whether a previous coefficient has an absolute value greater than zero, video decoder300may be considered as selecting at least one of a quantization or dequantization factor for the current coefficient based on whether the previous coefficient has an absolute value greater than zero.

Therefore, in some examples, video encoder200and video decoder300may be considered as selecting the context set from the plurality of context sets based on the selected quantization or dequantization factor. For instance, video encoder200determines the quantization factor and then determines the context set from the quantization factor. In some examples, video encoder200and video decoder300may be considered as selecting the quantization or dequantization factor based on the selected context set. For instance, video decoder300selects the context set from the plurality of context sets and then determines the dequantization factor from the context set.

On the encoder side (e.g., video encoder200), the trellis to decide the coefficient levels is illustrated inFIG. 5. For instance,FIG. 5illustrates an encoder trellis diagram for coefficient significance driven state machine.FIG. 5illustrates the transition from state to state of the example illustrated inFIG. 4as described above.

As described above, in some examples, the states 0 and 1 use quantizer Q0, and states 2 and 3 use quantizer Q1. For significance map coding, the context set used to code is selected based on the quantizer used, i.e., Qx where x=state>>1. In other words, video encoder200or video decoder300may determine the quantizer (e.g., quantization or dequantization factor) and select the context set from the plurality of context sets, or vice-versa. In some examples, the greater than 1flag (gt1) context set only uses one context set that is identical to the context set used for coding of gt1 flag used in coefficient coding in core experiments (CE) 7.1.2, as part of JVET-J0014.

In one or more example techniques described in this disclosure, to overcome the technical problem noted above, with a technical solution rooted in video coding technology, in a coefficient group regular coded level bins (e.g., sig, gt1, gt2, gt3, gt4) are coded with the same contexts as in CE 7.2.1 (except for gt1, i.e., single set) coefficient by coefficient in the first pass; this is followed by grouped remaining level coding for all coefficients in a coefficient group. The context index derivation is identical to the method used in CE7.2.1 where full coefficient levels are replaced by coefficient levels reconstructed using up to the highest coded regular coded bin (resulting index is equal to the index derived from full coefficient value). The remaining level coding is identical to the scheme used in CE7.1.2 and CE7.2.1, as part of JVET-J0014, where a Rice Parameter used for coding of bypass coded remainder level bins are derived from full value of neighboring reconstructed levels.

For example, in CE 7.2.0, as part of JVET-J0014, the following contexts are used. For significant coefficients (e.g., non-zero), the contexts are sig[0][18], sig[1][18]. For coefficients with value greater than 1 (gt1), the contexts are gt1[0][21], gt1[1][21]. For coefficients with value greater than 2, 3, and 4 (gt2, gt3, gt4), the context is gtX (X being 2, 3, or 4) [21]. In summary, sig and gt1 has 2 sets of contexts, and all the rest have one set.

In examples described in this disclosure, the contexts for significant (sig) are sig[0][18], sig[1][18]. For gt1, the context is gt1[21], and for gtX, the context is gtX[21]. Accordingly, in this disclosure, gt1 has one set. Contexts are neighboring information that maps to a probability that is used during entropy coding. If there are two distinct cases (different statistics), the probabilities should not mix and need separate contexts; otherwise it is better to mix them (e.g., in a single context).

The change in the bitstream format is illustrated by the example ofFIG. 6. For example,FIG. 6illustrates the proposed syntax, grouped bypass coded bins.

In some examples, the gt0, gt1, gt2, gt3, gt4, or some subset like gt0, gt1 or gt0, gt1, gt2, like bit planes can be bitplane by bitplane coded using contexts from the info available up to the ith bitplane that is available (similar to the one described in JEM noted above) followed by remaining level bins and sign bins. This way, there is no interleaving of gtX {X: 0, . . . 4} regular coded bins at the coefficient level. Also, a restriction on a number of total number of regular coded bins can be imposed; that is, once the limit of regular coded bin count is reached within a bitplane, the remainder of the bitplane is also regular coded, and the next bitplane starts with bypass coding of bins.

FIG. 7is a block diagram illustrating an example video encoder200that may perform the techniques of this disclosure.FIG. 7is provided for purposes of explanation and should not be considered limiting of the techniques as broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes video encoder200in the context of video coding standards such as the HEVC video coding standard and the H.266 (e.g., VVC) video coding standard in development. However, the techniques of this disclosure are not limited to these video coding standards, and are applicable generally to video encoding and decoding.

In the example ofFIG. 7, video encoder200includes video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, decoded picture buffer (DPB)218, and entropy encoding unit220.

Video encoder200stores reconstructed blocks in DPB218. For instance, in examples where operations of filter unit216are not needed, reconstruction unit214may store reconstructed blocks to DPB218. In examples where operations of filter unit216are needed, filter unit216may store the filtered reconstructed blocks to DPB218. Motion estimation unit222and motion compensation unit224may retrieve a reference picture from DPB218, formed from the reconstructed (and potentially filtered) blocks, to inter-predict blocks of subsequently encoded pictures. In addition, intra-prediction unit226may use reconstructed blocks in DPB218of a current picture to intra-predict other blocks in the current picture.

Video encoder200represents an example of a device configured to perform example techniques described in this disclosure. For example, video data memory or some other memory may store contexts for a plurality of context sets, each context set including one or more contexts. Quantization unit208may perform quantization using the example techniques described in this disclosure. As one example, quantization unit208may be configured to determine quantization factor Q0 and Q1. Quantization unit208may determine a quantization factor for the current coefficient based on whether a previous coefficient has an absolute value greater than zero. In some examples, quantization unit208may select the quantization factor for the current coefficient based on the quantization factor used when quantizing the previous coefficient and whether the previous coefficient has an absolute value greater than zero, as described above. Quantization unit208may apply the selected quantization factor for the current coefficient.

Entropy encoding unit220may be configured to select, for coding a current coefficient (e.g., significance flag of the current coefficient), a context from the plurality of context sets stored in video data memory or elsewhere based on whether a previous coefficient has an absolute value greater than zero. One way in which entropy encoding unit220may determine the context set is based on the quantization factor used by quantization unit208. For instance, if quantization unit208utilized Q0 based on whether the previous coefficient has an absolute value greater than zero, then entropy encoding unit220may utilize the first context set. If quantization unit208utilized Q1 based on whether the previous coefficient has an absolute value greater than zero, then entropy encoding unit220may utilize the second context set. Since the selection of Q0 or Q1 is based on whether the previous coefficient has an absolute value greater than zero, entropy encoding unit220may be considered as selecting a context set based on whether a previous coefficient has an absolute value greater than zero.

Entropy encoding unit220may determine context from selected context set. For instance, entropy encoding unit220may determine context based on values of neighboring coefficients. The values of neighboring coefficients may be indicative of an offset into the selected context set from which entropy encoding unit220determines the context. Entropy encoding unit220may encode the current coefficient based on the determined context (e.g., perform CABAC operations based on the determined context). In some examples, the current coefficient is a current coefficient of a coding group that includes a plurality of coefficients. Entropy encoding unit220may be configured to encode a significance flag of each respective coefficients in the plurality of coefficients that follow the current coefficient before encoding subsequent flags used to indicate the value of the current coefficient.

In the example ofFIG. 8, video decoder300includes coded picture buffer (CPB) memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and decoded picture buffer (DPB)314. Prediction processing unit304includes motion compensation unit316and intra-prediction unit318. Prediction processing unit304may include addition units to perform prediction in accordance with other prediction modes. As examples, prediction processing unit304may include a palette unit, an intra-block copy unit (which may form part of motion compensation unit316), an affine unit, a linear model (LM) unit, or the like. In other examples, video decoder300may include more, fewer, or different functional components.

Video decoder300may store the reconstructed blocks in DPB314. As discussed above, DPB314may provide reference information, such as samples of a current picture for intra-prediction and previously decoded pictures for subsequent motion compensation, to prediction processing unit304. Moreover, video decoder300may output decoded pictures from DPB for subsequent presentation on a display device, such as display device118ofFIG. 1.

In this manner, video decoder300represents an example of a device configured to perform example techniques described in this disclosure. For example, memory of or coupled to video decoder300may store contexts for a plurality of context sets, each context set including one or more contexts. Entropy decoding unit302may be configured to select, for coding a current coefficient (e.g., significance flag of the current coefficient), a context from the plurality of context sets stored in memory based on whether a previous coefficient has an absolute value greater than zero. For example, if the previous coefficient has an absolute value greater than zero, then entropy decoding unit302may select a first context set. If the previous coefficient has an absolute value that is not greater than zero, then entropy decoding unit302may select a second context set.

Entropy decoding unit302may determine a context from the selected context set. For instance, entropy decoding unit302may determine the context based on values of neighboring coefficients. The values of neighboring coefficients may be indicative of an offset into the selected context set from which entropy decoding unit302determines the context. Entropy decoding unit302may decode the current coefficient based on the determined context (e.g., perform CABAC operations based on the determined context). In some examples, the current coefficient is a current coefficient of a coding group that includes a plurality of coefficients. Entropy decoding unit302may be configured to decode a significance flag of each respective coefficient in the plurality of coefficients that follows the current coefficient before decoding subsequent flags used to indicate the value of the current coefficient.

Inverse quantization unit306may perform dequantization using the example techniques described in this disclosure. As one example, inverse quantization unit306may be configured to determine dequantization factor Q0 and Q1. Inverse quantization unit306may determine a dequantization factor for the current coefficient based on whether a previous coefficient has an absolute value greater than zero. In some examples, inverse quantization unit306may select the dequantization factor for the current coefficient based on the dequantization factor used when dequantizing the previous coefficient and whether the previous coefficient has an absolute value greater than zero, as described above. Inverse quantization unit306may apply the selected dequantization factor for the current coefficient.

One way in which inverse quantization unit306may determine the dequantization factor is based on the context set selected by entropy decoding unit302. For instance, if entropy decoding unit302selected a first context set based on whether the previous coefficient has an absolute value greater than zero, then inverse quantization unit306may utilize the dequantization factor Q0. If entropy decoding unit302selected a second context set based on whether the previous coefficient has an absolute value greater than zero, then inverse quantization unit306may utilize the dequantization factor Q1. Since the selection of the first or second context set is based on whether the previous coefficient has an absolute value greater than zero, inverse quantization unit306may be considered as selecting a dequantization factor based on whether a previous coefficient has an absolute value greater than zero.

FIG. 9is a flowchart illustrating an example method of operation in accordance with one or more example techniques described in this disclosure. For ease of illustration, the techniques are generally described with respect to a video coder, examples of which include video encoder200and video decoder300. At portions of the description, further reference to components of video encoder200and video decoder300is made.

A video coder may select, for coding a current coefficient, a context set from a plurality of context sets based on whether a previous coefficient has an absolute value greater than zero (400). For example, entropy encoding unit220, for coding a significance flag, may select a context set (e.g., a first context set or a second context set) from a plurality of context sets based on whether a previous coefficient has an absolute value greater than zero. As one example, entropy encoding unit220may select the context set from the plurality of context sets based on a quantization factor selected by quantization unit208(e.g., if quantization unit208selected Q0 for the current coefficient, entropy encoding unit220selects a first context set, and if quantization unit208selected Q1 for the current coefficient, entropy encoding unit220selects a second context set).

Entropy decoding unit302may determine the significance flag of the previous coefficient to determine whether the previous coefficient has an absolute value greater than zero. Entropy decoding unit302may select a context set from the plurality of context sets based on the significance flag of the previous coefficient (e.g., based on whether the previous coefficient has an absolute value greater than zero).

The video coder may determine a context from the selected context set (402). For example, entropy encoding unit220and entropy decoding unit302may determine the context from the selected context set based on values of neighboring coefficients. As one example, entropy encoding unit220and entropy decoding unit302may determine if none, one, or both of the right and below coefficients are significant (e.g., absolute value greater than zero), and determine an offset in the selected context set based on the determination if none, one, or both of the right and below coefficients are significant. Entropy encoding unit220and entropy decoding unit302may determine the context based on the offset.

The video coder may code a value for the current coefficient based on the determined context (404). As one example, the video coder may apply CABAC coding techniques to code a significance flag for the current coefficient based on the determined context. In examples where the video coder is video encoder200, video encoder200may encode a value for the current coefficient based on the determined context. In examples where the video coder is video decoder300, video decoder300may decode a value for the current coefficient based on the determined context.

In some examples, the current coefficient is a current coefficient of a coding group that includes a plurality of coefficients. The video coder (e.g., via entropy encoding unit220or entropy decoding unit302) may be configured to code a significance flag of each respective coefficients in the plurality of coefficients that follow the current coefficient before coding subsequent flags used to indicate the value of the current coefficient.

As illustrated inFIG. 9, the video coder may select at least one of a quantization or dequantization factor for the current coefficient based on whether the previous coefficient has an absolute value greater than zero (406). In some examples, the video coder (e.g., inverse quantization unit306) may be configured to select the dequantization factor based on the selected context set selected by entropy decoding unit302. For example, if entropy decoding unit302selected the first context set, then inverse quantization unit306may select the dequantization factor Q0, and if entropy decoding unit302selected the second context set, then inverse quantization unit306may select the dequantization factor Q1.

In some examples, the video coder may select at least one of the quantization or dequantization factor based on a quantization or dequantization factor used when quantizing or dequantizing the previous coefficient (e.g., previously coded coefficient in coding order) and whether the previous coefficient has an absolute value greater than zero. For example, as illustrated inFIG. 4, the video coder may be configured to utilize the state when the previous coefficient was coded and whether the previous coefficient has an absolute value greater than zero to determine the state to transition to, and the state to transition to may indicate the quantization or dequantization factor that is to be used.

The video coder may at least one of apply the selected quantization or dequantization factor for the current coefficient (408). For example, quantization unit208may apply the quantization factor to generate the quantized coefficients that are entropy encoded with entropy encoding unit220. Inverse quantization unit306may apply the dequantization factor to generate coefficients that are fed to inverse transform processing unit308.