Methods for quantization parameter control for video coding with joined pixel/transform based quantization

An example device for processing video data includes memory configured to store the video data and one or more processors coupled to the memory. The one or more processors are configured to adjust a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value and determine a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component. The one or more processors are configured to determine an integer chroma QP offset based on the integer component and determine a fractional chroma QP offset based on the fractional component. The one or more processors are configured to determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset and process the video data based on the DRA chroma scale adjustment value.

TECHNICAL FIELD

BACKGROUND

SUMMARY

In general, this disclosure describes techniques related to the field of coding of video signals with High Dynamic Range (HDR) and Wide Color Gamut (WCG) representations. More specifically, the current disclosure describes techniques for signaling and operations applied to video data in certain color spaces that may enable more efficient compression of HDR and WCG video data. The techniques may improve the compression efficiency of hybrid-based video coding systems utilized for coding HDR & WCG video data.

In video coding, it is beneficial to have harmonized techniques for processing video data. When techniques are not harmonized one video decoder may process the video data differently than intended by a video encoder and differently than another video decoder. The techniques of this disclosure include techniques and constraints for more efficient implementation of dynamic range adjustment (DRA) with harmonized pixel/transform domain quantization and facilitate bit matching across different video decoders.

In one example, a method of processing video data includes adjusting a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value, determining a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component, determining an integer chroma QP offset based on the integer component of the chroma QP, determining a fractional chroma QP offset based on the fractional component of the chroma QP, determining a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset, and processing the video data based on the DRA chroma scale adjustment value.

In another example, a device includes memory configured to store video data and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors being configured to: adjust a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value; determine a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process the video data based on the DRA chroma scale adjustment value.

In another example, a device for processing video data includes means for adjusting a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value, means for determining a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component, means for determining an integer chroma QP offset based on the integer component of the chroma QP, means for determining a fractional chroma QP offset based on the fractional component of the chroma QP, means for determining a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset, and means for processing the video data based on the DRA chroma scale adjustment value.

In another example, a computer-readable storage medium is encoded with instructions that, when executed, cause a programmable processor to adjust a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value, determine a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process the video data based on the DRA chroma scale adjustment value.

As used herein determining may include obtaining, receiving, reading from memory, calculating or the like.

DETAILED DESCRIPTION

In video coding, it is beneficial to have harmonized techniques for processing video data. When techniques are not harmonized one video decoder may process the video data differently than intended by a video encoder and differently than another video decoder. The techniques of this disclosure include techniques and constraints for more efficient implementation of DRA with harmonized pixel/transform domain quantization and facilitate bit matching across different video decoders.

In some video codecs capable of dynamic range adjustment (DRA), mathematical operations associated with DRA may include logarithmic and/or exponential operations. These operations may be expensive to implement and may use an excessive amount of processing power. The techniques of this disclosure may replace the logarithmic and/or exponential operations with approximations of the operations, thereby reducing expense and saving processing power.

In some examples, source device102may output encoded video data to file server114or another intermediate storage device that may store the encoded video data generated by source device102. Destination device116may access stored video data from file server114via streaming or download. File server114may be any type of server device capable of storing encoded video data and transmitting that encoded video data to the destination device116. File server114may represent a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device116may access encoded video data from file server114through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both that is suitable for accessing encoded video data stored on file server114. File server114and input interface122may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

Although not shown inFIG.1, in some examples, video encoder200and video decoder300may each be integrated with an audio encoder and/or audio decoder, and may include appropriate MUX-DEMUX units, or other hardware and/or software, to handle multiplexed streams including both audio and video in a common data stream. If applicable, MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

Video encoder200and video decoder300may operate according to a video coding standard, such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC) or extensions thereto, such as the multi-view and/or scalable video coding extensions. Alternatively, video encoder200and video decoder300may operate according to other proprietary or industry standards, such as ITU-T H.266, also referred to as Versatile Video Coding (VVC). A recent draft of the VVC standard is described in Bross, et al. “Versatile Video Coding (Draft 8),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 17thMeeting: Brussels, BE, 7-17 Jan. 2020, JVET-Q2001-vC (hereinafter “VVC Draft 8”). The techniques of this disclosure, however, are not limited to any particular coding standard.

In some examples, a tile may be partitioned into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a tile may not be referred to as a tile.

The bricks in a picture may also be arranged in a slice. A slice may be an integer number of bricks of a picture that may be exclusively contained in a single network abstraction layer (NAL) unit. In some examples, a slice includes either a number of complete tiles or only a consecutive sequence of complete bricks of one tile.

To perform intra-prediction, video encoder200may select an intra-prediction mode to generate the prediction block. Some examples of EVC provide DC, bi-linear, planar, and thirty angular intra-prediction modes. In general, video encoder200selects an intra-prediction mode that describes neighboring samples to a current block (e.g., a block of a CU) from which to predict samples of the current block. Such samples may generally be above and to the left of the current block in the same picture as the current block, assuming video encoder200codes CTUs and CUs in raster scan order (left to right, top to bottom).

Some examples of VVC provide sixty-seven intra-prediction modes, including various directional modes, as well as planar mode and DC mode. In general, video encoder200selects an intra-prediction mode that describes neighboring samples to a current block (e.g., a block of a CU) from which to predict samples of the current block. Such samples may generally be above, above and to the left, or to the left of the current block in the same picture as the current block, assuming video encoder200codes CTUs and CUs in raster scan order (left to right, top to bottom).

As mentioned above, in some video codecs capable of dynamic range adjustment (DRA), mathematical operations associated with DRA may include logarithmic and/or exponential operations. DRA may be used by a video decoder, such as video decoder300, to adjust the dynamic range of decoded pixels. These operations may be expensive to implement as they may use an excessive amount of processing power. The techniques of this disclosure may replace the logarithmic and/or exponential operations with approximations of the operations, thereby reducing expense and saving processing power.

In accordance with the techniques of this disclosure, a method of processing video data includes adjusting a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value, determining a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component, determining an integer chroma QP offset based on the integer component of the chroma QP, determining a fractional chroma QP offset based on the fractional component of the chroma QP, determining a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset, and processing the video data based on the DRA chroma scale adjustment value. In this manner, mathematical operations that would otherwise be implemented as logarithmic and/or exponential operations are replaced with approximations of the operations, which may reduce expense and save processing power.

In accordance with the techniques of this disclosure, a device for processing video data includes memory configured to store video data and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors being configured to: adjust a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value; determine a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process the video data based on the DRA chroma scale adjustment value.

In accordance with the techniques of this disclosure, a device for processing video data includes means for adjusting a chroma DRA scale value based on a luma dynamic range adjustment (DRA) scale value, means for determining a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component, means for determining an integer chroma QP offset based on the integer component of the chroma QP, means for determining a fractional chroma QP offset based on the fractional component of the chroma QP, means for determining a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset, and means for processing the video data based on the DRA chroma scale adjustment value.

In accordance with the techniques of this disclosure, a computer-readable storage medium is encoded with instructions that, when executed, cause a programmable processor to adjust a chroma DRA scale value based on a luma dynamic range adjustment (DRA) scale value, determine a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process the video data based on the DRA chroma scale adjustment value.

In accordance with the techniques of this disclosure, a method of coding video data includes performing dynamic range adjustment (DRA) on the video data, and coding the video data based on the dynamic range adjustment, wherein the performing DRA comprises utilizing a constrained bitstream of the video data.

In accordance with the techniques of this disclosure, a method of coding video data includes performing DRA on the video data, and coding the video data based on the DRA, wherein the performing DRA comprises determining an integer approximation of an exponential function or a logarithmic function.

In accordance with the techniques of this disclosure, a method of coding video data includes performing DRA on the video data, and coding the video data based on the DRA, wherein the performing DRA comprises determining a sub-sampled representation of an exponential function or a logarithmic function.

In accordance with the techniques of this disclosure, a method of coding video data includes determining whether a scale value or a quantization parameter (QP) value is exactly tabulated, based upon the scale value or QP value not being exactly tabulated, approximating an exponential function or a logarithmic function through linear interpolation, determining an estimated scale value or QP value based on the approximation, and coding the video data based on the estimated scale value or QP value.

In accordance with the techniques of this disclosure, a method of coding video data includes performing DRA on the video data, and coding the video data based on the DRA, wherein the performing DRA comprises converting between scale values or QP values only using integer values.

In accordance with the techniques of this disclosure, a method of coding video data includes any of or any combination of the techniques of this disclosure.

In accordance with the techniques of this disclosure, a device includes memory configured to store video data communicatively coupled to one or more processors implemented in circuitry, the one or more processors being configured to perform any of the techniques of this disclosure.

In accordance with the techniques of this disclosure, a device includes at least one means for performing any of the techniques of this disclosure.

In accordance with the techniques of this disclosure, a computer-readable storage medium is encoded with instructions that, when executed, cause a programmable processor to perform any of the techniques of this disclosure.

FIG.2is a block diagram illustrating an example video encoder200that may perform the techniques of this disclosure.FIG.2is provided for purposes of explanation and should not be considered limiting of the techniques as broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes video encoder200according to the techniques of EVC, VVC (ITU-T H.266, under development), and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video encoding devices that are configured to other video coding standards.

In the example ofFIG.2, video encoder200includes video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, decoded picture buffer (DPB)218, and entropy encoding unit220. Any or all of video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, DPB218, and entropy encoding unit220may be implemented in one or more processors or in processing circuitry. For instance, the units of video encoder200may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, of FPGA. Moreover, video encoder200may include additional or alternative processors or processing circuitry to perform these and other functions.

Video encoder200stores reconstructed blocks in DPB218. For instance, in examples where operations of filter unit216are not needed, reconstruction unit214may store reconstructed blocks to DPB218. In examples where operations of filter unit216are needed, filter unit216may store the filtered reconstructed blocks to DPB218. Motion estimation unit222and motion compensation unit224may retrieve a reference picture from DPB218, formed from the reconstructed (and potentially filtered) blocks, to inter-predict blocks of subsequently encoded pictures. In addition, intra-prediction unit226may use reconstructed blocks in DPB218of a current picture to intra-predict other blocks in the current picture.

Video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform dynamic range adjustment (DRA) on the video data, and encode the video data based on the dynamic range adjustment, wherein the performing DRA comprises utilizing a constrained bitstream of the video data.

Video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform DRA on the video data, and encode the video data based on the DRA, wherein the performing DRA comprises determining an integer approximation of an exponential function or a logarithmic function.

Video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform DRA on the video data, and encode the video data based on the DRA, wherein the performing DRA comprises determining a sub-sampled representation of an exponential function or a logarithmic function.

Video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to determine whether a scale value or a quantization parameter (QP) value is exactly tabulated, based upon the scale value or QP value not being exactly tabulated, approximate an exponential function or a logarithmic function through linear interpolation, determine an estimated scale value or QP value based on the approximation, and encode the video data based on the estimated scale value or QP value.

Video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform DRA on the video data, and encode the video data based on the DRA, wherein the performing DRA comprises converting between scale values or QP values only using integer values.

Video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform any of or any combination of the techniques of this disclosure.

FIG.3is a block diagram illustrating an example video decoder300that may perform the techniques of this disclosure.FIG.3is provided for purposes of explanation and is not limiting on the techniques as broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes video decoder300according to the techniques of EVC, VVC (ITU-T H.266, under development), and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video coding devices that are configured to other video coding standards.

In the example ofFIG.3, video decoder300includes coded picture buffer (CPB) memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and decoded picture buffer (DPB)314. Any or all of CPB memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and DPB314may be implemented in one or more processors or in processing circuitry. For instance, the units of video decoder300may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, of FPGA. Moreover, video decoder300may include additional or alternative processors or processing circuitry to perform these and other functions.

Video decoder300may store the reconstructed blocks in DPB314. For instance, in examples where operations of filter unit312are not performed, reconstruction unit310may store reconstructed blocks to DPB314. In examples where operations of filter unit312are performed, filter unit312may store the filtered reconstructed blocks to DPB314. As discussed above, DPB314may provide reference information, such as samples of a current picture for intra-prediction and previously decoded pictures for subsequent motion compensation, to prediction processing unit304. Moreover, video decoder300may output decoded pictures (e.g., decoded video) from DPB314for subsequent presentation on a display device, such as display device118ofFIG.1. Prior to outputting the decoded pictures from DPB314, video decoder300may apply DRA in DRA unit322to the decoded pictures. As part of applying DRA, DRA unit322may determine a luma adjusted chroma DRA scale value based on a luma DRA scale value. DRA unit322may determine a chroma QP based on the luma adjusted chroma DRA scale. DRA unit322may determine an integer chroma QP offset based on an integer component of the chroma QP. DRA unit322may determine a fractional chroma QP offset based on a fractional component of the chroma QP. DRA unit322may determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset. DRA unit322may also process the video data based on the DRA chroma scale adjustment value.

In this manner, video decoder300represents an example of a video decoding device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to adjust a chroma DRA scale value based on a luma DRA scale value; determine a chroma QP based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process the video data based on the DRA chroma scale adjustment value.

Video decoder300also represents an example of a video decoding device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform dynamic range adjustment (DRA) on the video data, and decode the video data based on the dynamic range adjustment, wherein the performing DRA comprises utilizing a constrained bitstream of the video data.

Video decoder300also represents an example of a device configured to decode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform DRA on the video data, and decode the video data based on the DRA, wherein the performing DRA comprises determining an integer approximation of an exponential function or a logarithmic function.

Video decoder300represents an example of a device configured to decode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform DRA on the video data, and decode the video data based on the DRA, wherein the performing DRA comprises determining a sub-sampled representation of an exponential function or a logarithmic function.

Video decoder300represents an example of a device configured to decode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to determine whether a scale value or a quantization parameter (QP) value is exactly tabulated, based upon the scale value or QP value not being exactly tabulated, approximate an exponential function or a logarithmic function through linear interpolation, determine an estimated scale value or QP value based on the approximation, and decode the video data based on the estimated scale value or QP value.

Video decoder300represents an example of a device configured to decode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform DRA on the video data, and decode the video data based on the DRA, wherein the performing DRA comprises converting between scale values or QP values only using integer values.

Video decoder300represents an example of a device configured to decode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform any of or any combination of the techniques of this disclosure.

High dynamic range (HDR) and wide color gamut (WCG) are now discussed. Next generation video applications may operate with video data representing captured scenery with high dynamic range (HDR) and wide color gamut (WCG). Parameters of the utilized dynamic range and color gamut are two independent attributes of video content, and their specification for purposes of digital television and multimedia services are defined by several international standards. For example, ITU-R Rec. 709 defines parameters for high definition television (HDTV) such as Standard Dynamic Range (SDR) and standard color gamut (SCG) and ITU-R Rec.2020 specifies ultra-high definition television (UHDTV) parameters such as HDR and WCG. There are also other standards development organization (SDO) documents specifying these attributes in other systems, e.g., P3 color gamut is defined in SMPTE-231-2 and some parameters of HDR are defined STMPTE-2084. A brief description of dynamic range and color gamut for video data follows.

Dynamic range is typically defined as the ratio between the minimum and maximum brightness of the video signal. Dynamic range is also measured in terms of ‘f-stop’, where one f-stop corresponds to a doubling of a signal's dynamic range. In MPEG's definition, HDR content is such content that features brightness variation with more than 16 f-stops. In some definitions, levels between 10 and 16 f-stops are considered as intermediate dynamic range, but are considered HDR in other definitions. At the same time, the human visual system (HVS) is capable of perceiving a much larger dynamic range, however the HVS includes an adaptation mechanism to narrow a so-called simultaneous range.

Video application and services may be regulated by Rec.709 and provide SDR, typically supporting a range of brightness (or luminance) of around 0.1 to 100 candelas (cd) per m2 (often referred to as “nits”), leading to less than 10 f-stops. Next generation video services are expected to provide a dynamic range of up-to 16 f-stops and some initial parameters have been specified in SMPTE-2084 and Rec.2020.

FIG.4is a conceptual diagram illustrating an example dynamic range of SDR in HDTV, an expected HDR in UHDTV and the HVS dynamic range. For example, graph400shows luminance of starlight, moonlight, indoors, and sunlight in nits on a logarithmic scale. Graph402shows the range of human vision and the HVS simultaneous dynamic range (also called the steady-state dynamic range) with respect to graph400. Graph404depicts the SDR in HDTV for a display with respect to graph400and graph402. For example, the SDR for a display may overlap a portion of the luminance range of moonlight to indoors and may overlap a portion of the HVS simultaneous range. Graph406depicts the HDR in UHDTV for a display, with respect to graphs400-404. As can be seen, the HDR is much larger than the SDR of graph404and encompasses more of the HVS simultaneous range than the SDR of graph404.

FIG.5is a conceptual diagram illustrating an example color gamut graph. An aspect of leading to a more realistic video experience, other than HDR, is the color dimension, which is conventionally defined by the color gamut. In the example ofFIG.5, a visual representation of SDR color gamut (triangle500based on the BT.709 color red, green and blue color primaries), and the wider color gamut that for UHDTV (triangle502based on the BT.2020 color red, green and blue color primaries).FIG.5also depicts the so-called spectrum locus (delimited by the tongue-shaped area504), representing limits of natural colors. As illustrated byFIG.5, moving from BT.709 (triangle500) to BT.2020 (triangle502) color primaries aims to provide UHDTV services with about 70% more colors. The circle labeled D65 specifies the white color for the given specifications.

A few examples of color gamut specifications are shown in Table 1.

Compression of HDR video data is now discussed. HDR/WCG is typically acquired and stored at a very high precision per component (which may even be stored with floating point precision), with the 4:4:4 chroma format and a very wide color space (e.g., XYZ). This representation targets high precision and is (almost) mathematically lossless. However, this format features many redundancies and is not optimal for compression purposes. A lower precision format with an HVS-based assumption is typically utilized for state-of-the-art video applications.

FIG.6is a block diagram illustrating an example format conversion technique. Video encoder200may perform the format conversion techniques to transform linear RGB510to HDR′ data518. One example of a video data format conversion process for purposes of compression includes three major processes, as shown inFIG.6. The techniques ofFIG.6may be performed by source device12(which may be an example of video encoder200). Linear RGB data510may be HDR/WCG video data and may be stored in a floating point representation. Linear RGB data510may be compacted using a non-linear transfer function (TF)512for dynamic range compacting. Transfer function512may compact linear RGB data510using any number of non-linear transfer functions, e.g., the perceptual quantizer (PQ) TF as defined in SMPTE-2084. In some examples, color conversion process514converts the compacted data into a more compact or robust color space (e.g., a YUV or YCrCb color space) that is more suitable for compression by a hybrid video encoder. A hybrid video encoder, such as video encoder200, is a video encoder that utilizes prediction when encoding video data. This more compact data may be quantized using a floating-to-integer representation quantization unit516to produce converted HDR′ data518. In this example HDR′ data518is in an integer representation. The HDR′ data is now in a format more suitable for compression by a hybrid video encoder (e.g., video encoder200). The order of the processes depicted inFIG.6is given as an example, and may vary in other applications. For example, color conversion may precede the TF process. In addition, additional processing, e.g. spatial subsampling, may be applied to color components.

FIG.7is a conceptual diagram illustrating an example inverse HDR/WCG conversion. The techniques ofFIG.7may be performed by destination device14(which may be an example of video decoder300). Converted HDR′ data520may be obtained at destination device14through decoding video data using a hybrid video decoder (e.g., video decoder300). A hybrid video decoder is a video decoder that utilizes prediction when decoding video data. Destination device14may inverse quantize HDR′ data520through inverse quantization unit522. Then an inverse color conversion process524may be applied to the inverse quantized HDR′ data. The inverse color conversion process524may be the inverse of color conversion process514. For example, the inverse color conversion process524may convert the HDR′ data from a YCrCb format back to an RGB format. Inverse transfer function526may be applied to the data to add back the dynamic range that was compacted by transfer function512to recreate the linear RGB data528.

The order of the techniques ofFIGS.6and7, is provided as an example, and the order may vary in real-world applications, e.g., color conversion may precede the TF module, as well as additional processing, e.g., spatial subsampling may be applied to color components. These three components of the techniques ofFIGS.6and7are described in more detail below.

The techniques depicted inFIG.6are now be discussed in more detail. The TF is first described. Video encoder200may apply a TF to video data to compact the dynamic range of the video data and make it possible to represent the video data with a limited number of bits. This function is typically a one-dimensional (1D) non-linear function either reflecting an inverse of electro-optical transfer function (EOTF) of the end-user display as specified for SDR in Rec.709 or approximating the HVS perception to brightness changes as in the PQ TF specified in SMPTE-2084 for HDR. The inverse process of the opto-electronic transfer function (OETF) is the EOTF, which maps the code levels back to luminance. For example, video decoder300may apply an OETF that corresponds to the EOTF applied by video encoder200.

FIG.8is a conceptual diagram illustrating an example of EOTFs. Graph600ofFIG.8depicts code levels on the x-axis and units of linear luminance on the y-axis and example EOTFs.

The Specification, ST2084, defines the EOTF application as follows: TF is applied to normalized linear R, G, B values which results in nonlinear representations of R′G′B′. ST2084 defines normalization by NORM=10000, which is associated with a peak brightness of 10000 nits (cd/m2).

FIG.9is a graphical diagram illustrating example normalized output nonlinear values based on normalized linear input values. Graph700ofFIG.9depicts input values (linear color value) normalized to a range of 0 . . . 1 and normalized output values (nonlinear color value) using a PQ EOTF. As depicted inFIG.9, 1 percent (0.01) (low illumination) of dynamical range of the input signal is converted to 50% (0.5) of dynamical range of output signal.

Typically, an EOTF is defined as a function with floating point accuracy. Thus, no error is introduced to a signal with this non-linearity if video decoder300applies the inverse TF of the corresponding OETF. The inverse TF (OETF) specified in ST2084 is defined as inversePQ function:

With floating point accuracy, sequential application of EOTF and OETF may provide a perfect reconstruction without errors. However, this representation is not optimal for streaming or broadcasting services because floating point representation may include the use of a relatively large number of bits when compared to a fixed point representation. A more compact representation using fixed bit accuracy of nonlinear R′G′B′ data is described later herein.

Note, that EOTF and OETF are subjects of active research, and the TF utilized in some HDR video coding systems may be different from ST2084.

Color transform techniques are now described. RGB data is typically utilized as an input, since RGB data is typically produced by image capturing sensors. However, the RGB color space has high redundancy among RGB components and may not be optimal for compact representation. To achieve more compact and more robust representation, RGB components are typically converted to a more uncorrelated color space that is more suitable for compression, e.g., YCbCr. This color space separates the brightness in the form of luminance and color information in different un-correlated components.

With modern video coding systems, the typically used color space is YCbCr, as specified in ITU-R BT.709 or ITU-R BT.2020. The YCbCr color space in the BT.709 standard specifies the following conversion process from R′G′B′ to Y′CbCr (non-constant luminance representation):

The above can also be implemented using the following approximate conversion that avoids the division for the Cb and Cr components:
Y′=0.212600*R′+0.715200*G′+0.072200*B′
Cb=−0.114572*R′−0.385428*G′+0.500000*B′
Cr=0.500000*R′−0.454153*G′−0.045847*B′(4)

The ITU-R BT.2020 standard specifies the following conversion process from R′G′B′ to Y′CbCr (non-constant luminance representation):

The above can also be implemented using the following approximate conversion that avoids the division for the Cb and Cr components:
Y′=0.262700*R′+0.678000*G′+0.059300*B′
Cb=−0.139630*R′−0.360370*G′+0.500000*B′
Cr=0.500000*R′−0.459786*G′−0.040214*B′(6)

It should be noted, that both color spaces remain normalized, therefore, for the input values normalized in the range 0 . . . 1 the resulting values may be mapped to the range 0 . . . 1. Generally, color transforms implemented with floating point accuracy provide perfect reconstruction, thus this process may be lossless.

Quantization (or fixed point conversion) is now described in more detail. All processing stages described above are typically implemented in floating point accuracy representation, and thus may be considered lossless. However, floating point accuracy can be considered expensive for most consumer electronics applications. Input data in a target color space may be converted to a target bit-depth fixed point accuracy and thereby save bandwidth and memory. Certain studies show that 10-12 bits accuracy in combination with the PQ TF is sufficient to provide HDR data of 16 f-stops with distortion below the Just-Noticeable Difference. Data represented with 10 bits accuracy can be further coded with most of the state-of-the-art video coding solutions. This conversion process includes signal quantization and is an element of lossy coding and is a source of inaccuracy introduced to converted data.

An example of such quantization applied to code words in a target color space, such as YCbCr, is shown below. Input values YCbCr represented in floating point accuracy may be converted into a signal of fixed bit-depth BitDepthYfor the Y value and BitDepthc for the chroma values (Cb, Cr).
DY′=Clip1Y(Round((1<<(BitDepthY−8))*(219*Y′+16)))
DCb=Clip1C(Round((1<<(BitDepthC−8))*(224*Cb+128)))
DCr=Clip1C(Round((1<<(BitDepthC−8))*(224*Cr+128)))  (7)withRound(x)=Sign(x)*Floor(Abs(x)+0.5)Sign (x)=−1 if x<0, 0 if x=0, 1 if x>0Floor(x) the largest integer less than or equal to xAbs(x)=x if x>=0, −x if x<0Clip1Y(x)=Clip3(0, (1<<BitDepthY)−1, x)Clip1C(x)=Clip3(0, (1<<BitDepthc)−1, x)Clip3(x,y,z)=x if z<x, y if z>y, z otherwise

In the document, Dynamic Range Adjustment SEI to enable High Dynamic Range video coding with Backward-Compatible Capability, D. Rusanovskyy, A. K. Ramasubramonian, D. Bugdayci, S. Lee, J. Sole, M. Karczewicz, VCEG document COM16-C 1027-E, September 2015, the authors proposed to implement dynamic range adjustment (DRA) as a piece-wise linear function f(x) that is defined for a group of non-overlapped dynamic range partitions (ranges) {Ri} of input value x, were i is an index of the range with range of 0 to N−1, inclusive, and where N is the total number of ranges {Ri} utilized for defining the DRA function. Assuming that ranges of the DRA are defined by a minimum and a maximum x value that belongs to the range Ri, e.g. [xi, xi+1−1], where xiand xi+1denote minimum values of the ranges Riand Ri+1, respectively. Applied to the Y color component of the video (luma), DRA function Sy is defined through a scale Sy,iand offset Oy,iwhich are applied to every x∈[xi, xi+1−1], thus Sy={Sy,i, Oy,i}.

Thus, for any Ri, and every x∈[xi, xi+1−1], the output value X is calculated as follows:
X=Sy,i*(x−Oy,i)  (8)

For the inverse DRA mapping process for the luma component Y conducted at the decoder (e.g., video decoder300), the DRA function Sy is defined by inverse of scale Sy,iand offset Oy,ivalues which are applied to every X∈[Xi, Xi+1−1].

Thus, for any Ri, and every X∈[Xi, Xi+1−1], reconstructed value x is calculated as follows:
x=X/Sy,i+Oy,i(9)

The forward DRA mapping process (conducted by video encoder200) for chroma components Cb and Cr is defined as follows: an example is given with term “u” denoting sample of Cb color component that belongs to the range Ri, u∈[ui, ui+1−1], thus Su={Su,i, Ou,i}:
u=Su,i*(u−Oy,i)+Offset,  (10)

where Offset is equal to 2(bitdepth−1)and denotes the bi-polar Cb, Cr signal offset.

The inverse DRA mapping process conducted at the decoder, e.g., video decoder300, for chroma components Cb and Cr is defined as follows: An example is given with U denoting a sample of remapped Cb color components which belongs to the range Ri, U∈[Ui, Ui+1−1]:
u=(U−Offset)/Su,i+Oy,i(11)

where Offset is equal to 2(bitdepth−1)and denotes the bi-polar Cb, Cr signal offset.

Luma-driven chroma scaling (LCS) is now described. LCS was initially proposed in JCTVC-W0101 HDR CE2: Report on CE2.a-1 LCS, A. K. Ramasubramonian, J. Sole, D. Rusanovskyy, D. Bugdayci, M. Karczewicz. In that paper, a technique to adjust chroma information, e.g. Cb and Cr, by exploiting brightness information associated with the processed chroma sample was disclosed. Similar to the DRA approach discussed above, the LCS proposal was to apply to a chroma sample, a scale factor Sufor Cb and Svfor Cr. However, instead of defining DRA function as piece-wise linear function Su={Su,i,Ou,i} for a set of ranges {Ri} accessible by chroma value u or v as in Equations (8) and (9), the LCS approach proposed to utilize a luma value Y to derive a scale factor for a chroma sample. Video encoder200may perform forward LCS mapping of the chroma sample u (or v) through the following formula:
U=Su,i(Y)*(u−Offset)+Offset  (12)

Video decoder300may perform the inverse LCS process conducted through the following formula:
u=(U−Offset)/Su,i(Y)+Offset  (13)

In more detail, for a given pixel located at (x, y), chroma samples Cb(x, y) or Cr(x, y) may be scaled with a factor derived from the pixel's LCS function SCb(or SCr) accessed by the pixel's luma value Y′(x, y).

With the forward LCS, for chroma samples, Cb (or Cr) values, and their associated luma value Y′ may be an input to the chroma scale function SCb(or SCr) and Cb or Cr may be converted into Cb′ and Cr′ as shown in Equation 14. Video decoder300may apply the inverse LCS, and reconstructed Cb′ or Cr′ may be converted to Cb, or Cr as it shown in Equation (15).

Cb′(x,y)=SC⁢b(Y′(x,y))*C⁢b⁡(x,y),Cr′(x,y)=SCr(Y′(x,y))*C⁢r⁡(x,y)(14)Cb⁡(x,y)=Cb′(x,y)SC⁢b(Y′(x,y))⁢C⁢r⁡(x,y)=Cr′(x,y)SC⁢r(Y′(x,y))(15)
The values SCb(Y′(x,y)) and SCr(Y′(x,y)) denote luma adjusted chroma DRA scale values.

FIG.10is a graphical diagram illustrating an example of an LCS function. With LCS function800in the example ofFIG.10, chroma components of pixels with smaller values of luma are multiplied with smaller scaling factors.

The relationship between DRA sample scaling and quantization parameters is now described. To adjust a compression ratio, block transform based video coding schemes, such as HEVC and VVC, utilize a scalar quantizer which is applied to transform coefficients. The scalar is controlled with a Quantization Parameter (QP) with the relationship between the QP and the scalar quantizer being defined as following:
scale=exp(QP/6)*log(2.0))  (16)

In some examples, scale may also be referred to as scaler.

The inverse function defines the relationship between scalar quantizer and QP of HEVC as follows:
QP=log2(scale)*6;  (17)

DRA, which effectively scales the pixel data and taking into consideration transform properties, can be mapped for a large class of signals to the scale applied in the transform domain. Thus, the following relationship is defined:
dQP=log2(scaleDRA)*6;  (18)
where dQP is an approximate QP offset (e.g., introduced by HEVC) by deploying DRA on the input data.

DRA scale compensation for decoder QP handling is now described. DRA scales for color components may be adjusted to compensate QP handling in a video codec.

Parameters of DRA for 3 color components (e.g., Y, Cb, Cr) may be defined through the following variables as described above:
DRAy={Sy,i,Oy,i}
DRACb={SCb,i,OCb,i}
DRACr={SCr,i,OCr,i}  (19)

Video encoder200may signal DRA parameters relating to pixel processing through the coded bitstream and video decoder300may derive the DRA parameters from syntax elements signaled in the bitstream. These DRA parameters are further adjusted by taking into consideration information describing quantization of the transform coefficients.
DRA′Cb=fun(DRACb,QPx)
DRA′Cr=fun(DRACr,QPx),  (20)
where QPx are QP adjustments or manipulations conducted by a codec for a given block of pixels and signaled to the decoder in the bitstream or provided to the decoder as side information, e.g., as pre-tabulated information and fun( . . . ) represents some functional dependency.

The output of this process are adjusted DRA parameters (DRA′Cb, DRA′Cr) which are to be applied to the decoded samples (Cbdec, Crdec).
Cbo=fun(DRA′Cb,Cbdec)
Cro=fun(DRA′Cr,Crdec)  (21)

QP information may be adjusted to reflect the impact of DRA applied to pixels. For example, video decoder300may alter QP information to reflect the impact of the DRA applied to pixels of the decoded picture.
QP′Cb=fun(QPx,DRACb)
QP′Cr=fun(QPx,DRACr)  (22)
where QPx are QP parameters derived by video decoder300without taking into consideration scaling implemented by DRA processing to current processed pixels.

The output of this process may be adjusted QP values (Q′y, QP′Cb, QP′Cr) which are utilized in a decision making process by video decoder300. In some examples, only a subset of the techniques in the decoding algorithm will use the adjusted QP in the decision making process.

How DRA scales compensate for the chroma QP shift table is now discussed. Video decoder300may derive parameters mentioned below based on local QP information derived from syntax elements of the decoded bitstream and may further alter the parameters based on side information available to video decoder300.

Examples of such processing are discussed in the HEVC specification (which is available at https://www.itu.int/rec/T-REC-H.265) clause 8.6.1:The variables qPCband qPCrare derived as follows:
qPiCb=Clip3(−QpBdOffsetc, 57,QpY+pps_cb_qp_offset+slice_cb_qp_offset+CuQ pOffsetCb)  (8-257)
qPiCr=Clip3(−QpBdOffsetc,57,QpY+pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffsetCr)  (8-258)If ChromaArrayType is equal to 1, the variables qPCband qPCrare set equal to the value of Qpc as specified in Table 8-10 based on the index qPi equal to qPiCband qPiCr, respectively.Otherwise, the variables qPCband qPCrare set equal to Min(qPi, 51), based on the index qPi equal to qPiCband qPiCr, respectively.The chroma quantization parameters for the Cb and Cr components, Qp′Cband Qp′Cr, are derived as follows:
Qp′Cb=qPCb+QpBdOffsetc(8-259)
Qp′Cr=qPCr+QpBdOffsetc(8-260)

FIG.11is a conceptual diagram illustrating a table describing the specification of QpC as a function of qPi for ChromaArrayType equal to 1. For example, table810ofFIG.11may be Table 8-10 of the HEVC specification.

In such examples, DRA scale parameters for chroma components may be altered to reflect the QP shift introduced by such processing. The following example is given for the Cb component. Derivations for Cr components are similar.

Video decoder300may derive a chroma quantization parameter for Cb component from Table 8-10 achievable with QP information. Video decoder300may estimate:
estimateQP1=qPcb+QpBdPffsetC
updatedQP1=fun(Table8-10,estimateQP1)  (23)
shiftQP1=updatedQP1−estimateQP1;

A variable updatedQP1 may be further used in the decoding process and shiftQP1 provides estimates for impact on the QP introduced by Table 8-10.

To harmonize pixel-level quantization conducted by DRA and QP handling in the decoder (e.g., video decoder300), the DRA scaling function is altered as follows:
estimateQP2=qPcb+QpBdPffsetC+scale2QP(DRACb)  (24)

where scale2QP(DRACb) conducts conversion from Scale to QP, similarly as shown in Equation (18).
updatedQP2=fun(Table8-10,estimateQP2)  (25)
shiftQP2=updatedQP2−estimateQP2;

In some examples, such as in the case of cross-component DRA implementation (e.g., LCS), Equation (24) will include a QP offset value estimated from the DRA scale of the Y component and additional QP offset values estimated from the chromaticity scale (addnDRACb Scale) used to produce DRA for Cb component. E.g.,
estimateQP2=qPcb+QpBdPffsetC+scale2QP(DRAY)+scale2QP(addnDRACbScale)

A variable updatedQP2 provides an estimate for QP in the case where DRA would be conducted through transform domain scaling and shiftQP2 provides an estimate of the impact on the QP introduced by Table 8-10.

In some circumstances, estimated shiftQP1 would not be equal to shiftQP2. To compensate for this difference, scales of DRA can be altered with a multiplicator as follows:
shiftScale=Qp2Scale(shiftQP2−shiftQP1)
DRACb′=shiftScale*DRACb(26)
where function Qp2Scale converts QP variable to associated quantizer scale as shown in Eq. 26.

The output of this process may be an adjusted DRA scale which is applied to the decoded samples Cbdec.

In some examples, the output of the scale to the QP conversion function scale2QP(DRACb) and the resulting estimateQP2 are non-integer values. In order to address elements of Table 8-10, input and output QP values to Table 8-10 may be interpolated between integer entries as follows:
qp1=fun(Table8-10,(Int)(estimateQP2));
qp2=fun(Table8-10,(Int)(estimateQP2+1.0));  (27)
shiftQP2=qp1+(qp2−qp1)*(estimateQP2−(Int)(estimateQP2));
where (Int)(x) denotes an integer component of x.

In yet another example, entries of Table 8-10 (or similar tabulated information) may be defined through an analytical function, or may be explicitly signaled in the bitstream.

In yet another example, the shiftScale can be computed to compensate for the impact of Table8-10 shiftQP1 as follows:
shiftScale=Qp2Scale(shiftQP1)

In some examples, a QP index for initializing Equations 23 and 24 may be signaled through the bitstream in order to avoid parsing and processing dependencies.

Harmonization of DRA scaling (implementing pixel level quantization) and transform domain quantization of the hybrid video codec utilizes conversion of values of scale/offset to the quantization parameter (QP) and the inverse QP. This process involves usage of log and exponential expressions that are expensive to implement in computational platforms, such as a video codec, with integer arithmetic. Such expressions include:
scale=exp(QP/6*log(2.0))  (28)
QP=log2(scale)*6;  (29)
wherein log(·) is the natural logarithm.

This disclosure describes techniques and constraints for more efficient implementation of DRA with harmonized pixel/transform domain quantization.

In some examples, this disclosure describes introducing bitstream constraints to the DRA specification that allow more efficient implementation of DRA. In some examples, this disclosure describes introducing integer approximation of exponential and log functions, e.g., through a limited number of tabulated values. In some examples, this disclosure describes utilizing sub-sampled representations of exponential and/or logarithmic functions. In some examples, this disclosure describes utilizing linear interpolation techniques to approximate exponential and/or log functions to estimate scale or QP values that may not be exactly tabulated (e.g., that do not appear in a table). In some examples, this disclosure describes specifying the accuracy of integer representation required for conversion between scale and QP values.

For example, video decoder300may determine a luma adjusted chroma DRA scale value based on a luma DRA scale value; determine a chroma QP based on the luma adjusted chroma DRA scale, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process the video data based on the DRA chroma scale adjustment value. In this manner, mathematical operations that would otherwise be implemented as logarithmic and/or exponential operations are replaced with approximations of the operations, which may reduce expense and save processing power.

An example of specification text follows. Details within the example of the specification text may be changed and still fall within the scope of this disclosure.

Derivation of adjusted chroma DRA scales

Inputs to this process are: variable denoting luma scales, lumaScale, and chroma component index cIdx. Output of this process is: a variable denoting chroma scales, chromaScale.

Variables scaleDra and scaleDrallorm are derived as follows:

scaleDra denotes a luma adjusted chroma DRA scale value which is determined based on the luma DRA scale value lumaScale. dra_cb_scale_value and dra_cr_scale_value denote initial chroma (Cb and Cr, respectively) DRA scale values, e.g., without consideration of the luma DRA scaling effects.

The variable IndexScaleQP is derived by invoking clause 8.9.5 of MPEG-5 EVC, ISO/IEC 23094-1:2020 Information Technology—General Video Coding—Part 1: Essential Video Coding (available at https://www.iso.org/standard/57797.html) for input value inValue set equal to scaleDrallorm, scaleQp array and the size of the scaleQp array set equal to 54 as input.

Variable qpDraInt is derived as follows:
qpDraInt=2*IndexScaleQP−60

The variables qpDraInt and qpDraFrac are derived as follows:
tableNum=scaleDrallorm−scaleQp[IndexScaleQP]
tableDelta=scaleQp[IndexScaleQP+1]−scaleQp[IndexScaleQP]

scaleQp is a table that provides an integer approximation of the conversion from the the luma adjusted chroma DRA scale value to a chroma quantization parameter according to Equation 29.

When tableNum is equal to 0, the variable qpDraFrac is set equal to 0, and the variable qpDraInt is decreased by 1, otherwise the variables qpDraInt and qpDraFrac are derived as follows:
qpDraFrac=(tableNum<<10)/tableDelta
qpDraInt+=qpDraFrac>>9
qpDraFrac=(1<<9)−qpDraFrac%(1<<9)

qpDraInt and qpDraFrac denote integer and fractional components, respectively, of a chroma QP determined based on the luma adjusted chroma DRA scale scaleDra.
idx0=Clip3(−QpBdOffsetc,57,dra_table_idx−qpDraInt)
idx1=Clip3(−QpBdOffsetc,57,dra_table_idx−qpDraInt+1)
dra_table_idx may denote a starting position in the chroma QP table as parsed from the bitstream.
Qp0=ChromaQpTable[cIdx][idx0].
Qp1=ChromaQpTable[cIdx][idx1].
qpDraIntAdj=((Qp1−Qp0)*qpDraFrac)>>9
qpDraFracAdj=qpDraFrac−(((Qp1−Qp0)*qpDraFrac) %(1<<9))
The indices idx0 and idx1 are determined based on the integer component qpDraInt of the chroma QP and used to index the ChromaQpTable to determine an integer chroma QP offset. qpDraIntAdj may denote an integer chroma QP offset. qpDraFracAdj may denote a fractional chroma QP offset.
draChromaQpShift=chromaQpTable[cIdx][dra_table_idx]−Qp0−qpDraIntAdj−qpDraInt

When draChromaQpShift is less than 0, variable draChromaScaleShiftFrac is derived as follows:
draChromaScaleShiftFrac=qpScale[idx0]−qpScale[idx1]
otherwise, variable draChromaScaleShiftFrac is derived as follows:
draChromaScaleShiftFrac=qpScale[idx2]−qpScale[idx0]

Variable draChromaScaleShift is modified as follows:
draChromaScaleShift=draChromaScaleShift+(draChromaScaleShiftFrac*qpDraFracAdj+(1<<8))>>9

Output variable chromaScale is derived as follows:
chromaScale=(scaleDra*draChromaScaleShift)+(1<<17))>>18
draChromaScaleShift denotes a DRA chroma scale adjustment value which is used to modify the luma adjusted chroma DRA scale value scaleDra to derive a modified chroma scale chromaScale.

The entries of scaleQp and qpScale tables are initialized as follows:

chroma_qp_table_present_flag equal to 1 specifies that chroma QP mapping tables ChromaQpTable are signaled and used to derive Qpc instead of derivation defined by Table 8-19 as shown inFIG.12.

chroma_qp_table_present_flag equal to 0 specifies that the chroma QP mapping tables are not signaled and that Table 8-19 is used for deriving the chroma QP values Qpc.FIG.12is a conceptual diagram illustrating a table describing the specification of QpC as a function of qPi (sps_iqt_flag==1), chromaQpTable[ ] for chroma_qp_table_present_flag equal to 0. In some examples, table819ofFIG.12may represent Table 8-19.

dra_descriptor1 specifies the number of bits used to represent the integer part of the DRA scale parameters signaling. The value of dra_descriptor1 shall be in the range of 0 to 15, inclusive. In the current version of the specification value of syntax element dra_descriptor1 is restricted to 4; other values are reserved for future use.

dra_descriptor2 specifies the number of bits used to represent the fractional part of the DRA scale parameters signaling and the reconstruction process. The value of dra_descriptor2 shall be in the range of 0 to 15, inclusive. In the current version of the specification value of syntax element dra_descriptor2 is restricted to 9; other values are reserved for future use.

The variable numBitsDraScale is derived as follows:
numBitsDraScale=dra_descriptor1+dra_descriptor2
It is a requirement of bitstream conformance that the value of numBitsDraScale shall be greater than 0.

dra_number_ranges_minus1 plus 1 specifies the number of ranges signaled to describe the DRA table. The value of dra_number_ranges_minus1 shall be in the range of 0 to 31, inclusive.

dra_equal_ranges_flag equal to 1 specifies that the DRA table is derived using equal-sized ranges, with size specified by the syntax element dra delta range[0]. dra_equal_ranges flag equal to 0 specifies that the DRA table is derived using dra_number_ranges, with the size of each of the ranges specified by the syntax element dra_delta_range[j].

dra_global_offset specifies the starting codeword position utilized to derive DRA table and initializes the variable inDraRange[0] as follows:
inDraRange[0]=dra_global_offset
The number of bits used to signal dra_global_offset is BitDepthYbits.

dra_delta_range[j] specifies the size of the j-th range in codewords which is utilized to derive the DRA table. The value of dra_delta_range[j] shall be in the range of 1 to (1<<BitDepthY)−1, inclusive.

The variable inDraRange[j] for j in the range of 1 to dra_number_ranges_minus1+1, inclusive, is derived as follows:
inDraRange[j]=inDraRange[j−1]+(dra_equal_ranges_flag==1)?dra_delta_range[0]:dra_delta_range[j−1]
It is a requirement of bitstream conformance that inDraRange[j] shall be in the range 0 to (1<<BitDepthY)−1, inclusive.

dra_scale_value[j] specifies the DRA scale value associated with j-th range of the DRA table. The number of bits used to signal dra scale value[j] is equal to numBitsDraScale.

dra_cb_scale_value specifies the scale value for chroma samples of the Cb component utilized to derive the DRA table. The number of bits used to signal dra_cb_scale_value is equal to numBitsDraScale.

dra_cr_scale_value specifies the scale value for chroma samples of the Cr component utilized to derive the DRA table. The number of bits used to signal dra_cr_scale_value is equal to numBitsDraScale.

The values of dra scale value[j] for j in the range of 0 to dra_number_ranges_minus1, inclusive, dra cb scale value and dra_cr_scale_value shall not be equal to 0. In the current version of the specification, the value of syntax elements dra scale value[j], dra_cb_scale_value and dra_cr_scale_value shall be less than 4<<dra_descriptor2. other values are reserved for future use.

dra_table_idx specifies the access entry of the ChromaQpTable utilized to derived the chroma scale values. The value of dra_table_idx shall be in the range of 0 to 57, inclusive.

The variable numOutRangesL is set equal to dra_number_ranges_minus1+1. The variable outRangesL[adaptation_parameter_set_id][0] is set to 0, the variable outRangesL[i], for i in the range of 0 to numOutRangesL, inclusive, is derived as follows.
outDelta=dra_scale_value[i−1]*(inDraRange[i]−inDraRange[i−1])outRangesL[adaptation_parameter_set_id][i]=outRangesL[adaptation_parameter_set_id][i−1]+outDelta

The variables denoting DRA scale and offset values,

InvLumaScale[adaptation_parameter_set_id][i], and

DraOffsets[adaptation_parameter_set_id][i] respectively, for i in the range of 0 to dra_number_ranges_minus1, inclusive, are derived as follows:

The variable outRangesL[i], for i in the range of 0 to numOutRangesL, inclusive, are modified as follows:
outRangesL[adaptation parameter set id][i]=(outRangesL[adaptation_parameter_set_id][i]+(1<<(dra_descriptor2−1)))>>dra_descriptor2

The process of deriving output chroma DRA parameters in clause 8.9.7 of MPEG-5 EVC, ISO/IEC 23094-1:2020 Information Technology—General Video Coding—Part 1: Essential Video Coding (available at https://www.iso.org/standard/57797.html) is invoked with array of luma scale values dra scale val[ ], array of inverse luma scale value InvLumaScale[ ][ ], array of luma range values outRangesL[ ], and chroma index cIdx as inputs and array of chroma scale values outScalesC[ ][ ], array of chroma offset values outOffsetsC[ ][ ] and array of chroma range values outRangesC[ ][ ] as outputs.

End of example specification text.

An example software implementation follows. Details within the example software implementation may be changed and still fall within the scope of this disclosure.

End of the example software implementation.

FIG.13is a block diagram of a video encoder and video decoder system including DRA units. A video encoder, such as video encoder200, may include forward DRA unit240and coding core242. In some examples, coding core242may include the units depicted inFIG.2and may function as described above with respect toFIG.2. Video encoder200may also determine a plurality of adaptation parameter sets (APSs)244and a plurality of picture parameter sets (PPSs)246that may include information from forward DRA unit240.

A video decoder, such as video decoder300, may include coding core348and output DRA unit342. In some examples, coding core348may include the units depicted inFIG.3and may function as described above with respect toFIG.3. Video decoder300may also determine a plurality of APSs344and a plurality of PPSs346which may include information to be used by output DRA unit342.

According to the techniques of this disclosure, output DRA unit342may determine a luma adjusted chroma DRA scale value based on a luma DRA scale value. Output DRA unit342may determine a chroma QP based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component. Output DRA unit342may determine an integer chroma QP offset based on the integer component of the chroma QP. Output DRA unit342may also determine a fractional chroma QP offset based on the fractional component of the chroma QP. Output DRA unit342may determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset, and process the video data based on the DRA chroma scale adjustment value.

FIG.14is a flowchart illustrating example DRA chroma scale techniques according to this disclosure. Video decoder300may determine a luma adjusted chroma DRA scale value based on a luma DRA scale value (330). For example, video decoder300may determine the luma adjusted chroma DRA scale value through scaleDra=lumaScale*((cIdx==0)?dra_cb_scale_value: dra_cr_scale_value) wherein lumaScale denotes the luma DRA scale value, dra_cb_scale_value and dra_cr_scale_value denote initial chroma (Cb and Cr, respectively) DRA scale values, e.g., without consideration of the luma DRA scaling effects, and scaleDra denotes the chroma DRA scale value adjusted based on the luma DRA scale value.

Video decoder300may determine a chroma QP based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component (332). For example, video decoder300may derive the variable IndexScaleQP by invoking clause 8.9.5 of MPEG-5 EVC, ISO/IEC 23094-1:2020 Information Technology—General Video Coding—Part 1: Essential Video Coding (available at https://www.iso.org/standard/57797.html) the input value inValue set equal to scaleDrallorm, scaleQp array and the size of the scaleQp array set equal to 54 as an input. Video decoder300may derive the variable qpDraInt as follows:

qpDraInt=2*IndexScaleQP−60. Video decoder300may derive the variables qpDraInt and qpDraFrac as follows:
tableNum=scaleDrallorm−scaleQp[IndexScaleQP]
tableDelta=scaleQp[IndexScaleQP+1]−scaleQp[IndexScaleQP]
When tableNum is equal to 0, video decoder300may set the variable qpDraFrac equal to 0, and decrease the variable qpDraInt by 1, otherwise video decoder300may derive the variables qpDraInt and qpDraFrac as follows:
qpDraFrac=(tableNum<<10)/tableDelta
qpDraInt+=qpDraFrac>>9
qpDraFrac=(1<<9)−qpDraFrac %(1<<9).
The integer and fractional components of the chroma QP may be denoted by qpDraInt and qpDraFrac, respectively.

Video decoder300may determine an integer chroma QP offset based on the integer component of the chroma QP (334). For example, video decoder300may parse a syntax element dra_table_idx and determine idx0=Clip3(−QpBdOffsetC, 57, dra_table_idx−qpDraInt) and idx1=Clip3(−QpBdOffsetC, 57, dra_table_idx−qpDraInt+1) and determine (e.g., look up) value in a table(s), such as a ChromaQpTable when determining the integer chroma QP offset. The indices idx0 and idx1 are determined based on the integer component qpDraInt of the chroma QP and used to index the ChromaQpTable to determine an integer chroma QP offset. qpDraIntAdj may denote an integer chroma QP offset.

Video decoder300may determine a fractional chroma QP offset based on the fractional component of the chroma QP (336). For example, video decoder300may use the fractional component and the integer defined table ChromaQpTable to determine the fractional chroma QP offset. qpDraFracAdj may denote a fractional chroma QP offset.

Video decoder300may determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset (338). For example, video decoder300may determine the DRA chroma scale adjustment value by using the fractional chroma QP offset and the integer defined qpScale. draChromaScaleShift may denote a DRA chroma scale adjustment value. The DRA chroma scale adjustment value may be used to adjust a DRA chroma scale scaleDra.

Video decoder300may process the video data based on the DRA chroma scale adjustment value (340). For example, video decoder300may scale a chroma value of a chroma component of the video data based on the DRA chroma scale adjustment value.

In some examples, as part of processing the video data based on the DRA chroma scale adjustment value, the one or more processors are configured to determine a final chroma DRA scale value based on the adjusted chroma DRA scale value and the DRA chroma scale adjustment value and process the video data based on the final chroma DRA scale value.

In some examples, as part of determining the fractional chroma QP offset and as part of determining the integer chroma QP offset, video decoder300may determine (e.g., look up) entries in a chroma QP table. In some examples, as part of determining the fractional chroma QP offset, video decoder300may determine (e.g., look up) two entries in the chroma QP table, such as Qp0 and Qp1, and interpolate between the two entries, e.g., based on the fractional component of the chroma QP (qpDraFrac). In some examples, as part of determining the integer chroma QP offset, video decoder300may determine (e.g., look up) two entries in the chroma QP table and interpolate between the two entries.

In some examples, video decoder300may parse a syntax element indicative of a starting position in the chroma QP table. In some examples, the chroma QP table is predetermined. In some examples, the chroma QP table is signaled in a bitstream. dra_table_idx may denote a starting position in the chroma QP table as parsed from the bitstream.

In some examples, video decoder300may determine values of a plurality of DRA syntax elements, wherein the plurality of DRA syntax elements are constrained to be within a predetermined value range. In some examples, the plurality of syntax elements are configured to facilitate integer implementation. In some examples, video decoder300may determine a value based on one or more entries in one or more scale QP tables (such as scaleQp and/or qpScale), wherein the one or more entries are sub-sampled representations of an exponential or logarithmic function for conversion between QP and DRA scale values as shown in Equations 28 and 29. In some examples, video decoder300may determine the DRA luma scale value. In some examples, as part of determining the DRA luma scale value, video decoder300may parse a syntax element indicative of an integer component of the DRA luma scale value and of a fractional component of the DRA luma scale value. For example, video decoder300may determine a DRA luma scale value. For example, video decoder300may parse a syntax element(s) indicative of a DRA luma scale value. For example, video decoder300may parse lumaScale.

FIG.15is a flowchart illustrating an example method for encoding a current block according to the techniques of this disclosure. The current block may comprise a current CU. Although described with respect to video encoder200(FIGS.1and2), it should be understood that other devices may be configured to perform a method similar to that ofFIG.15.

In this example, video encoder200initially predicts the current block (350). For example, video encoder200may form a prediction block for the current block. Video encoder200may then calculate a residual block for the current block (352). To calculate the residual block, video encoder200may calculate a difference between the original, unencoded block and the prediction block for the current block. Video encoder200may then transform the residual block and quantize transform coefficients of the residual block (354). Next, video encoder200may scan the quantized transform coefficients of the residual block (356). During the scan, or following the scan, video encoder200may entropy encode the transform coefficients (358). For example, video encoder200may encode the transform coefficients using CAVLC or CABAC. Video encoder200may then output the entropy encoded data of the block (360).

FIG.16is a flowchart illustrating an example method for decoding a current block of video data according to the techniques of this disclosure. The current block may comprise a current CU. Although described with respect to video decoder300(FIGS.1and3), it should be understood that other devices may be configured to perform a method similar to that ofFIG.16.

Video decoder300may receive entropy encoded data for the current block, such as entropy encoded prediction information and entropy encoded data for transform coefficients of a residual block corresponding to the current block (370). Video decoder300may entropy decode the entropy encoded data to determine prediction information for the current block and to reproduce transform coefficients of the residual block (372). Video decoder300may predict the current block (374), e.g., using an intra- or inter-prediction mode as indicated by the prediction information for the current block, to calculate a prediction block for the current block. Video decoder300may then inverse scan the reproduced transform coefficients (376), to create a block of quantized transform coefficients. Video decoder300may then inverse quantize the transform coefficients and apply an inverse transform to the transform coefficients to produce a residual block (378). Video decoder300may ultimately decode the current block by combining the prediction block and the residual block (380). Video decoder300may also apply the techniques ofFIG.13to adjust the dynamic range of the current block.

By harmonizing DRA scaling in a manner that utilizes estimations of logarithmic and exponential operations rather than the logarithmic and exponential operations themselves, the techniques of this disclosure may reduce expense and save processing power while facilitating bit matching across different video decoders. This bit matching may allow different video decoders to process the video data in the same manner such that if a user were to switch video decoding devices, the video being processed by the video decoding device would not be processed differently by the devices.

This disclosure includes the following examples.

Clause 1A. A method of processing video data, the method comprising: performing dynamic range adjustment (DRA) on the video data; and coding the video data based on the dynamic range adjustment, wherein the performing DRA comprises utilizing a constrained bitstream of the video data.

Clause 2A. The method of clause 1A, wherein the bitstream is constrained by requiring a variable to be at least a predetermined value.

Clause 3A. The method of clause 1A or 2A, wherein the bitstream is constrained by requiring a value of a variable to be in a predetermined range.

Clause 4A. The method of any combination of clauses 1A-3A, wherein the bitstream is constrained by requiring a variable to be equal to a predetermined value.

Clause 5A. The method of any combination of clauses 1A-3A, wherein the bitstream is constrained by requiring a variable to be less than a predetermined value.

Clause 6A. A method of processing video data, the method comprising: performing DRA on the video data; and coding the video data based on the DRA, wherein the performing DRA comprises determining an integer approximation of an exponential function or a logarithmic function.

Clause 7A. The method of clause 6A, wherein the determining an integer approximation comprises determining the integer approximation from a table, the table comprising a limited number of integer values.

Clause 8A. A method of processing video data, the method comprising: performing DRA on the video data; and coding the video data based on the DRA, wherein the performing DRA comprises determining a sub-sampled representation of an exponential function or a logarithmic function.

Clause 9A. A method of processing video data, the method comprising: determining whether a scale value or a quantization parameter (QP) value is exactly tabulated; based upon the scale value or QP value not being exactly tabulated, approximating an exponential function or a logarithmic function through linear interpolation; determining an estimated scale value or QP value based on the approximation; and coding the video data based on the estimated scale value or QP value.

Clause 10A. A method of processing video data, the method comprising: performing DRA on the video data; and coding the video data based on the DRA, wherein the performing DRA comprises converting between scale values or QP values only using integer values.

Clause 11A. A method of processing video data, the method comprising any of or any combination of the techniques of this disclosure.

Clause 12A. The method of any combination of clauses 1A-11A, wherein processing comprises decoding or post-processing after decoding.

Clause 13A. The method of any combination of clauses 1A-12A, wherein processing comprises encoding or pre-processing prior encoding.

Clause 14A. A device for processing video data, the device comprising one or more means for performing the method of any combination of clauses 1A-13A.

Clause 15A. The device of clause 14A, wherein the one or more means comprise one or more processors implemented in circuitry.

Clause 16A. The device of clause 14A or 15A, further comprising a memory to store the video data.

Clause 17A. The device of any combination of clauses 14A-16A, further comprising a display configured to display decoded video data.

Clause 18A. The device of any combination of clauses 14A-17A, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

Clause 19A. The device of any combination of clauses 14A-18A, wherein the device comprises a video decoder or post-decoding processor.

Clause 20A. The device of any combination of clauses 14A-19A, wherein the device comprises a video encoder or pre-encoding processor.

Clause 21A. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any combination of clauses 1A-11A.

Clause 1B. A method of processing video data, the method comprising: adjusting a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value; determining a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determining an integer chroma QP offset based on the integer component of the chroma QP; determining a fractional chroma QP offset based on the fractional component of the chroma QP; determining a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and processing the video data based on the DRA chroma scale adjustment value.

Clause 2B. The method of clause 1B, wherein processing the video data based on the DRA chroma scale adjustment value comprises: determining a final chroma DRA scale value based on the adjusted chroma DRA scale value and the DRA chroma scale adjustment value; and processing the video data based on the final chroma DRA scale value.

Clause 3B. The method of clause 1B or clause 2B, wherein determining the fractional chroma QP offset and determining the integer chroma QP offset comprises determining entries in a chroma QP table.

Clause 4B. The method of clause 3B, wherein determining the fractional chroma QP offset comprises determining two entries in the chroma QP table and interpolating between the two entries.

Clause 5B. The method of clause 3B or 4B, further comprising: parsing a syntax element indicative of a starting position in the chroma QP table.

Clause 6B. The method of any combination of clauses 3B-5B, wherein the chroma QP table is predetermined.

Clause 7B. The method of any combination of clauses 3B-5B, wherein the chroma QP table is signaled in a bitstream.

Clause 8B. The method of any combination of clauses 3B-7B, wherein determining the integer chroma QP offset comprises determining two entries in the chroma QP table and interpolating between the two entries.

Clause 9B. The method of any combination of clauses 1B-8B, further comprising: determining values of a plurality of DRA syntax elements, wherein the plurality of DRA syntax elements are constrained to be within a predetermined value range.

Clause 10B. The method of any combination of clauses 1B-9B, wherein determining the DRA chroma scale adjustment value comprises: determining a value based on one or more entries in one or more scale QP tables, wherein the one or more entries are sub-sampled representations of an exponential or logarithmic function.

Clause 11B. The method of any combination of clauses 1B-10B, further comprising determining the luma DRA scale value, wherein determining the luma DRA scale value comprises: parsing a syntax element indicative of an integer component of the DRA luma scale value and of a fractional component of the DRA luma scale value.

Clause 12B. A device for processing video data, the device comprising: memory configured to store the video data; and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors being configured to: adjust a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value; determine a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process the video data based on the DRA chroma scale adjustment value.

Clause 13B. The device of clause 12B, wherein as part of processing the video data based on the DRA chroma scale adjustment value, the one or more processors are configured to: determine a final chroma DRA scale value based on the adjusted chroma DRA scale value and the DRA chroma scale adjustment value; and process the video data based on the final chroma DRA scale value.

Clause 14B. The device of clause 12B or clause 13B, wherein as part of determining the fractional chroma QP offset and as part of determining the integer chroma QP offset, the one or more processors are configured to determine entries in a chroma QP table.

Clause 15B. The device of clause 14B, wherein as part of determining the fractional chroma QP offset, the one or more processors are configured to determine two entries in the chroma QP table and interpolate between the two entries.

Clause 16B. The device of clause 14B or 15B, wherein the one or more processors are further configured to: parse a syntax element indicative of a starting position in the chroma QP table.

Clause 17B. The device of any combination of clauses 14B-16B, wherein the chroma QP table is predetermined.

Clause 18B. The device of any combination of clauses 14B-16B, wherein the chroma QP table is signaled in a bitstream.

Clause 19B. The device of any combination of clauses 12B-18B, wherein as part of determining the integer chroma QP, the one or more processors are configured to determine two entries in the chroma QP table and interpolate between the two entries.

Clause 20B. The device of any combination of clauses 12B-19B, wherein the one or more processors are further configured to: determine values of a plurality of DRA syntax elements, wherein the plurality of syntax elements are constrained to be within a predetermined value range.

Clause 21B. The device of any combination of clauses 12B-20B, wherein as part of determining the DRA chroma scale adjustment value the one or more processors are further configured to: determine a value based on one or more entries in one or more scale QP tables, wherein the one or more entries are sub-sampled representations of an exponential or logarithmic function.

Clause 22B. The device of any combination of clauses 12B-21B, wherein the one or more processors are further configured to determine the DRA luma scale value, wherein as part of determining the DRA luma scale value, the one or more processors are configured to: parse a syntax element indicative of an integer component of the DRA luma scale value and of a fractional component of the DRA luma scale value.

Clause 23B. The device of any combination of clauses 12B-22B, wherein the device comprises a wireless communication device.

Clause 24B. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to: adjust a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value; determine a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; determine an integer chroma QP offset based on the integer component of the chroma QP; determine a fractional chroma QP offset based on the fractional component of the chroma QP; determine a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and process video data based on the DRA chroma scale adjustment value.

Clause 25B. A device for processing video data, the device comprising means for adjusting a chroma dynamic range adjustment (DRA) scale value based on a luma DRA scale value; means for determining a chroma quantization parameter (QP) based on the luma adjusted chroma DRA scale value, wherein the chroma QP comprises an integer component and a fractional component; means for determining an integer chroma QP offset based on the integer component of the chroma QP; means for determining a fractional chroma QP offset based on the fractional component of the chroma QP; means for determining a DRA chroma scale adjustment value based on the integer chroma QP offset and the fractional chroma QP offset; and means for processing the video data based on the DRA chroma scale adjustment value.