MODEL PARAMETER DERIVATION OF LOCAL ILLUMINATION COMPENSATION IN THE LUMA MAPPING WITH CHROMA SCALING-MAPPED DOMAIN IN VIDEO CODING

An example device for decoding video data includes memory configured to store the video data and one or more processors implemented in circuitry and communicatively coupled to the memory. The one or more processors are configured to reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block and derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block. The one or more processors are configured to apply the LIC model parameters to motion-compensated prediction signals and decode the video data based on the application of the LIC model parameters.

TECHNICAL FIELD

BACKGROUND

SUMMARY

In general, this disclosure describes techniques for local illumination compensation (LIC) parameter derivation in a video coding process. In particular, this disclosure describes techniques for LIC parameter derivation in a luma mapping with chroma scaling (LMCS) mapped domain. The techniques of this disclosure may decrease decoding latency and increase decoding efficiency by removing the need to load an inverse mapping table from memory for LIC parameter derivation in LMCS.

In one example, a method includes reshaping a pixel domain reference template block using a forward mapping function into a mapped domain reference template block, deriving local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block, applying the LIC model parameters to motion-compensated prediction signals, and decoding the video data based on the application of the LIC model parameters.

In another example, a device includes memory configured to store the video data and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors being configured to: reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; apply the LIC model parameters to motion-compensated prediction signals; and decode the video data based on the application of the LIC model parameters.

In another example, a computer-readable storage medium stores instructions which, when executed, cause one or more processors to: reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; apply the LIC model parameters to motion-compensated prediction signals; and decode the video data based on the application of the LIC model parameters.

In another example, a device includes means for reshaping a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; means for deriving local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; means for applying the LIC model parameters to motion-compensated prediction signals; and means for decoding the video data based on the application of the LIC model parameters.one or more means for performing any of the techniques of this disclosure.

DETAILED DESCRIPTION

In some example video decoders, when local illumination compensation (LIC) and luma mapping with chroma scaling (LMCS) are both enabled, LIC is the only inter prediction mode that requires loading an inverse look-up table at the coding unit (CU) level to convert reconstruction signals back to the pixel domain to then derive LIC model parameters. Unlike the Versatile Video Coding (VVC) standard, which requires loading this inverse table at the coding tree unit (CTU)/virtual pipeline data unit (VPDU)/picture level within the loop filtering stage, this additional inverse mapping at the CU level introduces additional hardware-implementation burden to the motion compensation module of a video decoder, resulting in potentially longer latency during this motion compensation stage.

According to the techniques of this disclosure, rather than use the inverse mapping at the CU level, a video decoder may derive LIC parameters based on a mapped domain reference template block and a mapped domain neighboring reconstruction template block. In this manner, the inverse mapping table may not be read from memory to derive the LIC parameters, which may save processing power and reduce decoding latency.

System100as shown inFIG. 1is merely one example. In general, any digital video encoding and/or decoding device may be configured to perform techniques for LIC parameter derivation. Source device102and destination device116are merely examples of such coding devices in which source device102generates coded video data for transmission to destination device116. This disclosure refers to a “coding” device as a device that performs coding (encoding and/or decoding) of data. Thus, video encoder200and video decoder300represent examples of coding devices, in particular, a video encoder and a video decoder, respectively. In some examples, source device102and destination device116may operate in a substantially symmetrical manner such that each of source device102and destination device116includes video encoding and decoding components. Hence, system100may support one-way or two-way video transmission between source device102and destination device116, e.g., for video streaming, video playback, video broadcasting, or video telephony.

In some examples, source device102may output encoded video data to file server114or another intermediate storage device that may store the encoded video data generated by source device102. Destination device116may access stored video data from file server114via streaming or download.

File server114may be any type of server device capable of storing encoded video data and transmitting that encoded video data to the destination device116. File server114may represent a web server (e.g., for a website), a server configured to provide a file transfer protocol service (such as File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a hypertext transfer protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) server, and/or a network attached storage (NAS) device. File server114may, additionally or alternatively, implement one or more HTTP streaming protocols, such as Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), Real Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, or the like.

Destination device116may access encoded video data from file server114through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both that is suitable for accessing encoded video data stored on file server114. Input interface122may be configured to operate according to any one or more of the various protocols discussed above for retrieving or receiving media data from file server114, or other such protocols for retrieving media data.

Although not shown inFIG. 1, in some examples, video encoder200and video decoder300may each be integrated with an audio encoder and/or audio decoder, and may include appropriate MUX-DEMUX units, or other hardware and/or software, to handle multiplexed streams including both audio and video in a common data stream. If applicable, MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

Video encoder200and video decoder300may operate according to a video coding standard, such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC) or extensions thereto, such as the multi-view and/or scalable video coding extensions. Alternatively, video encoder200and video decoder300may operate according to other proprietary or industry standards, such as ITU-T H.266, also referred to as Versatile Video Coding (VVC). A draft of the VVC standard is described in Bross, et al. “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 18thMeeting: by teleconference, 15-24 Apr. 2020, JVET-R2001-vA (hereinafter “VVC Draft 9”). The techniques of this disclosure, however, are not limited to any particular coding standard.

In some examples, a CTU includes a coding tree block (CTB) of luma samples, two corresponding CTBs of chroma samples of a picture that has three sample arrays, or a CTB of samples of a monochrome picture or a picture that is coded using three separate color planes and syntax structures used to code the samples. A CTB may be an N×N block of samples for some value of N such that the division of a component into CTBs is a partitioning. A component is an array or single sample from one of the three arrays (luma and two chroma) that compose a picture in 4:2:0, 4:2:2, or 4:4:4 color format or the array or a single sample of the array that compose a picture in monochrome format. In some examples, a coding block is an M×N block of samples for some values of M and N such that a division of a CTB into coding blocks is a partitioning.

In some examples, a tile may be partitioned into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick that is a true subset of a tile may not be referred to as a tile.

The bricks in a picture may also be arranged in a slice. A slice may be an integer number of bricks of a picture that may be exclusively contained in a single network abstraction layer (NAL) unit. In some examples, a slice includes either a number of complete tiles or only a consecutive sequence of complete bricks of one tile.

In accordance with the techniques of this disclosure, a method includes reshaping a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; deriving local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; applying the LIC model parameters to motion-compensated prediction signals; and decoding the video data based on the application of the LIC model parameters.

In accordance with the techniques of this disclosure, a device includes memory configured to store the video data and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors being configured to: reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; apply the LIC model parameters to motion-compensated prediction signals; and decode the video data based on the application of the LIC model parameters.

In accordance with the techniques of this disclosure, a computer-readable storage medium stores instructions which, when executed, cause one or more processors to: reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; apply the LIC model parameters to motion-compensated prediction signals; and decode the video data based on the application of the LIC model parameters.

In accordance with the techniques of this disclosure, a device includes means for reshaping a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; means for deriving local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; means for applying the LIC model parameters to motion-compensated prediction signals; and means for decoding the video data based on the application of the LIC model parameters.one or more means for performing any of the techniques of this disclosure.

FIGS. 2A and 2Bare conceptual diagrams illustrating an example quadtree binary tree (QTBT) structure130, and a corresponding coding tree unit (CTU)132. The solid lines represent quadtree splitting, and dotted lines indicate binary tree splitting. In each split (i.e., non-leaf) node of the binary tree, one flag is signaled to indicate which splitting type (i.e., horizontal or vertical) is used, where 0 indicates horizontal splitting and 1 indicates vertical splitting in this example. For the quadtree splitting, there is no need to indicate the splitting type, because quadtree nodes split a block horizontally and vertically into 4 sub-blocks with equal size. Accordingly, video encoder200may encode, and video decoder300may decode, syntax elements (such as splitting information) for a region tree level of QTBT structure130(i.e., the solid lines) and syntax elements (such as splitting information) for a prediction tree level of QTBT structure130(i.e., the dashed lines). Video encoder200may encode, and video decoder300may decode, video data, such as prediction and transform data, for CUs represented by terminal leaf nodes of QTBT structure130.

In general, CTU132ofFIG. 2Bmay be associated with parameters defining sizes of blocks corresponding to nodes of QTBT structure130at the first and second levels. These parameters may include a CTU size (representing a size of CTU132in samples), a minimum quadtree size (MinQTSize, representing a minimum allowed quadtree leaf node size), a maximum binary tree size (MaxBTSize, representing a maximum allowed binary tree root node size), a maximum binary tree depth (MaxBTDepth, representing a maximum allowed binary tree depth), and a minimum binary tree size (MinBTSize, representing the minimum allowed binary tree leaf node size).

The root node of a QTBT structure corresponding to a CTU may have four child nodes at the first level of the QTBT structure, each of which may be partitioned according to quadtree partitioning. That is, nodes of the first level are either leaf nodes (having no child nodes) or have four child nodes. The example of QTBT structure130represents such nodes as including the parent node and child nodes having solid lines for branches. If nodes of the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), then the nodes can be further partitioned by respective binary trees. The binary tree splitting of one node can be iterated until the nodes resulting from the split reach the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). The example of QTBT structure130represents such nodes as having dashed lines for branches. The binary tree leaf node is referred to as a coding unit (CU), which is used for prediction (e.g., intra-picture or inter-picture prediction) and transform, without any further partitioning. As discussed above, CUs may also be referred to as “video blocks” or “blocks.”

In one example of the QTBT partitioning structure, the CTU size is set as 128×128 (luma samples and two corresponding 64×64 chroma samples), the MinQTSize is set as 16×16, the MaxBTSize is set as 64×64, the MinBTSize (for both width and height) is set as 4, and the MaxBTDepth is set as 4. The quadtree partitioning is applied to the CTU first to generate quad-tree leaf nodes. The quadtree leaf nodes may have a size from 16×16 (i.e., the MinQTSize) to 128×128 (i.e., the CTU size). If the quadtree leaf node is 128×128, the leaf quadtree node will not be further split by the binary tree, because the size exceeds the MaxBTSize (i.e., 64×64, in this example). Otherwise, the quadtree leaf node will be further partitioned by the binary tree. Therefore, the quadtree leaf node is also the root node for the binary tree and has the binary tree depth as 0. When the binary tree depth reaches MaxBTDepth (4, in this example), no further splitting is permitted. A binary tree node having a width equal to MinBTSize (4, in this example) implies that no further vertical splitting (that is, dividing of the width) is permitted for that binary tree node. Similarly, a binary tree node having a height equal to MinBTSize implies no further horizontal splitting (that is, dividing of the height) is permitted for that binary tree node. As noted above, leaf nodes of the binary tree are referred to as CUs, and are further processed according to prediction and transform without further partitioning.

FIG. 3is a block diagram illustrating an example video encoder200that may perform the techniques of this disclosure.FIG. 3is provided for purposes of explanation and should not be considered limiting of the techniques as broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes video encoder200according to the techniques of VVC (ITU-T H.266, under development), and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video encoding devices that are configured to other video coding standards.

In the example ofFIG. 3, video encoder200includes video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, decoded picture buffer (DPB)218, and entropy encoding unit220. Any or all of video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, DPB218, and entropy encoding unit220may be implemented in one or more processors or in processing circuitry. For instance, the units of video encoder200may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Moreover, video encoder200may include additional or alternative processors or processing circuitry to perform these and other functions.

Video encoder200stores reconstructed blocks in DPB218. For instance, in examples where operations of filter unit216are not needed, reconstruction unit214may store reconstructed blocks to DPB218. In examples where operations of filter unit216are needed, filter unit216may store the filtered reconstructed blocks to DPB218. Motion estimation unit222and motion compensation unit224may retrieve a reference picture from DPB218, formed from the reconstructed (and potentially filtered) blocks, to inter-predict blocks of subsequently encoded pictures. In addition, intra-prediction unit226may use reconstructed blocks in DPB218of a current picture to intra-predict other blocks in the current picture.

In the example ofFIG. 4, video decoder300includes coded picture buffer (CPB) memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and decoded picture buffer (DPB)314. Any or all of CPB memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and DPB314may be implemented in one or more processors or in processing circuitry. For instance, the units of video decoder300may be implemented as one or more circuits or logic elements as part of hardware circuitry, or as part of a processor, ASIC, or FPGA. Moreover, video decoder300may include additional or alternative processors or processing circuitry to perform these and other functions.

In some examples, motion compensation unit316may use the inter coding tool LIC along with LMCS. In such examples, motion compensation unit316may reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block. Motion compensation unit316may derive LIC model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block. Motion compensation unit316may apply the LIC model parameters to motion-compensated prediction signals.

Video decoder300may store the reconstructed blocks in DPB314. For instance, in examples where operations of filter unit312are not performed, reconstruction unit310may store reconstructed blocks to DPB314. In examples where operations of filter unit312are performed, filter unit312may store the filtered reconstructed blocks to DPB314. As discussed above, DPB314may provide reference information, such as samples of a current picture for intra-prediction and previously decoded pictures for subsequent motion compensation, to prediction processing unit304. Moreover, video decoder300may output decoded pictures (e.g., decoded video) from DPB314for subsequent presentation on a display device, such as display device118ofFIG. 1.

In this manner, video decoder300represents an example of a video decoding device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block, derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block, apply the LIC model parameters to motion-compensated prediction signals, and decode the video data based on the application of the LIC model parameters.

As discussed above, this disclosure is related to local illumination compensation (LIC) in video coding. The techniques of this disclosure may be applied to any existing video codecs, such as HEVC (High Efficiency Video Coding), VVC (Versatile Video Coding), Essential Video Coding (EVC) or be an efficient coding tool in any future video coding standards. In the following section of this disclosure, HEVC techniques and work in VVC related to LIC are discussed.

In addition, a newer video coding standard, namely High Efficiency Video Coding (HEVC) or ITU-T H.265, including its range extension, multiview extension (MV-HEVC) and scalable extension (SHVC), has recently been developed by the Joint Collaboration Team on Video Coding (JCT-VC) as well as Joint Collaboration Team on 3D Video Coding Extension Development (JCT-3V) of ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Motion Picture Experts Group (MPEG).

ITU-T VCEG (Q6/16) and ISO/IEC MPEG (JTC 1/SC 29/WG 11) are studying the potential need for standardization of future video coding technology with a compression capability that significantly exceeds that of the HEVC standard (including its current extensions and near-term extensions for screen content coding and high-dynamic-range coding). These two groups are working together on this exploration activity in a joint collaboration effort known as the Joint Video Exploration Team (JVET) to evaluate compression technology designs proposed by their experts in this area. The algorithm description of Versatile Video Coding and Test Model 9 (VTM 9) could also be referred to as JVET-R2002.

CU structure and motion vector prediction in HEVC are now discussed. In HEVC, the largest coding unit in a slice is a CTB or CTU. A CTB contains a quad-tree the nodes of which are coding units.

The size of a CTB can range from 16×16 to 64×64 in the HEVC main profile (although technically 8×8 CTB sizes can also be supported). A CU could be the same size of a CTB to as small as 8×8. Each CU is coded with one mode, e.g., inter or intra. When a CU is inter coded, the CU may be further partitioned into 2 or 4 PUs or become just one PU when further partitioning is not applied. When two PUs are present in one CU, they can be half size rectangles or two rectangles, one of which is ¼ the size of the CU and the other of which is ¾ the size of the CU. When a CU is inter coded, each PU has one set of motion information, which is derived with a unique inter prediction mode.

Motion vector prediction is now discussed. In the HEVC standard, there are two inter prediction modes, named merge (skip is considered as a special case of merge) and advanced motion vector prediction (AMVP) modes, respectively, for a PU.

In either AMVP or merge mode, an MV candidate list is maintained for multiple motion vector predictors. The MV(s), as well as reference indices in the merge mode, of the current PU, are generated by taking one candidate from the MV candidate list.

The MV candidate list may contain up to 5 candidates for the merge mode and two candidates for the AMVP mode. A merge candidate may contain a set of motion information, e.g., motion vectors corresponding to both reference picture lists (list 0 (L0) and list 1 (L1)) and the reference indices. If a merge candidate is identified by a merge index, video decoder300may determine the reference pictures used for the prediction of the current block, as well as the associated motion vectors. On the other hand, under AMVP mode for each potential prediction direction from either list 0 or list 1, a reference index needs to be explicitly signaled (e.g., by video encoder200), together with an MV predictor (MVP) index to the MV candidate list since the AMVP candidate contains only a motion vector. In AMVP mode, the predicted motion vectors can be further refined. The candidates for both modes may be derived similarly from the same spatial and temporal neighboring blocks.

FIGS. 5A-5Bare conceptual diagrams illustrating examples of spatial neighboring candidates for merge mode and advanced motion vector prediction (AMVP) mode, respectively. Spatial MV candidates may be derived from the neighboring blocks shown onFIGS. 5A-B, for a specific PU (PUO400), although the techniques for generating the candidates from the blocks differ for merge and AMVP modes.

For example, in merge mode, video decoder300may derive up to four spatial MV candidates in the order shown inFIG. 5A. The order is as follows: left (0), above (1), above right (2), below left (3), and above left (4), as shown inFIG. 5A. For example, video encoder200or video decoder300may derive up to four spatial MV candidates in the order shown inFIG. 5A.

In AVMP mode, the neighboring blocks are divided into two groups: a left group consisting of the block 0 and 1 to the left of PUO402, and an above group consisting of the blocks 2, 3, and 4 above PUO402, as shown onFIG. 5B. For each group, the potential candidate in a neighboring block referring to the same reference picture as that indicated by the signaled reference index has the highest priority to be chosen to form a final candidate of the group. For example, video encoder200or video decoder300may select the candidate in a neighboring block referring to the same reference picture as that indicated by the signaled reference index to form a final candidate. In some cases, all neighboring blocks do not contain a motion vector pointing to the same reference picture. Therefore, if such a candidate cannot be found, video decoder300may scale the first available candidate to form the final candidate. In this manner, any temporal distance differences can be compensated.

Temporal Motion Vector Prediction in HEVC is now discussed. A temporal motion vector predictor (TMVP) candidate, if enabled and available, is added into the MV candidate list after spatial motion vector candidates. For example, video encoder200or video decoder300may add a TMVP candidate into the MV candidate list after the spatial motion vector candidates. The process of motion vector derivation for a TMVP candidate is the same for both merge and AMVP modes. However, in some examples, the target reference index for the TMVP candidate in the merge mode is set to 0.

FIG. 6Ais a conceptual diagram illustrating an example of a temporal motion vector prediction (TMVP) candidate. The primary block location for TMVP candidate derivation is the bottom right block outside of the collocated PU as shown inFIG. 6Aas a block “T”410, to compensate for the bias to the above and left blocks used to generate spatial neighboring candidates. However, if that block is located outside of the current CTB row (e.g., block414) or motion information is not available, video encoder200or video decoder300may substitute the center block412of the PU for the bottom right block outside of the collocated PU.

Video encoder200or video decoder300may derive the motion vector for the TMVP candidate from the co-located PU of the co-located picture, indicated in the slice level. The motion vector for the co-located PU is called a collocated MV.

FIG. 6Bis a conceptual diagram illustrating an example of motion vector scaling. Similar to temporal direct mode in AVC, to derive the TMVP candidate motion vector, co-located MV424needs to be scaled to compensate for temporal distance differences, as shown inFIG. 6B. For example, current temporal distance422is different than co-located temporal distance420. Therefore, video encoder200or video decoder300may scale co-located MV424in proportion to the differences in current temporal distance422and co-located temporal distance420.

Other Aspects of Motion Prediction in HEVC are now discussed. Several aspects of merge and AMVP modes are described below. Motion vector scaling: the value of motion vectors is proportional to the distance of pictures in presentation time. A motion vector associates two pictures, the reference picture, and the picture containing the motion vector (e.g., the containing picture). When a motion vector is utilized to predict the other motion vector, video encoder200or video decoder300calculate the distance of the containing picture and the reference picture based on the Picture Order Count (POC) values.

For a motion vector to be predicted, both the motion vector's associated containing picture and reference picture may be different. Therefore, video encoder200or video decoder300may calculate a new distance (based on POC). Video encoder200or video decoder300scale the motion vector based on these two POC distances. For a spatial neighboring candidate, the containing pictures for the two motion vectors are the same, while the reference pictures are different. In HEVC, motion vector scaling applies to both TMVP and AMVP for spatial and temporal neighboring candidates.

If a motion vector candidate list is not complete, video encoder200or video decoder300generate artificial MV candidates and insert the artificial MV candidates at the end of the list until the MV candidate list has all the candidates (e.g., the list is full). In merge mode, there are two types of artificial MV candidates: combined candidates derived only for B-slices and zero candidates used only for AMVP if the first type does not provide enough artificial candidates to fill the MV candidate list.

For each pair of candidates that are already in the candidate list and have necessary motion information, bi-directional combined motion vector candidates are derived by a combination of the motion vector of the first candidate referring to a picture in list 0 and the motion vector of a second candidate referring to a picture in list 1.

The pruning process for candidate insertion is now discussed. Candidates from different blocks may happen to be the same, which decreases the efficiency of a merge/AMVP candidate list. A pruning process may be applied to solve this problem. For example, video encoder200or video decoder300may compare one candidate against the others in the current candidate list to avoid inserting identical candidates to a certain extent. To reduce the complexity, only limited pruning is applied instead of comparing each potential candidate with all the other existing candidates.

Local illumination compensation is now discussed. An overview of illumination compensation proposed for HEVC is presented. In JCTVC-0041, a partition-based illumination compensation (PBIC) was proposed. PBIC is different from weighted prediction (WP), for which video encoder200may indicate and signal parameters at a slice level. With PBIC, video encoder200may enable/disable PBIC and signal PBIC model parameters at a PU level to handle local illumination variation.

Similar to WP, illumination compensation (IC) also has a scaling factor (also denoted by a) and an offset (also denoted by b), and a right shift number which is fixed to be 6. An IC flag is coded (e.g., by video encoder200) for each PU to indicate whether IC applies for a current PU or not. If IC applies for the current PU, video encoder200may signal a set of IC parameters (e.g., a and b) to video decoder300and video decoder300may use the set of IC parameters for motion compensation. In a bi-prediction case, video encoder200may signal two scaling factors (one for each prediction direction) and one offset. To save bits spent on IC parameters, a chroma component shares the scaling factors with a luma component and a fixed offset of 128 is used.

An overview of IC in 3D-HEVC is now provided. In 3D-HEVC, IC is enabled for inter-view prediction. IC in 3D-HEVC is different from WP and PBIC for which video encoder200signals IC parameters explicitly. For IC in 3D-HEVC, a video coder (e.g., video decoder300) derives IC parameters based on neighboring samples of a current CU and neighboring samples of a reference block.

IC applies to 2N×2N partition mode only. For AMVP mode, video encoder200signals one IC flag for each CU that is predicted from an inter-view reference picture. For merge mode, to save bits, video encoder200signals an IC flag only when the merge index of the PU is not equal to 0. IC does not apply to a CU that is only predicted from temporal reference pictures.

The linear IC model used in inter-view prediction is shown in Eq. (1):

Here, PUcis the current PU, (i,j) is the coordinate of pixels in PUc, (dvx, dvy) is the disparity vector of PUc. p(i, j) is the prediction of PUc, r is the PU's reference picture from neighboring view, and a and b are parameters of the linear IC model.

FIG. 7is a conceptual diagram illustrating examples of neighboring pixels used to estimate parameters in an IC model with the reference block of the current block being found by using a disparity vector of the current PU. To estimate parameters a and b for a PU, video decoder300use two set of pixels as shown inFIG. 7:

1) available reconstructed neighboring pixels in the left column and above row of current CU430(the CU that contains current PU) (indicated through grey circles); and

2) Corresponding neighboring pixels of current CU's reference block440(indicated through grey circles). A reference block of the current CU is found by using a disparity vector of the current PU.

For example, Recneg432and Recrefneig442denote a used neighboring pixel set of current CU430and reference block440of current CU430, respectively, and 2N denotes the pixel number in Recneigand Recrefneig. Then, a and b can be calculated as:

In some cases, only a is used in a linear model and b is always set equal to 0, or only b is used and a is always set equal to 1. For example, video encoder200or video decoder300may use only a in a linear model or use only b.

Local illumination compensation (LIC) in JVET is now discussed. LIC is based on a linear model for illumination changes, using a scaling factor a (with a shift number fixed to be 6) and an offset b. LIC is enabled or disabled adaptively for each inter-mode coded coding unit (CU).

FIG. 8is a conceptual diagram illustrating examples of neighboring samples used for deriving IC parameters. When LIC applies for a CU, video encoder200or video decoder300employ a least square error method to derive the parameters a and b by using the neighboring samples of the current CU and their corresponding reference samples. More specifically, as illustrated inFIG. 8, the subsampled (2:1 subsampling) neighboring samples (shown as circles with slashes) of the CU and the corresponding pixels (shown as circles with a checkboard pattern and identified by motion information of the current CU or sub-CU) in the reference picture are used. The IC parameters are derived and applied for each prediction direction separately. For example, PU450and PU452are depicted as well as subsampled neighboring samples of the current CU (which includes PU450and PU452). Reference block454, which is the reference block for PU450in list0, and subsampled neighboring samples of reference block454are also shown.

When a CU is coded with merge mode, the LIC flag is copied from neighboring blocks, in a way similar to motion information copy in merge mode. When a CU is otherwise encoded (e.g., not using merge mode), video encoder200signals an LIC flag to video decoder300to indicate whether LIC applies or not.

Weighted prediction (WP) is now discussed. In HEVC, WP is supported, where a scaling factor (denoted by a), a shift number (denoted by s) and an offset (denoted by b) are used in the motion compensation. Suppose the pixel value in position (x, y) of the reference picture is p(x, y), then p′(x, y)=((a*p(x, y)+(1<<(s−1)))>>s)+b instead of p(x, y) is used as the prediction value in motion compensation.

When WP is enabled, for each reference picture of current slice, video encoder200signals a flag to be received by video decoder300to indicate whether WP applies for the reference picture or not. If WP applies for one reference picture, video encoder200sends a set of WP parameters (e.g., a, s and b) to video decoder300and video decoder300uses the set of WP parameters for motion compensation from the reference picture. To flexibly turn on/off WP for luma and chroma components, video encoder200may separately signal WP flag and WP parameters for luma and chroma components. In WP, one same set of WP parameters is used for all pixels in one reference picture.

FIG. 9is a block diagram illustrating an example luma mapping with chroma scaling (LMCS) architecture. In VVC, a coding tool called LMCS is added as a new processing block before the loop filters. LMCS has two main components: 1) in-loop mapping of the luma component based on adaptive piecewise linear models; 2) for the chroma components, luma-dependent chroma residual scaling is applied.FIG. 9shows the LMCS architecture from the perspective of a decoder, such as video decoder300. For example, video decoder300may implement LMCS as depicted inFIG. 9. For example, video decoder300may process the inverse quantization and inverse transform500, perform luma intra prediction504and add the luma prediction together with the luma residual in reconstruction502in the mapped domain. Video decoder200may process loop filters506and516(such as deblocking filter, adaptive loop filter, and sample adaptive offset), perform motion compensated510and514, perform chroma intra prediction512, add the chroma prediction together with the chroma residual522, and store decoded pictures as reference pictures508and520in the original (e.g., non-mapped) domain. Forward mapping of the luma signal in forward reshape524, inverse mapping of the luma signal in inverse reshape526, and a luma-dependent chroma scaling process528are LMCS functional blocks. Like most other tools in VVC, video encoder200can enable/disable LMCS at the sequence level using an SPS flag.

Luma mapping with a piecewise linear model is now discussed. The in-loop mapping of the luma component adjusts the dynamic range of the input signal by redistributing the codewords across the dynamic range to improve compression efficiency. Luma mapping makes use of a forward mapping function, FwdMap, and a corresponding inverse mapping function, InvMap. Video encoder200signals the FwdMap function using a piecewise linear model with 16 equal pieces. The InvMap function does not need to be signaled as video decoder300may derive the InvMap function from the FwdMap function.

Video encoder200signals the luma mapping model in the adaptation parameter set (APS) syntax structure with aps_params_type set equal to 1 (LMCS_APS). Up to four LMCS APS's may be used in a coded video sequence. In this example, only one LMCS APS may be used for a picture. Video encoder200may signal the luma mapping model using the piecewise linear model. The piecewise linear model partitions the input signal's dynamic range into 16 equal pieces, and for each piece, the linear mapping parameters of the piece may be expressed using the number of codewords assigned to that piece. For example, with a 10-bit input, each of the 16 pieces will have 64 codewords assigned to the piece by default. The signaled number of codewords is used to calculate the scaling factor and adjust the mapping function accordingly for that piece. At the slice level, video encoder200signals an LMCS enable flag to indicate if the LMCS process as depicted inFIG. 9is applied to the current slice. If LMCS is enabled for the current slice, video encoder200signals an aps_id in the slice header to identify the APS that carries the luma mapping parameters.

Each i-th piece, i=0 . . . 15, of the FwdMap piecewise linear model is defined by two input pivot points InputPivot[ ] and two output (mapped) pivot points MappedPivot[ ].

The InputPivot[ ] and MappedPivot[ ] are computed as follows (assuming 10-bit video):1) OrgCW=642) For i=0:16, InputPivot[i]=i*OrgCW3) For i=0:16, MappedPivot[i] is calculated as follows:MappedPivot[0]=0;for(i=0; i<16; i++)MappedPivot[i+1]=MappedPivot[i]+SignalledCW[i]
where SignalledCW[i] is the signaled number of codewords for the i-th piece.

InFIG. 9, forward reshape524and inverse reshape526are shown. These boxes represent the forward reshaping of data from the pixel domain (also called the original domain) to a mapped domain and the inverse reshaping of data from the mapped domain to the pixel domain, respectively. As shown inFIG. 9, for an inter-coded block, motion-compensated prediction (e.g., motion compensation510) is performed in the original domain and then the motion-compensated prediction signal is converted to the mapped domain (e.g., by forward reshape524). In other words, after the motion-compensated prediction block Ypredis calculated based on the reference signals in DPB508, video decoder300applies the FwdMap function (e.g., forward reshape524) to map or reshape the luma prediction block in the original domain to the mapped domain, Ytpred=FwdMap(Ypred). For an intra-coded block, the FwdMap function is not applied because intra prediction504is performed in the mapped domain. After reconstructed block Yris calculated, video decoder300applies the InvMap function (e.g., inverse reshape526) to convert the reconstructed luma values in the mapped domain back to the reconstructed luma values in the original domain (Ŷi=InvMap(Yr)). The InvMap function (e.g., inverse reshape526) is applied to both intra- and inter-coded luma blocks.

The luma mapping process (forward and/or inverse mapping) can be implemented using either look-up-tables (LUTs) or using on-the-fly computation. If LUTs are used, then FwdMapLUT and InvMapLUT can be pre-calculated and pre-stored for use at the tile group level, and forward and inverse mapping can be simply implemented as FwdMap(Ypred)=FwdMapLUT[Ypred] and InvMap(Yr)=InvMapLUT[Yr], respectively. Alternatively, on-the-fly computation may be used. Take forward mapping function FwdMap as an example. In order to determine the piece to which a luma sample belongs, video decoder300may right shift the sample value by 6 bits (which corresponds to 16 equal pieces). Then, video decoder300may retrieve the linear model parameters for that piece and apply the linear model parameters on-the-fly to compute the mapped luma value. Let i be the piece index, a1, a2 be InputPivot[i] and InputPivot[i+1], respectively, and b1, b2 be MappedPivot[i] and MappedPivot[i+1], respectively. The FwdMap function may be as follows:

The InvMap function can be computed on-the-fly in a similar manner. Generally, the pieces in the mapped domain are not of equal size. Therefore, the most straightforward inverse mapping process would require video decoder300to make comparisons in order to determine to which piece the current sample value belongs. Such comparisons increase decoder complexity. For this reason, VVC imposes a bitstream constraint on the values of the output pivot points MappedPivot[i] as follows. Assume the range of the mapped domain (for 10-bit video, this range is [0, 1023]) is divided into 32 equal pieces. If MappedPivot[i] is not a multiple of 32, then MappedPivot[i+1] and MappedPivot[i] cannot belong to the same piece of the 32 equal-sized pieces, e.g., MappedPivot[i+1]>>(BitDepthY−5) shall not be equal to MappedPivot[i]>>(BitDepthY−5). Thanks to such a bitstream constraint, the InvMap function can also be carried out using a simple right bit-shift by 5 bits (which corresponds to 32 equal-sized pieces) in order to determine the piece to which the sample value belongs.

Luma-dependent chroma residual scaling is now discussed. Chroma residual scaling is designed to compensate for the interaction between the luma signal and the luma signal's corresponding chroma signals. Video encoder200signals whether chroma residual scaling is enabled or not at the slice level. If luma mapping is enabled, video encoder200signals an additional flag to indicate if luma-dependent chroma residual scaling is enabled or not. When luma mapping is not used, luma-dependent chroma residual scaling is disabled. Further, luma-dependent chroma residual scaling is always disabled for chroma blocks whose area is less than or equal to 4.

Chroma residual scaling depends on the average value of top and/or left reconstructed neighboring luma samples of the current virtual pipeline data unit (VPDU). If the current CU is inter 128×128, inter 128×64, or inter 64×128, then video decoder300uses the chroma residual scaling factor derived for the CU associated with the first VPDU for all chroma transform blocks in that CU. Denote avgYr as the average of the reconstructed neighboring luma samples (seeFIG. 9). Video decoder300computes the value of CscaleInvthrough the following steps:

1) Find the index YIdxof the piecewise linear model to which avgYr belongs based on the InvMap function.

2) CScaleIn=cScaleInv[YIdx], where cScaleInv[ ] is a 16-piece LUT pre-computed table based on the value of SignalledCW[i] and an offset value signaled in the APS for chroma residual scaling process.

Unlike luma mapping, which is performed on a sample basis, CScaleInvis a constant value for the entire chroma block. With CScaleInv, chroma residual scaling is applied as follows:

Video encoder 200:CResScale=CRes*CScale=CRes/CScaleInv

Video decoder 300:CRes=CResScale/CScale=CResScale*CScaleInv

FIG. 10is a block diagram illustrating an example LMCS architecture when local illumination compensation (LIC) is used. In the example ofFIG. 10similar blocks as those inFIG. 9are numbered the same. When LMCS is enabled, the neighboring reconstruction samples (Recneig) of the luma component relative to the current CU need to be converted back from the mapped domain to the pixel domain before motion compensation operates.FIG. 10shows the LMCS architecture with LIC enabled. In the luma motion compensation loop, video decoder300may use an inverse lookup table at the CU level to map reconstructed luma signals (Yr) back to the pixel domain (denoted as Recneig) before video decoder300can derive the LIC model parameters a and b in the pixel domain. This inverse mapping is represented by inverse reshape530between the reconstruction502and the motion compensation510.

When LIC and LMCS are both enabled, LIC is the only inter prediction mode that requires a video coder, such as video decoder300, to load the inverse look-up table at the CU level to convert reconstruction signals back to the pixel domain for LIC model parameter derivation, as LMCS is a picture level tool. Unlike the standardization of VVC which requires loading this inverse table at the CTU/VPDU/Picture level within the loop filtering stage, this additional inverse mapping at the CU level would introduce additional hardware-implementation burden to the motion compensation module of video decoder300, resulting in potentially longer latency during this motion compensation stage.

FIG. 11is a block diagram illustrating an example LMCS architecture when LIC is used according to the techniques of this disclosure. In the example ofFIG. 11similar blocks as those inFIG. 9are numbered the same. LIC model parameter derivation in an alternative domain is now discussed. According to the techniques of this disclosure, video decoder300may use the same derivation process of LIC parameters, but apply the derivation process directly in the mapped domain. Video decoder300may apply the resulting model parameters directly to the motion-compensated samples in the pixel domain. For example, the CU-level forward lookup table may be used to convert or reshape a pixel domain reference template block from the pixel domain to the mapped domain to determine a mapped domain reference template block. For example, video decoder300may use a forward mapping function (e.g., forward reshape532) to reshape the pixel domain reference template block to a mapped domain reference template block. AsFIG. 11shows, the mapped-domain reference template block (e.g., shown exiting forward reshape532and entering motion compensation510) and the mapped-domain neighboring reconstruction template block (e.g., shown exiting reconstruction502and entering motion compensation510) are both used as if they were operating on pixel-domain signals to derive the LIC model parameters a and b. For example, both Recrefneigand Recneigare input to motion compensation510, unlike the example ofFIG. 9. The derived model parameters (together with the shift number that is equal to N, where N is an integer number (an integer shift), e.g., 6) are applied to the motion-compensated prediction signals in the pixel domain by motion compensation510. As a result, the inverse lookup table at the CU level can be avoided completely, which may decrease the latency and improve the decoding efficiency of video decoder300. For example, video decoder300may use a mapped-domain reference template block and the mapped-domain neighboring reconstruction template block as if they were operating on pixel-domain signals to derive the LIC model parameters a and b. Video decoder300may apply the LIC model parameters a and b to the motion-compensated prediction signals in the pixel domain.

In another example, video decoder300may apply the derived model parameters to mapped motion-compensated prediction signals, instead of being applied to pixel-domain motion-compensated prediction signals.

The same techniques of LIC model parameter derivation in the mapped domain can be extended to other dynamic range mappings (denoted as an alternative domain). Given a forward and inverse mapping function for the dynamic range mapping, the same architecture as the aforementioned architecture ofFIG. 11can be applied directly. For example, video decoder300may use a gamma function FwdMap(x)=A*xras the forward mapping function, where A and r are constant values, respectively. For example, the mapping functions can also be of polynomial form, such as FwdMap(x)=Σi∈{0, 1, . . . n}aixifor forward mapping, where airepresents a real number which is a coefficient of the polynomial function of x. Such coefficients may be trained on a frame-by-frame basis and be signaled in an APS. The inverse mapping function can be easily derived based on backward querying by FwdMap(x). For example, FwdMap(x0) FwdMap(x1), . . . and FwdMap(xi) are all mapped to a certain value, e.g., yj, then video decoder300may determine one of the entries in inverse function InvMap(yj) to be one of the x0, x1, . . . and xi, or a weighted average of them.

FIG. 12is a flowchart illustrating LIC parameter derivation techniques according to this disclosure. Video decoder300may reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block (550). For example, video decoder300may map luma components of a reference template block using LMCS to create a mapped domain reference template block. Video decoder300may derive LIC model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block (552). For example, rather than load an inverse mapping table from memory, video decoder300may determine an inverse mapping function based on the table and apply the inverse mapping function to the mapped domain reference template block and the mapped domain neighboring reconstruction template block before deriving the LIC parameters, video decoder300may derive the LIC parameters based on the mapped domain reference template block and a mapped domain neighboring reconstruction template block.

Video decoder300may apply the LIC model parameters to motion-compensated prediction signals (554). For example, video decoder300may apply the LIC model parameters to prediction samples after video decoder300motion compensates the prediction samples.

Video decoder300may decode the video data based on the application of the LIC model parameters (556). For example, video decoder300may decode the LIC-compensated motion-compensated prediction signals.

In some examples, applying the LIC model parameters further includes applying an integer shift to the motion-compensated prediction signals. In some examples, the integer shift is a right shift by 6.

In some examples, the motion-compensated prediction signals are in the pixel domain. In some examples, the motion-compensated prediction signals are in the mapped domain.

In some examples, the forward mapping function includes FwdMap(Y_pred)=((b2−b1)/(a2−a1))*(Y_pred−a1)+b1, where Y_pred is a luma prediction signal, i is a piece index, a1 is an input pivot point of i, a2 is an input pivot point of i+1, b1 is a mapped pivot point of i, and b2 is a mapped pivot point of i+1. In some examples, the forward mapping function includes a gamma function. In some examples, the gamma function includes FwdMap(x)=A*xr, where A and r are constant values. In some examples, the forward mapping function comprises a polynomial function. In some examples, the polynomial function comprises FwdMap(x)=Σi∈{0, 1, . . . , n} aixi, where airepresents a real number which is a coefficient of the polynomial function of x.

In some examples, the LIC model parameters are applied on a coding unit (CU) basis. For example, video decoder300may apply the LIC model parameters on a coding unit basis. In some examples, video decoder300may determine that luma mapping with chroma scaling is enabled for the CU.

FIG. 13is a flowchart illustrating an example method for encoding a current block in accordance with the techniques of this disclosure. The current block may comprise a current CU. Although described with respect to video encoder200(FIGS. 1and3), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 13.

In this example, video encoder200initially predicts the current block (350). For example, video encoder200may form a prediction block for the current block. Video encoder200may, for example, form the prediction block using any of the various LIC and LMCS techniques described above. Video encoder200may then calculate a residual block for the current block (352). To calculate the residual block, video encoder200may calculate a difference between the original, unencoded block and the prediction block for the current block. Video encoder200may then transform the residual block and quantize transform coefficients of the residual block (354). Next, video encoder200may scan the quantized transform coefficients of the residual block (356). During the scan, or following the scan, video encoder200may entropy encode the transform coefficients (358). For example, video encoder200may encode the transform coefficients using CAVLC or CABAC. Video encoder200may then output the entropy encoded data of the block (360).

FIG. 14is a flowchart illustrating an example method for decoding a current block of video data in accordance with the techniques of this disclosure. The current block may comprise a current CU. Although described with respect to video decoder300(FIGS. 1 and 4), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 14.

Video decoder300may receive entropy encoded data for the current block, such as entropy encoded prediction information and entropy encoded data for transform coefficients of a residual block corresponding to the current block (370). Video decoder300may entropy decode the entropy encoded data to determine prediction information for the current block and to reproduce transform coefficients of the residual block (372). Video decoder300may predict the current block (374), e.g., using an intra- or inter-prediction mode as indicated by the prediction information for the current block, to calculate a prediction block for the current block. Video decoder300may, for example, form the prediction block using any of the various LIC and LMCS techniques described above. As part of predicting the current block, video decoder300may use the same prediction techniques as, or inverse prediction techniques of, those used inFIG. 12. Video decoder300may then inverse scan the reproduced transform coefficients (376), to create a block of quantized transform coefficients. Video decoder300may then inverse quantize the transform coefficients and apply an inverse transform to the transform coefficients to produce a residual block (378). Video decoder300may ultimately decode the current block by combining the prediction block and the residual block (380).

By deriving LIC model parameters from a mapped domain reference template block and a mapped domain neighboring reconstruction template block, video decoder300may avoid loading an inverse mapping table from memory, thereby reducing decoding latency and increasing processing efficiency.

The following numbered clauses illustrate one or more aspects of the devices and techniques described in this disclosure.

Clause 1A. A method of coding video data, the method comprising: reshaping a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; deriving local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; applying the LIC model parameters to motion-compensated prediction signals; and coding the video data based on the application of the LIC model parameters.

Clause 2A. The method of clause 1A, wherein applying the LIC model parameters further comprises applying an integer shift.

Clause 3A. The method of clause 2A, wherein the integer shift is a right shift by 6.

Clause 4A. The method of clause 1A, wherein the motion-compensated prediction signals are in the pixel domain.

Clause 5A. The method of clause 1A, wherein the motion-compensated prediction signals are in the mapped domain.

Clause 6A. The method of clause 1A, wherein the forward mapping function comprises FwdMap(Y_pred)=((b2−b1)/(a2−a1))*(Y_pred−a1)+b1.

Clause 7A. The method of clause 1A, wherein the forward mapping function comprises a gamma function.

Clause 8A. The method of clause 7A, wherein the gamma function comprises FwdMap(x)=A*xr, where A and r are constant values.

Clause 9A. The method of clause 1A, wherein the forward mapping function comprises a polynomial function.

Clause 10A. The method of clause 9A, wherein the polynomial function comprises FwdMap(x)=Σi∈{0, 1, . . . , n} aixi, where airepresents a real number which is a coefficient of the polynomial function of x.

Clause 11A. The method of clause 1A, further comprising: deriving an inverse mapping function based on the forward mapping function; and applying the inverse mapping function to reshape the mapped domain neighboring reconstruction template block.

Clause 12A. The method of any of clauses 1A-11A, wherein coding comprises decoding.

Clause 13A. The method of any of clauses 1A-12A, wherein coding comprises encoding.

Clause 14A. A device for coding video data, the device comprising one or more means for performing the method of any of clauses 1A-13A.

Clause 15A. The device of clause 14A, wherein the one or more means comprise one or more processors implemented in circuitry.

Clause 16A. The device of any of clauses 14A and 15A, further comprising a memory to store the video data.

Clause 17A. The device of any of clauses 14A-16A, further comprising a display configured to display decoded video data.

Clause 18A. The device of any of clauses 14A-17A, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

Clause 19A. The device of any of clauses 14A-18A, wherein the device comprises a video decoder.

Clause 20A. The device of any of clauses 14A-19A, wherein the device comprises a video encoder.

Clause 21A. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 1A-11A.

Clause 1B. A method of decoding video data, the method comprising: reshaping a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; deriving local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; applying the LIC model parameters to motion-compensated prediction signals; and decoding the video data based on the application of the LIC model parameters.

Clause 2B. The method of clause 1B, wherein applying the LIC model parameters further comprises applying an integer shift to the motion-compensated prediction signals.

Clause 3B. The method of clause 2B, wherein the integer shift is a right shift by 6.

Clause 4B. The method of any of clauses 1B-3B, wherein the motion-compensated prediction signals are in a pixel domain.

Clause 5B. The method of any of clauses 1B-3B, wherein the motion-compensated prediction signals are in a mapped domain.

Clause 6B. The method of any of clauses 1B-5B, wherein the forward mapping function comprises: FwdMap(Y_pred)=((b2−b1)/(a2−a1))*(Y_pred−a1)+b1, where Y_pred is a luma prediction signal, i is a piece index, a1 is an input pivot point of a2 is an input pivot point of i+1, b1 is a mapped pivot point of i, and b2 is a mapped pivot point of i+1.

Clause 7B. The method of any of clauses 1B-5B, wherein the forward mapping function comprises a gamma function.

Clause 8B. The method of clause 7B, wherein the gamma function comprises: FwdMap(x)=A*xr, where A and r are constant values.

Clause 9B. The method of any of clauses 1B-5B, wherein the forward mapping function comprises a polynomial function.

Clause 10B. The method of clause 9B, wherein the polynomial function comprises FwdMap(x)=Σi∈{0, 1, . . . , n} aixi, where airepresents a real number which is a coefficient of the polynomial function of x.

Clause 11B. The method of any of clauses 1B-10B, wherein the LIC model parameters are applied on a coding unit (CU) basis.

Clause 12B. The method of clause 11B, further comprising: determining that luma mapping with chroma scaling is enabled for the CU.

Clause 13B. A device for decoding video data, the device comprising: memory configured to store the video data; and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors being configured to: reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; apply the LIC model parameters to motion-compensated prediction signals; and decode the video data based on the application of the LIC model parameters.

Clause 14B. The device of clause 13B, wherein as part of applying the LIC model parameters, the one or more processors are further configured to: apply an integer shift to the motion-compensated prediction signals.

Clause 15B. The device of clause 14B, wherein the integer shift is a right shift by 6.

Clause 16B. The device of any of clauses 13B-15B, wherein the motion-compensated prediction signals are in a pixel domain.

Clause 17B. The device of any of clauses 13B-15B, wherein the motion-compensated prediction signals are in a mapped domain.

Clause 18B. The device of any of clauses 13B-17B, wherein the forward mapping function comprises: FwdMap(Y_pred)=((b2−b1)/(a2−a1))*(Y_pred−a1)+b1, where Y_pred is a luma prediction signal, i is a piece index, a1 is an input pivot point of a2 is an input pivot point of i+1, b1 is a mapped pivot point of i, and b2 is a mapped pivot point of i+1.

Clause 19B. The device of any of clauses 13B-17B, wherein the forward mapping function comprises a gamma function.

Clause 20B. The device of clause 19B, wherein the gamma function comprises: FwdMap(x)=A*xr, where A and r are constant values.

Clause 21B. The device of any of clauses 13B-178B, wherein the forward mapping function comprises a polynomial function.

Clause 22B. The device of clause 21B, wherein the polynomial function comprises: FwdMap(x)=aixi, where airepresents a real number which is a coefficient of the polynomial function of x

Clause 23B. The device of any of clauses 13B-22B, wherein the one or more processors apply the LIC model parameters on a coding unit (CU) basis.

Clause 24B. The device of clause 23B, wherein the one or more processors are further configured to: determine that luma mapping with chroma scaling is enabled for the CU.

Clause 25B. The device of any of clauses 13B-24B, further comprising: a display configured to display the video data.

Clause 26B. The device of any of clauses 13B-25B, further comprising: a camera configured to capture the video data.

Clause 27B. The device of any of clauses 13B-26B, wherein the device comprises a mobile telephone.

Clause 28B. A non-transitory computer readable storage medium having instructions stored thereon which, when executed, cause one or more processors to: reshape a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; derive local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; apply the LIC model parameters to motion-compensated prediction signals; and decode the video data based on the application of the LIC model parameters.

Clause 29B. A device for decoding video data, the device comprising: means for reshaping a pixel domain reference template block using a forward mapping function into a mapped domain reference template block; means for deriving local illumination compensation (LIC) model parameters from the mapped domain reference template block and a mapped domain neighboring reconstruction template block; means for applying the LIC model parameters to motion-compensated prediction signals; and means for decoding the video data based on the application of the LIC model parameters.