Coding video data using derived chroma mode

An example device for decoding video data includes a memory for storing video data, and one or more processors implemented in circuitry and configured to construct an intra-prediction candidate list for a current chroma block of the video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes, determine cost (e.g., sum of absolute transform difference (SATD)) values for each of the candidate intra-prediction modes in the intra-prediction candidate list for the current chroma block, and generate a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values (e.g., the candidate intra-prediction mode having the lowest cost value).

TECHNICAL FIELD

BACKGROUND

SUMMARY

In general, this disclosure describes techniques related to intra-prediction for image and/or video coding, and more particularly, intra-prediction of chrominance (chroma) blocks of video data. According to the techniques of this disclosure, a decoder may derive a chroma coding mode, without the use of explicit signaling. These techniques may be used in the context of advanced video codecs, such as extensions of HEVC or the next generation of video coding standards.

In one example, a method of decoding video data includes constructing an intra-prediction candidate list for a current chroma block of video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes, determining cost values, such as sum of absolute transform difference (SATD) values, for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block, and generating a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values, e.g., the candidate intra-prediction mode having a lowest cost value.

In another example, a device for decoding video data includes a memory for storing video data, and one or more processors implemented in circuitry and configured to construct an intra-prediction candidate list for a current chroma block of the video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes, determine cost values for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block, and generate a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values.

In another example, a device for decoding video data includes means for constructing an intra-prediction candidate list for a current chroma block of video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes, means for determining cost values for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block, and means for generating a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values.

In another example, a computer-readable storage medium is encoded with instructions that, when executed, cause a programmable processor to construct an intra-prediction candidate list for a current chroma block of video data indicating candidate intra-prediction modes for the current chroma block, determine cost values for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block, and generate a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values.

DETAILED DESCRIPTION

In general, the techniques of this disclosure relate to coding (encoding or decoding) of image and/or video data. This disclosure primarily describes techniques in the context of video coding, but it should be understood that these techniques may also be applied to image coding. Thus, where video coding techniques are described, it should be understood that image coding techniques may generally be substituted. Video data includes a series of pictures, which generally include color pictures. A raw color picture may be represented by pixels in a red-green-blue (RGB) color space. A video encoder or front-end unit may process (e.g., convert) a raw color picture to a luminance (luma) and chrominance (chroma) color space, e.g., one set of luma information and two sets of chroma information, one for blue-hue chroma and another for red-hue chroma. This luminance and chrominance color space may also be referred to as YUV or YCbCr. The video encoder may then encode the luma and chroma.

Moreover, video and image coding techniques generally include dividing a picture into blocks and coding each block by predicting the block and coding data representing differences (that is, a residual) between the block and its prediction. A video coder may predict a block using intra-prediction (using information from only the same picture) or inter-prediction (using information from other, previously coded pictures).

The techniques of this disclosure generally include techniques for determining an intra-prediction mode to be applied to predict a chroma block of video data. Rather than coding data signaling the intra-prediction mode, this disclosure describes techniques for determining (or deriving) the intra-prediction mode for a chroma block of video data implicitly, i.e., without explicit signaling (by a source device) or receipt of signaled data (by a destination device) of the mode. By deriving the coding mode in this manner, the techniques of this disclosure may reduce an amount of data signaled in a bitstream including coded video data, without overly increasing the complexity of the video coding process. In this manner, the techniques may improve the field of video coding, in that bitrate may be reduced without overly increasing an amount of processing needed to perform these techniques. Moreover, the techniques of this disclosure reduce the processing requirements of previously proposed techniques for decoder-derived chroma intra-prediction modes, thereby further improving the field of video coding.

In particular, Yu Han, Jicheng An, Jianhua Zheng, “Decoder-Side Direct Mode Prediction,” JVET-E0027, describes a decoder-side derived direct mode (DDM) technique for the JVET exploration model (JEM). As discussed in greater detail below, the DDM technique of JVET-E0027 includes an iterative search process and a process for downsampling a luma block to calculate a sum of absolute transform difference (SATD) value. This disclosure recognizes that the iterative search at the encoder and decoder sides may introduce too much complexity for both software and hardware implementations of the DDM techniques. Additionally, this disclosure recognizes that even using a six-tap filter to downsample a luma block for the SATD calculation, the complexity of using a downsampled block may still be too much for larger coding blocks, e.g., where the coding block size is equal to 64×64. The techniques of this disclosure reduce these complexities, thereby improving the field of video coding.

Video coding standards include ITU-T H.261, ISO/IEC MPEG-1 Visual, ITU-T H.262 or ISO/IEC MPEG-2 Visual, ITU-T H.263, ISO/IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC), including its Scalable Video Coding (SVC) and Multi-view Video Coding (MVC) extensions. In addition, a new video coding standard, namely High Efficiency Video Coding (HEVC), has been developed by the Joint Collaboration Team on Video Coding (JCT-VC) of ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Motion Picture Experts Group (MPEG). The HEVC specification is available from phenix.int-evry.fr/jct/doc_end_user/documents/14_Vienna/wg11/JCTVC-N1003-v1.zip. The specification of HEVC and its extensions including Format Range (RExt), Scalability (SHVC), and Multi-View (MV-HEVC) Extensions and Screen Content Extensions is available from phenix.int-evey.fr/jct/doc_end_user/current_document.php?id=10481.

ITU-T VCEG (Q6/16) and ISO/IEC MPEG (JTC 1/SC 29/WG 11) are studying the potential need for standardization of future video coding technology with a compression capability that significantly exceeds that of the current HEVC standard (including its current extensions and near-term extensions for screen content coding and high-dynamic-range coding). The groups are working together on this exploration activity in a joint collaboration effort known as the Joint Video Exploration Team (JVET) to evaluate compression technology designs proposed by their experts in this area. The JVET reference software, i.e., Joint Exploration Model 4 (JEM 4), is available from jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/tags/HM-16.6-JEM-4.0/. J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, J. Boyce “Algorithm description of Joint Exploration Test Model 4”, JVET-D1001, Chengdu, October 2016, provides an algorithm description for JEM4.

FIG. 1is a block diagram illustrating an example video encoding and decoding system100that may perform the techniques of this disclosure. The techniques of this disclosure are generally directed to coding (encoding and/or decoding) video data. In general, video data includes any data for processing a video. Thus, video data may include raw, uncoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

In some examples, source device102may output encoded video data to file server114or another intermediate storage device that may store the encoded video generated by source device102. Destination device116may access stored video data from file server114via streaming or download. File server114may be any type of server device capable of storing encoded video data and transmitting that encoded video data to the destination device116. File server114may represent a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device116may access encoded video data from file server114through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both that is suitable for accessing encoded video data stored on file server114. File server114and input interface122may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

Although not shown inFIG. 1, in some examples, video encoder200and video decoder300may each be integrated with an audio encoder and/or audio decoder, and may include appropriate MUX-DEMUX units, or other hardware and/or software, to handle multiplexed streams including both audio and video in a common data stream. If applicable, MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

Video encoder200and video decoder300may operate according to a video coding standard, such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC) or extensions thereto, such as the multi-view and/or scalable video coding extensions. Alternatively, video encoder200and video decoder300may operate according to other proprietary or industry standards, such as the Joint Exploration Test Model (JEM). The techniques of this disclosure, however, are not limited to any particular coding standard.

As another example, video encoder200and video decoder300may be configured to operate according to JEM. According to JEM, a video coder (such as video encoder200) partitions a picture into a plurality of coding tree units (CTUs). Video encoder200may partition a CTU according to a tree structure, such as a quadtree-binary tree (QTBT) structure. The QTBT structure of JEM removes the concepts of multiple partition types, such as the separation between CUs, PUs, and TUs of HEVC. A QTBT structure of JEM includes two levels: a first level partitioned according to quadtree partitioning, and a second level partitioned according to binary tree partitioning. A root node of the QTBT structure corresponds to a CTU. Leaf nodes of the binary trees correspond to coding units (CUs).

In some examples, video encoder200and video decoder300may use a single QTBT structure to represent each of the luminance and chrominance components, while in other examples, video encoder200and video decoder300may use two or more QTBT structures, such as one QTBT structure for the luminance component and another QTBT structure for both chrominance components (or two QTBT structures for respective chrominance components).

In accordance with the techniques of this disclosure, video encoder200and video decoder300may derive intra-prediction modes for chroma blocks, without explicitly coding data representing the intra-prediction modes for the chroma blocks. For example, as discussed above, NET-E0027 describes a six-tap downsampling filter and an iterative search process for deriving intra-prediction modes for chroma blocks. In accordance with the techniques of this disclosure, however, video encoder200and video decoder300may use different down-sample ratios for blocks (corresponding luma or chroma neighboring blocks) for decoder-side intra-prediction mode derivation. In one example, the ratio may depend on the sizes of coding blocks. That is, video encoder200and video decoder300may select the ratio as a function of the size of a coding block. In another example, video encoder200and video decoder300may apply different filtering methods for various blocks.

Additionally or alternatively, video encoder200and video decoder300may perform techniques that are simpler than the iterative search techniques described in NET-E0027. In particular, video encoder200and video decoder300may check the cost (e.g., sum of absolute transform distance (SATD)) with one or more direct modes, and/or one or more modes from spatial neighboring luma/chroma blocks. These modes may be denoted as a first set of candidates. In one example, video encoder200and video decoder300may further check certain modes if these modes are not included in the first set of candidates, such as DC, planar, horizontal, and/or vertical modes. Alternatively, video encoder200and video decoder300may further check a second set of candidates, where the second set of candidates may include angular modes close to one or more angular modes in the first set of candidates (where “close” may be, e.g., within one unit of angle, e.g., as shown inFIG. 5as discussed below).

In another example, video encoder200and video decoder300may check only a single set of candidates that depends on decoded information, including but not limited to any or all of the following: luma intra prediction mode, block size, transform type, and/or transform coefficients. For example, if the luma block is coded using DC or Planar mode, and the number of coded coefficients is small (e.g., no non-zero coefficients, or only a few non-zero coefficients), only DC or Planar mode can be considered as the candidate mode for DDM.

In some examples, when decoder-side derived direct mode is applied, the indication of chroma modes may be as follows. In one example, video encoder200and video decoder300may code a one bit flag that indicates whether the DDM is either CCLM or derived direction mode. In one example, video encoder200and video decoder300may code a one bit flag to indicate whether the DDM is either CCLM or derived direction mode or direct modes from luma blocks. In one example, video encoder200and video decoder300may construct the candidate list for chroma intra prediction mode list to only contain two modes: one is CCLM and the other is the derived direction mode.

In some examples, rather than using luma blocks to calculate a distortion cost, video encoder200and video decoder300may use a template that identifies neighboring chroma blocks to use for calculating the distortion cost. The template may be predefined, or video encoder200may construct the template and signal data representative of the template, e.g., in a video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header, block header, or the like.

In some examples, video encoder200and video decoder300may derive chroma derived modes in units of sub-blocks. That is, video encoder200and video decoder300may divide a luma block into multiple sub-blocks. For each sub-block, the video coder (video encoder200or video decoder300) may derive a sub-block DDM. The video coder may then apply this sub-block DDM to the co-located chroma sub-block to perform intra prediction. In one example, when deriving the DDM for each sub-block, the video coder only uses the reconstructed luma reference samples (located at the above/left neighborhood of the current block) to perform intra prediction for each candidate intra mode.

FIGS. 2A and 2Bare a conceptual diagrams illustrating an example quadtree binary tree (QTBT) structure130, and a corresponding coding tree unit (CTU)132. J. An, Y.-W. Chen, K. Zhang, H. Huang, Y.-W. Huang, and S. Lei., “Block partitioning structure for next generation video coding,” International Telecommunication Union, COM16-C966, September 2015, proposed a QTBT structure for future video coding standards beyond HEVC, similar to that ofFIGS. 2A and 2B.

In the example ofFIG. 2A, the solid lines represent quadtree splitting, and dotted lines indicate binary tree splitting. There are two binary tree splitting types: symmetric horizontal splitting and symmetric vertical splitting. In each split (i.e., non-leaf) node of the binary tree, one flag is signaled to indicate which splitting type (i.e., horizontal or vertical) is used, where 0 indicates horizontal splitting and 1 indicates vertical splitting in this example. For the quadtree splitting, there is no need to indicate the splitting type, since quadtree nodes split a block horizontally and vertically into 4 sub-blocks with equal size. Accordingly, video encoder200may encode, and video decoder300may decode, syntax elements (such as splitting information) for a region tree level of QTBT structure130(i.e., the solid lines) and syntax elements (such as splitting information) for a prediction tree level of QTBT structure130(i.e., the dashed lines). Video encoder200may encode, and video decoder300may decode, video data, such as prediction and transform data, for CUs represented by terminal leaf nodes of QTBT structure130.

In general, CTU132ofFIG. 2Bmay be associated with parameters defining sizes of blocks corresponding to nodes of QTBT structure130at the first and second levels. These parameters may include a CTU size (representing a size of CTU132in samples), a minimum quadtree size (MinQTSize, representing a minimum allowed quadtree leaf node size), a maximum binary tree size (MaxBTSize, representing a maximum allowed binary tree root node size), a maximum binary tree depth (MaxBTDepth, representing a maximum allowed binary tree depth), and a minimum binary tree size (MinBTSize, representing the minimum allowed binary tree leaf node size).

The root node of a QTBT structure corresponding to a CTU (or coding tree block (CTB)) may have four child nodes at the first level of the QTBT structure, each of which may be partitioned according to quadtree partitioning. That is, nodes of the first level are either leaf nodes (having no child nodes) or have four child nodes. The example of QTBT structure130represents such nodes as including the parent node and child nodes having solid lines for branches. If nodes of the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), they can be further partitioned by respective binary trees. The binary tree splitting of one node can be iterated until the nodes resulting from the split reach the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). The example of QTBT structure130represents such nodes as having dashed lines for branches. The binary tree leaf node is referred to as a coding unit (CU), which is used for prediction (e.g., intra-picture or inter-picture prediction) and transform, without any further partitioning. As discussed above, CUs may also be referred to as “video blocks” or “blocks.”

In one example of the QTBT partitioning structure, the CTU size is set as 128×128 (luma samples and two corresponding 64×64 chroma samples), the MinQTSize is set as 16×16, the MaxBTSize is set as 64×64, the MinBTSize (for both width and height) is set as 4, and the MaxBTDepth is set as 4. The quadtree partitioning is applied to the CTU first to generate quad-tree leaf nodes. The quadtree leaf nodes may have a size from 16×16 (i.e., the MinQTSize) to 128×128 (i.e., the CTU size). If the leaf quadtree node is 128×128, it will not be further split by the binary tree, since the size exceeds the MaxBTSize (i.e., 64×64, in this example). Otherwise, the leaf quadtree node will be further partitioned by the binary tree. Therefore, the quadtree leaf node is also the root node for the binary tree and has the binary tree depth as 0. When the binary tree depth reaches MaxBTDepth (4, in this example), no further splitting is permitted. When the binary tree node has width equal to MinBTSize (4, in this example), it implies no further horizontal splitting is permitted. Similarly, a binary tree node having a height equal to MinBTSize implies no further vertical splitting is permitted for that binary tree node. As noted above, leaf nodes of the binary tree are referred to as CUs, and are further processed according to prediction and transform without further partitioning.

In addition, the QTBT block structure supports the feature that luma and chroma may have separate QTBT structures. Currently in JEM, for P and B slices, the luma and chroma CTUs in one CTU share the same QTBT structure. For I slice, the luma CTU is partitioned into CUs by a QTBT structure, and chroma CTUs are partitioned into chroma CUs by another QTBT structure. This means that, in JEM, a CU in an I slice includes a coding block of luma component or coding blocks of two chroma component, and a CU in P and B slice a CU includes coding blocks of all three color components.

In JEM, six chroma modes per PU are allowed. The DM mode indicates that a chroma PU utilizes the same prediction mode as a corresponding luma PU. For I slice, the QTBT structure for luma and chroma may be different. When DM mode is used in I slice, the luma prediction mode of the PU covering the top-left position is inherited.

FIGS. 3A and 3Bare conceptual diagrams illustrating example QTBT structures for luma and chroma blocks. In particular,FIG. 3Adepicts an example luma QTBT structure134A, whileFIG. 3Bdepicts an example chroma QTBT structure134B. The left partition of the chroma CTU138(filled with grey) inFIG. 3Band its corresponding luma block136(filled with grey) with finer partitions inFIG. 3Aare depicted. L(i) marked in each partition indicates the luma intra prediction mode index equal to i. In this case, when the left partition is coded with DM mode, it picks the LM mode from the top-left corresponding luma block, i.e., intra prediction mode index equal to 1 will be used to code/decode the left partition of the chroma block.

Table 1 below specifies the mode arrangement used in signaling the chroma mode of JEM. In order to remove possible redundancy in the signaling, arising when derived refers to one of the modes always present, angular (66 when there are 67 intra modes in total) mode (named “alternative mode”) is used to substitute the duplicate mode, as shown in Table 1.

TABLE 1Specification of Chroma Intra PredictionModes and Associated NamesChromaChroma intra alternativeIntramode, if default modepredictionis equal to the derivedmodePrimary modemodeDefault0INTRA_PLANARINTRA_ANGULAR66modes1INTRA_ANGULAR50INTRA_ANGULAR662INTRA_ANGULAR18INTRA_ANGULAR663INTRA_DCINTRA_ANGULAR664LMN/A5Derived mode (DM)N/A

FIG. 4is a conceptual diagram illustrating an example of intra-prediction for a 16×16 block140. A video coder, such as video encoder200or video decoder300, generally performs intra-prediction on a block, such as block140, using its spatially neighboring samples142, which represent reconstructed image samples. To perform intra-prediction, the video coder may predict samples of block140using above-neighboring and/or left-neighboring samples of neighboring samples142. For example, the video coder may use neighboring samples142to form predicted samples in a particular direction, such as along prediction direction144.

HEVC, as one example, includes 35 intra-prediction modes, including 33 directional (or angular) modes and two non-directional modes: DC and planar mode.FIG. 5is a conceptual diagram illustrating various prediction modes of HEVC. Table 2 specifies the various intra-prediction modes of HEVC.

FIG. 6is a conceptual diagram illustrating generation of a prediction block using planar mode. Planar mode is typically the most frequently used intra-prediction mode for coding video data. To perform planar prediction for an N×N block150, for each sample pxylocated at (x, y) (i.e., sample152), the video coder calculates a prediction value using four specific neighboring reconstructed samples, i.e., reference samples154A,154B,154C, and154D (reference samples154), using a bilinear filter. The four reference samples154include the top-right reconstructed sample TR154A, the bottom-left reconstructed sample BL154B, the reconstructed sample154C located at the same column (rx,−1) of the current sample denoted by T, and the sample154D at the same row (r−1,y) of the current sample denoted by L. The planar mode can be formulated as:
pxy=(N−x−1)·L+(N−y−1)·T+x·TR+y·BL(1)

For DC mode, the video coder fills the prediction block with the average value of the neighboring reconstructed samples. Generally, both Planar and DC modes are applied for modeling smoothly varying and constant image regions.

FIG. 7is a conceptual diagram illustrating an example of predicting a block160using an angular intra-prediction mode. For angular intra-prediction modes (which, in HEVC, include 33 different prediction directions), the intra-prediction process is described as follows. For each given angular intra-prediction, the intra prediction direction (or angle) can be identified accordingly. For example, according toFIG. 5, intra mode 18 corresponds to a pure horizontal prediction direction, and intra mode 26 corresponds to a pure vertical prediction direction.

Given a specific intra prediction direction, for each sample of the prediction block, the video coder projects the coordinate (x, y) of the sample to the row/column of neighboring reconstructed samples along the prediction direction. For example, as shown inFIG. 7, sample162is predicted using reference samples164,166. Suppose (x,y) is projected to the fractional position α between two neighboring reconstructed samples L164and R166. Then the video coder calculates the prediction value for (x, y) using a two-tap bi-linear interpolation filter, formulated as follows in HEVC:
pxy=(1−α)·L+α·R(2)

To avoid floating point operations, in HEVC, the above calculation is actually approximated using integer arithmetic as:
pxy=((32−a)·L+a·R+16)>>5  (3)
where a is an integer equal to 32*α.

Quite often, coding structures in the chroma signal follow those of the co-located luma signal. Taking advantage of this behavior, HEVC introduced a mechanism to indicate the cases when a chroma block (or prediction unit (PU) in HEVC) utilizes the same prediction mode as the corresponding luma block (or PU). Table 3 below specifies the mode arrangement used in signaling the chroma mode in conventional HEVC. In the case that derived mode is indicated for a PU of HEVC, the prediction is performed by using the corresponding luma PU mode. In order to remove the possible redundancy in the signaling arising when the derived mode refers to one of the modes always present, angular (34) mode (named alternative mode) is used to substitute the duplicate mode as shown in Table 3, in HEVC.

TABLE 3Specification of Chroma Intra PredictionModes and Associated NamesChromaChroma intra AlternativeIntramode, if the defaultpredictionmode is equal to themodePrimary modederived modeDefault0INTRA_PLANARINTRA_ANGULAR34modes1INTRA_ANGULAR26INTRA_ANGULAR342INTRA_ANGULAR10INTRA_ANGULAR343INTRA_DCINTRA_ANGULAR344Derived mode (DM)N/A

In chroma mode coding according to HEVC, a one-bit (1-b) syntax element (0) is assigned to the most often occurring derived mode, while three-bit (3-b) syntax elements (100, 101, 110, 111) are assigned to the remaining four modes. Only the first bin (or bit) is coded with one context model, and the remaining two bins (if needed) are bypass coded.

JEM has extended the 33 angular modes of HEVC to 65 angular modes, plus DC and planar mode. JEM also includes techniques for signaling intra-prediction modes using six most probable modes (MPMs) of an MPM candidate list, as discussed in EE5: Enhanced Cross-component Linear Model Intra-prediction, K. Zhang, J. Chen, L. Zhang, M. Karczewicz, “EE5: Enhanced Cross-component Linear Model Intra-prediction,” JVET-E0077. According to JEM, a video coder first codes one flag (MPM flag) for a block to indicate whether the intra-prediction mode for the block is from the MPM candidate list. If the intra-prediction mode is from the MPM candidate list, the video coder then codes an index into the MPM candidate list that identifies the intra-prediction mode used to predict the block in the MPM candidate list. If the intra-prediction mode is not from the MPM candidate list, the video coder codes an index into the remaining intra-prediction modes that identifies the intra-prediction mode used to predict the block from the remaining intra-prediction modes.

JEM also includes techniques for performing adaptive multiple core transforms. In addition to DCT-II and 4×4 DST-VII, which have been employed in HEVC, an Adaptive Multiple Transform (AMT) scheme is used in JEM for residual coding for both inter and intra coded blocks. According to the AMT scheme, a video coder uses multiple selected transforms from the DCT and DST families, other than the transforms of HEVC. The newly introduced transform matrices are DST-VII, DCT-VIII, DST-I and DCT-V.

For intra residue coding, due to the different residual statistics of different intra prediction modes, JEM describes a mode-dependent transform candidate selection process. JEM defines three transform sub-sets, as shown in Table 4, and the video coder selects the transform subset based on the intra prediction mode, as specified in Table 5.

With the sub-set conception, the video coder identifies a transform subset using the intra-prediction mode of a coding unit (CU) with the CU-level AMT flag being equal to 1. After that, for each of the horizontal and vertical transform, the video coder selects one of the two transform candidates in the identified transform subset based on data that is explicitly signaled with lag. For inter-prediction residual, however, only one transform set, which includes DST-VII and DCT-VIII, is used for all inter modes and for both horizontal and vertical transforms.

FIG. 8is a conceptual diagram illustrating example sets of reference samples used for linear model prediction mode for video coding. JEM describes a linear model (LM) prediction mode for video coding. Although the cross-complement redundancy is significantly reduced in YCbCr color space, correlation between three color components still exists. Various methods have been studied to improve the video coding performance by further reduce the correlation.

In 4:2:0 chroma video coding, a method named Linear Model (LM) prediction mode has been well studied during development of HEVC standard. With LM prediction mode, the chroma samples are predicted based on downsampled reconstructed luma samples of the same block by using a linear model as follows:
predC(i,j)=α·recL(i,j)+β  (4)
where predC(i, j) represents the prediction of chroma samples in a block and recL(i, j) represents the downsampled reconstructed luma samples of the same block. Parameters α and β are derived from causal reconstructed samples around the current block, and one example for selected reference samples is depicted inFIG. 8. Denote the chroma block size by N×N, then both i and j are within the range [0, N).

Parameters α and β in equation (4) are derived by minimizing regression error between the neighboring reconstructed luma and chroma samples around the current block.

And the parameters α and β are solved as follows

α=I⁢⁢Σ⁢⁢xi·yi-Σ⁢⁢xi·Σ⁢⁢yiI⁢⁢Σ⁢⁢xi·xi-Σ⁢⁢xi·Σ⁢⁢xi(6)β=(Σ⁢⁢yi-α·Σ⁢⁢xi)⁢/⁢I(7)
where xiis a downsampled reconstructed Luma reference sample, y, represents reconstructed Chroma reference samples, and I is an amount of the reference samples. For a target N×N chroma block, when both left and above causal samples are available, total involved samples number I is equal to 2N; when only left or above causal samples are available, total involved samples number I is equal to N.

In summary, when LM prediction mode is applied, the following steps are invoked in order:a) Downsampling the neighboring luma samplesb) Derive the linear parameters (i.e., α and β)c) Downsampling the current luma block and derive the prediction from the downsampled luma block and linear parameters

To further improve the coding efficiency, a video coder may use a downsampling filter of (1, 2, 1) and (1, 1) to derive neighboring samples xiand downsampled luma samples recL(1, j) within the corresponding luma block.

In JEM, the LM prediction mode is extended to the prediction between two chroma components, i.e., Cr component is predicted from Cb component. Instead of using the reconstructed sample signal, the cross component prediction is applied in residual domain. This is implemented by adding a weighted reconstructed Cb residual to the original Cr intra prediction to form the final Cr prediction:
pred*Cr(i,j)=predCr(i,j)+α·resiCb′(i,j)  (8)

The scaling factor α is derived in as in LM mode. The only difference is an addition of a regression cost relative to a default α value in the error function so that derived scaling factor is biased towards the default value (−0.5).

In JEM, LM prediction mode is added as one additional chroma intra prediction mode. At encoder side, one more RD cost check for chroma component is added for selecting the chroma intra prediction mode. The prediction from luma to chroma is called cross-component linear mode (CCLM).

JEM describes the following for entropy coding of chroma prediction modes. In chroma mode coding, a 1-b syntax element (0) is assigned to the most often occurring derived mode, two bins (10) are assigned to LM mode, and 4-b syntax elements (1100, 1101, 1110, 1111) are assigned to the remaining four modes. The video coder codes the first two bins with one context model, and the remaining two bins (if needed) using bypass coding. Table 6 below indicates the bin string for each chroma mode, according to JEM.

Two proposals related to chroma coding were adopted during the 5thJVET meeting held from Jan. 12, 2017. The first related to improved CCLM modes, and the second related to multiple derived modes.

With respect to the improved CCLM modes, multiple linear models (MMLM) mode was adopted. According to this mode, a video coder groups samples of a block into multiple sets. The video coder calculates a threshold (Threshold) value as the average value of the neighboring reconstructed Luma samples. A neighboring sample with Rec′L[x,y]<=Threshold is classified into group 1, while a neighboring sample with Rec′L[x,y]>Threshold is classified into group 2 and two CCLM models are used for 2 groups of samples. These techniques are described in JVET-E0077.

With respect to multiple derived modes, the modes include cross-component linear model mode, multiple intra prediction modes derived from co-located luma coding blocks, and modes from spatial neighbors. The modes from spatial neighbors include five chroma prediction modes from left, above, below-left, above-right, and above-left spatial neighboring blocks of merge mode; planar and DC modes; derived modes are added, those intra modes are obtained by adding −1 or +1 to the angular modes which are already included into the list; default modes are added in the order of: Vertical (Mode 18), Horizontal (Mode 50), Mode 2, Mode 34, Mode 66, Mode 10, Mode 26; and then if any of the four default modes (Planar, Horizontal, Vertical and DC modes) is not included in the list, the missing default modes are used to replace the last one or more candidates. These techniques are described in L. Zhang, W.-J. Chien, J. Chen, X. Zhao, M. Karczewicz, “Multiple Direct Modes for chroma intra coding”, JVET-E0062.

FIG. 9is a conceptual diagram illustrating an example process for downsampling luma blocks according to the DDM techniques. To produce a luma value corresponding to chroma position170, a video coder applies the ({1 2 1} {1 2 1}) filter to luma samples at luma positions172A,172B,172C,172D,172E, and172F, adds these values together, and divides by eight. Thus, the video coder multiplies the values of luma samples172B and172E by two, and adds these values to the values of luma samples172A,172C,172D, and172F, then divides this total by eight to produce the downsampled luma value at chroma position170. By downsampling the reconstructed luma block in this manner, the video coder reduces searching complexity.

The video coder then calculates the sum of absolute transform difference (SATD) between a prediction block and the reconstruction of the down-sampled luma block, for each of the intra modes to be tested as DDM. The video coder selects the intra mode that yields the minimum SATD the best mode, i.e., DDM.

The video coder then modifies the intra chroma prediction mode list by adding the DDM mode into the intra chroma prediction mode list, which includes one cross-component linear model (CCLM) mode, one proposed DDM mode, one DM mode derived from the co-located luma block, and three default modes (planar, vertical, and horizontal; if one of these is identical to the DM mode, it is replaced by the DC mode). The binarization for the syntax element intra chroma prediction mode is specified in Table 7. Four context models are used to code the intra chroma prediction mode.

Considering the trade-off between complexity and performance, an iterative search algorithm is used for DDM prediction, as proposed in JVET-E0027. Firstly, according to JVET-E0027, the initial mode candidate list consists of planar, DC, and every 4-th mode of the 65 angular intra directions. Then, the SATD is calculated for all the candidate intra modes, and the one that minimizes the SATD is selected as the starting intra mode for the next search. If the selected intra mode is planar or DC, it is set to DDM mode, and the searching process finishes. Otherwise, JVET-E0027 proposes searching the two neighboring modes (with interval 2) of the starting intra mode. The best mode is used as the starting intra mode for the next search. In the last search, the two nearest neighboring modes (with interval 1) of the starting intra mode are checked. Finally, set DDM mode to the intra mode which minimizes the SATD.

FIG. 10is a block diagram illustrating an example video encoder200that may perform the techniques of this disclosure.FIG. 10is provided for purposes of explanation and should not be considered limiting of the techniques as broadly exemplified and described in this disclosure. For purposes of explanation, this disclosure describes video encoder200in the context of video coding standards such as the HEVC video coding standard and the H.266 video coding standard in development. However, the techniques of this disclosure are not limited to these video coding standards, and are applicable generally to video encoding and decoding.

In the example ofFIG. 10, video encoder200includes video data memory230, mode selection unit202, residual generation unit204, transform processing unit206, quantization unit208, inverse quantization unit210, inverse transform processing unit212, reconstruction unit214, filter unit216, decoded picture buffer (DPB)218, and entropy encoding unit220.

When mode selection unit202determines to perform intra-prediction for a chroma block, mode selection unit202may avoid actually signaling an indication of the intra-prediction mode for the chroma block. Instead, it may be presumed that a video decoder, such as video decoder300, will derive the intra-prediction mode using other information of the bitstream. Moreover, mode selection unit202may select the intra-prediction mode using the same information, in order to ensure that the same intra-prediction mode will be selected by the decoder.

According to the techniques of this disclosure, to derive an intra-prediction mode for a chroma block, mode selection unit202(or intra-prediction unit226thereof) may construct an intra-prediction candidate list for the chroma block. The intra-prediction candidate list may include candidate intra-prediction modes for the chroma block. Mode selection unit202may then calculate sum of absolute transform difference (SATD) values for each of the candidate intra-prediction modes in the intra-prediction candidate list for the chroma block. Mode selection unit202may then provide an indication of the intra-prediction mode having the lowest SATD value to intra-prediction unit226to cause intra-prediction unit226to generate a prediction block for the chroma block.

Filter unit216may perform one or more filter operations on reconstructed blocks. For example, filter unit216may perform deblocking operations to reduce blockiness artifacts along edges of CUs. As illustrated by dashed lines, operations of filter unit216may be skipped in some examples.

Video encoder200stores reconstructed blocks in DPB218. For instance, in examples where operations of filter unit224are not needed, reconstruction unit214may store reconstructed blocks to DPB218. In examples where operations of filter unit224are needed, filter unit216may store the filtered reconstructed blocks to DPB218. Motion estimation unit222and motion compensation unit224may retrieve a reference picture from DPB218, formed from the reconstructed (and potentially filtered) blocks, to inter-predict blocks of subsequently encoded pictures. In addition, intra-prediction unit226may use reconstructed blocks in DPB218of a current picture to intra-predict other blocks in the current picture.

In this manner, video encoder200represents an example of a device configured to encode video data including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to construct an intra-prediction candidate list for a current chroma block of the video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes, determine cost values for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block; and generate a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values.

In the example ofFIG. 11, video decoder300includes coded picture buffer (CPB) memory320, entropy decoding unit302, prediction processing unit304, inverse quantization unit306, inverse transform processing unit308, reconstruction unit310, filter unit312, and decoded picture buffer (DPB)314. Prediction processing unit304includes motion compensation unit316and intra-prediction unit318. Prediction processing unit304may include addition units to perform prediction in accordance with other prediction modes. As examples, prediction processing unit304may include a palette unit, an intra-block copy unit (which may form part of motion compensation unit318), an affine unit, a linear model (LM) unit, or the like. In other examples, video decoder300may include more, fewer, or different functional components.

When prediction processing unit304determines that a chroma block is intra-predicted, entropy decoding unit302may not decode an indication of the intra-prediction mode for the chroma block. Instead, prediction processing unit304may derive the intra-prediction mode using other information of the bitstream.

According to the techniques of this disclosure, to derive an intra-prediction mode for a chroma block, prediction processing unit304(or intra-prediction unit318thereof) may construct an intra-prediction candidate list for the chroma block. The intra-prediction candidate list may include candidate intra-prediction modes for the chroma block. Prediction processing unit304may then calculate sum of absolute transform difference (SATD) values for each of the candidate intra-prediction modes in the intra-prediction candidate list for the chroma block. Prediction processing unit304may then provide an indication of the intra-prediction mode having the lowest SATD value to intra-prediction unit318to cause intra-prediction unit318to generate a prediction block for the chroma block.

Filter unit312may perform one or more filter operations on reconstructed blocks. For example, filter unit312may perform deblocking operations to reduce blockiness artifacts along edges of the reconstructed blocks. As illustrated by dashed lines, operations of filter unit312are not necessarily performed in all examples.

Video decoder300may store the reconstructed blocks in DPB314. As discussed above, DPB314may provide reference information, such as samples of a current picture for intra-prediction and previously decoded pictures for subsequent motion compensation, to prediction processing unit304. Moreover, video decoder300may output decoded pictures from DPB for subsequent presentation on a display device, such as display device118ofFIG. 1.

In this manner, video decoder300represents an example of a video decoding device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to construct an intra-prediction candidate list for a current chroma block of the video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes, determine cost values for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block; and generate a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values.

FIG. 12is a flowchart illustrating an example method for encoding a current block. The current block may comprise a current CU. Although described with respect to video encoder200(FIGS. 1 and 10), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 12.

In this example, video encoder200initially constructs an intra-prediction candidate list for a current chroma block (344). The intra-prediction candidate list indicates (e.g., includes) candidate intra-prediction modes for the chroma block. In some examples, video encoder200may add one or more default intra-prediction modes to the intra-prediction candidate list, where the default intra-prediction modes may include DC mode, planar mode, horizontal mode, and vertical mode. In some examples, to construct the intra-prediction candidate list, video encoder200may add a first set of one or more angular intra-prediction modes to the intra-prediction candidate list corresponding to angular intra-prediction modes of one or more neighboring blocks to the current chroma block or a luma block co-located with the current chroma block, and then add a second set of one or more angular intra-prediction modes to the intra-prediction candidate list, each of the second set of one or more angular intra-prediction modes having an angle close to an angle of at least one of the intra-prediction modes of the first set (where close may be, e.g., within one unit of angle, as shown inFIG. 5). In some examples, video encoder200may add at least one direct intra-prediction mode to the intra-prediction candidate list, and add an intra-prediction mode of a neighboring block to the current chroma block to the intra-prediction candidate list.

Video encoder200may then calculate cost values, e.g., sum of absolute transform difference (SATD values) for the intra-prediction mode candidates in the candidate list (346). For example, video encoder200may downsample a reconstructed luma block co-located with the current chroma block. In some examples, video encoder200may select a downsampling ratio for the luma block from a plurality of different available downsampling ratios, e.g., as a function of a size of the luma block or of the chroma block. In some examples, video encoder200may calculate the SATD values relative to a reference block generated from one or more neighboring chroma blocks indicated by a template, rather than from the downsampled luma block. Then, for each candidate intra-prediction mode in the intra-prediction candidate list, video encoder200may generate a prediction block using the candidate intra-prediction mode corresponding to the downsampled reconstructed luma block, and calculate the SATD value for the candidate intra-prediction mode between the generated prediction block and the downsampled reconstructed luma block or the generated reference block.

Video encoder200may then select the intra-prediction candidate having the lowest SATD value as the intra-prediction mode for the current chroma block (348). Video encoder200then predicts the current chroma block (350). For example, video encoder200may form a prediction block for the current block using the selected intra-prediction mode. Video encoder200may then calculate a residual block for the current block (352). To calculate the residual block, video encoder200may calculate a difference between the original, uncoded block and the prediction block for the current block. Video encoder200may then transform and quantize coefficients of the residual block (354). Next, video encoder200may scan the quantized transform coefficients of the residual block (356). During the scan, or following the scan, video encoder200may entropy encode the coefficients (358). For example, video encoder200may encode the coefficients using CAVLC or CABAC. Video encoder200may then output the entropy coded data of the block (360).

In this manner, the method ofFIG. 12represents an example of a method of encoding a chroma block of video data, including constructing an intra-prediction candidate list for a current chroma block of video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes; determining cost values for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block; and generating a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values. Video encoder200may also use the generated prediction block to decode and reconstruct the current chroma block for subsequent prediction of other blocks in the same picture, and therefore, this method may also be described as a method of decoding video data performed by video encoder200.

FIG. 13is a flowchart illustrating an example method for decoding a current block of video data. The current block may comprise a current CU. Although described with respect to video decoder300(FIGS. 1 and 11), it should be understood that other devices may be configured to perform a method similar to that ofFIG. 13.

In this example, video decoder300initially constructs an intra-prediction candidate list for a current chroma block (364). The intra-prediction candidate list indicates (e.g., includes) candidate intra-prediction modes for the chroma block. In some examples, video decoder300may add one or more default intra-prediction modes to the intra-prediction candidate list, where the default intra-prediction modes may include DC mode, planar mode, horizontal mode, and vertical mode. In some examples, to construct the intra-prediction candidate list, video decoder300may add a first set of one or more angular intra-prediction modes to the intra-prediction candidate list corresponding to angular intra-prediction modes of one or more neighboring blocks to the current chroma block or a luma block co-located with the current chroma block, and then add a second set of one or more angular intra-prediction modes to the intra-prediction candidate list, each of the second set of one or more angular intra-prediction modes having an angle close to an angle of at least one of the intra-prediction modes of the first set (where close may be, e.g., within one unit of angle, as shown inFIG. 5). In some examples, video decoder300may add at least one direct intra-prediction mode to the intra-prediction candidate list, and add an intra-prediction mode of a neighboring block to the current chroma block to the intra-prediction candidate list.

Video decoder300may then calculate cost values, e.g., sum of absolute transform difference (SATD values) for the intra-prediction mode candidates in the candidate list (366). For example, video decoder300may downsample a reconstructed luma block co-located with the current chroma block. In some examples, video decoder300may select a downsampling ratio for the luma block from a plurality of different available downsampling ratios, e.g., as a function of a size of the luma block or of the chroma block. In some examples, video decoder300may calculate the SATD values relative to a reference block generated from one or more neighboring chroma blocks indicated by a template, rather than from the downsampled luma block. Then, for each candidate intra-prediction mode in the intra-prediction candidate list, video decoder300may generate a prediction block using the candidate intra-prediction mode corresponding to the downsampled reconstructed luma block, and calculate the SATD value for the candidate intra-prediction mode between the generated prediction block and the downsampled reconstructed luma block or the generated reference block.

Video decoder300may then select the intra-prediction candidate having the lowest SATD value as the intra-prediction mode for the current chroma block (368). Video decoder300may receive entropy coded data for the current block, such as entropy coded prediction information and entropy coded data for coefficients of a residual block corresponding to the current block (370). Video decoder300may entropy decode the entropy coded data to determine prediction information for the current block and to reproduce coefficients of the residual block (372). Video decoder300may predict the current block (374), e.g., using an intra- or inter-prediction mode as indicated by the prediction information for the current block, to calculate a prediction block for the current block. Video decoder300may then inverse scan the reproduced coefficients (376), to create a block of quantized transform coefficients. Video decoder300may then inverse quantize and inverse transform the coefficients to produce a residual block (378). Video decoder300may ultimately decode the current block by combining the prediction block and the residual block (380).

In this manner, the method ofFIG. 13represents an example of a method of decoding a chroma block of video data, including constructing an intra-prediction candidate list for a current chroma block of video data indicating candidate intra-prediction modes for the current chroma block, wherein the intra-prediction candidate list indicates a subset of allowed luminance (luma) candidate intra-prediction modes; determining cost values for each of the candidate intra-prediction modes indicated by the intra-prediction candidate list for the current chroma block; and generating a prediction block for the current chroma block using one of the candidate intra-prediction modes indicated by the intra-prediction candidate list according to the cost values.