Patent Description:
The present application generally relates to video coding and compression, and in particular but not limited to, methods and devices for quantization and de-quantization design in video coding.

Various video coding techniques may be used to compress video data. Video coding is performed according to one or more video coding standards. For example, video coding standards include versatile video coding (WC), joint exploration test model (JEM), high-efficiency video coding (H. <NUM>/HEVC), advanced video coding (H. <NUM>/AVC), moving picture experts group (MPEG) coding, or the like. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, or the like) that take advantage of redundancy present in video images or sequences. An important goal of video coding techniques is to compress video data into a form that uses a lower bit rate, while avoiding or minimizing degradations to video quality.

Document <CIT> relates to escape color coding for palette coding mode. Document <NPL> relates to scaling process for transform coefficients.

In general, this disclosure describes examples of techniques relating to quantization and de-quantization design in video coding.

According to a first aspect of the present disclosure, there is provided a method for video decoding including: deriving, from a bitstream and for a first coding block, a quantization parameter and a quantized level, deriving a scale value by looking up a scale level table based on the quantization parameter and in accordance with a determination that the first coding block is coded in a transform skip mode, obtaining a residual sample for a sample in the first coding block based on the quantized level, the scale value and a plurality of bit-shifts by applying a first de-quantization operation, wherein the first de-quantization operation is also applied to obtain a reconstructed sample for an escape sample in a second coding block coded in a palette mode and wherein a block size of the first coding block or the second coding block is not used for determining parameters of the first de-quantization operation, wherein the first de-quantization operation is as follows: <MAT> wherein pLevel is a quantized level, pSample corresponds to a reconstructed value of a residual sample in the transform skip mode or an escape sample in the palette mode, decScale[ ] is a scale value, QPis a quantization parameter, % denotes a modulo operation, QP%<NUM> represents an operation of QPmodulo <NUM>.

According to a second aspect of the present disclosure, there is provided a computing device, comprising: one or more processors, a non-transitory storage coupled to the one or more processors and a plurality of programs stored in the non-transitory storage that, when executed by the one or more processors, cause the one or more processors to perform the before said method.

According to a third aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing a bitstream for execution by a computing device having one or more processors, wherein the bitstream, when executed by the one or more processors, cause the one or more processors to perform the before said method. According to a fourth aspect of the present disclosure, there is provided a computer program product, comprising instructions stored therein, wherein, when the instructions are executed by a processor, the instructions cause the processor to perform the before said method.

A more particular description of the examples of the present disclosure will be rendered by reference to specific examples illustrated in the appended drawings. Given that these drawings depict only some examples and are not therefore considered to be limiting in scope, the examples will be described and explained with additional specificity and details through the use of the accompanying drawings.

Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth in order to assist in understanding the subject matter presented herein. But it will be apparent to one of ordinary skill in the art that various alternatives may be used. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

Throughout the disclosure, the terms "first," "second," "third," and etc. are all used as nomenclature only for references to relevant elements, e.g. devices, components, compositions, steps, and etc., without implying any spatial or chronological orders, unless expressly specified otherwise. For example, a "first device" and a "second device" may refer to two separately formed devices, or two parts, components or operational states of a same device, and may be named arbitrarily.

The terms "module," "sub-module," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," or "sub-unit" may include memory (shared, dedicated, or group) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. The module or circuit may include one or more components that are directly or indirectly connected. These components may or may not be physically attached to, or located adjacent to, one another. As used herein, the term "if" or "when" may be understood to mean "upon" or "in response to" depending on the context. These terms, if appear in a claim, may not indicate that the relevant limitations or features are conditional or optional. For example, a method may comprise steps of: i) when or if condition X is present, function or action X' is performed, and ii) when or if condition Y is present, function or action Y' is performed. The method may be implemented with both the capability of performing function or action X', and the capability of performing function or action Y'. Thus, the functions X' and Y' may both be performed, at different times, on multiple executions of the method.

<FIG> is a block diagram illustrating an exemplary video encoder in accordance with some implementations of the present disclosure. In the encoder <NUM>, a video frame is partitioned into a plurality of video blocks for processing. For each given video block, a prediction is formed based on either an inter prediction approach or an intra prediction approach. In inter prediction, one or more predictors are formed through motion estimation and motion compensation, based on pixels from previously reconstructed frames. In intra prediction, predictors are formed based on reconstructed pixels in a current frame. Through mode decision, a best predictor may be chosen to predict a current block.

A prediction residual, representing the difference between a current video block and its predictor, is sent to a Transform circuitry <NUM>. Transform coefficients are then sent from the Transform circuitry <NUM> to a Quantization circuitry <NUM> for entropy reduction. Quantized coefficients are then fed to an Entropy Coding circuitry <NUM> to generate a compressed video bitstream. As shown in <FIG>, prediction-related information <NUM> from an inter prediction circuitry and/or an Intra Prediction circuitry <NUM>, such as video block partition info, motion vectors, reference picture index, and intra prediction mode, are also fed through the Entropy Coding circuitry <NUM> and saved into a compressed video bitstream <NUM>.

In the encoder <NUM>, decoder-related circuitries are also needed in order to reconstruct pixels for the purpose of prediction. First, a prediction residual is reconstructed through an Inverse Quantization <NUM> and an Inverse Transform circuitry <NUM>. This reconstructed prediction residual is combined with a Block Predictor <NUM> to generate un-filtered reconstructed pixels for a current video block.

To improve coding efficiency and visual quality, an in-loop filter is used. For example, a deblocking filter is available in AVC, HEVC as well as the now-current version of WC. In HEVC, an additional in-loop filter called SAO (sample adaptive offset) is defined to further improve coding efficiency. In the now-current version of the WC standard, yet another in-loop filter called ALF (adaptive loop filter) is being actively investigated, and it has a good chance of being included in the final standard.

<FIG> is a block diagram illustrating an exemplary block-based video decoder <NUM> which may be used in conjunction with many video coding standards. This decoder <NUM> is similar to the reconstruction-related section residing in the encoder <NUM> of <FIG>. In the decoder <NUM>, an incoming video bitstream <NUM> is first decoded through an Entropy Decoding <NUM> to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed through an Inverse Quantization <NUM> and an Inverse Transform <NUM> to obtain a reconstructed prediction residual. A block predictor mechanism, implemented in an Intra/inter Mode Selector <NUM>, is configured to perform either an Intra Prediction <NUM>, or a Motion Compensation <NUM>, based on decoded prediction information. A set of unfiltered reconstructed pixels are obtained by summing up the reconstructed prediction residual from the Inverse Transform <NUM> and a predictive output generated by the block predictor mechanism, using a summer <NUM>. In situations where the In-Loop Filter <NUM> is turned on, a filtering operation is performed on these reconstructed pixels to derive a final reconstructed Video Output <NUM>.

Video coding/decoding standards mentioned above, such as WC, JEM, HEVC, MPEG-<NUM>, Part <NUM>, are conceptually similar. For example, they all use block-based processing. In a Joint Video Experts Team (JVET) meeting, the JVET defined the first draft of the Versatile Video Coding (WC) and the WC Test Model <NUM> (VTM1) encoding method. It was decided to include a quadtree with nested multi-type tree using binary and ternary splits coding block structure as the initial new coding feature of VVC. A quadtree is a tree in which a parent node can be split into four child nodes, each of which may become another parent node for another split into four new child nodes.

In WC, the picture partitioning structure divides the input video into blocks called coding tree units (CTUs). A CTU is split using a quadtree with nested multi-type tree structure into coding units (CUs), with a leaf CU defining a region sharing the same prediction mode, for example, intra or inter. In the disclosure, the term "unit" defines a region of an image covering all components; the term "block" is used to define a region covering a particular component, for example, luma, and may differ in spatial location when considering the chroma sampling format such as <NUM>:<NUM>:<NUM>.

A picture may be divided into a sequence of coding tree units (CTUs). The CTU concept is same to that of the HEVC. For a picture that has three sample arrays, a CTU consists of an N×N block of luma samples together with two corresponding blocks of chroma samples. <FIG> shows a picture divided into CTUs in accordance with some implementations of the present disclosure. The maximum allowed size of the luma block in a CTU is specified to be <NUM>×<NUM>. And the maximum size of the luma transform blocks is <NUM>×<NUM>.

In HEVC, a CTU is split into CUs by using a quaternary-tree structure denoted as coding tree to adapt to various local characteristics. The decision whether to code a picture area using inter-picture (temporal) or intra-picture (spatial) prediction is made at a leaf CU level. Each leaf CU can be further split into one, two, or four prediction units (PUs) according to a PU splitting type. Inside one PU, a same prediction process is applied and relevant information is transmitted to a decoder on a PU basis. After obtaining a residual block by applying the prediction process based on the PU splitting type, a leaf CU can be partitioned into transform units (TUs) according to another quaternary-tree structure similar to the coding tree for the CU. One of key feature of the HEVC structure is that it has the multiple partition conceptions including CU, PU, and TU.

In WC, a quadtree with nested multi-type tree using binary and ternary splits segmentation structure replaces the concepts of multiple partition unit types. For example, it removes the separation of the CU, PU, and TU concepts except as needed for CUs that have a size too large for the maximum transform length, and supports more flexibility for CU partition shapes. In a coding tree structure, a CU may have either a square or rectangular shape. A CTU is first partitioned by a quaternary tree, that is quadtree, structure. Then leaf nodes of the quaternary tree may be further partitioned by a multi-type tree structure.

<FIG> is a schematic diagram illustrating multi-type tree splitting modes in accordance with some implementations of the present disclosure. As shown in <FIG>, there are four splitting types in multi-type tree structure, vertical binary splitting (SPLIT_BT_VER), horizontal binary splitting (SPLIT_BT_HOR), vertical ternary splitting (SPLIT_TT_VER), and horizontal ternary splitting (SPLIT_TT_HOR). Leaf nodes of the multi-type tree are called CUs, and unless the CU is too large for the maximum transform length, this segmentation is used for prediction and transform processing without any further partitioning. Thus, the CU, PU and TU may have a same block size in the quadtree with nested multi-type tree coding block structure. Exception occurs when maximum supported transform length is smaller than the width or height of the color component of the CU.

<FIG> illustrates a signaling mechanism of partition splitting information in a quadtree with nested multi-type tree coding tree structure in accordance with some implementations of the present disclosure. A CTU is treated as a root of a quaternary tree and is first partitioned by a quaternary tree structure. Each quaternary tree leaf node, when sufficiently large to allow it, is then further partitioned by a multi-type tree structure. In the multi-type tree structure, a first flag (mtt_split_cu_flag) is signaled to indicate whether the node is further partitioned. When a node is further partitioned, a second flag (mtt_split_cu_vertical_flag) is signaled to indicate a splitting direction, and then a third flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is a binary split or a ternary split. Based on the values of mtt split_cu vertical flag and mtt_split_cu_binary_flag, the multi-type tree slitting mode (MttSplitMode) of a CU is derived as shown in Table <NUM>.

<FIG> shows a CTU divided into multiple CUs with a quadtree with nested multi-type tree coding block structure in accordance with some implementations of the present disclosure. As shown in <FIG>, bold block edges represent quadtree partitioning and the remaining edges represent multi-type tree partitioning. The quadtree with nested multi-type tree partition provides a content-adaptive coding tree structure comprised of CUs. The size of a CU may be as large as the CTU or as small as <NUM>×<NUM> in units of luma samples. For the case of the <NUM>:<NUM>:<NUM> chroma format, the maximum chroma coding block (CB) size is <NUM>×<NUM> and the minimum chroma CB size is <NUM>×<NUM>.

In WC, the maximum supported luma transform size is <NUM>×<NUM> and the maximum supported chroma transform size is <NUM>×<NUM>. When the width or height of a CB is larger than the maximum transform width or height, the CB is automatically split in the horizontal and/or vertical direction to meet the transform size restriction in that direction. Following parameters are defined and specified by Sequence Parameter Set (SPS) syntax elements for the quadtree with nested multi-type tree coding tree scheme.

In one example of the quadtree with nested multi-type tree coding tree structure, CTU size is set as <NUM>×<NUM> luma samples with two corresponding <NUM>×<NUM> blocks of <NUM>:<NUM>:<NUM> chroma samples. MinQTSize is set as <NUM>×<NUM>, MaxBtSize is set as <NUM>×<NUM>, and MaxTtSize is set as <NUM>×<NUM>. MinBtSize and MinTtSize, for both width and height, are set as <NUM>×<NUM>, and MaxMttDepth is set as <NUM>. The quaternary tree partitioning is applied to the CTU first to generate quaternary tree leaf nodes. The quaternary tree leaf nodes may have a size from <NUM>×<NUM>, that is MinQTSize, to <NUM>×<NUM>, that is the CTU size. If a leaf QT node is <NUM>×<NUM>, it will not be further split by the binary tree since the size exceeds MaxBtSize and MaxTtSize, that is, <NUM>×<NUM>. Otherwise, the leaf quadtree node could be further partitioned by the multi-type tree. Therefore, the quaternary tree leaf node is also the root node for the multi-type tree and it has multi-type tree depth, that is mttDepth, as <NUM>. When the multi-type tree depth reaches MaxMttDepth, that is, <NUM>, no further splitting is considered. When the multi-type tree node has width equal to MinBtSize and smaller than or equal to <NUM> * MinTtSize, no further horizontal splitting is considered. Similarly, when the multi-type tree node has height equal to MinBtSize and smaller than or equal to <NUM> * MinTtSize, no further vertical splitting is considered.

<FIG> shows an example of a block coded in a palette mode in accordance with some implementations of the present disclosure. To allow <NUM>×<NUM> Luma block and <NUM>×<NUM> Chroma pipelining design in WC hardware decoders, TT split is forbidden when either width or height of a luma coding block is larger than <NUM>, as shown in <FIG>. TT split is also forbidden when either width or height of a chroma coding block is larger than <NUM>.

In WC, the coding tree scheme supports the ability for luma and chroma to have a separate block tree structure. This CU splitting structure is termed dual tree structure or dual coding tree structure. The CU splitting structure shared by both the luma and chroma is termed as single tree structure or single coding tree structure. For P and B slices, the luma and chroma coding tree blocks (CTBs) in one CTU have to share the same coding tree structure. However, for I slices, the luma and chroma can have separate block tree structures. When separate block tree mode is applied, luma CTB is partitioned into CUs by one coding tree structure, and the chroma CTBs are partitioned into chroma CUs by another coding tree structure. This means that a CU in an I slice may consist of a coding block of the luma component or coding blocks of two chroma components, and a CU in a P or B slice always consists of coding blocks of all three color components unless the video is monochrome.

In addition to DCT-II which has been employed in HEVC, an MTS scheme is used for residual coding both inter and intra coded blocks. It uses multiple selected transforms from DCT8/DST7. The newly introduced transform matrices are DST-VII and DCT-VIII. Table <NUM> shows basic functions of a selected DST/DCT.

In order to keep orthogonality of a transform matrix, transform matrices are quantized more accurately than the transform matrices in HEVC. To keep intermediate values of transformed coefficients within the <NUM>-bit range, after horizontal and after vertical transform, all the coefficients are to have <NUM>-bit.

In order to control MTS scheme, separate enabling flags are specified at SPS level for intra and inter, respectively. When MTS is enabled at SPS, a CU level flag is signaled to indicate whether MTS is applied or not. Here, MTS is applied only for luma. The MTS CU level flag is signaled when following conditions are satisfied: first, both width and height are smaller than or equal to <NUM>; second, CBF flag is equal to one.

If MTS_CU_flag equals to zero, then DCT2 is applied in both directions. However, if MTS_CU_flag is equal to one, then two other flags are additionally signaled to indicate the transform type for the horizontal and vertical directions, respectively. Transform and signaling mapping table as shown in Table <NUM>. Unified the transform selection for ISP and implicit MTS is used by removing the intra-mode and block-shape dependencies. If current block is ISP mode or if the current block is intra block and both intra and inter explicit MTS is on, then only DST7 is used for both horizontal and vertical transform cores. When it comes to transform matrix precision, <NUM>-bit primary transform cores are used. Therefore, all the transform cores used in HEVC are kept as the same, including <NUM>-point DCT2 and DST7, <NUM>-point, <NUM>-point, and <NUM>-point DCT2. Also, other transform cores including <NUM>-point DCT2, <NUM>-point DCT8, <NUM>-point, <NUM>-point, <NUM>-point DST7 and DCT8, use <NUM>-bit primary transform cores.

To reduce the complexity of large size DST7 and DCT8, High frequency transform coefficients are zeroed out for the DST7 and DCT8 blocks with size (width or height, or both width and height) equal to <NUM>. Only the coefficients within the 16x16 lower-frequency region are retained.

As in HEVC, the residual of a block can be coded with transform skip mode. To avoid the redundancy of syntax coding, the transform skip flag is not signaled when the CU level MTS_CU_flag is not equal to zero. The block size limitation for transform skip is the same to that for MTS in JEM4, which indicate that transform skip is applicable for a CU when both block width and height are equal to or less than <NUM>. Note that implicit MTS transform is set to DCT2 when LFNST or MIP is activated for the current CU. Also, the implicit MTS can be still enabled when MTS is enabled for inter coded blocks.

In some examples, Maximum QP may be extended from <NUM> to <NUM>, and signaling of initial Quantization Parameter (QP) may be changed accordingly. The initial value of SliceQpY is modified at a slice segment layer when a non-zero value of slice_qp_delta is coded. Specifically, the value of init_qp_minus26 may be modified to be in the range of ( - <NUM> + QpBdOffsetY ) to +<NUM>. When the size of a transform block is not a power of <NUM>, the transform coefficients are processed along with a modification to the QP or QP levelScale table rather than by multiplication by <NUM>/<NUM> (or <NUM>/<NUM>), to compensate for an implicit scaling by the transform process. For transform skip block, minimum allowed QP is defined as <NUM> because quantization step size becomes <NUM> when QP is equal to <NUM>.

In some examples, a fixed look-up table is used to convert a luma quantization parameter QPY to a chroma quantization parameter QPC. In WC, a more flexible luma-to-chroma QP mapping is used. Instead of having a fixed table, the luma-to-chroma QP mapping relationship is signaled in SPS using a flexible piecewise linear model, with the only constraint on the linear model being that the slope of each piece cannot be negative. That is, as luma QP increases, chroma QP must stay flat or increase, but cannot decrease. The piecewise linear model is defined by: <NUM>) number of pieces in the model; and <NUM>) input (luma) and output (chroma) delta QPs for that piece. The input range of the piecewise linear model is [-QpBdOffsetY, <NUM>] and the output range of the piecewise linear model is [-QpBdOffsetC, <NUM>]. The QP mapping relationship can be signaled separately for Cb, Cr, and joint Cb/Cr coding, or signaled jointly for all three types of residual coding.

Same as in HEVC, CU-level QP adaptation is allowed in WC. Delta QP values for luma and chroma components can be signaled separately. For the chroma components, the allowed chroma QP offset values are signaled in the form of offset lists in picture parameter set (PPS) in a similar manner as in HEVC. The lists are defined separately for Cb, Cr, and joint Cb/Cr coding. Up to <NUM> offset values are allowed for each of Cb, Cr, and joint Cb/Cr lists. At CU-level, an index is signaled to indicate which one of the offset values in the offset list is used to adjust the chroma QP for that CU.

Transformation processes require resulting coefficients to be scaled by a certain factor, followed by a shift operation during quantization and de-quantization processes. The scale factor is defined as follows: <MAT> where M and N are the width and height of a transform block.

In some examples, dimensions of a block are powers of <NUM>, i.e. M=<NUM>m and N=<NUM>n. This means that when M is equal to N, or indeed when M·N is power of <NUM>, the factor can be applied by a right-shift. Blocks satisfying such conditions are referred as term "normal block. " When M·N is not a power of <NUM>, different scale and shift values are used to compensate. The scale values are defined as in Table <NUM>. Blocks satisfying such conditions are referred as term "compensated block.

For blocks coded in transform skip mode, scale and shift operations defined for normal blocks are performed.

The derivation of a scaled transform coefficient is illustrated in Table <NUM>. The definitions of all the variables in Table <NUM> can be found in the version <NUM> of VVC draft spec.

Given a QP value used, quantization and de-quantization processes for transform coefficients may be described as follows. Quantization may be described by equation (<NUM>): <MAT>.

De-quantization may be described by equation (<NUM>): <MAT> where the variable rectNonTsFlag may be obtained by equation (<NUM>): <MAT> and the variable transformShift' may be obtained by equation (<NUM>): <MAT>.

Here, pCoeff is a transform coefficient value; pLevel is a quantized value or quantization level; pcoeff' is reconstructed transform coefficient values from de-quantization process.

The variable rectNonTsFlag represents if a current block is classified as a "normal block" or a "compensated block. " When it has a value of false or <NUM>, the current block is classified as a normal block. When it has a value of true or <NUM>, the current block is classified as a compensated block.

The variable transformShift represents a bit-shift that is used to compensate the dynamic range increase due to <NUM>-dimenional (2D) transforms, which is equal to <NUM> - bitDepth - (log<NUM>(W) + log<NUM>(H))/<NUM>, where W and H are the width and height of the current transform unit, bitDepth is a coding bit-depth. Depending on the value of rectNonTsFlag, the actual value used in the shift operation, transformShift', may take a same value as transformShift, or a value equal to (transformShift - <NUM>).

encScale[ ][ ] and decScale[ ][ ] are quantization and dequantization scaling values which are in <NUM>-bit and <NUM>-bit precision respectively, and defined as shown in Table <NUM>. Depending on the value of rectNonTsFlag, a set of scaling factors used for the "normal blocks" or "compensated blocks" are selected and used for the current block.

Under a transform skip mode, prediction residuals are quantized and coded directly without transform operations performed. More specifically, its quantization and de-quantization processes can be described as follows.

Quantization may be described by equation (<NUM>): <MAT>.

De-quantization may be described by equation (<NUM>): <MAT> where pResi and pResi' are original and reconstructed prediction residual sample values; pLevel is a quantized value, or quantization level; encScale[ ] and decScale[ ] are the quantization and dequantization scaling values which are in <NUM>-bit and <NUM>-bit precision respectively, and defined the same as those used for "normal blocks" shown in Table <NUM>.

VTM6 supports the palette mode for screen content coding in <NUM>:<NUM>:<NUM> color format. When the palette mode is enabled and if a CU size is smaller than or equal to 64x64, a flag is transmitted at the CU level to indicate whether the palette mode is used for the CU. Palette mode is signaled as a prediction mode other than an intra prediction, inter prediction, and intra block copy (IBC) mode.

If the palette mode is utilized for a CU, the sample values in the CU may be represented by a small set of representative color values. The set is referred as "palette. " For pixels with values close to the palette colors, palette indices may be signaled to convey their values to the decoder. It is also possible to specify a sample whose values are not close to any palette color by signaling an escape symbol index, followed by escape values. The escape values are the sample's quantized component values. This is illustrated in <FIG>.

For quantization and de-quantization of the escape value, the following equations describe the corresponding processes that are applied at the encoder and the decoder, respectively.

De-quantization may be described by equation (<NUM>): <MAT> where pEsca and pEsca' are original and reconstructed escape values; pLevel is a quantized value, or quantization level; encScale[ ] and decscale[ ] are quantization and dequantization scaling values which are respectively in <NUM>-bit and <NUM>-bit precision, and defined the same as those used for "normal blocks" shown in Table <NUM>.

For coding of the palette, a palette predictor including a list of colors is maintained. The predictor is initialized to <NUM> (i.e. empty list) at the beginning of each slice for non-wavefront case and at the beginning of each CTU row for wavefront case. For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette in the CU. The reuse flags are sent using run-length coding of zeros. After this, the number of new palette entries and the component values for the new palette entries are signaled. After coding a CU under palette mode, the palette predictor is updated using the current palette, and entries from the palette predictor that are not reused in the current palette are added at the end until the maximum palette size allowed is reached to form the new palette predictor. An escape flag is signaled for each CU to indicate if escape symbols are present in the current CU. If escape symbols are present, the palette table is augmented by one and the last index is assigned to represent the escape symbol.

<FIG> show horizontal and vertical traverse scans in accordance with some implementations of the present disclosure. Palette indices of samples in a CU form a palette index map. The index map is coded using horizontal and/or vertical traverse scans as shown in <FIG>. The scan order is explicitly signaled in the bitstream using the palette_transpose_flag.

<FIG> shows coding of palette indices in accordance with some implementations of the present disclosure. The palette indices are coded using two main palette sample modes: INDEX and COPY_ABOVE. The mode is signaled using a flag except for the top row where only the horizontal scan is used, the first column where only the vertical scan is used, or for palette sample locations where the previous mode was COPY_ABOVE. In the COPY_ABOVE mode, the palette index of the sample in the row above is copied. In the INDEX mode, the palette index is explicitly signaled. For both INDEX and COPY_ABOVE modes, a run value is signaled which specifies the number of pixels that is coded using the same associated mode.

The coding order for index map is as follows. First, the number of index values associated to INDEX runs is signaled. This is followed by signalling of the actual index values for the entire CU using truncated binary coding. Then the palette mode (INDEX or COPY_ABOVE) and run length for each run are signaled in an interleaved manner. Finally, the quantized escape mode colors for the entire CU are grouped together and coded with exponential Golomb coding.

For slices with dual luma/chroma tree, palette is applied on luma (Y component) and chroma (Cb and Cr components) separately. For slices of single tree, palette will be applied on Y, Cb, Cr components jointly, i.e., each entry in the palette contains Y, Cb, Cr values. For deblocking, the block boundary of a palette coded block is not deblocked.

Three different designs of quantization scheme are available and applied to regular transform, transform skip, and palette mode, respectively. Each different quantization design is associated with different shift and scale operations. For blocks where the regular transform is applied, the shift and scale operations are block shape dependent. For blocks where transform skip is applied, the shift and scale operations are not dependent on block shape. For blocks coded in palette mode, only scale operation is performed, and the operation is not dependent on block shape. Such non-unified design may not be optimal from standardization point of view.

In some examples, methods are provided to simplify and further improve transform and quantization. The quantization and de-quantization operations used under transform skip mode and palette mode may be unified by applying the quantization and de-quantization operations used under transform skip mode to palette mode.

In some examples, the quantization and de-quantization operations used under transform skip mode and palette mode are unified by applying the current quantization and de-quantization operations under palette mode to transform skip mode.

In some examples, the quantization and de-quantization operations of the regular transform mode, transform skip mode and palette mode are all unified by applying the quantization and de-quantization operations for regular transform mode to all modes, including transform skip mode and palette mode as well.

In some examples, a same quantization and de-quantization process is applied to both prediction residuals under transform skip mode and escape values under palette mode. The quantization and de-quantization operations that are used under transform skip mode is applied to the palette mode as well. For example, the equations (<NUM>) and (<NUM>) corresponding to QuantTS(pResi) and <MAT> are used for escape values under palette mode, where pResi would be replaced with the original escape values pEsca, and the output from <MAT> in this case would be the reconstructed escape values pEsca'. Specifically, the quantization operation is described as: <MAT> where transformShift = <NUM> - bitDepth - (log<NUM>(W) + log<NUM>(H))/<NUM>, pLevel is the quantized level, pEsca is the escape color value, QP is the quantization parameter, encscale[ ] is the scale value, % denotes a modulo operation, QP%<NUM> represents an operation of QP modulo <NUM>, W is a width of the CU, H is a height of the CU, and bitDepth is a coding bit-depth.

Further, the dequantization operation is accordingly described as: <MAT> where transformShift = <NUM> - bitDepth - (log<NUM>(W) + log<NUM>(H))/<NUM>, pLevel is the quantized level, pEsca' is the reconstructed escape color value for the CU, QP is the quantization parameter, decScale[ ] is the scale value, % denotes the modulo operation, QP%<NUM> represents the operation of QP modulo <NUM>, W is the width of the CU, H is the height of the CU, and bitDepth is the coding bit-depth.

In some examples, the quantization and de-quantization operations that are used for escape values under the palette mode are applied to prediction residuals under the transform skip mode as well. For example, the equations (<NUM>) and (<NUM>) corresponding to QuantE(pEsca) and <MAT> are used for prediction residual values under the transform skip mode, where pEsca would be replaced with the prediction residual values pResi, and the output from <MAT> in this case would be the reconstructed prediction residual values pResi'.

Specifically, the quantization operation is described as: <MAT> where pLevel is the quantized level, pResi is the prediction residual sample value, encScale[ ] is the scale value, QP is the quantization parameter, % denotes the modulo operation, QP%<NUM> represents the operation of QP modulo <NUM>.

Further, the de-quantization operation is described as: <MAT> where pLevel is the quantized level, pResi' is the reconstructed prediction residual sample value, decScale[ ] is the scale value, QP is the quantization parameter, % denotes the modulo operation, QP%<NUM> represents the operation of QP modulo <NUM>.

In some examples, the quantization and de-quantization processes for the regular transform mode are used for the transform skip mode and palette mode as well. For example, the equations (<NUM>), (<NUM>), (<NUM>) and (<NUM>) are also used for the quantization/de-quantization process under transform skip mode and palette mode.

In some examples, when the Quant(pCoeff) and Quant-<NUM>(pLevel) functions are used for prediction residual values pResi under the transform skip mode, pCoeff would be replaced with pResi · <NUM>transformShift, and the output from Quant-<NUM>(pLevel) in this case would be equal to pResi' · <NUM>transformShift. As a result, the reconstructed prediction residual value pResi' is derived by Quant-<NUM>(pLevel) /<NUM>transformShift. Accordingly, except some shift operations, the same quantization and de-quantization functions for transform skip coefficients are used for prediction residual values under transform skip mode. Specifically, the quantization operation may be described as: <MAT> <MAT> <MAT> where pLevel is the quantized level, pResi is the prediction residual sample value, encScale[ ] [ ] is the scale value, QP is the quantization parameter, % denotes the modulo operation, QP%<NUM> represents the operation of QP modulo <NUM>, W is the width of the CU, H is the height of the CU, bitDepth is the coding bit-depth, rectNonTsFlag equals <NUM> when the CU is classified as a normal block, and rectNonTsFlag equals <NUM> when the CU is classified as a compensated block.

Further, the de-quantization operation may be described as: <MAT> <MAT> <MAT> where pLevel is the quantized level, pResi' is the reconstructed prediction residual sample value, decScale[ ] [ ] is the scale value, QP is the quantization parameter, % denotes the modulo operation, QP%<NUM> represents the operation of QP modulo <NUM>, W is the width of the CU, H is the height of the CU, bitDepth is the coding bit-depth, rectNonTsFlag equals <NUM> when the CU is classified as a normal block, and rectNonTsFlag equals <NUM> when the CU is classified as a compensated block.

In some examples, when the Quant(pCoeff) and Quant-<NUM>(pLevel) functions are used for escape values pEsca under the palette mode, pCoeff would be replaced with pReca · <NUM>transformshift, and the output from Quant-<NUM>(pLevel) in this case would be equal to pEsca' · <NUM>transformShift. As a result, the reconstructed escape value pEsca' is derived by Quant-<NUM>(pLevel) /<NUM>transformShift. So, except some shift operations, the same quantization and de-quantization functions for transform skip coefficients are also used for escape values under the palette mode.

Specifically, the quantization operation may be described as: <MAT> <MAT> <MAT> wherein pLevel is the quantized level, pEsca is the escape color value, encScale[] [ ] is the scale value, QP is the quantization parameter, % denotes the modulo operation, QP%<NUM> represents the operation of QP modulo <NUM>, W is the width of the CU, H is the height of the CU, bitDepth is the coding bit-depth, rectNonTsFlag equals <NUM> when the CU is classified as a normal block, and rectNonTsFlag equals <NUM> when the CU is classified as a compensated block.

It is worth noting that the quantization and dequantization processes according to Quant(pCoeff) and Quant-<NUM>(pLevel) may not be used to support lossless coding in WC because perfect reconstruction is not guaranteed even with a quantization step size of <NUM> using these two functions.

In some examples, when the above method is used to unify the quantization and dequantization processes of different sample values, a separate flag, for example, trans_quant_bypass_flag, may be signaled in bitstream to indicate if a given image area, for example, a CU is coded in lossless coding mode. If such a flag indicates a given block is coded in lossless mode, the corresponding quantization and dequantization processes are bypassed for the coding of the block.

<FIG> is a block diagram illustrating an apparatus for video coding in accordance with some implementations of the present disclosure. The apparatus <NUM> may be a terminal, such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.

As shown in <FIG>, the apparatus <NUM> may include one or more of the following components: a processing component <NUM>, a memory <NUM>, a power supply component <NUM>, a multimedia component <NUM>, an audio component <NUM>, an input/output (I/O) interface <NUM>, a sensor component <NUM>, and a communication component <NUM>.

The processing component <NUM> usually controls overall operations of the apparatus <NUM>, such as operations relating to display, a telephone call, data communication, a camera operation and a recording operation. The processing component <NUM> may include one or more processors <NUM> for executing instructions to complete all or a part of steps of the above method. Further, the processing component <NUM> may include one or more modules to facilitate interaction between the processing component <NUM> and other components. For example, the processing component <NUM> may include a multimedia module to facilitate the interaction between the multimedia component <NUM> and the processing component <NUM>.

The memory <NUM> is configured to store different types of data to support operations of the apparatus <NUM>. Examples of such data include instructions, contact data, phonebook data, messages, pictures, videos, and so on for any application or method that operates on the apparatus <NUM>. The memory <NUM> may be implemented by any type of volatile or non-volatile storage devices or a combination thereof, and the memory <NUM> may be a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic memory, a flash memory, a magnetic disk or a compact disk.

The multimedia component <NUM> includes a screen providing an output interface between the apparatus <NUM> and a user. In some examples, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen receiving an input signal from a user. The touch panel may include one or more touch sensors for sensing a touch, a slide and a gesture on the touch panel. The touch sensor may not only sense a boundary of a touching or sliding actions, but also detect duration and pressure related to the touching or sliding operation. In some examples, the multimedia component <NUM> may include a front camera and/or a rear camera. When the apparatus <NUM> is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data.

The audio component <NUM> is configured to output and/or input an audio signal. For example, the audio component <NUM> includes a microphone (MIC). When the apparatus <NUM> is in an operating mode, such as a call mode, a recording mode and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory <NUM> or sent via the communication component <NUM>. In some examples, the audio component <NUM> further includes a speaker for outputting an audio signal.

The I/O interface <NUM> provides an interface between the processing component <NUM> and a peripheral interface module. The above peripheral interface module may be a keyboard, a click wheel, a button, or the like. These buttons may include but not limited to, a home button, a volume button, a start button and a lock button.

The sensor component <NUM> includes one or more sensors for providing a state assessment in different aspects for the apparatus <NUM>. For example, the sensor component <NUM> may detect an on/off state of the apparatus <NUM> and relative locations of components. For example, the components are a display and a keypad of the apparatus <NUM>. The sensor component <NUM> may also detect a position change of the apparatus <NUM> or a component of the apparatus <NUM>, presence or absence of a contact of a user on the apparatus <NUM>, an orientation or acceleration/deceleration of the apparatus <NUM>, and a temperature change of apparatus <NUM>. The sensor component <NUM> may include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component <NUM> may further include an optical sensor, such as a CMOS or CCD image sensor used in an imaging application. In some examples, the sensor component <NUM> may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component <NUM> is configured to facilitate wired or wireless communication between the apparatus <NUM> and other devices. The apparatus <NUM> may access a wireless network based on a communication standard, such as WiFi, <NUM>, or a combination thereof. In an example, the communication component <NUM> receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example, the communication component <NUM> may further include a Near Field Communication (NFC) module for promoting short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra-Wide Band (UWB) technology, Bluetooth (BT) technology and other technology.

In an example, the apparatus <NUM> may be implemented by one or more of Application Specific Integrated Circuits (ASIC), Digital Signal Processors (DSP), Digital Signal Processing Devices (DSPD), Programmable Logic Devices (PLD), Field Programmable Gate Arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic elements to perform the above method.

A non-transitory computer readable storage medium may be, for example, a Hard Disk Drive (HDD), a Solid-State Drive (SSD), Flash memory, a Hybrid Drive or Solid-State Hybrid Drive (SSHD), a Read-Only Memory (ROM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk and etc..

<FIG> is a flowchart illustrating an exemplary process of quantization design in video coding in accordance with some implementations of the present disclosure. The process may be applied in an encoder.

In step <NUM>, the processor <NUM> determines a quantization parameter for residual data of a CU.

In step <NUM>, the processor <NUM> derives a scale value by scaling the quantization parameter by a scale factor.

In step <NUM>, the processor <NUM> determines a plurality of coefficients associated with the CU.

In some examples, the plurality of coefficients may comprise a transform coefficient, an escape color value for the CU, and a prediction residual sample value.

In step <NUM>, the processor <NUM> determines a plurality of parameters associated with the CU.

In some examples, the plurality of parameters comprises a bit-shift that is determined based on a coding bit-depth, and a width and a height of the CU, and the scale factor is determined based on the width and the height of the CU.

In step <NUM>, the processor <NUM> obtains a plurality of bit-shifts by bit-shifting the plurality of parameters.

In step <NUM>, the processor <NUM> obtains a quantized level based on the scale value, the plurality of coefficients, and the plurality of bit-shifts.

In some examples, when the processor <NUM> determines the plurality of coefficients associated with the CU, the processor <NUM> further determines an escape color value for the CU. The escape color value may be a value for a pixel in the CU with a color not in a preset plurality of colors selected from the CU. And when the processor <NUM> obtains the quantized level based on the scale value, the plurality of coefficients, and the plurality of bit-shifts, the processor <NUM> further obtains the quantized level based on the scale value, the escape color value, and the plurality of bit-shifts.

In some examples, the processor <NUM> further determines a prediction residual sample associated with the CU. If the processor <NUM> determines the plurality of coefficients associated with the CU, the processor <NUM> further determines a prediction residual sample value corresponding to the prediction residual sample. When the processor <NUM> obtains the quantized level based on the scale value, the plurality of coefficients, and the plurality of bit-shifts, the processor <NUM> further obtains the quantized level based on the scale value, the prediction residual sample value, and the plurality of bit-shifts.

<FIG> is a flowchart illustrating an exemplary process of de-quantization design in video coding in accordance with some implementations of the present disclosure. The process may be applied in a decoder.

In step <NUM>, the processor <NUM> receives a video bitstream comprising a quantization parameter and a quantized level.

In step <NUM>, the processor <NUM> determines a plurality of parameters associated with a CU.

In some examples, the plurality of parameters may comprise a bit-shift that is determined based on a coding bit-depth, and a width and a height of the CU, and the scale factor is determined based on the width and the height of the CU.

In step <NUM>, the processor <NUM> obtains a plurality of coefficients associated with the CU based on the quantized level, the scale value, and the plurality of bit-shifts.

In some examples, the plurality of coefficients may comprise a reconstructed transform coefficient, a reconstructed escape color value for the CU, and a reconstructed prediction residual sample value.

In some examples, when the processor <NUM> obtains the plurality of coefficients associated with the CU based on the quantized level, the scale value, and the plurality of bit-shifts, the processor <NUM> further obtains a reconstructed escape color value for the CU based on the quantized level, the scale value, and the plurality of bit-shifts. And the reconstructed escape color value is a value for a pixel in the CU with a color not in a preset plurality of colors selected from the CU.

In some examples, the processor <NUM> further determines a prediction residual sample associated with the CU. When the processor <NUM> obtains the plurality of coefficients associated with the CU based on the quantized level, the scale value, and the plurality of bit-shifts, the processor <NUM> obtains the reconstructed prediction residual sample value for the CU based on the quantized level, the scale value, and the plurality of bit-shifts. And the reconstructed prediction residual sample value may be corresponding to the prediction residual sample.

In some examples, there is provided a computing device for video coding. The apparatus includes a processor <NUM>; and a memory <NUM> configured to store instructions executable by the processor; where the processor, upon execution of the instructions, is configured to perform a method as illustrated in <FIG>.

In some other examples, there is provided a non-transitory computer readable storage medium <NUM>, having instructions stored therein. When the instructions are executed by a processor <NUM>, the instructions cause the processor to perform a method as illustrated in <FIG>.

Claim 1:
A method for video decoding, comprising:
deriving, from a bitstream and for a first coding block, a quantization parameter and a quantized level;
deriving a scale value by looking up a scale level table based on the quantization parameter; and
in accordance with a determination that the first coding block is coded in a transform skip mode, obtaining a residual sample for a sample in the first coding block based on the quantized level, the scale value and a plurality of bit-shifts by applying a first de-quantization operation;
wherein the first de-quantization operation is also applied to obtain a reconstructed sample for an escape sample in a second coding block coded in a palette mode; and
wherein a block size of the first coding block or the second coding block is not used for determining parameters of the first de-quantization operation,
wherein the first de-quantization operation is as follows: <MAT>
wherein pLevel is a quantized level, pSample corresponds to a reconstructed value of a residual sample in the transform skip mode or an escape sample in the palette mode, decScale[ ] is a scale value, QP is a quantization parameter, % denotes a modulo operation, QP%<NUM> represents an operation of QP modulo <NUM>.