Patent ID: 12225238

DETAILED DESCRIPTION OF THE INVENTION

Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals even if occurring in different figures.

In the following description, a plurality of details is set forth to provide a more throughout explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present invention. In addition, features of the different embodiments described herein after may be combined with each other, unless specifically noted otherwise.

The following description of the figures starts with a presentation of a description of an encoder and a decoder of a block-based predictive codec for coding pictures of a video in order to form an example for a coding framework into which embodiments of the present invention may be built in.

The respective encoder and decoder are described with respect toFIGS.1ato3. The herein described embodiments of the concept of the present invention could be built into the encoder and decoder ofFIGS.1a,1band2, respectively, although the embodiments described withFIGS.4to6, may also be used to form encoders and decoders not operating according to the coding framework underlying the encoder and decoder ofFIGS.1a,1band2.

FIG.1ashows an apparatus (e. g. a video encoder and/or a picture encoder) for predictively coding a picture12into a data stream14exemplarily using transform-based residual coding. The apparatus, or encoder, is indicated using reference sign10.FIG.1bshows also the apparatus for predictively coding a picture12into a data stream14, wherein a possible prediction module44is shown in more detail.FIG.2shows a corresponding decoder20, i.e. an apparatus20configured to predictively decode the picture12′ from the data stream14also using transform-based residual decoding, wherein the apostrophe has been used to indicate that the picture12′ as reconstructed by the decoder20deviates from picture12originally encoded by apparatus10in terms of coding loss introduced by a quantization of the prediction residual signal.FIG.1a,1bandFIG.2exemplarily use transform based prediction residual coding, although embodiments of the present application are not restricted to this kind of prediction residual coding. This is true for other details described with respect toFIGS.1a,1band2, too, as will be outlined hereinafter.

The encoder10is configured to subject the prediction residual signal to spatial-to-spectral transformation and to encode the prediction residual signal, thus obtained, into the data stream14. Likewise, the decoder20is configured to decode the prediction residual signal from the data stream14and subject the prediction residual signal, thus obtained, to spectral-to-spatial transformation.

Internally, the encoder10may comprise a prediction residual signal former22which generates a prediction residual24so as to measure a deviation of a prediction signal26from the original signal, i.e. from the picture12, wherein the prediction signal26can be interpreted as a linear combination of a set of one or more predictor blocks according to an embodiment of the present invention. The prediction residual signal former22may, for instance, be a subtractor which subtracts the prediction signal from the original signal, i.e. from the picture12. The encoder10then further comprises a transformer28which subjects the prediction residual signal24to a spatial-to-spectral transformation to obtain a spectral-domain prediction residual signal24′ which is then subject to quantization by a quantizer32, also comprised by the encoder10. The thus quantized prediction residual signal24″ is coded into bitstream14. To this end, encoder10may optionally comprise an entropy coder34which entropy codes the prediction residual signal as transformed and quantized into data stream14.

The prediction signal26is generated by a prediction stage36of encoder10on the basis of the prediction residual signal24″ encoded into, and decodable from, data stream14. To this end, the prediction stage36may internally, as is shown inFIG.1a, comprise a dequantizer38which dequantizes prediction residual signal24″ so as to gain spectral-domain prediction residual signal24″′, which corresponds to signal24′ except for quantization loss, followed by an inverse transformer40which subjects the latter prediction residual signal24″′ to an inverse transformation, i.e. a spectral-to-spatial transformation, to obtain prediction residual signal24″″ which corresponds to the original prediction residual signal24except for quantization loss. A combiner42of the prediction stage36then recombines, such as by addition, the prediction signal26and the prediction residual signal24″″ so as to obtain a reconstructed signal46, i.e. a reconstruction of the original signal12. Reconstructed signal46may correspond to signal12′. A prediction module44of prediction stage36then generates the prediction signal26on the basis of signal46by using, for instance, spatial prediction, i.e. intra-picture prediction, and/or temporal prediction, i.e. inter-picture prediction, as shown inFIG.1bin more detail.

Likewise, decoder20, as shown inFIG.2, may be internally composed of components corresponding to, and interconnected in a manner corresponding to, prediction stage36. In particular, entropy decoder50of decoder20may entropy decode the quantized spectral-domain prediction residual signal24″ from the data stream, whereupon dequantizer52, inverse transformer54, combiner56and prediction module58, interconnected and cooperating in the manner described above with respect to the modules of prediction stage36, recover the reconstructed signal on the basis of prediction residual signal24″ so that, as shown inFIG.2, the output of combiner56results in the reconstructed signal, namely picture12′.

Although not specifically described above, it is readily clear that the encoder10may set some coding parameters including, for instance, prediction modes, motion parameters and the like, according to some optimization scheme such as, for instance, in a manner optimizing some rate and distortion related criterion, i.e. coding cost. For example, encoder10and decoder20and the corresponding modules44,58, respectively, may support different prediction modes such as intra-coding modes and inter-coding modes. The granularity at which encoder and decoder switch between these prediction mode types may correspond to a subdivision of picture12and12′, respectively, into coding segments or coding blocks. In units of these coding segments, for instance, the picture may be subdivided into blocks being intra-coded and blocks being inter-coded.

Intra-coded blocks are predicted on the basis of a spatial, already coded/decoded neighborhood (e. g. a current template) of the respective block (e. g. a current block) as is outlined in more detail below. Several intra-coding modes may exist and be selected for a respective intra-coded segment including directional or angular intra-coding modes according to which the respective segment is filled by extrapolating the sample values of the neighborhood along a certain direction which is specific for the respective directional intra-coding mode, into the respective intra-coded segment. The intra-coding modes may, for instance, also comprise one or more further modes such as a DC coding mode, according to which the prediction for the respective intra-coded block assigns a DC value to all samples within the respective intra-coded segment, and/or a planar intra-coding mode according to which the prediction of the respective block is approximated or determined to be a spatial distribution of sample values described by a two-dimensional linear function over the sample positions of the respective intra-coded block with driving tilt and offset of the plane defined by the two-dimensional linear function on the basis of the neighboring samples.

Compared thereto, inter-coded blocks may be predicted, for instance, temporally. For inter-coded blocks, motion vectors may be signaled within the data stream14, the motion vectors indicating the spatial displacement of the portion of a previously coded picture (e. g. a reference picture) of the video to which picture12belongs, at which the previously coded/decoded picture is sampled in order to obtain the prediction signal for the respective inter-coded block. This means, in addition to the residual signal coding comprised by data stream14, such as the entropy-coded transform coefficient levels representing the quantized spectral-domain prediction residual signal24″, data stream14may have encoded thereinto coding mode parameters for assigning the coding modes to the various blocks, prediction parameters for some of the blocks, such as motion parameters for inter-coded segments, and optional further parameters such as parameters for controlling and signaling the subdivision of picture12and12′, respectively, into the segments. The decoder20uses these parameters to subdivide the picture in the same manner as the encoder did, to assign the same prediction modes to the segments, and to perform the same prediction to result in the same prediction signal.

FIG.3illustrates the relationship between the reconstructed signal, i.e. the reconstructed picture12′, on the one hand, and the combination of the prediction residual signal24″″ as signaled in the data stream14, and the prediction signal26, on the other hand. As already denoted above, the combination may be an addition. The prediction signal26is illustrated inFIG.3as a subdivision of the picture area into intra-coded blocks which are illustratively indicated using hatching, and inter-coded blocks which are illustratively indicated not-hatched. The subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of square blocks or non-square blocks, or a multi-tree subdivision of picture12from a tree root block into a plurality of leaf blocks of varying size, such as a quadtree subdivision or the like, wherein a mixture thereof is illustrated inFIG.3in which the picture area is first subdivided into rows and columns of tree root blocks which are then further subdivided in accordance with a recursive multi-tree subdivisioning into one or more leaf blocks.

Again, data stream14may have an intra-coding mode coded thereinto for intra-coded blocks80, which assigns one of several supported intra-coding modes to the respective intra-coded block80. For inter-coded blocks82, the data stream14may have one or more motion parameters coded thereinto. Generally speaking, inter-coded blocks82are not restricted to being temporally coded. Alternatively, inter-coded blocks82may be any block predicted from previously coded portions beyond the current picture12itself, such as previously coded pictures of a video to which picture12belongs, or picture of another view or an hierarchically lower layer in the case of encoder and decoder being scalable encoders and decoders, respectively.

The prediction residual signal24″″ inFIG.3is also illustrated as a subdivision of the picture area into blocks84. These blocks might be called transform blocks in order to distinguish same from the coding blocks80and82. In effect,FIG.3illustrates that encoder10and decoder20may use two different subdivisions of picture12and picture12′, respectively, into blocks, namely one subdivisioning into coding blocks80and82, respectively, and another subdivision into transform blocks84. Both subdivisions might be the same, i.e. each coding block80and82, may concurrently form a transform block84, butFIG.3illustrates the case where, for instance, a subdivision into transform blocks84forms an extension of the subdivision into coding blocks80,82so that any border between two blocks of blocks80and82overlays a border between two blocks84, or alternatively speaking each block80,82either coincides with one of the transform blocks84or coincides with a cluster of transform blocks84. However, the subdivisions may also be determined or selected independent from each other so that transform blocks84could alternatively cross block borders between blocks80,82. As far as the subdivision into transform blocks84is concerned, similar statements are thus true as those brought forward with respect to the subdivision into blocks80,82, i.e. the blocks84may be the result of a regular subdivision of picture area into blocks (with or without arrangement into rows and columns), the result of a recursive multi-tree subdivisioning of the picture area, or a combination thereof or any other sort of blockation. Just as an aside, it is noted that blocks80,82and84are not restricted to being of quadratic, rectangular or any other shape.

FIG.3further illustrates that the combination of the prediction signal26and the prediction residual signal24″″ directly results in the reconstructed signal12′. However, it should be noted that more than one prediction signal26may be combined with the prediction residual signal24″″ to result into picture12′ in accordance with alternative embodiments.

InFIG.3, the transform blocks84shall have the following significance. Transformer28and inverse transformer54perform their transformations in units of these transform blocks84. For instance, many codecs use some sort of DST (discrete sine transform) or DCT (discrete cosine transform) for all transform blocks84. Some codecs allow for skipping the transformation so that, for some of the transform blocks84, the prediction residual signal is coded in the spatial domain directly. However, in accordance with embodiments described below, encoder10and decoder20are configured in such a manner that they support several transforms. For example, the transforms supported by encoder10and decoder20could comprise:DCT-II (or DCT-III), where DCT stands for Discrete Cosine TransformDST-IV, where DST stands for Discrete Sine TransformDCT-IVDST-VIIIdentity Transformation (IT)

Naturally, while transformer28would support all of the forward transform versions of these transforms, the decoder20or inverse transformer54would support the corresponding backward or inverse versions thereof:Inverse DCT-II (or inverse DCT-III)Inverse DST-IVInverse DCT-IVInverse DST-VIIIdentity Transformation (IT)

The subsequent description provides more details on which transforms could be supported by encoder10and decoder20. In any case, it should be noted that the set of supported transforms may comprise merely one transform such as one spectral-to-spatial or spatial-to-spectral transform, but it is also possible, that no transform is used by the encoder or decoder at all or for single blocks80,82,84.

As already outlined above,FIGS.1ato3have been presented as an example where the inventive concept described herein may be implemented in order to form specific examples for encoders and decoders according to the present application. Insofar, the encoder and decoder ofFIGS.1a,1band2, respectively, may represent possible implementations of the encoders and decoders described herein before.FIGS.1a,1band2are, however, only examples. An encoder according to embodiments of the present application may, however, perform block-based encoding of a picture12using the concept outlined in more detail before or hereinafter and being different from the encoder ofFIG.1aor1bsuch as, for instance, in that the sub-division into blocks80is performed in a manner different than exemplified inFIG.3and/or in that no transform (e.g. transform skip/identity transform) is used at all or for single blocks. Likewise, decoders according to embodiments of the present application may perform block-based decoding of picture12′ from data stream14using a coding concept further outlined below, but may differ, for instance, from the decoder20ofFIG.2in that same sub-divides picture12′ into blocks in a manner different than described with respect toFIG.3and/or in that same does not derive the prediction residual from the data stream14in transform domain, but in spatial domain, for instance and/or in that same does not use any transform at all or for single blocks.

According to an embodiment the inventive concept described in the following can concern the transformer28/inverse transformer40and the quantizer32/dequantizer38of the encoder or the inverse transformer54and the dequantizer52of the decoder. According to an embodiment, the transformer28, the inverse transformer40,54, the quantizer32and/or the dequantizer38,52can be disabled for lossless coding of a block of a picture, wherein lossless coding is indicated by coding parameters, like a quantization parameter and a transform mode. The inventive concept may also concern further processing of the prediction residual24″ and/or the prediction signal26and/or the prediction residual corrected predictive reconstruction46, wherein the focus is on the function of the decoder/encoder at lossless coding.

The quantization step size, i.e. the quantization accuracy, can be varied depending on the selected transform and transform block size as described below. The description is written from the decoder perspective and the decoder-side scaling52(multiplication) with the quantization step size can be seen as being the inverse (non-reversible) of the encoder-side division by the step size.

On the decoder side, the scaling52, i.e. the dequantization, of (quantized) transform coefficient levels in current video coding standards like H.265/HEVC is designed for transform coefficients resulting from DCT/DST integer transforms with higher precision as illustrated inFIG.4. In there, the variable bitDepth specifies the bit depth of the image samples, e.g. 8 or 10-bit. The variables log 2TbW and log 2TbH specify the binary logarithm of the transform block width and height, respectively.FIG.4shows a decoder-side scaling52and inverse transform54in recent video coding standards such as H.265/HEVC.

It should be noted that, at the decoder, the two 1D DCT/DST-based integer transforms1281introduce an additional factor of 212·√{square root over (2log 2TbW+log 2TbH)}, which needs to be compensated by scaling with the inverse. For non-square blocks with an odd log 2TbH+log 2TbW, the scaling includes a factor of √{square root over (2)}. This can be taken into account by either adding a scale factor of 181/256 or using a different set of levelScale values that incorporate that factor for this case, e.g. levelScale[ ]={29, 32, 36, 40, 45, 51}. For the identity transform or transform skip case1282, this does not apply.

(24·levelScale[QP⁢%⁢6]·2⌊QP6⌋210)

It can be seen that the step size or scaling factor becomes smaller then 1 for QPs less than 4 because levelScale for these QPs is less than 64=26. For the transform coefficients, this is not a problem since the integer forward transform1281increases the precision of the residual signal and consequently the dynamic range. However, for the residual signal in case of the identity transform or transform skip1282, there is no increase in dynamic range. In this case, the scaling factor less than 1 could introduce a distortion for the QPs<4 which is not there for QP 4, which has a scaling factor of 1. This is contradictory to the quantizer design intent where decreasing the QP should decrease the distortion.

Varying the quantization step size depending on the selected transform, e.g. whether the transform is skipped or not, could be used to derive a different quantization step size for transform skip1282. Especially for the lowest QPs 0, 1, 2 and 3, this would solve the problem of having a quantization step size/scaling factor less than 1 for the lowest QPs. In one example shown inFIG.5, the solution could be to clip53the quantization parameter to the minimum allowed value of four (QP′), resulting in a quantization step size that cannot be lower than one. In addition to that, the size-dependent normalization541with bdShift1 and the final rounding542to the bit depth with bdShift2, used by the transform, can be moved to the transform path54. This would reduce the transform skip scaling to a downshift by 10-bit with rounding. In another example, a bitstream restriction can be defined that does not allow an encoder to use QP values that result in a scaling factor of less than 1 for transform skip instead of clipping the QP value to 4.FIG.5shows an improved decoder-side scaling52and inverse transform54according the present invention.

At the other end of the bit-rate range, i.e. for lower bit rates, the quantization step size for the identity transform1282may be decreased by an offset, resulting in a higher fidelity for block that does not apply a transform or that does apply the identity transform1282. This would enable the encoder to select appropriate QP values for transform skip blocks to achieve higher compression efficiency. This aspect is not limited to the identity transform/transform skip1282, it can also be used to modify the QP for other transform types1281by an offset. An encoder would, e.g., determine this offset in a way that increases the coding efficiency, e.g. by maximizing perceived visual quality or minimizing objective distortion like a square error for a given bitrate, or by reducing the bitrate for a given quality/distortion. This (in terms of the applied criterion) optimal derivation from the slice QP depends, for example, on the content, bit-rate or complexity operation point, and further factors such as selected transform and transform block size. The present invention describes methods for signaling the QP offset for the case of multiple transforms. Without loss of generality, given two alternative transforms, a fixed QP offset may be transmitted by the encoder in a high-level syntax structure (such as sequence parameter set, picture parameter set, tile group header, slice header, or similar) for each of the two alternative transforms. Alternatively, the QP offset is, e.g., transmitted by the encoder for each transform block when the encoder has selected the alternative transform. A combination of the two approaches is the signaling of a basis QP offset in a high-level syntax structure and an additional offset for each transform block that uses the alternative transform. The offset can be a value that is added or subtracted to a basis QP or an index into a set of offset values. That set can be predefined or signaled in a high-level syntax structure.the QP offset relative to a basis QP for the identity transform may be signaled in a high-level syntax structure, e.g on sequence, picture, tile group, tile, or slice level.alternatively, the QP offset relative to a basis QP for the identity transform may be signaled for each coding unit or predefined set of coding units.alternatively, the QP offset relative to a basis QP, for the identity transform is signaled for each transform unit that applies the identity transform.

FIG.6shows features and functionalities of a decoder, for example, of a decoder20or36shown inFIGS.1ato2. Also the following description is mainly written from the decoder perspective it is clear that an encoder may comprise parallel features.

A picture12is encoded into a data stream14by the encoder and the decoder is configured to provide a reconstructed picture based on the data stream14, wherein the reconstructed picture equals the picture12or has no recognizable minimal differences to the picture12in case of lossless coding. The picture12can be divided into portions100, i.e. blocks, and regions104. A region104comprises a plurality of portions100. A predetermined portion100is within a predetermined region104. The data stream14may comprise portion individual information, like a plurality98of coding parameters, and region individual information, like optionally a lossless coding syntax element102. The plurality98of coding parameters relate to a predetermined portion100of the picture12and control a prediction residual transform mode, e.g. within the transformer28and the inverse transformer40,54shown inFIGS.1ato2, and a quantization accuracy, e.g. within the quantizer28and the dequantizer40,54shown inFIGS.1ato2, with respect to the predetermined portion100. The prediction residual transform mode may be controlled by a transform mode indicating syntax element981. The quantization accuracy may be controlled by a quantization parameter (QP)982.

A decoder for decoding the picture12from the data stream14is configured to check106whether a plurality98of coding parameters, is indicative of a coding parameter setting corresponding, either because immediately signaling such a coding, or by leading to such a coding by the decoder being configured to interpret or change such a setting to a setting leading to lossless residual coding, such as mapping QP<4 to QP=4 in case of TM=transform skip, to a lossless prediction residual coding1061. An encoder for encoding the picture12into the data stream14may also be configured to check whether the plurality98of coding parameters, is indicative of a coding parameter setting corresponding to the lossless prediction residual coding1061. (QP, TM)=(4,transform skip) or (QP, TM)=(1 . . . 4, transform skip) may represent a coding parameter setting corresponding to lossless prediction residual coding1061. A coding parameter setting of a quantization parameter982to a quantization accuracy equal or finer than a predetermined quantization accuracy and/or a transform mode indicating syntax element981to a transform skip as transform mode may correspond to the lossless prediction residual coding1061.

According to an embodiment, the decoder/encoder is configured to read/signal the plurality98of coding parameters from/into the data stream14and check whether the prediction residual transform mode indicated by the plurality98of coding parameters, e.g. indicated by the transform mode indicating syntax element981, corresponds to a transform skip mode and whether a quantization accuracy indicated by the plurality98of coding parameters, e.g. indicated by the quantization parameter982, corresponds to a quantization step size finer than a predetermined quantization step size, e.g., corresponds to QP<4, corresponding to no quantization and, if yes, change the quantization step size to the predetermined quantization step size, e.g. QP=4. If the transform mode is set to transform skip, the decoder/encoder is configured to map a quantization parameter982finer than the predetermined quantization accuracy (QP<4) to a quantization parameter982equal the predetermined quantization accuracy to enable a lossless coding. The predetermined quantization accuracy might represent no quantization32,52or a bypassing or disabling of a quantization32,52.

Responsive to the plurality98of coding parameters being indicative of the coding parameter setting corresponding to the lossless prediction residual coding1061, the decoder/encoder is configured to set110one or more predetermined coding options relating to one or more tools of the decoder/encoder for processing a prediction residual corrected predictive reconstruction (e.g. in the prediction-loop of the encoder) with respect to the predetermined portion100so that the one or more tools are disabled with respect to the predetermined portion100. The prediction residual corrected predictive reconstruction may represent the output, i.e. the reconstructed signal46, of the combiner56or42, respectively, as shown inFIGS.1ato2. The one or more tools, e.g., relate to deblocking, SAO, ALF and may be positioned downstream the output46of the combiner56or42, respectively.

According to an embodiment the decoder/encoder is configured to set the one or more predetermined coding options with respect to the predetermined portion100so that the one or more tools are disabled110with respect to the predetermined portion100if the plurality of coding parameters are indicative of the coding parameter setting corresponding to the lossless prediction residual coding1061and to a predetermined tool state if the plurality of coding parameters are not indicative of the coding parameter setting corresponding to the lossless prediction residual coding, i.e. in a lossy coding case1062.

The decoder is optionally configured to read from the data stream14a lossless coding syntax element102(e.g., determined and encoded by the encoder) which indicates whether a predetermined region104of the picture, which covers or contains the predetermined portion100, is coded into the data stream14using, not exclusively, but for portions fulfilling the check106, lossless coding or lossy coding. The decoder is configured to set110the one or more predetermined coding options so that the one or more tools are disabled with respect to the predetermined portion100if the lossless coding syntax element102indicates that the predetermined region104of the picture12is coded into the data stream14using lossless coding1061, and if the plurality98of coding parameters are indicative of the coding parameter setting corresponding to the lossless prediction residual coding1061. Additionally, the decoder is configured to set120the one or more predetermined coding options to a predetermined tool state if the plurality98of coding parameters do not indicate, i.e. are not equal to, the coding parameter setting corresponding to the lossless prediction residual coding1061or the lossless coding syntax element102indicates that the predetermined region104of the picture12is coded into the data stream14using lossy coding1062. The lossless coding syntax element102is signaled for a region104of the picture12and for each portion100within the region104the decoder/encoder checks106whether the plurality98of coding parameters indicate for the individual portion100a lossless coding1061or a lossy coding1062. Thus it is possible that some of the portions are decoded/encoded differently than indicated by the lossless coding syntax element102for the whole region104.

According to an embodiment, the decoder is configured to determine the predetermined tool state depending on one or more syntax elements108, e.g. syntax elements relating to SAO, ALF or the like, in the data stream14. In case of lossless coding1061the decoder is configured to skip122reading the one or more tool syntax elements108, since the one or more tools are disabled110. Optionally, at least one of the one or more syntax elements108is absent from the data stream14if the one or more predetermined coding options with respect to the predetermined portion100are set so that the one or more tools are disabled110with respect to the predetermined portion100, compare aspect 3.4.

According to an embodiment, the decoder is configured to set130one or more further coding options with respect to the predetermined portion100, e.g., described with respect to the following aspects 1 to 3.4 in the description below, to a default state responsive to the plurality98of coding parameters being indicative of the coding parameter setting corresponding to the lossless prediction residual coding1061. The default state may represent a reduction of a filtering or a disabling of a filtering, cf. aspect 3.3, in terms of low-pass filtering for a derivation of a prediction signal, e.g. the prediction signal26shown inFIGS.1ato2, for the predetermined portion100and/or a perfectly invertible transform, cf. aspect 3.2, to be performed on a prediction residual signal, e.g. the prediction residual signal24shown inFIGS.1ato2. The further coding options may relate to a binarization, cf. aspect 3.1, of prediction residual data into bin strings and context-adaptive binary entropy decoding of the bin strings and/or a usage of a perfectly invertible transform, cf. aspect 3.2, on a prediction residual24″′,24″″ or a prediction residual corrected reconstruction46of the predetermined portion100and/or a disabling or reduction of a filtering, cf. aspect 3.3, for a derivation of the prediction signal26for the predetermined portion and/or a disabling of a processing of a prediction residual corrected predictive reconstruction46with respect to the predetermined portion100or a prediction residual re-quantization, cf. aspect 3.4. Similarly, for an encoder the further coding options may relate to a binarization of prediction residual data into bin strings and context-adaptive binary entropy encoding of the bin strings and/or a usage of a perfectly invertible transform on a prediction residual24″′,24″″ or a prediction residual corrected reconstruction46of the predetermined portion in a prediction-loop of the encoder and/or a disabling or reduction of a filtering for a derivation of a prediction signal26for the predetermined portion and/or a disabling of a processing of a prediction residual corrected predictive reconstruction46with respect to the predetermined portion100or a prediction residual quantization.

An encoder can be configured to determine and encode the plurality98of coding parameters, lossless coding syntax element102and/or the tool syntax elements108in the data stream14. The encoder can comprise parallel features and/or functionalities as described with regard to the decoder. This applies at least for the prediction-loop36of the encoder which has the same features and/or functionalities as the decoder. But it is clear, that features relating to the inverse transformer54or the dequantizer52of the decoder can similarly also be applied to the transformer28and/or the quantizer32of the encoder.

One or more of the following aspects may be integrated in the decoder/encoder described with regard toFIG.6. Alternatively, the following aspects can be implemented individually in a decoder or encoder.

Aspect 1: Reduction of a Number of Syntax Elements for Lossless Image or Video Coding

The basic approach of this aspect is to make a subblock-wise transform quantization bypass coding flag obsolete since, with the corrections described above, its functionality is completely provided by the subblock-wise transform skipping functionality when a coding quantization parameter (QP) of 4 (unity step-size) is employed and post reconstruction modifications of block samples by coding tools such as in-loop filters are disabled. The deactivation of such post filters can be achieved by the conditioning on the transform skip mode and a QP that is lower or equal than four.

In the following paragraphs, different possible implementations are specified.

An embodiment of aspect 1 is by means of a coding block-wise (here, transform block-wise and color component-wise) one-bit transform skipping indicator, called transform_skip_flag. This indicator is transmitted as part of the transform unit coding syntax, for example, as shown in table 1. The plurality98of coding parameters may comprise the transform_skip_flag, i.e. the transform mode indicating syntax element981, controlling the prediction residual transform mode with respect to the predetermined portion100of the picture12. The transform_skip_flag defines a transform skip as the transform mode, i.e. the prediction residual transform mode, for the predetermined portion100.

TABLE 1transform unit coding syntaxDescriptortransform_unit( x0, y0, tbWidth, tbHeight, treeType, subTuIndex, chType ) {if( IntraSubPartitionsSplitType != ISP_NO_SPLIT &&treeType = = SINGLE_TREE && subTuIndex = = NumIntraSubPartitions − 1 ) {xC = CbPosX[ chType ][ x0 ][ y0 ]yC = CbPosY[ chType ][ x0 ][ y0 ]wC = CbWidth[ chType ][ x0 ][ y0 ] / SubWidthChC = CbHeight[ chType ][ x0 ][ y0 ] / SubHeightC} else {xC = x0yC = y0wC = tbWidth / SubWidthChC = tbHeight / SubHeightC}chromaAvailable = treeType != DUAL_TREE_LUMA && sps_chroma_format_idc != 0 &&( IntraSubPartitionsSplitType = = ISP_NO_SPLIT ∥( IntraSubPartitionsSplitType != ISP_NO_SPLIT &&subTuIndex = = NumIntraSubPartitions − 1 ) )if( ( treeType = = SINGLE_TREE ∥ treeType = = DUAL_TREE_CHROMA ) &&sps_chroma_format_idc != 0 &&( ( IntraSubPartitionsSplitType = = ISP_NO_SPLIT && !( cu_sbt_flag &&( ( subTuIndex = = 0 && cu_sbt_pos_flag ) ∥( subTuIndex = = 1 && !cu_sbt_pos_flag ) ) ) ) ∥( IntraSubPartitionsSplitType != ISP_NO_SPLIT &&( subTuIndex = = NumIntraSubPartitions − 1 ) ) ) ) {tu_cb_coded_flag[ xC][ yC ]ae(v)tu_cr_coded_flag[ xC ][ yC ]ae(v)}if( treeType = = SINGLE_TREE ∥ treeType = = DUAL_TREE_LUMA) {if( ( IntraSubPartitionsSplitType = = ISP_NO_SPLIT && !( cu_sbt_flag &&( ( subTuIndex = = 0 && cu_sbt_pos_flag ) ∥( subTuIndex = = 1 && !cu_sbt_pos_flag ) ) ) &&( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTRA ∥( chromaAvailable && ( tu_cb_coded_flag[ xC ][ yC ] ∥tu_cr_coded_flag[ xC][ yC ] ) ) ∥CbWidth[ chType ][ x0][ y0 ] > MaxTbSizeY ∥CbHeight[ chType ][ x0][ y0 ] > MaxTbSizeY ) ) ∥( IntraSubPartitionsSplitType != ISP_NO_SPLIT &&( subTuIndex < NumIntraSubPartitions − 1 ∥ !InferTuCbfLuma ) ) )tu_y_coded_flag [ x0 ][ y0 ]ae(v)if( IntraSubPartitionsSplitType != ISP_NO_SPLIT )InferTuCbfLuma = InferTuCbfLuma && !tu_y_coded_flag[ x0 ][ y0 ]}if( ( CbWidth[ chType][ x0 ][ y0 ] > 64 ∥ CbHeight[ chType ][ x0 ][ y0 ] > 64 ∥tu_y_coded_flag[ x0 ][ y0 ] ∥ ( chromaAvailable && ( tu_cb_coded_flag[ xC ][ yC ] ∥tu_cr_coded_flag[ xC ][ yC ] ) ) && treeType != DUAL_TREE_CHROMA &&pps_cu_qp_delta_enabled_flag && !IsCuQpDeltaCoded ) {cu_qp_delta_absae(v)if( cu_qp_delta_abs )cu_qp_delta_sign_flagae(v)}if( ( Cb Width[ chType][ x0][ y0 ] > 64 ∥ CbHeight[ chType][ x0][ y0 ] >64 ∥( chromaAvailable && ( tu_cb_coded_flag[ xC ][ yC ] ∥tu_cr_coded_flag[ xC ][ yC ] ) ) ) &&treeType != DUAL_TREE_LUMA && sh_cu_chroma_qp_offset_enabled_flag &&!IsCuChromaQpOffsetCoded ) {cu_chroma_qp_offset_flagae(v)if( cu_chroma_qp_offset_flag && pps_chroma_qp_offset_list_len_minus1 > 0 )cu_chroma_qp_offset_idxae(v)}if( sps_joint_cbcr_enabled_flag && ( ( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTRA&& ( tu_cb_coded_flag[ xC ][ yC ] ∥ tu_cr_coded_flag[ xC ][ yC ] ) ) ∥( tu_cb_coded_flag[ xC ][ yC ] && tu_cr_coded_flag[ xC ][ yC ] ) ) &&chromaAvailable )tu_joint_cbcr_residual_flag[ xC ][ yC ]ae(v)if( tu_y_coded_flag[ x0 ][ y0 ] && treeType != DUAL_TREE_CHROMA ) {if( sps_transform_skip_enabled_flag && !BdpcmFlag[ x0 ][ y0 ][ 0 ] &&tbWidth <= MaxTsSize && tbHeight <= MaxTsSize &&( IntraSubPartitionsSplitType = = ISP_NO_SPLIT ) && !cu_sbt_flag )transform_skip_flag[ x0 ][ y0 ][ 0 ]ae(v)if( !transform_skip_flag[ x0 ][ y0 ][ 0 ] ∥ sh_ts_residual_coding_disabled_flag )residual_coding( x0, y0, Log2( tbWidth ), Log2( tbHeight ), 0 )elseresidual_ts_coding( x0, y0, Log2( tbWidth ), Log2( tbHeight ), 0 )}if( tu_cb_coded_flag[ xC ][ yC ] && treeType != DUAL_TREE_LUMA ) {if( sps_transform_skip_enabled_flag && !BdpcmFlag[ x0 ][ y0 ][ 1 ] &&wC >= MaxTsSize && hC >= MaxTsSize && !cu_sbt_flag )transform_skip_flag[ xC ][ yC ][ 1 ]ae(v)if( !transform_skip_flag[ xC ][ yC ][ 1 ] ∥ sh_ts_residual_coding_disabled_flag )residual_coding( xC, yC, Log2( wC ), Log2( hC ), 1 )elseresidual_ts_coding( xC, yC, Log2( wC ), Log2( hC ), 1 )}if( tu_cr_coded_flag[ xC ][ yC ] && treeType != DUAL_TREE_LUMA &&!( tu_cb_coded_flag[ xC ][ yC ] && tu_joint_cbcr_residual_flag[ xC ][ yC ] ) ) {if( sps_transform_skip_enabled_flag && !BdpcmFlag[ x0 ][ y0 ][ 2 ] &&wC <= MaxTsSize && hC <= MaxTsSize && !cu_sbt_flag )transform_skip_flag[ xC ][ yC ][ 2 ]ae(v)if( !transform_skip_flag[ xC ][ yC ][ 2 ] ∥ sh_ts_residual_coding_disabled_flag )residual_coding( xC, yC, Log2( wC ), Log2( hC ), 2 )elseresidual_ts_coding( xC, yC, Log2( wC ), Log2( hC ), 2 )}}

Depending on this transform_skip_flag (value 1 if transform skipping is used, value 0 otherwise), an actual quantization parameter qP982may be determined, for example, as specified in equations (436)-(438), for palette coding as described under item 1.1, as well as equation (1153), for transform coefficients scaling as described under item 1.2, otherwise. The plurality98of coding parameters may comprise this quantization parameter qP982controlling the quantization accuracy with respect to the predetermined portion100of the picture12.

1.1 Decoding Process for Palette Mode

Inputs to this process are:a location (xCbComp, yCbComp) specifying the top-left sample of the current coding block relative to the top-left sample of the current picture,a variable treeType specifying whether a single or a dual tree is used and if a dual tree is used, it specifies whether the current tree corresponds to the luma or chroma components,a variable cIdx specifying the colour component of the current block,two variables nCbW and nCbH specifying the width and height of the current coding block, respectively.

Output of this process is an array recSamples[x][y], with x=0 . . . nCbW−1, y=0 . . . nCbH−1 specifying reconstructed sample values for the block.

Depending on the value of treeType, the variables startComp, numComps and maxNumPalettePredictorSize are derived as follows:If treeType is equal to SINGLE_TREE:
startComp=0  (422)
numComps=sps_chroma_format_idc==?1:3  (423)
maxNumPalettePredictorSize=63  (424)Otherwise, treeType is equal to DUAL_TREE_LUMA:
startComp=0  (425)
numComps=1  (426)
maxNumPalettePredictorSize=31  (427)Otherwise, treeType is equal to DUAL_TREE_CHROMA:
startComp=1  (428)
numComps=2  (429)
maxNumPalettePredictorSize=31  (430)

Depending on the value of cIdx, the variables nSubWidth and nSubHeight are derived as follows:If cIdx is greater than 0 and startComp is equal to 0, nSubWidth is set equal to SubWidthC and nSubHeight is set equal to SubHeightC.Otherwise, nSubWidth is set equal to 1 and nSubHeight is set equal to 1.

The (nCbW×nCbH) block of the reconstructed sample array recSamples at location (xCbComp, yCbComp) is represented by recSamples[x][y] with x=0 . . . nCbW−1 and y=0 . . . nCbH−1, and the value of recSamples [x][y] for each x in the range of 0 to nCbW−1, inclusive, and each y in the range of 0 to nCbH−1, inclusive, is derived asThe variables xL, yL, xCbL, and yCbL are derived as follows:
xL=x*nSubWidth  (431)
yL=y*nSubHeight  (432)
xCbL=xCbComp*nSubWidth  (433)
yCbL=yCbComp*nSubHeight  (434)The variable blsEscapeSample is derived as follows:If PaletteIndexMap[xCbL+xL][yCbL+yL] is equal to MaxPaletteIndex and palette_escape_val_present_flag is equal to 1, blsEscapeSample is set equal to 1.Otherwise, blsEscapeSample is set equal to 0.If blsEscapeSample is equal to 0, the following applies:
recSamples[x][y]=CurrentPaletteEntries[cIdx][PaletteIndexMap[xCbL+xL][yCbL+yL]](435)Otherwise (blsEscapeSample is equal to 1), the following ordered steps apply:1. The quantization parameter qP is derived as follows:If cIdx is equal to 0,
qP=Max(QpPrimeTsMin,Qp′Y)  (436)Otherwise, if cIdx is equal to 1,
qP=Max(QpPrimeTsMin,Qp′Cb)  (437)Otherwise (cIdx is equal to 2),
qP=Max(QpPrimeTsMin,Qp′Cr)  (438)
1.2 Scaling Process for Transform Coefficients

Inputs to this process are:a luma location (xTbY, yTbY) specifying the top-left sample of the current luma transform block relative to the top-left luma sample of the current picture,a variable nTbW specifying the transform block width,a variable nTbH specifying the transform block height,a variable predMode specifying the prediction mode of the coding unit,a variable cIdx specifying the colour component of the current block.

Output of this process is the (nTbW)×(nTbH) array d of scaled transform coefficients with elements d[x][y].

The quantization parameter qP and the variable QpActOffset are derived as follows:If cIdx is equal to 0, the following applies:
qP=Qp′Y(1142)
QpActOffset=cu_act_enabled_flag[xTbY][yTbY]?−5:0  (1143)Otherwise, if TuCResMode[xTbY][yTbY] is equal to 2, the following applies:
qP=Qp′CbCr(1144)
QpActOffset=cu_act_enabled_flag[xTbY][yTbY]?1:0  (1145)Otherwise, if cIdx is equal to 1, the following applies:
qP=Qp′Cb(1146)
QpActOffset=cu_act_enabled_flag[xTbY][yTbY]?1:0  (1147)Otherwise (cIdx is equal to 2), the following applies:
qP=Qp′Cr(1148)
QpActOffset=cu_act_enabled_flag[xTbY][yTbY]?3:0  (1149)

The quantization parameter qP is modified and the variables rectNonTsFlag and bdShift are derived as follows:If transform_skip_flag[xTbY][yTbY][cIdx] is equal to 0, the following applies:
qP=Clip3(0,63+QpBdOffset,qP+QpActOffset)  (1150)
rectNonTsFlag=(((Log 2(nTbW)+Log 2(nTbH))&1)==1)?1:0  (1151)
bdShift=BitDepth+rectNonTsFlag+((Log 2(nTbW)+Log 2(nTbH))/2)−5+sh_dep_quant_used_flag  (1152)Otherwise (transform_skip_flag[xTbY][yTbY][cIdx] is equal to 1), the following applies:
qP=Clip3(QpPrimeTsMin,63+QpBdOffset,qP+QpActOffset)  (1153)
rectNonTsFlag=0  (1154)
bdShift=10  (1155)

According to an embodiment, the value of qP is limited to be greater than or equal to a minimum of QpPrimeTsMin, a constant (across the given video sequence) specified in equation (69):

sps_internal_bitdepth_minus_input_bitdepth specifies the minimum allowed quantization parameter for transform skip mode as follows:
QpPrimeTsMin=4+6*sps_internal_bitdepth_minus_input_bitdepth  (69)

The value of sps_internal_bitdepth_minus_input_bitdepth shall be in the range of 0 to 8, inclusive.

In other words, the decoder/encoder is configured to read/signal the plurality98of coding parameters from/into the data stream14and check whether the prediction residual transform mode indicated by the plurality of coding parameters corresponds to a transform skip mode and to a quantization step size finer than a predetermined quantization step size, i.e. the QpPrimeTsMin, corresponding to no quantization, and, if yes, change the quantization step size to the predetermined quantization step size. QpPrimeTsMin, finally, is governed by a sequence-wise transmitted (in the sequence header) parameter sps_internal_bitdepth_minus_input_bitdepth. The predetermined quantization step size may be dependent on an internal bitdepth and an input bitdepth, e.g., it may be dependent on a difference between the internal bit-depth and the input bit-depth. The decoder/encoder may be configured to deduce a minimum for a quantization step size scale parameter, i.e. the predetermined quantization step size and/or the QpPrimeTsMin, based on the difference, and adhere to the minimum quantization step size scale parameter for portions100in a prediction residual transform skip mode. The decoder/encoder may be configured to in adhering to the minimum quantization step size scale parameter for portions100coded in a prediction residual transform skip mode, change a signaled quantization step size scale parameter982signaled in the data stream14for the portions100, to equal the minimum quantization step size scale parameter in case of the signaled quantization step size scale parameter falling below the minimum quantization step size scale parameter. Note that, when this parameter equals zero (which is a commonly encountered case), QpPrimeTsMin results in a value of 4.

Thus, using element sps_internal_bitdepth_minus_input_bitdepth, element transform_skip_flag, and variable QpPrimeTsMin, it is possible to achieve lossless coding of prediction residual corrected predictive reconstructed picture samples in a specific picture region, such as a coding or transform unit, and a specific color component.

To reach lossless coding even in the presence of in-loop “post-reconstruction” deblocking filtering, the deblocking filtering parameters are also conditioned on the actual quantization parameter qP, in a manner that the filtering is, effectively, bypassed when the qP value is less than or equal to QpPrimeTsMin. This is achieved by a corresponding specification of the deblocking parameters ß′ and tc′ to both equal zero, as shown in Table 2 and equations (1280) and (1282) of item 1.3. Note that this may entail transmitting specific appropriate values for sh_luma_beta_offset_div2 for equation (1280) and sh_luma_tc_offset_div2 for equation (1282) in cases where QpPrimeTsMin of equation (69) is greater than 4 (which results from sps_internal_bitdepth_minus_input_bitdepth>0). The decoder/encoder is configured to infer that one or more predetermined coding options relating to one or more tools of the decoder/encoder for processing a prediction residual corrected predictive reconstruction with respect to the predetermined portion100or for prediction residual re-quantization/quantization are to be set110so that the one or more tools are disabled with respect to the predetermined portion100, if the predetermined portion100of the picture12is coded into the data stream14using lossless coding1061, and by deriving120the one or more predetermined coding options from the plurality98of coding parameters, if the predetermined portion100of the picture is coded into the data stream14using lossy coding1062.

1.3 Decision process for luma block edges

Inputs to this process are:a picture sample array recPicture,a location (xCb, yCb) specifying the top-left sample of the current coding block relative to the top-left sample of the current picture,a location (xBl, yBl) specifying the top-left sample of the current block relative to the top-left sample of the current coding block,a variable edgeType specifying whether a vertical (EDGE_VER) or a horizontal (EDGE_HOR) edge is filtered,a variable bS specifying the boundary filtering strength,a variable maxFilterLengthP specifying the maximum filter length,a variable maxFilterLengthQ specifying the maximum filter length.

Outputs of this process are:the variables dE, dEp and dEq containing decisions,the modified filter length variables maxFilterLengthP and maxFilterLengthQ,the variable tC.

The sample values pi,kand qj,kwith i=0 . . . Max(2, maxFilterLengthP), j=0 . . . Max(2, maxFilterLengthQ) and k=0 and 3 are derived as follows:If edgeType is equal to EDGE_VER, the following applies:
qj,k=recPicture[xCb+xBl+j][yCb+yBl+k](1273)
pi,k=recPicture[xCb+xBl−i−1][yCb+yBl+k](1274)Otherwise (edgeType is equal to EDGE_HOR), the following applies:
qj,k=recPicture[xCb+xBl+k][yCb+yBl+j](1275)
pi,k=recPicture[xCb+xBl+k][yCb+yBl−i−1]  (1276)

The variable qpOffset is derived as follows:If sps_ladf_enabled_flag is equal to 1, the following applies:The variable lumaLevel of the reconstructed luma level is derived as follow:
lumaLevel=((p0,0+p0,3+q0,0+q0,3)>>2),  (1277)The variable qpOffset is set equal to sps_ladf_lowest_interval_qp_offset and modified as follows:

for( i = 0; i < sps_num_ladf_intervals_minus2 + 1; i++ ) {(1278)if( lumaLevel > SpsLadfIntervalLowerBound[ i + 1 ] )qpOffset = sps_ladf_qp_offset[ i ]elsebreak}Otherwise, qpOffset is set equal to 0.

The variables QpQand QpPare set equal to the QpYvalues of the coding units which include the coding blocks containing the sample q0,0and p0,0, respectively.

The variable qP is derived as follows:
qP=((QpQ+QpP+1)>>1)+qpOffset  (1279)

The value of the variable β′ is determined as specified in table 2 based on the quantization parameter Q derived as follows:
Q=Clip3(0,63,qP+(sh_luma_beta_offset_div2<<1))  (1280)
where sh_luma_beta_offset_div2 is the value of the syntax element sh_luma_beta_offset_div2 for the slice that contains sample q0,0.

The variable β is derived as follows:
β=β′*(1<<(BitDepth−8))  (1281)

The value of the variable tC′ is determined as specified in table 2 based on the quantization parameter Q derived as follows:
Q=Clip3(0,65,qP+2*(bS−1)+(sh_luma_tc_offset_div2<<1))  (1282)
where sh_luma_tc_offset_div2 is the value of the syntax element sh_luma_tc_offset_div2 for the slice that contains sample q0,0.

The variable tCis derived as follows:
roundOffset=1<<(9−BitDepth)  (1283)
tC=BitDepth<10?(tC′+roundOffset)>>(10−BitDepth):tC′*(1<<(BitDepth−10))  (1284)

TABLE 2Derivation of threshold variables β′ and tC′ from input QQ012345β′000000tC′000000Q67891011β′000000tC′000000Q121314151617β′000067tC′000000Q181920212223β′8910111213tC′344445Q242526272829β′141516171820tC′555778Q303132333435β′222426283032tC′91010111314Q363738394041β′343638404244tC′151719212425Q424344454647β′464850525456tC′293336414551Q484950515253β′586062646668tC′5764718089100Q545556575859β′707274767880tC′112125141157177198Q606162636465β′82848688——tC′222250280314352395

Aspect 2: Introduction of a New “Global” lossless_coding Flag

To streamline the VVC configuration for lossless coding ability, it is proposed to introduce a new one-bit flag, to be called lossless_coding herein, into the sequence, frame, picture, sub-picture, tile-group, or slice header syntax of an image or video coding bit-stream. In other words, a new “global” (relative to the subblock level) flag is proposed to be introduced, which can have a value of 0 (no lossless coding operation, i. e., normal mode) or 1 (lossless mode) and which controls the operation (activation or deactivation or algorithmic details) of at least two coding tools provided (i. e., specified) by the affected image or video codec. More specifically, the operation of at least 2 of the following list of tools depends on the lossless_coding flag:entropy coder for any residual (spatial or transform) signal coefficientsjoint inter-component transform (ICT) in terms of transform matrix operationsmoothing filtering operations of some predictive coding tools (to be disabled)in-loop filters: deblocking, shape adaptive offset, adaptive loop filter (ALF), reshaper (disabled)quantization of residual spatial or transform coefficients (disabled)

Aspect 3: Desired Effect of “Global” lossless_coding Flag on Individual Coding Tools

This third aspect specifies in detail how the value of the lossless_coding flag or the inferred meaning of the transform mode (indicating transform skipping functionality) and QP (indicating a particular step-size), as mentioned in Aspect 1 above, changes the behavior of the entropy coding, inter-component transform, prediction, and in-loop filtering tools for a given picture region or portion associated with said lossless_coding flag or said inferred meaning.

3.0 Quantization

A quantization parameter (QP) equals to four and the usage of the transform skip mode results in the lossless quantization mode of the residual signal. A bitdepth correction may be used when the input bitdepth and the internal bitdepth differs. A difference between the input bitdepth and the internal bitdepth can be realized by signaling the input bitdepth in the high-level syntax. Alternatively, the difference between the input and the internal bitdepth can be transmitted in the high-level syntax. Another alternative is the signaling of the minimum allowed QP for transform skip in the high-level syntax, either absolutely or relative to the base QP.

An embodiment is related to a video decoder configured to perform video decoding from a data stream14at an internal bit-depth and video output at an input bit-depth or internal bit-depth. The video decoder is configured to read from the data stream a syntax element which indicates a difference between the internal bit-depth and the input bit-depth. Parallel a video encoder is configured to perform video encoding into a data stream14at an internal bit-depth and receive video input at an input bit-depth or internal bit-depth. The video encoder is configured to encode into the data stream14the syntax element which indicates the difference between the internal bit-depth and the input bit-depth.

A bit-depth transition may be performed from an internally-reconstructed video version46to the input bit-depth. The decoder may be configured to perform this transition before the video output and the encoder may be configured to perform this transition in the prediction-loop36.

According to an embodiment, the decoder/encoder is configured to deduce a minimum for a quantization step size scale parameter, e.g. the quantization parameter982, based on the difference, e.g., owing to a difference of non-zero, another QP than 4 may result in lossless coding or almost lossless coding, and adhere to the minimum quantization step size scale parameter for portions of the video coded in a prediction residual transform skip mode. The decoder/encoder is configured to map a quantization step size scale parameter982finer than the minimum, e.g., QP<4, to a quantization step size scale parameter982equal the minimum to enable a lossless coding. For lossless coding, the minimum might be associated with no quantization32,52or a bypassing or disabling of a quantization32,52. The decoder/encoder may be configured to in adhering to the minimum quantization step size scale parameter for video portions100coded in a prediction residual transform skip mode, change a signaled quantization step size scale parameter signaled in the data stream14for the video portions100, to equal the minimum quantization step size scale parameter in case of the signaled quantization step size scale parameter falling below the minimum quantization step size scale parameter. The decoder/encoder may adhere to the minimum quantization step size scale parameter merely within video regions104for which the data stream14signals a lossless coding mode1061.

Another embodiment is related to a video decoder configured to perform video decoding from a data stream14at an internal bit-depth and video output at an input bit-depth or internal bit-depth, and read from the data stream14a syntax element which indicates a minimum for a quantization step size scale parameter (QP minimum), e.g., owing to a difference of non-zero between internal and input bit-depth, on which difference the decoder might be informed as well, e.g., in addition to the QP minimum, either by way of transmitting the difference or by transmitting both values, i.e. the input bit-depth and the internal bit-depth. Another QP than 4 may result in, or almost in, lossless coding. Parallel, a video encoder may be configured to perform video encoding into the data stream14at an internal bit-depth and receive video input at an input bit-depth or internal bit-depth, and encode into the data stream14a syntax element which indicates a minimum for a quantization step size scale parameter. The decoder/encoder may be configured to in adhere to the minimum quantization step size scale parameter for video portions100coded in a prediction residual transform skip mode, and optionally change a signaled quantization step size scale parameter signaled in the data stream14for the video portions100, to equal the minimum quantization step size scale parameter in case of the signaled quantization step size scale parameter falling below the minimum quantization step size scale parameter. The decoder/encoder may adhere to the minimum quantization step size scale parameter merely within video regions104for which the data stream14signals a lossless coding mode1061.

Another embodiment is related to a video decoder/encoder configured to derive/encode from/into a data stream14an indication of an internal bit-depth and an input bit-depth or a difference between same, perform video decoding/encoding from/into the data stream14at the internal bit-depth and the decoder is configured to video output at the input bit-depth and the encoder is configured to receive video input at the input bit-depth. Additionally, the decoder/encoder is configured to check whether the internal bit-depth falls below the input bit-depth and change the internal bit-depth to correspond to the input bit-depth. Optionally, the decoder/encoder is configured to derive/encode from/into the data stream14an indication of a lossless coded video portion, and perform the checking and the changing within the lossless coded video portion and use the internal bit-depth, e.g., as derived from the data stream14, for a lossy coded video portion. The internal bit-depth may be signaled in the data stream be the encoder.

3.1. Entropy Coding

Since lossless coding, e.g. with a QP of 4, typically produces significant higher bitrates compared to lossy coding, the entropy coding engine can switch to a high-throughput mode to prevent processing bottlenecks. This could be done e.g. by using different codes in binarization or processing more bins after binarization in bypass coding mode instead of regular coding mode in CABAC that uses context modelling and binary arithmetic coding.

An embodiment is related to a decoder (encoder), configured to determine for a predetermined portion100of a picture12, whether same is (to be) coded into a data stream14using lossless coding1061or lossy coding1062, and decode (encode) a prediction residual from (into) the data stream14for the predetermined portion100using binarization of prediction residual data into bin strings and context-adaptive binary entropy decoding (encoding) of the bin strings in a first manner (called residual_ts_coding( ) in Table 1), if the predetermined portion100of the picture12is (to be) coded into the data stream14using lossless coding1061, and in a second manner (called residual_coding( ) in Table 1), if the predetermined portion100of the picture12is (to be) coded into the data stream14using lossy coding1062, wherein the first and second manners differ so that a computational complexity is reduced in the first manner compared to the second manner.

The determination whether the predetermined portion100of a picture12is coded into a data stream14using lossless coding1061or lossy coding1062can be based on the data stream14, for example, like described with regard to the decoder/encoder inFIG.6or by reading a portion-wise transform quantization bypass coding flag or differently. According to an embodiment, the decoder is configured to perform the determining by reading from the data stream14a lossless coding syntax element, e.g.102, which indicates whether the predetermined portion100of the picture12, or a predetermined region104containing the predetermined portion100, is coded into the data stream14using lossless coding1061or lossy coding1062and performing the determination depending on the lossless coding syntax element. The encoder may encode this lossless coding syntax element.

The computational complexity may be reduced in the first manner compared to the second manner by at least one ofa number of bins of the bin strings coded using equi-probability bypass mode is greater in the first manner than in the second manner, anddifferent binarization codes, e.g. Expontial Golomb code, truncated unary code or the like, being used in the first and second manner.

According to an embodiment, an amount of bins of the bin strings belonging to a unary code or truncated unary code of the bin strings is lower in the first manner than in the second manner.

3.2 Inter-Component Transform (ICT)

The coefficients of the ICT inverse transform matrix do not allow for lossless coding since there is no corresponding forward transform matrix which, in the absence of residual signal quantization, results in perfectly lossless reconstruction. Hence, it is proposed to, when lossless_coding equals 1, a different ICT inverse transform matrix (specified as a list of coefficients or mathematical operations such as shifts or multiplications) is to be employed in the codec than when lossless_coding equals 0 (normal operation). Specifically, when lossless_coding equals zero, the conventional ICT upmix operation, e. g.,
cb=c1+c2,
cr=sign*(c1−c2),
may be employed, where c1 and c2 are two transmitted residual block signals and cb and cr are the associated output residual signals resulting from the inverse ICT. When lossless_coding equals one, however, a lifting transform [4, 5] or a modulo transform [6] operation may be applied to c1 and c2 instead of the abovementioned operations, which allows for perfect reconstruction of the cb and cr prediction residuals in the absence of quantization and in-loop filtering. A different but equivalent way to describe this perfect reconstruction property is to state that the transform is, mathematically, perfectly invertible even in practical applications running on, e. g., computers where, usually, transform operations use rounding-to-integer steps forbidding mathematically perfect reconstruction. For example, the forward lossless ICT
c1=cb+sign*cr,
c2=cb−sign*INT(c1/2)
along with the corresponding inverse lossless ICT
cb′=c2+sign*INT(c1/2),
cr′=sign*(c1−cb′),
where INT( ) denotes a floor (round towards minus infinity), ceiling (round towards plus infinity), or rounding (round to nearest integer) operator and sign equals 1 or −1, achieves perfect reconstruction of both cb and cr (i. e., cb′=cb, cr′=cr). Hence, the above inverse lossless ICT operation resulting in cb′ and cr′ is advantageously applied in the decoder when lossless coding is desired. Note that perfect reconstruction can also be achieved by switching the above forward and inverse lossless operations such that the forward lossless ICT is given by
c1=cr+sign*INT(cb/2),
c2=sign*(cb−c1)
and the corresponding inverse losslessICTis given by
cb′=c1+sign*c2,
cr′=c1−sign*INT(cb′/2).

Also note that the + and − signs in the above equations may differ in particular implementations while leading to equivalent results (i. e., cb′=cb, cr′=cr). Finally, it is worth noting that slightly different formulations, e. g., a formulation equivalent to the integer mid-side (M/S) processing in HD-AAC, described in [5], may be employed as lossless inverse transform in the decoding process.

A decoder (encoder), according to an embodiment, is configured to determine for a predetermined portion100of a picture12, whether same is (to be) coded into a data stream14using lossless coding1061or lossy coding1062, and perform on a prediction residual24″,24″ or a prediction residual corrected reconstruction46(e.g., in a prediction-loop of the encoder) of the predetermined portion100a perfectly invertible transform, if the predetermined portion100of the picture12is coded into the data stream14using lossless coding1061, and a non-perfectly invertible transform, if the predetermined portion100of the picture12is coded into the data stream14using lossy coding1062.

The determination whether the predetermined portion100of a picture12is coded into a data stream14using lossless coding1061or lossy coding1062can be based on the data stream14, for example, like described with regard to the decoder/encoder inFIG.6or by reading a portion-wise transform quantization bypass coding flag or differently. According to an embodiment, the decoder is configured to perform the determining by reading from the data stream14a lossless coding syntax element, e.g.102, which indicates whether the predetermined portion100of the picture12, or a predetermined region104containing the predetermined portion100, is coded into the data stream14using lossless coding1061or lossy coding1062and performing the determination depending on the lossless coding syntax element. The encoder may encode this lossless coding syntax element.

The perfectly invertible transform and the non-perfectly invertible transform may be inter-color-component transforms or are spectrally decomposing intra-color-component transforms.

3.3 Smoothing Filters in Some Predictors

For lossless coding with high-bitrates, filters that are applied to the prediction signal to attenuate compression artifacts, e.g. smoothing filters to reduce quantization artifacts, could be not beneficial. So in case of transform skip with quantization skipping, these filters can be disabled for lossless coding.

A decoder (encoder), according to an embodiment, is configured to determine, for a predetermined portion100of a picture12, whether same is (to be) coded into the data stream14using lossless coding1061or lossy coding1062, and derive a prediction signal26for the predetermined portion100in a first manner, if the predetermined portion100of the picture12is (to be) coded into the data stream14using lossless coding1061, and in a second manner, if the predetermined portion100of the picture12is (to be) coded into the data stream14using lossy coding1062, wherein the first and second manners differ so that the prediction signal26is less filtered, e.g. by an interpolation filter, in the first manner than in the second manner or unfiltered in the first manner while being filtered in the second manner.

The determination whether the predetermined portion100of a picture12is coded into a data stream14using lossless coding1061or lossy coding1062can be based on the data stream14, for example, like described with regard to the decoder/encoder inFIG.6or by reading a portion-wise transform quantization bypass coding flag or differently. According to an embodiment, the decoder is configured to perform the determining by reading from the data stream14a lossless coding syntax element, e.g.102, which indicates whether the predetermined portion100of the picture12, or a predetermined region104containing the predetermined portion100, is coded into the data stream14using lossless coding1061or lossy coding1062and performing the determination depending on the lossless coding syntax element. The encoder may encode this lossless coding syntax element.

According to an embodiment, the prediction signal26is less filtered in the first manner than in the second manner or unfiltered in the first manner while being filtered in the second manner in terms of low-pass filtering, e.g. the prediction signal26has, in a higher frequency half out of an overall spatial frequency spectrum of the prediction signal26, higher energy when derived based on the first manner than in the second manner.

3.4 In-Loop Filters and Quantization

When lossless_coding equals 1, all in-loop filters (e.g., deblocking, shape adaptive offset (SAO), reshaper) and the quantization are disabled, and their related syntax element(s), incl. their individual sequence or frame-wise activation flags and any QPs or delta-QPs, shall not be present in a coded bit-stream.

A decoder (encoder), according to an embodiment, is configured to determine, for a predetermined portion100of a picture12, whether same is (to be) coded into the data stream14using lossless coding1061or lossy coding1062, and infer that one or more predetermined coding options relating to one or more tools of the decoder (encoder) for processing a prediction residual corrected predictive reconstruction46with respect to the predetermined portion100or for prediction residual re-quantization52,38(quantization52) are to be set so that the one or more tools are disabled with respect to the predetermined portion100, if the predetermined portion100of the picture12is coded into the data stream14using lossless coding1061, and by deriving the one or more predetermined coding options from a plurality98of coding parameters, if the predetermined portion100of the picture12is coded into the data stream14using lossy coding1062.

The determination whether the predetermined portion100of a picture12is coded into a data stream14using lossless coding1061or lossy coding1062can be based on the data stream14, for example, like described with regard to the decoder/encoder inFIG.6or by reading a portion-wise transform quantization bypass coding flag or differently. According to an embodiment, the decoder is configured to perform the determining by reading from the data stream14a lossless coding syntax element, e.g.102, which indicates whether the predetermined portion100of the picture12, or a predetermined region104containing the predetermined portion100, is coded into the data stream14using lossless coding1061or lossy coding1062and performing the determination depending on the lossless coding syntax element. The encoder may encode this lossless coding syntax element.

In the following different embodiments of methods are described.

FIG.7shows a method200for decoding (encoding) a picture into a data stream, comprising determining106for a predetermined portion of the picture, whether same is (to be) coded into the data stream using lossless coding or lossy coding, and decoding (encoding) a prediction residual from (into) the data stream for the predetermined portion using binarization of prediction residual data into bin strings and context-adaptive binary entropy encoding of the bin strings in a first manner210, if the predetermined portion of the picture is (to be) coded into the data stream using lossless coding, and in a second manner220, if the predetermined portion of the picture is (to be) coded into the data stream using lossy coding, wherein the first210and second220manners differ so that a computational complexity is reduced in the first manner210compared to the second manner220.

FIG.8shows a method300for decoding (encoding) a picture from (into) a data stream, comprising determining106for a predetermined portion of the picture, whether same is (to be) coded into the data stream using lossless coding or lossy coding, and performing on a prediction residual or a prediction residual corrected reconstruction of the predetermined portion a perfectly invertible transform310, if the predetermined portion of the picture is (to be) coded into the data stream using lossless coding, and a non-perfectly invertible transform320, if the predetermined portion of the picture is (to be) coded into the data stream using lossy coding.

FIG.9shows a method400for decoding (encoding) a picture from (into) a data stream, comprising determining106for a predetermined portion of the picture, whether same is (to be) coded into the data stream using lossless coding or lossy coding, and deriving a prediction signal for the predetermined portion in a first manner410, if the predetermined portion of the picture is (to be) coded into the data stream using lossless coding, and in a second manner420, if the predetermined portion of the picture is (to be) coded into the data stream using lossy coding, wherein the first and second manners differ so that the prediction signal is less filtered, e.g. using an interpolation filter, in the first manner410than in the second manner420or unfiltered in the first manner410while being filtered in the second manner420.

FIG.10shows a method500for decoding (encoding) a picture from (into) a data stream, comprising determining106for a predetermined portion of the picture, whether same is (to be) coded into the data stream using lossless coding or lossy coding, and inferring that one or more predetermined coding options relating to one or more tools of the decoder/encoder for processing a prediction residual corrected predictive reconstruction with respect to the predetermined portion or for prediction residual quantization are to be set so that the one or more tools are disabled510with respect to the predetermined portion, if the predetermined portion of the picture is (to be) coded into the data stream using lossless coding, and by deriving520the one or more predetermined coding options from the plurality of coding parameters, if the predetermined portion of the picture is (to be) coded into the data stream using lossy coding.

FIG.11ashows a method600comprising performing610video decoding from a data stream at an internal bit-depth and video outputting620at an input bit-depth or internal bit-depth and reading630from the data stream a syntax element which indicates a difference between the internal bit-depth and the input bit-depth.FIG.11bshows a parallel method600comprising performing612video encoding into a data stream at an internal bit-depth and receiving622video input at an input bit-depth or internal bit depth and encoding632into the data stream a syntax element which indicates a difference between the internal bit-depth and the input bit-depth.

FIG.12ashows a method700comprising performing610video decoding from a data stream at an internal bit-depth and video outputting620at an input bit-depth or internal bit-depth and reading730from the data stream a syntax element which indicates a minimum for a quantization step size scale parameter.FIG.12bshows a parallel method700comprising performing612video encoding into a data stream at an internal bit-depth and receiving622video input at an input bit-depth or internal bit-depth and encoding732into the data stream a syntax element which indicates a minimum for a quantization step size scale parameter.

FIG.13ashows a method800comprising deriving810from a data stream an indication of an internal bit-depth and an input bit-depth or a difference between same, performing610video decoding from the data stream at the internal bit-depth and video outputing620at the input bit-depth, checking820whether the internal bit-depth falls below the input bit-depth and changing830the internal bit-depth to correspond to the input bit-depth.FIG.13bshows a parallel method800comprising encoding812into a data stream video an indication of an internal bit-depth and an input bit-depth or a difference between same, performing612video encoding into the data stream at the internal bit-depth and receiving622video input at the input bit-depth, checking820whether the internal bit-depth falls below the input bit-depth and changing830the internal bit-depth to correspond to the input bit-depth.

Implementation Alternatives

Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.

Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.

A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods may be performed by any hardware apparatus.

The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

The apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and/or in software.

The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

The methods described herein, or any components of the apparatus described herein, may be performed at least partially by hardware and/or by software.

While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.

REFERENCES

[1] ITU-T, Recommendation H.265 and ISO/IEC, Int. Standard 23008-2, “High efficiency video coding,” Geneva, Switzerland, February 2018. Online: http://www.itu.int/rec/T-REC-H.265.[2] JVET/Fraunhofer HHI, “VVCSoftware_VTM: VVC VTM reference software,” July 2019. Online: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM.[3] C. Helmrich, H. Schwarz, T. Nguyen, C. Rudat, D. Marpe, and T. Wiegand, “CE7: Joint Chroma Residual Coding with Multiple Modes (Tests CE7-2.1, CE7-2.2),” Joint Video Experts Group, document JVET-00105, Gothenburg, June 2019.[4] I. Daubechies and W. Sweldens, “Factoring Wavelet Transforms into Lifting Steps,”J. Four. Anal. Appl., September1996.[5] R. Geiger, R. Yu, J. Herre, S. Rahardja, S.-W. Kim, X. Lin, and M. Schmidt, “ISO/IEC MPEG-4 High-Definition Scalable Advanced Audio Coding,”J. Aud. Eng. Soc., vol. 55, no. 1/2, pp. 27-43, January 2007.[6] S. Srinivasan, “Modulo transforms—An alternative to lifting,”IEEE Trans. on Signal Processing, vol. 54, no. 5, pp. 1864-1874, June 2006.