Patent ID: 12244864

DETAILED DESCRIPTION OF THE EMBODIMENTS

Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals even if occurring in different figures.

In the following description, a plurality of details is set forth to provide a more throughout explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present invention. In addition, features of the different embodiments described herein after may be combined with each other, unless specifically noted otherwise.

In the following, various examples are described which may assist in achieving a more effective compression when using matrix-based intra prediction. The matrix-based intra prediction may be added to other intra-prediction modes heuristically designed, for instance, or may be provided exclusively.

In order to ease the understanding of the following examples of the present application, the description starts with a presentation of possible encoders and decoders fitting thereto into which the subsequently outlined examples of the present application could be built.FIG.1shows an apparatus for block-wise encoding a picture10into a datastream12. The apparatus is indicated using reference sign14and may be a still picture encoder or a video encoder. In other words, picture10may be a current picture out of a video16when the encoder14is configured to encode video16including picture10into datastream12, or encoder14may encode picture10into datastream12exclusively.

As mentioned, encoder14performs the encoding in a block-wise manner or block-base. To this, encoder14subdivides picture10into blocks, units of which encoder14encodes picture10into datastream12. Examples of possible subdivisions of picture10into blocks18are set out in more detail below. Generally, the subdivision may end-up into blocks18of constant size such as an array of blocks arranged in rows and columns or into blocks18of different block sizes such as by use of a hierarchical multi-tree subdivisioning with starting the multi-tree subdivisioning from the whole picture area of picture10or from a pre-partitioning of picture10into an array of tree blocks wherein these examples shall not be treated as excluding other possible ways of subdivisioning picture10into blocks18.

Further, encoder14is a predictive encoder configured to predictively encode picture10into datastream12. For a certain block18this means that encoder14determines a prediction signal for block18and encodes the prediction residual, i.e. the prediction error at which the prediction signal deviates from the actual picture content within block18, into datastream12.

Encoder14may support different prediction modes so as to derive the prediction signal for a certain block18. The prediction modes, which are of importance in the following examples, are intra-prediction modes according to which the inner of block18is predicted spatially from neighboring, already encoded samples of picture10. The encoding of picture10into datastream12and, accordingly, the corresponding decoding procedure, may be based on a certain coding order20defined among blocks18. For instance, the coding order20may traverse blocks18in a raster scan order such as row-wise from top to bottom with traversing each row from left to right, for instance. In case of hierarchical multi-tree based subdivisioning, raster scan ordering may be applied within each hierarchy level, wherein a depth-first traversal order may be applied, i.e. leaf notes within a block of a certain hierarchy level may precede blocks of the same hierarchy level having the same parent block according to coding order20. Depending on the coding order20, neighboring, already encoded samples of a block18may be located usually at one or more sides of block18. In case of the examples presented herein, for instance, neighboring, already encoded samples of a block18are located to the top of, and to the left of block18.

Intra-prediction modes may not be the only ones supported by encoder14. In case of encoder14being a video encoder, for instance, encoder14may also support inter-prediction modes according to which a block18is temporarily predicted from a previously encoded picture of video16. Such an inter-prediction mode may be a motion-compensated prediction mode according to which a motion vector is signaled for such a block18indicating a relative spatial offset of the portion from which the prediction signal of block18is to be derived as a copy. Additionally or alternatively, other non-intra-prediction modes may be available as well such as inter-view prediction modes in case of encoder14being a multi-view encoder, or non-predictive modes according to which the inner of block18is coded as is, i.e. without any prediction.

Before starting with focusing the description of the present application onto intra-prediction modes, a more specific example for a possible block-based encoder, i.e. for a possible implementation of encoder14, as described with respect toFIG.2with then presenting two corresponding examples for a decoder fitting toFIGS.1and2, respectively.

FIG.2shows a possible implementation of encoder14ofFIG.1, namely one where the encoder is configured to use transform coding for encoding the prediction residual although this is nearly an example and the present application is not restricted to that sort of prediction residual coding. According toFIG.2, encoder14comprises a subtractor22configured to subtract from the inbound signal, i.e. picture10or, on a block basis, current block18, the corresponding prediction signal24so as to obtain the prediction residual signal26which is then encoded by a prediction residual encoder28into a datastream12. The prediction residual encoder28is composed of a lossy encoding stage28aand a lossless encoding stage28b. The lossy stage28areceives the prediction residual signal26and comprises a quantizer30which quantizes the samples of the prediction residual signal26. As already mentioned above, the present example uses transform coding of the prediction residual signal26and accordingly, the lossy encoding stage28acomprises a transform stage32connected between subtractor22and quantizer30so as to transform such a spectrally decomposed prediction residual26with a quantization of quantizer30taking place on the transformed coefficients where presenting the residual signal26. The transform may be a DCT, DST, FFT, Hadamard transform or the like. The transformed and quantized prediction residual signal34is then subject to lossless coding by the lossless encoding stage28bwhich is an entropy coder entropy coding quantized prediction residual signal34into datastream12. Encoder14further comprises the prediction residual signal reconstruction stage36connected to the output of quantizer30so as to reconstruct from the transformed and quantized prediction residual signal34the prediction residual signal in a manner also available at the decoder, i.e. taking the coding loss is quantizer30into account. To this end, the prediction residual reconstruction stage36comprises a dequantizer38which perform the inverse of the quantization of quantizer30, followed by an inverse transformer40which performs the inverse transformation relative to the transformation performed by transformer32such as the inverse of the spectral decomposition such as the inverse to any of the above-mentioned specific transformation examples. Encoder14comprises an adder42which adds the reconstructed prediction residual signal as output by inverse transformer40and the prediction signal24so as to output a reconstructed signal, i.e. reconstructed samples. This output is fed into a predictor44of encoder14which then determines the prediction signal24based thereon. It is predictor44which supports all the prediction modes already discussed above with respect toFIG.1.FIG.2also illustrates that in case of encoder14being a video encoder, encoder14may also comprise an in-loop filter46with filters completely reconstructed pictures which, after having been filtered, form reference pictures for predictor44with respect to inter-predicted block.

As already mentioned above, encoder14operates block-based. For the subsequent description, the block bases of interest is the one subdividing picture10into blocks for which the intra-prediction mode is selected out of a set or plurality of intra-prediction modes supported by predictor44or encoder14, respectively, and the selected intra-prediction mode performed individually. Other sorts of blocks into which picture10is subdivided may, however, exist as well. For instance, the above-mentioned decision whether picture10is inter-coded or intra-coded may be done at a granularity or in units of blocks deviating from blocks18. For instance, the inter/intra mode decision may be performed at a level of coding blocks into which picture10is subdivided, and each coding block is subdivided into prediction blocks. Prediction blocks with encoding blocks for which it has been decided that intra-prediction is used, are each subdivided to an intra-prediction mode decision. To this, for each of these prediction blocks, it is decided as to which supported intra-prediction mode should be used for the respective prediction block. These prediction blocks will form blocks18which are of interest here. Prediction blocks within coding blocks associated with inter-prediction would be treated differently by predictor44. They would be inter-predicted from reference pictures by determining a motion vector and copying the prediction signal for this block from a location in the reference picture pointed to by the motion vector. Another block subdivisioning pertains the subdivisioning into transform blocks at units of which the transformations by transformer32and inverse transformer40are performed. Transformed blocks may, for instance, be the result of further subdivisioning coding blocks. Naturally, the examples set out herein should not be treated as being limiting and other examples exist as well. For the sake of completeness only, it is noted that the subdivisioning into coding blocks may, for instance, use multi-tree subdivisioning, and prediction blocks and/or transform blocks may be obtained by further subdividing coding blocks using multi-tree subdivisioning, as well.

A decoder54or apparatus for block-wise decoding fitting to the encoder14ofFIG.1is depicted inFIG.3. This decoder54does the opposite of encoder14, i.e. it decodes from datastream12picture10in a block-wise manner and supports, to this end, a plurality of intra-prediction modes. The decoder54may comprise a residual provider156, for example. All the other possibilities discussed above with respect toFIG.1are valid for the decoder54, too. To this, decoder54may be a still picture decoder or a video decoder and all the prediction modes and prediction possibilities are supported by decoder54as well. The difference between encoder14and decoder54lies, primarily, in the fact that encoder14chooses or selects coding decisions according to some optimization such as, for instance, in order to minimize some cost function which may depend on coding rate and/or coding distortion. One of these coding options or coding parameters may involve a selection of the intra-prediction mode to be used for a current block18among available or supported intra-prediction modes. The selected intra-prediction mode may then be signaled by encoder14for current block18within datastream12with decoder54redoing the selection using this signalization in datastream12for block18. Likewise, the subdivisioning of picture10into blocks18may be subject to optimization within encoder14and corresponding subdivision information may be conveyed within datastream12with decoder54recovering the subdivision of picture10into blocks18on the basis of the subdivision information. Summarizing the above, decoder54may be a predictive decoder operating on a block-bases and besides intra-prediction modes, decoder54may support other prediction modes such as inter-prediction modes in case of, for instance, decoder54being a video decoder. In decoding, decoder54may also use the coding order20discussed with respect toFIG.1and as this coding order20is obeyed both at encoder14and decoder54, the same neighboring samples are available for a current block18both at encoder14and decoder54. Accordingly, in order to avoid unnecessary repetition, the description of the mode of operation of encoder14shall also apply to decoder54as far the subdivision of picture10into blocks is concerned, for instance, as far as prediction is concerned and as far as the coding of the prediction residual is concerned. Differences lie in the fact that encoder14chooses, by optimization, some coding options or coding parameters and signals within, or inserts into, datastream12the coding parameters which are then derived from the datastream12by decoder54so as to redo the prediction, subdivision and so forth.

FIG.4shows a possible implementation of the decoder54ofFIG.3, namely one fitting to the implementation of encoder14ofFIG.1as shown inFIG.2. As many elements of the encoder54ofFIG.4are the same as those occurring in the corresponding encoder ofFIG.2, the same reference signs, provided with an apostrophe, are used inFIG.4in order to indicate these elements. In particular, adder42′, optional in-loop filter46′ and predictor44′ are connected into a prediction loop in the same manner that they are in encoder ofFIG.2. The reconstructed, i.e. dequantized and retransformed prediction residual signal applied to added42′ is derived by a sequence of entropy decoder56which inverses the entropy encoding of entropy encoder28b, followed by the residual signal reconstruction stage36′ which is composed of dequantizer38′ and inverse transformer40′ just as it is the case on encoding side. The decoder's output is the reconstruction of picture10. The reconstruction of picture10may be available directly at the output of adder42′ or, alternatively, at the output of in-loop filter46′. Some post-filter may be arranged at the decoder's output in order to subject the reconstruction of picture10to some post-filtering in order to improve the picture quality, but this option is not depicted inFIG.4.

Again, with respect toFIG.4the description brought forward above with respect toFIG.2shall be valid forFIG.4as well with the exception that merely the encoder performs the optimization tasks and the associated decisions with respect to coding options. However, all the description with respect to block-subdivisioning, prediction, dequantization and retransforming is also valid for the decoder54ofFIG.4.

The embodiments described below make use of a so-called matrix-based intra-prediction. The general concept shall be outlined below. The concept is sometimes called ALWIP (Affine-linear weighted intra prediction) in the following, as an alternative synonym for MIP (Matrix-based Intra Prediction).

In ALWIP or MIP mode, for predicting the samples of a rectangular block of width W and height H, Affine-linear weighted intra prediction (ALWIP) (or MIP) may take one line of H reconstructed neighbouring boundary samples left of the block and one line of W reconstructed neighbouring boundary samples above the block as input. If the reconstructed samples are unavailable, they may be generated as it is done in the conventional intra prediction. TheFIGS.5.1to5.4show matrix-based intra-prediction of samples104of a predetermined block18.

A generation of the prediction signal (e.g., the values for the complete block18) may be based on at least some of the following three steps:1. Out of the boundary samples17, samples102(e.g., four samples in the case of W=H=4 and/or eight samples in other case) may be extracted by averaging or downsampling (e.g., step811). It is possible to reduce, at step811, the number of samples17aand17cneighboring the block18. As shown, e.g. at100inFIG.5.1, after having subdivided the row17cand the column17ain groups110of two samples each, one single sample might be maintained per group110(e.g., the average of the samples of the group110or a simple choice among the samples of the group110). Or as shown, e.g. at122inFIG.5.4, the boundary samples may be grouped into groups120of four consecutive samples, wherein in this case also only one sample might be maintained per group120(e.g., selected among the four samples, or the average of the four samples).2. A matrix vector multiplication, followed by addition of an offset, may be carried out with the averaged samples (or the samples remaining from downsampling) as an input. The result may be a reduced prediction signal on a subsampled set of samples in the original block. (e.g., step812)3. The prediction signal at the remaining position may be generated, e.g. by upsampling, from the prediction signal on the subsampled set, e.g., by linear interpolation (e.g., step813).

Thanks to steps1. (811) and/or3. (813), the total number of multiplications needed in the computation of the matrix-vector product may be such that it is always smaller or equal than 4*W*H. Moreover, the averaging operations on the boundary and the linear interpolation of the reduced prediction signal are carried out by solely using additions and bit-shifts. In other words, for example, at most four multiplications per sample are needed for the ALWIP modes.

In some examples, the matrices (e.g.,17M) and offset vectors (e.g., bk) needed to generate the prediction signal may be taken from sets (e.g., three sets), e.g., S0, S1, S2, of matrices which may be stored, for example, in storage unit(s) of the decoder and of the encoder.

In some examples the set S0may comprise (e.g., consist of) n0(e.g., n0=16 or n0=18 or another number) matrices A0i, i∈{0, . . . , n0−1} each of which may have 16 rows and 4 columns and 18 offset vectors b0i, i∈{0, . . . , n0−1} each of size16to perform the technique according toFIG.5.1. Matrices and offset vectors of this set are used for blocks18of size 4×4. Once the boundary vector has been reduced to a Pred=4 vector (as for step811ofFIG.5.1), it is possible to map the Pred=4 samples of the reduced set of samples102directly into the Q=16 samples of the 4×4 block18to be predicted.

In some examples, the set S1may comprise (e.g., consist of) n1(e.g., n1=8 or n1=18 or another number) matrices A1i, i∈{0, . . . , n1−1}, each of which may have 16 rows and 8 columns and18offset vectors b1i, i∈{0, . . . , n1−1} each of size16to perform the technique according toFIG.5.2or5.3. Matrices and offset vectors of this set S1may be used for blocks of sizes 4×8, 4×16, 4×32, 4×64, 16×4, 32×4, 64×4, 8×4 and 8×8. Additionally, it may also be used for blocks of size W×H with max(W,H)=4 and min(W,H)=4, i.e. for blocks of size 4×16 or 16×4, 4×32 or 32×4 and 4×64 or 64×4. The 16×8 matrix refers to the reduced version of the block18, which is a 4×4 block, as obtained inFIGS.5.2and5.3.

Additionally or alternatively, the set S2may comprise (e.g., consists of) n2(e.g., n2=6 or n2=18 or another number) matrices A2i, i∈{0, . . . , n2−1}, each of which may have 64 rows and 8 columns and of 18 offset vectors b2i, i∈{0, . . . ,n2−1} of size 64. The 64×8 matrix refers to the reduced version of the block18, which is an 8×8 block, e.g. as obtained inFIG.5.4. Matrices and offset vectors of this set may be used for blocks of sizes 8×16, 8×32, 8×64, 16×8, 16×16, 16×32, 16×64, 32×8, 32×16, 32×32, 32×64, 64×8, 64×16, 64×32, 64×64.

Matrices and offset vectors of that set or parts of these matrices and offset vectors may be used for all other block-shapes.

1.1 Averaging or Downsampling of the Boundary

Here, features are provided regarding step811.

The boundary samples (17a,17c) may be averaged and/or downsampled (e.g., from P samples to Pred<P samples).

In a first step, the input boundaries bdrytop(e.g.,17c) and bdryleft(e.g.,17a) may be reduced to smaller boundaries bdryredtopand bdryredleftto arrive at the reduced set102. Here, bdryredtopand bdryredleftboth consists of 2 samples in the case of a 4×4-block and both consist of 4 samples in other cases.

In the case of a 4×4-block, it is possible to define

bdryredtop[0]=(bdrytop[0]+bdrytop[1]+1)>>1,bdryredtop[1]=(bdrytop[2]+bdrytop[3]+1)>>1,
and define bdryredleftanalogously. Accordingly, bdryredtop[0], bdryredtop[1], bdryredleft[0] bdryredleft[1] are average values obtained e.g. using bit-shifting operations.

In all other cases (e.g., for blocks of wither width or height different from 4), if the block-width W is given as W=4*2k, for 0≤i<4 one defines

bdryredtop[i]=((∑j=02k-1⁢bdrytop[i*2k+j])+1⁢<<(k-1))>>k.
and defines bdryredleftanalogously.

In still other cases, it is possible to downsample the boundary (e.g., by selecting one particular boundary sample from a group of boundary samples) to arrive at a reduce number of samples. For example, bdryredtop[0] may be chosen among bdrytop[0] and bdrytop[1], and bdryredtop[1] may be chosen among bdrytop[2] and bdrytop[3]. It is also possible to define bdryredleftanalogously.

The two reduced boundaries bdryredtopand bdryredlefta may be concatenated to a reduced boundary vector bdryred(associated to the reduced set102), also indicated with17P. The reduced boundary vector bdryredmay be thus of size four (Pred=4) for blocks of shape 4×4 (example ofFIG.5.1) and of size eight (Pred=8) for blocks of all other shapes (examples ofFIG.5.2-5.4).

Here, if mode <18 (or the number of matrixes in the set of matrixes), it is possible to define
bdryred=[bdryredtop,bdryredleft]

If mode ≥18, which corresponds to the transposed mode of mode −17, it is possible to define
bdryred=[bdryredleft,bdryredtop]

Hence, according to a particular state (one state: mode <18; one other state: mode ≥18) it is possible to distribute the predicted values of the output vector along a different scan order (e.g., one scan order: [bdryredtop, bdryredleft]; one other scan order: [bdryredleft, bdryredtop]).

Other strategies may be carried out. In other examples, the mode index ‘mode’ is not necessarily in the range 0 to 35 (other ranges may be defined). Further, it is not necessary that each of the three sets S0, S1, S2has 18 matrices (hence, instead of expressions like mode ≥18, it is possible to mode ≥n0, n1, n2, which are the number of matrixes for each set of matrixes S0, S1, S2, respectively). Further, the sets may have different numbers of matrixes each (for example, it may be that S0has 16 matrixes S1has eight matrixes, and S2has six matrixes).

The mode and transposed information are not necessarily stored and/or transmitted as one combined mode index ‘mode’: in some examples there is the possibility of signalling explicitly as a transposed flag and the matrix index (0-15 for S0, 0-7 for S1and 0-5 for S2).

In some cases, the combination of the transposed flag and matrix index may be interpreted as a set index or mode index. For example, there may be one bit operating as transposed flag, and some bits indicating the matrix index, collectively indicated as “set index” or “mode index”.

1.2 Generation of the Reduced Prediction Signal by Matrix Vector Multiplication

Here, features are provided regarding step812.

Out of the reduced input vector bdryred(boundary vector17P) one may generate a reduced prediction signal predred. The latter signal may be a signal on the downsampled block of with Wredand height Hred. Here, Wredand Hredmay be defined as:

Wr⁢e⁢d=4,Hr⁢e⁢d=4;if⁢max⁢(W,H)≤8,Wr⁢e⁢d=min⁢(W,8),Hr⁢e⁢d=min⁢(H,8);else.

The reduced prediction signal predredmay be computed by calculating a matrix vector-product and adding an offset:

p⁢r⁢e⁢dr⁢e⁢d=A·bdryr⁢e⁢d+b.

Here, A is a matrix (e.g., prediction matrix17M) that may have Wred*Hredrows and 4 columns if W=H=4 and 8 columns in all other cases and b is a vector that may be of size Wred*Hred.

If W=H=4, then A may have 4 columns and 16 rows and thus 4 multiplications per sample may be needed in that case to compute predred. In all other cases, A may have 8 columns and one may verify that in these cases one has 8*Wred*Hred≤4*W*H, i.e. also in these cases, at most 4 multiplications per sample are needed to compute predred.

The matrix A and the vector b may be taken from one of the sets S0, S1, S2as follows. One defines an index idx=idx(W,H) by setting idx(W,H)=0, if W=H=4, idx(W,H)=1, if max(W,H)=8 and idx(W,H)=2 in all other cases. Moreover, one may put m=mode, if mode <18 and m=mode−17, else. Then, if idx≤1 or idx=2 and min(W,H)=4, one may put A=Aidxmand b=bidxm. In the case that idx=2 and min(W,H)=4, one lets A be the matrix that arises by leaving out every row of Aidxm, that, in the case W=4, corresponds to an odd x-coordinate in the downsampled block, or, in the case H=4, corresponds to an odd y-coordinate in the downsampled block. If mode ≥18, one replaces the reduced prediction signal by its transposed signal. In alternative examples, different strategies may be carried out. For example, instead of reducing the size of a larger matrix (“leave out”), a smaller matrix of S1(idx=1) with Wred=4 and Hred=4 is used. I.e., such blocks are now assigned to S1instead of S2.

Other strategies may be carried out. In other examples, the mode index ‘mode’ is not necessarily in the range 0 to 35 (other ranges may be defined). Further, it is not necessary that each of the three sets S0, S1, S2has 18 matrices (hence, instead of expressions like mode <18, it is possible to mode <n0, n1, n2, which are the number of matrixes for each set of matrixes S0, S1, S2, respectively). Further, the sets may have different numbers of matrixes each (for example, it may be that S0has 16 matrixes S1has eight matrixes, and S2has six matrixes).

1.3 Linear Interpolation to Generate the Final Prediction Signal

Here, features are provided regarding step812.

Interpolation of the subsampled prediction signal, on large blocks a second version of the averaged boundary may be needed. Namely, if min(W,H)=8 and W=H, one writes W=8*2l, and for 0≤i<8 defines

bdryred⁢IItop[i]=((∑j=02l-1⁢bdrytop[i*2l+j])+1⁢<<(l-1))>>l.

If min(W,H)=8 and H=W, one defines bdryred analogously.

In addition or alternative, it is possible to have a “hard downsampling”, in which the bdryredlltop[i] is equal to

bdryred⁢IItop[i]=bdrytop[(i+1)*2l-1].

Also, bdryredllleftcan be defined analogously.

At the sample positions that were left out in the generation of predred(e.g., step813in examples ofFIGS.5.2-5.4), the final prediction signal may arise by linear interpolation from predred. This linear interpolation may be unnecessary, in some examples if W=H=4 (e.g., example ofFIG.5.1).

The linear interpolation may be given as follows (other examples are notwithstanding possible). It is assumed that W=H. Then, if H=Hred, a vertical upsampling of predred may be performed. In that case, predredmay be extended by one line to the top as follows. If W=8, predredmay have width Wred=4 and may be extended to the top by the averaged boundary signal bdryredtop, e.g. as defined above. If W=8, predredis of width Wred=8 and it is extended to the top by the averaged boundary signal bdryredu, e.g. as defined above. One may write predred[x][−1] for the first line of predred. Then the signal predredups,veron a block of width Wredand height 2*Hredmay be given as

predredups,ver[x][2*y+1]=predred[x][y],predredups,ver[x][2*y]=(predred[x][y-1]+predred[x][y]+1)>>1,
where 0≤x<Wredand 0≤y<Hred. The latter process may be carried out k times until 2k*Hred=H. Thus, if H=8 or H=16, it may be carried out at most once. If H=32, it may be carried out twice. If H=64, it may be carried out three times. Next, a horizontal upsampling operation may be applied to the result of the vertical upsampling. The latter upsampling operation may use the full boundary left of the prediction signal. Finally, if H=W, one may proceed analogously by first upsampling in the horizontal direction (if required) and then in the vertical direction.

This is an example of an interpolation using reduced boundary samples for the first interpolation (horizontally or vertically) and original boundary samples for the second interpolation (vertically or horizontally). Depending on the block size, only the second or no interpolation is required. If both horizontal and vertical interpolation is required, the order depends on the width and height of the block.

However, different techniques may be implemented: for example, original boundary samples may be used for both the first and the second interpolation and the order may be fixed, e.g. first horizontal then vertical (in other cases, first vertical then horizontal).

Hence, the interpolation order (horizontal/vertical) and the use of reduced/original boundary samples may be varied.

1.4 Illustration of an Example of the Entire ALWIP Process

The entire process of averaging, matrix-vector-multiplication and linear interpolation is illustrated for different shapes inFIGS.5.1-5.4. Note, that the remaining shapes are treated as in one of the depicted cases.1. Given a 4×4 block, ALWIP may take two averages along each axis of the boundary by using the technique ofFIG.5.1. The resulting four input samples enter the matrix-vector-multiplication 19. The matrices are taken from the set S0. After adding an offset, this may yield the 16 final prediction samples. Linear interpolation is not necessary for generating the prediction signal. Thus, a total of (4*16)/(4*4)=4 multiplications per sample are performed. See, for example,FIG.5.1.2. Given an 8×8 block, ALWIP may take four averages along each axis of the boundary. The resulting eight input samples enter the matrix-vector-multiplication 19, by using the technique ofFIG.5.2. The matrices are taken from the set S1. This yields 16 samples on the odd positions of the prediction block. Thus, a total of (8*16)/(8*8)=2 multiplications per sample are performed. After adding an offset, these samples may be interpolated, e.g., vertically by using the top boundary and, e.g., horizontally by using the left boundary. See, for example,FIG.5.2.3. Given an 8×4 block, ALWIP may take four averages along the horizontal axis of the boundary and the four original boundary values on the left boundary by using the technique ofFIG.5.3. The resulting eight input samples enter the matrix-vector-multiplication 19. The matrices are taken from the set S1. This yields 16 samples on the odd horizontal and each vertical positions of the prediction block. Thus, a total of (8*16)/(8*4)=4 multiplications per sample are performed. After adding an offset, these samples are interpolated horizontally by using the left boundary, for example. See, for example,FIG.5.3.The transposed case is treated accordingly.4. Given a 16×16 block, ALWIP may take four averages along each axis of the boundary. The resulting eight input samples enter the matrix-vector-multiplication 19 by using the technique ofFIG.5.4. The matrices are taken from the set S2. This yields 64 samples on the odd positions of the prediction block. Thus, a total of (8*64)/(16*16)=2 multiplications per sample are performed. After adding an offset, these samples are interpolated vertically by using the top boundary and horizontally by using the left boundary, for example. See, for example,FIG.5.4.For larger shapes, the procedure may be essentially the same and it is easy to check that the number of multiplications per sample is less than two.For W×8 blocks, only horizontal interpolation is necessary as the samples are given at the odd horizontal and each vertical positions. Thus, at most (8*64)/(16*8)=4 multiplications per sample are performed in these cases.Finally for W×4 blocks with W>8, let Axbe the matrix that arises by leaving out every row that correspond to an odd entry along the horizontal axis of the downsampled block. Thus, the output size may be 32 and again, only horizontal interpolation remains to be performed. At most (8*32)/(16*4)=4 multiplications per sample may be performed.The transposed cases may be treated accordingly.

According to the embodiments proposed below, the MIP modes are applied in a manner which renders the usage of MIP even more efficient than compared to the usage so far anticipated in the current VVC version. The embodiments relate to three different aspects which may be applied separately from each other or may be combined pairwise or altogether. Compared to the current VVC implementation, it is first proposed that all MIP modes, i.e. also the 0thMIP mode, can be used in a transposed way where the parity of the mode determines the transposed condition. In doing so, the number of MIP modes for MipSizeId=0 may be reduced from 35 to 32 and to reduce the number of MIP modes for MipSizeId=1 from 19 to 16. The MIP-mode may then be coded with a fixed-length code for small blocks. Second, it is proposed to remove the restriction that MIP may not be used on blocks whose aspect ratio is greater or equal to four. Additionally, it is proposed to introduce a separate context for the MTS-index depending on the MIP flag.

It is reported that the proposed method yields −0.06% BD-rate change over the VTM-6.0 in the AI configuration and 0.04% BD-rate savings over the VTM-6.0 in the RA configuration if the proposed context is not added. Here, the encoder runtime is reported to be 101% for the AI configuration and to be 100% for the RA configuration. The decoder runtime is reported to be 100% for the AI configuration and to be 99% for the RA configuration. If furthermore the proposed context is added, it is reported that the proposed method yields −0.09% BD-rate change over the VTM-6.0 in the AI configuration and −0.05% BD-rate change over the VTM-6.0 in the RA configuration. Here, the encoder runtime is reported to be 100% for the AI configuration and to be 100% for the RA configuration. The decoder runtime is reported to be 101% for the AI configuration and to be 99% for the RA configuration.

It should be noted that the following description of theFIGS.6to9primarily describes features usable in a picture decoder or video decoder (i.e., an apparatus for decoding). However, it is clear, that any of the features described herein can also be used in the context of a picture encoder or video encoder.

FIG.6shows an apparatus54for decoding a predetermined block18of a picture using intra-prediction.

The apparatus54is configured to read, from a data stream12, a mode index200using a binarization code202, the mode index pointing to one out of a list204of matrix-based intra-prediction modes. The list204of matrix-based intra-prediction modes consists of an even number of matrix-based intra-prediction modes, wherein the matrix-based intra-prediction modes of the list204are grouped into pairs212of matrix-based intra-prediction modes. Each pair212consists of a first matrix-based intra-prediction mode and a second matrix-based intra-prediction mode. The apparatus54is configured to read, from the data stream12, the mode index200using the binarization code202in a manner so that for each pair212of matrix-based intra-prediction modes the first matrix-based intra-prediction mode is assigned a first codeword and the second matrix-based intra-prediction mode is assigned a second codeword and both codewords are equal in length.

Optionally, the binarization code202is a variable length code, the variable length code comprises codewords of different lengths. Alternatively, the binarization code may be a truncated binary code and the number of matrix-based intra-prediction modes is not a power of two, so that the truncated binary code has codewords of different lengths. A matrix-based intra-prediction mode associated with a first pair212of matrix-based intra-prediction modes may be assigned a codeword different in length as a codeword assigned to matrix-based intra-prediction mode associated with a second pair212of matrix-based intra-prediction modes. However, both codewords of a pair212of matrix-based intra-prediction modes are equal in length.

According to an embodiment, the apparatus54may be configured to read the mode index200from the data stream12using an equi-probability bypass mode of a context adaptive binary arithmetic decoder.

Similarly, to the apparatus54(i.e. a decoder) for decoding the predetermined block18of the picture using intra-prediction, an apparatus (i.e. an encoder) for encoding the predetermined block18of the picture using intra-prediction can be configured to encode the mode index200into the data stream12using the binarization code202and optionally using the equi-probability bypass mode of a context adaptive binary arithmetic encoder.

The decoder and the encoder are configured to predict samples108of the predetermined block18by computing a matrix-vector product206between an input vector102derived from reference samples17in a neighbourhood of the predetermined block18and a prediction matrix19associated with the matrix-based intra-prediction mode k pointed to by the mode index200. The computation of the matrix-vector product206results in an output vector208. Furthermore, the samples108of the predetermined block18are predicted by associating components210of the output vector208obtained by the matrix-vector product206onto sample positions104of the predetermined block18. This prediction of the samples108of the predetermined block18may be performed as described with regard toFIGS.5.1to5.4.

For each pair212of matrix-based intra-prediction modes, the prediction matrix19associated with a first matrix-based intra-prediction mode of the respective pair212of matrix-based intra-prediction modes is equal to the prediction matrix19associated with a second matrix-based intra-prediction mode of the respective pair212of matrix-based intra-prediction modes. Thus, for matrix-based intra-prediction modes 2k and 2k+1, the same prediction matrix19is used. For each pair212of matrix-based intra-prediction modes, the encoder and the decoder are configured so that, if the matrix-based intra-prediction mode pointed to by the mode index200is the first matrix-based intra-prediction mode of the respective pair212of matrix-based intra-prediction modes, e.g. a mode with odd mode index 2k+1, an association of the reference samples17in the neighbourhood of the predetermined block18with components214of the input vector112and of the sample positions104of the predetermined block18with the components210of the output vector208is transposed relative to the association in case of the matrix-based intra-prediction mode pointed to by the mode index200being the second matrix-based intra-prediction mode of the respective pair212of matrix-based intra-prediction modes, e.g. a mode with even mode index 2k.

The decoder/encoder might be configured to determine whether the matrix-based intra-prediction mode pointed to by the mode index200is the first matrix-based intra-prediction mode of the respective pair of matrix-based intra-prediction modes or the second matrix-based intra-prediction mode of the respective pair212of matrix-based intra-prediction modes, based on the parity of the mode index200. The parity of the mode index200might indicate whether the input vector102and the output vector208are used in a transposed way or not for the prediction of the samples108of the predetermined block18. That is, as shown inFIG.7, if a certain component of the components 1 to n of the input vector102is associated with position (x,y) with (0,0) denoting the upper left corner sample AA of the predetermined block18in the former case, then it is associated with (y,x) in the latter case. The same applies to the components (AA, AB, AC, BA, CA, . . . ) of the output vector208.

Each pair212consists of a first matrix-based intra-prediction mode and a second matrix-based intra-prediction mode, which modes are related to each other by the same prediction matrix19and only differ among each other in terms of the input vector102and the output vector208being transposed or not. According to an embodiment, a last significant bit, i.e. a last bin, of the binarization code202or, as an alternative, a most significant bit, i.e. a first bin, of the binarization code202—such as a truncated binary code or fixed length binary code—of the index200, might indicate whether the MIP mode is to be used in a transposed way or not. Alternatively, or seen from a different perspective, the mode index is composed of a syntax element which represents a sub-index onto a pair of corresponding, mutually transposed MIP modes, which might also be binarized using a truncated binary code, with a fixed number of bits and an additional syntax element which represents a transposition flag, which additional syntax element indicates whether transposition is to be applied or not. In other words, the index200would be composed of a first syntax element indicting a MIP mode pair and an additional flag indicating whether the matrix underlying the MIP mode pair is to be applied in a transposed way or not. The additional bit or flag might be decoded/encoded from/into the data stream12before or after the sub-index syntax element is decoded/encoded from/into the data stream12.

According to an embodiment, the decoder/encoder is configured to index the prediction matrix19out of a plurality of prediction matrices using the integer part of the mode index200divided by 2. This is based on the idea, that both matrix-based intra-prediction modes of a pair212use the same prediction matrix19for the prediction of the samples108of the predetermined block18, for which reason the prediction matrix19is already sufficiently indicated by pointing with the mode index200to the relevant pair212in the list204.

As shown inFIGS.6and7, the decoder/encoder might be configured to set217an inter-sample distance216of the sample positions104of the predetermined block18and an inter-sample distance218of the reference samples17in the neighbourhood of the predetermined block18horizontally according to a first ratio of a horizontal dimension220of the predetermined block18relative to a horizontal default dimension and/or vertically according to a second ratio of a vertical dimension222of the predetermined block18relative to a vertical default dimension. This enables the usage of the list204of matrix-based intra-prediction modes for a plurality of block dimensions. The apparatus might fill spaces between the predicted samples by interpolation. The inter-sample distance setting217of the inter-sample distance216of the sample positions104of the predetermined block18and of the inter-sample distance218of the reference samples17in the neighbourhood of the predetermined block18enables an improved distribution of the predicted samples108in the predetermined block18and of the reference samples17in the neighbourhood of the predetermined block18. Thus, the predicted samples might be equally distributed enabling an improved interpolation of samples of the predetermined block18.

According to an embodiment, the decoder/encoder is configured to order the matrix-based intra-prediction modes in the list204of matrix-based intra-prediction modes equally for the plurality of block dimensions. Alternatively, the order might be adapted to, for instance, the block being wider than high or vice versa, i.e. higher than wide, or quadratic. This ordering may increase the coding efficiency and reduce the bitstream, since matrix-based intra-prediction modes for common block dimensions may be associated with short codewords and matrix-based intra-prediction modes for rare block dimensions may be associated with longer codewords.

Optionally, the plurality of block dimensions includes at least one block dimension corresponding to an aspect-ratio of larger than 4. The matrix-based intra-prediction might be optimized such that the predetermined block18with an aspect-ratio of the horizontal dimension220to the vertical dimension222is larger than 4. That is, the plurality of block dimensions includes a predetermined block with an at least four times larger horizontal dimension220than the vertical dimension222and/or a predetermined block with an at least four times larger vertical dimension222than the horizontal dimension220.FIG.7might show a predetermined block18with a block dimension corresponding to an aspect-ratio of larger than 4.

In the current VVC working draft, for each block18on which MIP may be used, the number of MIP modes is an odd integer N=2n+1. Here, the 0-th mode may not be transposed while all other modes may be transposed: For i=n, MIP mode i is the transposed of MIP mode i-n. It is asserted that the restriction that the 0-th mode may not be transposed had originally been made due to the fact that in the first adopted version of MIP from the 15-th JVET meeting in Geneva, the MIP modes were signalled by an MPM (most-probable mode) scheme with 3 MPMs, [1]. In this scheme, the total number of MIP modes was an odd integer of the form N=2k+3, where k is an integer. However, at the 16-th JVET meeting in Gothenburg, the latter signalization scheme for the MIP modes was replaced by a truncated binary code, [2]. It is asserted that as a consequence, the number of MIP modes does not need to be odd anymore and that thus the special treatment of the 0-th MIP mode regarding transposed can be removed.

Thus, it is proposed that in Clause 8.4.5.2.1 of the current VVC-draft, for an MIP-mode predModeIntra, the transposed flag isTransposed which determines whether the mode is transposed is defined as
transposed_flag=predModeIntra&1.

Consequently, it is proposed that in Clause 8.4.5.2.1, the index modeld200which determines which matrix19is to be used for the given MIP mode predModeIntra is defined as
modeld=predModelntra>>1.

By this manner, signalling a mode and its transposed mode is equally expensive in terms of VLC length. No asymmetry results. No MPM list construction is necessary either.

Furthermore, is it proposed to reduce the number of MIP modes from 35 to 32 for MipSizeId=0 and from 19 to 16 for MipSizeId=1. For MipSizeId=2, it is proposed to use 12 instead of 11 MIP modes, since it is proposed that also the 0-th MIP mode can be transposed. Consequently, in the proposed method, 16 instead of 18 MIP matrices 19 are needed for MipSizeId=0 and 8 instead of 10 MIP matrices 19 are needed for MipSizeId=1. For MipSizeId=2, as before, 6 MIP matrices 19 are needed.

Next, in the current VVC working draft, MIP is excluded for blocks18whose aspect ratio is greater or equal to four. It is asserted that this restriction is unnecessary since the MIP of the current VVC works for these blocks18without any additional design changes. It is asserted that the restriction for MIP regarding the block shape was mainly made to save encoder runtime and that it is thus undesirable to be normative. Thus, it is proposed to remove the latter restriction on the usage of MIP and to allow MIP for all block shapes, as shown inFIG.8.

FIG.8shows an apparatus54for decoding a predetermined block of a picture using intra-prediction, configured to predict each of predetermined intra-predicted blocks18of the picture by reading, from a data stream12, a mode index200, the mode index200pointing to one out of a list204of matrix-based intra-prediction modes. The apparatus54is configured to predict samples108of the respective predetermined intra-predicted block18by computing a matrix-vector product206between an input vector102derived from reference samples17in a neighbourhood of the respective predetermined intra-predicted block18and a prediction matrix19associated with the matrix-based intra-prediction mode (k) pointed to by the mode index200and by associating components of an output vector208obtained by the matrix-vector product206onto sample positions104of the respective predetermined intra-predicted block18. The predetermined intra-predicted blocks18comprise blocks an aspect-ratio of which is larger than 4.

Furthermore, the apparatus54might be configured to set an inter-sample distance216of the sample positions104of the respective predetermined intra-predicted block and an inter-sample distance218of the reference samples17in the neighbourhood of the respective predetermined intra-predicted block18horizontally according to a first ratio of a horizontal dimension220of the predetermined block18relative to a horizontal default dimension and/or vertically according to a second ratio of a vertical dimension222of the predetermined block relative to a vertical default dimension. Optionally, the apparatus54is configured to use the list of matrix-based intra-prediction modes for a plurality of block dimensions. According to an embodiment, the apparatus might be configured to order the matrix-based intra-prediction modes in the list204of matrix-based intra-prediction modes equally for the plurality of block dimensions or adapt the order to, for instance, the block being wider than high or vice versa or quadratic.

An apparatus (encoder) for encoding a predetermined block of a picture using intra-prediction can have the same or similar features as the apparatus54(decoder) for decoding a predetermined block of a picture using intra-prediction, wherein the encoder is configured to encode, into the data stream12, the mode index200.

According to an embodiment, the mode index200is read/encoded as described with regard toFIG.6.

Optionally, the decoder/encoder is configured to perform the prediction of the samples108of the respective predetermined intra-predicted block18as described with regard to one of theFIGS.5.1to7.

According to an embodiment, shown inFIG.9, it is proposed to use a separate context for the MTS-flag, i.e. a transformation flag300, if the corresponding coding unit, e.g. a predetermined intra-predicted block18, is using MIP. The MTS flag300may indicate whether the residual transformation for that block18should be the default one, such as a DCT-II applied horizontally and vertically, or if not in which case subsequent syntax elements may optionally signal whether, for instance, a DST-VII or DCT-VIII is applied instead, vertically and/or horizontally. It is asserted that there is a statistical difference between the usage of MTS for MIP and for non-MIP blocks which can thus be exploited by separating the corresponding context. In particular, as MIP modes have to have been learned with a particular residual transform, here the default one, other transforms are more unlikely to yield better results than compared to heuristically designed modes such as the DC, planar and angular modes.

FIG.9shows an apparatus54for decoding a predetermined block18of a picture using intra-prediction.

The apparatus54is configured to predict310each of predetermined intra-predicted blocks18, e.g. the MIP blocks, of the picture by reading, from a data stream12, a mode index200, the mode index200pointing to one out of a list204of matrix-based intra-prediction modes, and by predicting samples108of the respective predetermined intra-predicted block18by computing a matrix-vector product206between an input vector102derived from reference samples17in a neighbourhood of the respective predetermined intra-predicted block18and a prediction matrix19associated with the matrix-based intra-prediction mode (k) pointed to by the mode index200and associating components of an output vector208obtained by the matrix-vector product206onto sample positions104of the respective predetermined intra-predicted block18. This intra-prediction310of the predetermined intra-predicted blocks18may be performed as described with regard to one ofFIGS.5.1to8. The apparatus54is configured to obtain a prediction signal340by this prediction310.

Furthermore, the apparatus54is configured to predict310each of further predetermined intra-predicted blocks18, e.g. “normal” intra blocks, of the picture using one out of a set of normal intra-prediction modes comprising an angular prediction mode, a DC prediction mode and a planar prediction mode to obtain a prediction signal340.

For a predetermined block18out of the predetermined intra-predicted blocks and the further predetermined intra-predicted blocks, the apparatus54is configured to decode a transformation flag300, e.g. MTS flag, from the data stream12. This transformation flag300may be coded for both predetermined intra-predicted blocks and further predetermined intra-predicted blocks, i.e. MIP blocks and normal intra blocks. The transformation flag300indicates whether a prediction residual for the predetermined block18is coded into the data stream12using a default transformation320or a different transformation322. The default transformation320might be a DCT-II applied horizontally and vertically and the different transformation322might be a DST-VII or a DCT-DCT-VIII horizontally and a DST-VII or DCT-DCT-VIII vertically.

The apparatus54is configured to decode the transformation flag300from the data stream12using context adaptive binary arithmetic coding350using a probability model selected depending on whether the predetermined block is one of the predetermined intra-predicted blocks or one of further predetermined intra-predicted blocks. Optionally, the apparatus54is configured to use as probability model for decoding the transformation flag300a first probability model and update the first probability model depending on the transformation flag300if the predetermined block18is one of the predetermined intra-predicted blocks and use as probability model for decoding the transformation flag a second probability model and update the second probability model depending on the transformation flag300if the predetermined block18is one of the further predetermined intra-predicted blocks.

According to an embodiment, the apparatus54is configured to, if the transformation flag300indicates that the prediction residual302for the predetermined block18is coded into the data stream12using the different transformation322, decode one or more further syntax elements from the data stream12indicating the different transformation. The one or more further syntax elements may indicate the different transformation322out of a set of separable transforms using a first one of a DST-VII and a DCT-DCT-VIII horizontally and a second one of DST-VII and DCT-DCT-VIII vertically.

The apparatus54is configured to decode the prediction residual302for the predetermined block18from the data stream12and re-transform the prediction residual302using a reverse transformation which reverses the default transformation320if the transformation flag300indicates that the prediction residual302for the predetermined block18is coded into the data stream12using the default transformation320, and which reverses the different transformation322if the transformation flag300indicates that the prediction residual302for the predetermined block18is coded into the data stream12using the different transformation322, so as to obtain a prediction residual signal330. The apparatus54is configured to correct the prediction signal340using the prediction residual signal330.

An apparatus (encoder) for encoding a predetermined block of a picture using intra-prediction can have the same or similar features as the apparatus54(decoder) for decoding a predetermined block of a picture using intra-prediction, wherein the encoder is configured to perform the re-transform in a prediction loop.

According to an embodiment, the encoder is configured to encode the prediction residual302for the predetermined block18into the data stream12by transforming the prediction residual signal330using the default transformation320if the transformation flag300indicates that the prediction residual302for the predetermined block18is to be coded into the data stream12using the default transformation320, and using the different transformation322if the transformation flag300indicates that the prediction residual302for the predetermined block18is to be coded into the data stream12using the different transformation322, so as to obtain a transformed prediction residual signal. Furthermore, the encoder is configured to encode the transformed prediction residual signal into the data stream12, so that the prediction signal340is correctable by a re-transform of the transformed prediction residual signal.

Additionally, the encoder is configured to encode the transformation flag300into the data stream12using context adaptive binary arithmetic coding350using the probability model selected depending on whether the predetermined block18is one of the predetermined intra-predicted blocks or one of further predetermined intra-predicted blocks.

The encoder and/or decoder may comprise features and/or functionalities as described with regard to one of theFIGS.5.1to8.

In the tables depicted below, experimental results according to the common test conditions [3] and the test conditions of the CE for intra, [4], are presented. In Table 1 and Table 2, results are reported for the proposed cleanup but without the addition of the extra context for the MTS flag. In Table 3 and Table 4, results are reported for the proposed cleanup with the proposed addition of the extra context for the MTS flag. No optimizations at the encoder were made in comparison to the VTM-6.0 anchor in any of the tests.

TABLE 1Result of the incorporated MIP changes: Reference is VTM-6.0anchor, test is VTM-6.0 with the proposed cleanup and withoutthe addition of the extra context, Al configurationYUVenc timedec timeClass A1−0.08%−0.03%−0.03%101%101%Class A2−0.06%−0.04%−0.06%101%101%Class B−0.07%−0.08%−0.12%100%101%Class C−0.03%−0.18%−0.01%101%97%Class E−0.09%0.08%0.02%100%99%Overall−0.06%−0.06%−0.05%101%100%Class D−0.02%−0.04%0.07%101%99%Class F−0.03%−0.05%−0.01%100%100%

TABLE 2Result of the incorporated MIP changes: Reference is VTM-6.0anchor, test is VTM-6.0 with the proposed cleanup and withoutthe addition of the extra context, RA configurationYUVenc timedec timeClass A1−0.03%−0.11%−0.12%100%100%Class A2−0.02%0.12%−0.07%101%97%Class B−0.05%0.03%−0.12%101%100%Class C−0.03%0.08%−0.01%100%102%Class EOverall−0.04%0.03%−0.08%100%99%Class D−0.04%−0.12%−0.12%101%102%Class F−0.01%−0.05%0.00%102%102%

TABLE 3Result of the incorporated MIP changes: Reference is VTM-6.0 anchor, test is VTM-6.0 with the proposed cleanup andwith the addition of the extra context, Al configurationYUVenc timedec timeClass A1−0.12%−0.16%−0.10%96%99%Class A2−0.08%−0.10%−0.08%100%102%Class B−0.08%−0.07%−0.12%103%102%Class C−0.04%−0.18%−0.07%100%99%Class E−0.13%−0.07%−0.05%101%105%Overall−0.09%−0.11%−0.08%100%101%Class D−0.02%0.05%−0.09%99%98%Class F−0.03%−0.10%−0.12%100%101%

TABLE 4Result of the incorporated MIP changes: Reference is VTM-6.0 anchor, test is VTM 6.0 with the proposed cleanup andwith the addition of the extra context, RA configurationYUVenc timedec timeClass A1−0.05%−0.05%−0.08%100%99%Class A2−0.03%−0.02%0.01%100%99%Class B−0.07%0.01%−0.04%101%102%Class C−0.03%0.07%−0.08%99%100%Class EOverall−0.05%0.01%−0.05%100%99%Class D−0.05%−0.11%0.00%101%100%Class F−0.06%0.06%−0.03%100%100%

In the present application, an alignment of the transposed usage of the MIP modes and of the number of MIP modes with the newly adopted signalling of the MIP mode is proposed. Moreover, a cleanup regarding an unnecessary restriction for the usage of MIP is proposed. Finally, a separate context for the MTS index depending on the MIP flag is proposed. Any or all of these changes may favourably adopted into the next working draft of the VVC.

Implementation Alternatives:

Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.

Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.

A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

The apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and/or in software.

The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

The methods described herein, or any components of the apparatus described herein, may be performed at least partially by hardware and/or by software.

The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

REFERENCES

[1] B. Bross et al., “Versatile Video Coding (Draft 5)”, JVET-N1001-v8, Geneva, Switzerland, March 2019[2] B. Bross et al., “Versatile Video Coding (Draft 6)”, JVET-O2001, Gothenburg, Sweden, July 2019[3] F. Bossen et al., “JVET common test conditions and software reference configurations for SDR video”, JVET-N1010, Geneva, Switzerland, March 2019[4] G. Van der Auwera et al., “Description of Core Experiment 3: Intra Prediction and Mode Coding”, JVET-02023, Gothenburg, Sweden, July 2019