Patent Description:
Contemporary perceptual (i.e., lossy) block transform image and video codecs (coders/decoders) can reach very good visual reconstruction quality even at relatively low bit-rates. At very low bitrates, however, artifacts such as blurring and discontinuities around the block boundaries, often referred to as "blocking", appear. To mitigate these typically annoying artifacts, deblocking postprocessing algorithms are utilized in modern codecs such as H. <NUM> / HEVC, H. <NUM> / WC, and AV1.

In video coding, a typical deblocking post-processor operates as an in-loop filter on each decoded image or frame, i.e., on each inter-picture prediction (also known as motion compensation) source image/frame before coding of the next image/frame in the encoding loop. The deblocking postfilter analyzes the boundary pixel values of each reconstructed sub-block of the decoded image in terms of potential discontinuities. If a weak discontinuity is found, it is assumed to be caused by the low-rate coding itself and not to be part of the original image and, thus, this discontinuity is reduced by way of smoothing of the pixel values (e. , the addition of adaptive pixel value offsets).

A similar in-loop filter is the Sample Adaptive Offset (SAO) method used in HEVC, which classifies the decoded pixels per (sub-)block based on their values and determines additive offsets for each pixel class. These additive offsets are then signaled to, and applied in, the decoder per (sub-)block. In doing so, the SAO filter act as a deringing filter.

Details on the HEVC deblocking filter are given in https://ieeexplore. org/document/<NUM><NPL>, <NPL>.

<CIT>) discloses a control of the deblocking filter strength by considering the block-size and whether non-zero residual coefficients exist in the block.

<CIT>) discloses to apply different filters for different block sizes such as CU, PU or/and TU sizes. Accordingly, the deblocking filtering strength is adjusted based on the block size, which implies that the amount of modification applied to pixels by the deblocking filter is varied depending on the block size.

<CIT>) discloses to set the boundary strength of two adjacent blocks to a first strength value if any block of the two adjacent blocks contains a non-zero prediction residual in the encoding data.

<NUM> / WC, the maximum transform block size has doubled compared to the size al-towed in HEVC, which was found to necessitate the use of stronger deblocking filters (i.e., deblocking postprocessors modifying a wider range of pixels) especially around the large-block boundaries. Such more aggressive deblocking filters, however, increase the risk of smoothing out - and, thus, potentially erasing - original image content which was not caused by the low-rate coding.

It is, therefore, concluded that very strong deblocking filtering is desirable for some low-rate coded high-resolution image and video content and that it is essential to allow for highly selective control of the application of said strong deblocking filtering. Naturally, a bit-flag may be signaled for each sub-block (e.g., each coding tree unit, CTU) to indicate to the receiver (i.e., decoder) whether to allow the application of the strong deblocking filter. This approach, however, would lead to many additional signaling bits being included in the bit-stream, thereby increasing the coding bit-rate to an unacceptable level especially at very low bit-rates.

Thus, a more efficient solution is required. Accordingly, it is an object of the present invention to improve existing artefact-filtering and to provide an efficient signaling of varying filtering strengths without the drawbacks mentioned above.

According to a first aspect of the invention, this problem is solved with a decoder having the features of claim <NUM>, an encoder having the features of claim <NUM>, a method for decoding according to claim <NUM>, a method for encoding according to claim <NUM>, a computer readable digital storage medium according to claim <NUM> and a data stream according to claim <NUM>.

Enabling disclosure for the protected invention is provided with the embodiments described in relation to the first aspect and especially <FIG>. The other figures, aspects, and embodiments are provided for illustrative purposes and do not represent embodiments of the invention unless when combined with all of the features respectively defined in the independent claims.

The inventive decoder of the first aspect is configured for block-based decoding of picture data using a deblocking or deringing filter. The decoder is configured to reconstruct, in a blockwise manner, a picture from a data stream using prediction and using a prediction residual coded in the data stream to obtain a reconstructed version of the picture. Predictive coding may, for instance, be executed by means of a spatial intra-picture prediction and/or by means of a temporal inter-picture prediction. Intra-picture prediction may be applied to still images and moving images, while inter-picture prediction may only be applied to moving images. For images with low visual activity, e.g. with few image details, the prediction typically works very efficiently. As a result, the corresponding prediction residuals may comprise very little signal energy and may therefore often be fully quantized to zero. In doing so, these zero-coded prediction residuals can be exempt from transmission. For images with higher visual activity, e.g. with more image details, the prediction may typically exhibit relatively high signal variance in its prediction residual, thus requiring a transmission of at least one (coarsely) quantized prediction residual which is not fully zero. This may also be referred to as non-zero coding of the respective prediction residual. Said non-zero coded prediction residuals may be candidates for causing visible blocking or deringing in the reconstructed (i.e. decoded) version of the picture. Thus, the decoder is configured to apply the deblocking or deringing filter to the reconstructed version of the picture. In this regard, the inventive decoder is furthermore configured to locally vary a strength of the deblocking or deringing filter. In other words, the decoder may control the amount of the deblocking or deringing that is applied to the decoded picture. This may result in an improved image quality over conventional deblocking or deringing filters without said control. Said strength of the deblocking or deringing filter may be quantitively measured. Strength measures may, for instance, be a width of a block's circumferential portions which are affected by the filter, or differently speaking a measure for a reach up to which the filter causes filtering from the block border of the blocks, wherein the larger the strength, the larger the width. Additionally or alternatively, a mean energy of a difference between the filtered version and the unfiltered version of the reconstructed picture to which the filter is applied may be used to measure the filter strength, wherein the larger the strength, the larger the mean energy. The inventive decoder may selectively decide whether to apply said filter control, i.e. whether to vary the filter strength or not, depending on a pre-selection of candidate pictures or candidate picture areas (e.g. blocks), respectively. Said candidate pictures or candidate picture areas (e.g. blocks) may be selected depending on a first measure locally measuring a mean block size, and a second measure locally measuring a frequency of non-zero coding of the prediction residual. The frequency of non-zero coding is meant to describe how often a non-zero coding of a prediction residual in the respective picture or picture area (e.g. block) was applied. In other words, depending on the number of non-zero coded prediction residuals and depending on the mean block size (e.g. a number of blocks or sub-blocks) the decoder may vary the filter strength of the deblocking or deringing filter.

The inventive encoder of the first aspect is configured for block-based encoding of picture data using a deblocking or deringing filter as an in-loop filter. The encoder is configured to encode, in a blockwise manner, a picture into a data stream using prediction and to encode a prediction residual into the data stream with providing a reconstructed version of the picture in a prediction loop of the encoder. The encoder is further configured to apply the deblocking or deringing filter onto the reconstructed version of the picture, and to locally vary a strength of the deblocking or deringing filter depending on a first measure locally measuring a mean block size, and a second measure locally measuring a frequency of non-zero coding of the prediction residual. In other words, the encoder may compute an optimum block-based partitioning of the picture in a rate-distortion loop. Based on this computation, the encoder may select a candidate picture or a candidate picture area (e.g. block), respectively, based on the mean block size and the number of non-zero coded prediction residuals. These selected candidate pictures or candidate picture areas (e.g. blocks) may then be subject to the varying filter strength. In other words, if a candidate picture or candidate picture area (e.g. block) was selected by the encoder, it may apply the deblocking or deringing filter with varying filter strength to said selected candidate picture or candidate picture area (e.g. block), i.e. the quantity of deblocking or deringing may be selectively controlled by the encoder and, thus, the quality of picture coding may be improved compared to conventional encoders.

According to a second aspect of the invention, a deblocking filter is suggested, wherein said deblocking filter is configured to filter a block of a picture in order to reduce blocking or ringing artefacts. Accordingly, the deblocking filter may also be referred to as a deringing filter. The deblocking filter according to the second aspect may be combined with the encoder and/or the decoder and/or the methods according to the first aspect. Alternatively, the deblocking filter according to the second aspect may be combined with encoders and/or decoders and/or methods being different from the first aspect.

The deblocking filter according to the second aspect may be configured to filter a block of a picture that is processed in a block-based manner. Said filtering may be exploited for reducing blocking or ringing artefacts which may appear upon block-based coding of the picture. The picture may be partitioned into several blocks and subblocks. The deblocking filter may be applied to one or more of said blocks and subblocks for reducing blocking or ringing artefacts upon coding of the picture. Each block may have a block border, which may correspond to the outer circumferential borderline of said block. The blocks may be square or generally rectangular, depending on the applied partitioning scheme. Accordingly, also the border of each block may be square or rectangular, respectively. The border may comprise several portions, for example, portions extending along an edge (also referred to as edge border portions) and portions extending around a corner (also referred to as corner border portions). If a picture is partitioned into a plurality of blocks, said blocks may be contiguously arranged, i.e. the blocks may abut each other. Accordingly, a first block may be surrounded by one or more other blocks. The content (e.g. pixels contained in a block) of neighboring blocks may differ from each other, for instance if there is a transition from a dark picture region into a light picture region. Accordingly, there may be a dissimilarity between a picture content contained inside a first block and a picture content contained outside said first block. Said picture content outside the first block may be contained inside a surrounding second block and may therefore also be referred to as a surrounding picture content. The dissimilarities may represent a difference between the picture content contained inside the first block and the surrounding picture content contained outside the first block. Said dissimilarities may also be referred to as an offset between the picture content contained inside the first block and the surrounding picture content contained outside the first block. The higher the dissimilarities the higher the magnitude of the offset value. The dissimilarities may cause blocking or ringing artefacts upon coding the picture. Thus, they have to be smoothened, which may also be referred to as deblocking or deringing, which may be performed by the inventive deblocking filter. Therefore, the inventive deblocking filter may be configured to determine, for each of at least eight border portions of a border of the block, a dissimilarity between an unfiltered content of the block and a surrounding picture content around the block across the respective border portion. Said at least eight border portions include four corner border portions, each arranged at a corner of the block, and four edge border portions, each arranged at intermediary portions of the border between the corners of the block. The deblocking filter may perform a deblocking filtering using different filter characteristics depending on the current picture content, i.e. depending on the aforementioned dissimilarities. Said different filter characteristics may be adjusted by means of adjustable parameters, which may depend on the current picture dissimilarities. Accordingly, the inventive deblocking filter may be configured to parametrize a deblocking filtering process of the block using the dissimilarities determined for the at least eight border portions in order to obtain a filtered content of the block.

In the following, embodiments of the present invention are described in more detail with reference to the figures, in which.

Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals.

Method steps which are depicted by means of a block diagram and which are described with reference to said block diagram may also be executed in an order different from the depicted and/or described order. Furthermore, method steps concerning a particular feature of a device may be replaceable with said feature of said device, and the other way around.

In this document, the first aspect of the invention will first be described with reference to <FIG>. Afterwards, the second aspect of the invention will be described subsequently with reference to <FIG>.

The following description of the figures starts with a presentation of a description of an encoder and a decoder of a block-based predictive codec for coding pictures of a video in order to form an example for a coding framework into which embodiments of the present invention may be built in. The respective encoder and decoder are described with respect to <FIG>. Thereinafter the description of embodiments of the concept of the present invention is presented along with a description as to how such concepts could be built into the encoder and decoder of <FIG> and <FIG>, respectively, although the embodiments described with the subsequent <FIG> and following, may also be used to form encoders and decoders not operating according to the coding framework underlying the encoder and decoder of <FIG> and <FIG>.

<FIG> shows an apparatus for predictively coding a picture <NUM> into a data stream <NUM> exemplarily using transform-based residual coding. The apparatus, or encoder, is indicated using reference sign <NUM>. <FIG> shows a corresponding decoder <NUM>, i.e. an apparatus <NUM> configured to predictively decode the picture <NUM>' from the data stream <NUM> also using transform-based residual decoding, wherein the apostrophe has been used to indicate that the picture <NUM>' as reconstructed by the decoder <NUM> deviates from picture <NUM> originally encoded by apparatus <NUM> in terms of coding loss introduced by a quantization of the prediction residual signal. <FIG> and <FIG> exemplarily use transform based prediction residual coding, although embodiments of the present application are not restricted to this kind of prediction residual coding. This is true for other details described with respect to <FIG> and <FIG>, too, as will be outlined hereinafter.

The encoder <NUM> is configured to subject the prediction residual signal to spatial-to-spectral transformation and to encode the prediction residual signal, thus obtained, into the data stream <NUM>. Likewise, the decoder <NUM> is configured to decode the prediction residual signal from the data stream <NUM> and subject the prediction residual signal thus obtained to spectral-to-spatial transformation.

Internally, the encoder <NUM> may comprise a prediction residual signal former <NUM> which generates a prediction residual <NUM> so as to measure a deviation of a prediction signal <NUM> from the original signal, i.e. from the picture <NUM>. The prediction residual signal former <NUM> may, for instance, be a subtractor which subtracts the prediction signal from the original signal, i.e. from the picture <NUM>. The encoder <NUM> then further comprises a transformer <NUM> which subjects the prediction residual signal <NUM> to a spatial-to-spectral transformation to obtain a spectral-domain prediction residual signal <NUM>' which is then subject to quantization by a quantizer <NUM>, also comprised by the encoder <NUM>. The thus quantized prediction residual signal <NUM>" is coded into bitstream <NUM>. To this end, encoder <NUM> may optionally comprise an entropy coder <NUM> which entropy codes the prediction residual signal as transformed and quantized into data stream <NUM>. The prediction signal <NUM> is generated by a prediction stage <NUM> of encoder <NUM> on the basis of the prediction residual signal <NUM>" encoded into, and decodable from, data stream <NUM>. To this end, the prediction stage <NUM> may internally, as is shown in <FIG>, comprise a dequantizer <NUM> which dequantizes prediction residual signal <NUM>" so as to gain spectral-domain prediction residual signal <NUM>‴, which corresponds to signal <NUM>' except for quantization loss, followed by an inverse transformer <NUM> which subjects the latter prediction residual signal <NUM>‴ to an inverse transformation, i.e. a spectral-to-spatial transformation, to obtain prediction residual signal 24ʺʺ, which corresponds to the original prediction residual signal <NUM> except for quantization loss. A combiner <NUM> of the prediction stage <NUM> then recombines, such as by addition, the prediction signal <NUM> and the prediction residual signal 24ʺʺ so as to obtain a reconstructed signal <NUM>, i.e. a reconstruction of the original signal <NUM>. Reconstructed signal <NUM> may correspond to signal <NUM>'. A prediction module <NUM> of prediction stage <NUM> then generates the prediction signal <NUM> on the basis of signal <NUM> by using, for instance, spatial prediction, i.e. intra-picture prediction, and/or temporal prediction, i.e. inter-picture prediction.

Likewise, decoder <NUM>, as shown in <FIG>, may be internally composed of components corresponding to, and interconnected in a manner corresponding to, prediction stage <NUM>. In particular, entropy decoder <NUM> of decoder <NUM> may entropy decode the quantized spectral-domain prediction residual signal <NUM>" from the data stream, whereupon dequantizer <NUM>, inverse transformer <NUM>, combiner <NUM> and prediction module <NUM>, interconnected and cooperating in the manner described above with respect to the modules of prediction stage <NUM>, recover the reconstructed signal on the basis of prediction residual signal <NUM>" so that, as shown in <FIG>, the output of combiner <NUM> results in the reconstructed signal, namely picture <NUM>'.

Although not specifically described above, it is readily clear that the encoder <NUM> may set some coding parameters including, for instance, prediction modes, motion parameters and the like, according to some optimization scheme such as, for instance, in a manner optimizing some rate and distortion related criterion, i.e. coding cost. For example, encoder <NUM> and decoder <NUM> and the corresponding modules <NUM>, <NUM>, respectively, may support different prediction modes such as intra-coding modes and inter-coding modes. The granularity at which encoder and decoder switch between these prediction mode types may correspond to a subdivision of picture <NUM> and <NUM>', respectively, into coding segments or coding blocks. In units of these coding segments, for instance, the picture may be subdivided into blocks being intra-coded and blocks being inter-coded. Intra-coded blocks are predicted on the basis of a spatial, already coded/decoded neighborhood of the respective block as is outlined in more detail below. Several intra-coding modes may exist and be selected for a respective intra-coded segment including directional or angular intra-coding modes according to which the respective segment is filled by extrapolating the sample values of the neighborhood along a certain direction which is specific for the respective directional intra-coding mode, into the respective intra-coded segment. The intra-coding modes may, for instance, also comprise one or more further modes such as a DC coding mode, according to which the prediction for the respective intra-coded block assigns a DC value to all samples within the respective intra-coded segment, and/or a planar intra-coding mode according to which the prediction of the respective block is approximated or determined to be a spatial distribution of sample values described by a two-dimensional linear function over the sample positions of the respective intra-coded block with driving tilt and offset of the plane defined by the two-dimensional linear function on the basis of the neighboring samples. Compared thereto, inter-coded blocks may be predicted, for instance, temporally. For inter-coded blocks, motion vectors may be signaled within the data stream, the motion vectors indicating the spatial displacement of the portion of a previously coded picture of the video to which picture <NUM> belongs, at which the previously coded/decoded picture is sampled in order to obtain the prediction signal for the respective inter-coded block. This means, in addition to the residual signal coding comprised by data stream <NUM>, such as the entropy-coded transform coefficient levels representing the quantized spectral-domain prediction residual signal <NUM>", data stream <NUM> may have encoded thereinto coding mode parameters for assigning the coding modes to the various blocks, prediction parameters for some of the blocks, such as motion parameters for inter-coded segments, and optional further parameters such as parameters for controlling and signaling the subdivision of picture <NUM> and <NUM>', respectively, into the segments. The decoder <NUM> uses these parameters to subdivide the picture in the same manner as the encoder did, to assign the same prediction modes to the segments, and to perform the same prediction to result in the same prediction signal.

<FIG> illustrates the relationship between the reconstructed signal, i.e. the reconstructed picture <NUM>', on the one hand, and the combination of the prediction residual signal <NUM>"" as signaled in the data stream <NUM>, and the prediction signal <NUM>, on the other hand. As already denoted above, the combination may be an addition. The prediction signal <NUM> is illustrated in <FIG> as a subdivision of the picture area into intra-coded blocks which are illustratively indicated using hatching, and inter-coded blocks which are illustratively indicated not-hatched. The subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of square blocks or non-square blocks, or a multi-tree subdivision of picture <NUM> from a tree root block into a plurality of leaf blocks of varying size, such as a quadtree subdivision or the like, wherein a mixture thereof is illustrated in <FIG> in which the picture area is first subdivided into rows and columns of tree root blocks which are then further subdivided in accordance with a recursive multi-tree subdivisioning into one or more leaf blocks.

Again, data stream <NUM> may have an intra-coding mode coded thereinto for intra-coded blocks <NUM>, which assigns one of several supported intra-coding modes to the respective intra-coded block <NUM>. For inter-coded blocks <NUM>, the data stream <NUM> may have one or more motion parameters coded thereinto. Generally speaking, inter-coded blocks <NUM> are not restricted to being temporally coded. Alternatively, inter-coded blocks <NUM> may be any block predicted from previously coded portions beyond the current picture <NUM> itself, such as previously coded pictures of a video to which picture <NUM> belongs, or picture of another view or an hierarchically lower layer in the case of encoder and decoder being scalable encoders and decoders, respectively.

The prediction residual signal <NUM>"" in <FIG> is also illustrated as a subdivision of the picture area into blocks <NUM>. These blocks might be called transform blocks in order to distinguish same from the coding blocks <NUM> and <NUM>. In effect, <FIG> illustrates that encoder <NUM> and decoder <NUM> may use two different subdivisions of picture <NUM> and picture <NUM>', respectively, into blocks, namely one subdivisioning into coding blocks <NUM> and <NUM>, respectively, and another subdivision into transform blocks <NUM>. Both subdivisions might be the same, i.e. each coding block <NUM> and <NUM>, may concurrently form a transform block <NUM>, but <FIG> illustrates the case where, for instance, a subdivision into transform blocks <NUM> forms an extension of the subdivision into coding blocks <NUM>, <NUM> so that any border between two blocks of blocks <NUM> and <NUM> overlays a border between two blocks <NUM>, or alternatively speaking each block <NUM>, <NUM> either coincides with one of the transform blocks <NUM> or coincides with a cluster of transform blocks <NUM>. However, the subdivisions may also be determined or selected independent from each other so that transform blocks <NUM> could alternatively cross block borders between blocks <NUM>, <NUM>. As far as the subdivision into transform blocks <NUM> is concerned, similar statements are thus true as those brought forward with respect to the subdivision into blocks <NUM>, <NUM>, i.e. the blocks <NUM> may be the result of a regular subdivision of picture area into blocks (with or without arrangement into rows and columns), the result of a recursive multi-tree subdivisioning of the picture area, or a combination thereof or any other sort of blockation. Just as an aside, it is noted that blocks <NUM>, <NUM> and <NUM> are not restricted to being of quadratic, rectangular or any other shape.

<FIG> further illustrates that the combination of the prediction signal <NUM> and the prediction residual signal 24ʺʺ directly results in the reconstructed signal <NUM>'. However, it should be noted that more than one prediction signal <NUM> may be combined with the prediction residual signal 24ʺʺ to result into picture <NUM>' in accordance with alternative embodiments.

In <FIG>, the transform blocks <NUM> shall have the following significance. Transformer <NUM> and inverse transformer <NUM> perform their transformations in units of these transform blocks <NUM>. For instance, many codecs use some sort of DST or DCT for all transform blocks <NUM>. Some codecs allow for skipping the transformation so that, for some of the transform blocks <NUM>, the prediction residual signal is coded in the spatial domain directly. However, in accordance with embodiments described below, encoder <NUM> and decoder <NUM> are configured in such a manner that they support several transforms. For example, the transforms supported by encoder <NUM> and decoder <NUM> could comprise:.

Naturally, while transformer <NUM> would support all of the forward transform versions of these transforms, the decoder <NUM> or inverse transformer <NUM> would support the corresponding backward or inverse versions thereof:.

The subsequent description provides more details on which transforms could be supported by encoder <NUM> and decoder <NUM>. In any case, it should be noted that the set of supported transforms may comprise merely one transform such as one spectral-to-spatial or spatial-to-spectral transform.

As already outlined above, <FIG> have been presented as an example where the inventive concept described further below may be implemented in order to form specific examples for encoders and decoders according to the present application. Insofar, the encoder and decoder of <FIG> and <FIG>, respectively, may represent possible implementations of the encoders and decoders described herein below. <FIG> and <FIG> are, however, only examples. An encoder according to embodiments of the present application may, however, perform block-based encoding of a picture <NUM> using the concept outlined in more detail below and being different from the encoder of <FIG> such as, for instance, in that same is no video encoder, but a still picture encoder, in that same does not support inter-prediction, or in that the sub-division into blocks <NUM> is performed in a manner different than exemplified in <FIG>. Likewise, decoders according to embodiments of the present application may perform block-based decoding of picture <NUM>' from data stream <NUM> using the coding concept further outlined below, but may differ, for instance, from the decoder <NUM> of <FIG> in that same is no video decoder, but a still picture decoder, in that same does not support intra-prediction, or in that same sub-divides picture <NUM>' into blocks in a manner different than described with respect to <FIG> and/or in that same does not derive the prediction residual from the data stream <NUM> in transform domain, but in spatial domain, for instance.

<FIG> shows a decoder <NUM> according to an exemplary embodiment of the present application according to the first aspect of the invention. The decoder <NUM> may use the above described concept of block based decoding of picture data, i.e. of a still picture or a moving picture <NUM>'.

The decoder <NUM> as depicted in <FIG> may comprise a similar internal structure as the one described above with reference to <FIG>. Thus, equal or equivalent elements or elements with equal or equivalent functionality are denoted in <FIG> and <FIG> by equal or equivalent reference numerals. However, the decoder <NUM> of <FIG> may differ from the decoder of <FIG> in that it may additionally comprise a deblocking or deringing filter <NUM> for filtering and attenuating blocking and/or ringing artefacts, wherein blocking may be regarded as a particular case of a ringing artefact.

As described above, the reconstructed version of the picture, i.e. the decoded picture <NUM>' may be obtained by combining the residual signal <NUM> and the prediction residual <NUM>" in the combiner <NUM>. The decoder <NUM> of <FIG> may additionally apply the deblocking or deringing filter <NUM> to the reconstructed version of the picture, i.e. to the decoded picture <NUM>', upon combination of the residual signal <NUM> and the prediction residual <NUM>".

According to the inventive principle, the decoder <NUM> may locally vary a strength of the deblocking or deringing filter <NUM>. In other words, the decoder <NUM> may decide about the filter strength that shall be applied to the decoded picture <NUM>', e.g. whether a weak or a strong deblocking or deringing filter function shall be applied, or even if a deblocking or deringing filter shall be applied at all.

This decision about the applicable filter strength of the deblocking or deringing filter <NUM> may be based on a first and a second measure. The first measure may represent a locally measured mean block size. The second measure may represent a frequency of non-zero coding of the prediction residual <NUM>", i.e. the number of non-zero coded prediction residuals <NUM>".

The decoder <NUM> may determine the variable filter strength on a block-wise basis. Therefore, the decoder <NUM> may be configured to partition the picture <NUM> into blocks and to perform the reconstruction of the picture <NUM> by using said blocks, similar as described above with reference to <FIG>.

<FIG> shows an example, wherein the picture <NUM> may be partitioned into one or more blocks <NUM>, <NUM>, <NUM>, <NUM>. These blocks <NUM>, <NUM>, <NUM>, <NUM> may also be referred to as coding blocks. The decoder <NUM> may perform the reconstruction of the picture <NUM>, i.e. the decoding of the picture <NUM>, by using said coding blocks <NUM>, <NUM>, <NUM>, <NUM>.

Furthermore, the coding blocks <NUM>, <NUM>, <NUM>, <NUM> may be subject to sub-partitioning into one or more sub-blocks 181a-<NUM> and 182a-182d, respectively. The term "blocks" in general as used herein may accordingly refer to coding blocks <NUM>, <NUM>, <NUM>, <NUM> and/or to sub-blocks 181a-<NUM>, 182a-182d. The above mentioned first measure may be designed to locally measure the size of the blocks. Accordingly, the size of the blocks may, for instance, be measured in terms of coding blocks <NUM>, <NUM>, <NUM>, <NUM> and/or in terms of sub-blocks 181a-<NUM>, 182a-182d.

The partitioning mode for partitioning the coding blocks <NUM>, <NUM>, <NUM>, <NUM> into one or more sub-blocks 181a-<NUM>, 182a-182d may be signaled in the data stream <NUM> by means of a coding tree, which may also be referred to as a partitioning tree or a split tree. A tree root block, which may correspond to a coding block <NUM>, <NUM>, <NUM>, <NUM>, may be split into one or more leaf blocks, which may correspond to the sub-blocks 181a-<NUM> and 182a-182d.

Accordingly, as exemplarily depicted in <FIG>, the decoder <NUM> may be configured to perform the partitioning of the picture <NUM> into blocks by subjecting each of the plurality of tree root blocks <NUM>, <NUM>, <NUM>, <NUM> to a recursive multi-tree sub-divisioning so that the blocks form leaf blocks 181a-<NUM>, 182a-182d of the plurality of tree root blocks <NUM>, <NUM>, <NUM>, <NUM>. The decoder <NUM> may determine the first measure and the second measure locally for each tree root block <NUM>, <NUM>, <NUM>, <NUM>.

For example, in HEVC the coding blocks <NUM>, <NUM>, <NUM>, <NUM> may also be referred to as Coding Tree Units (CTU), and the sub-blocks 181a-<NUM>, 182a-182d may also be referred to as Coding Units (CU). A non-limiting exemplary embodiment shall be described in the following using the HEVC standard. However, the principle of the present application is not restricted to the HEVC standard.

Let us assume a usage of the principle of this application in an image or video codec defining a block size of L × L as the largest possible coding block size. Such a coding block <NUM>, <NUM>, <NUM>, <NUM>, also called CTU above, can be subject to sub-partitioning into multiple square or rectangular sub-blocks 181a-<NUM>, 182a-182d, each of size M × N pixel units. Examples are depicted in <FIG> which show, as non-limiting examples, several possibilities of partitioning a coding block <NUM>.

For example, <FIG> shows an example in which the coding block <NUM> is not further partitioned into sub-blocks. Accordingly, coding block <NUM> may only comprise one single sub-block (sub-block <NUM>) and may, thus, be the same as sub-block <NUM>. <FIG> therefore implies the absence of sub-partitioning.

Some further examples for CTU segmentations, including rectangular sub-blocks, are shown in <FIG>. At both the encoder and decoder side, the case of partitioning a CTU <NUM>, <NUM>, <NUM>, <NUM> into one or more sub-blocks (CUs) 181a-<NUM>, 182a-182d can be identified by way of the CTU's coding tree signaled in the bit-stream. For example, splitting a block by a quad tree may lead to four square sub-blocks, while splitting a block by a (generalized) binary tree may lead to two (generalized) rectangular sub-blocks.

<FIG> shows an exemplary partitioning of coding block <NUM> into four square sub-blocks (<NUM> to <NUM>) splitted by a quad tree. <FIG> shows an exemplary partitioning of coding block <NUM> into seven square sub-blocks (<NUM> to <NUM>) splitted by a quad tree. <FIG> shows an exemplary partitioning of coding block <NUM> into seven sub-blocks, wherein sub-block <NUM> is a square sub-block splitted by a quad tree, wherein sub-blocks <NUM> to <NUM> are generalized rectangular sub-blocks vertically splitted by a binary tree, and wherein sub-blocks <NUM> and <NUM> are generalized rectangular sub-blocks horizontally splitted by a binary tree. <FIG> shows an exemplary partitioning of coding block <NUM> into seven sub-blocks, wherein sub-blocks <NUM> and <NUM> are generalized rectangular sub-blocks vertically splitted by a binary tree, and wherein sub-blocks <NUM> to <NUM> are square sub-blocks splitted by a quad tree.

In other words, the decoder <NUM> may be configured to read partitioning information (e.g. quad tree, binary tree) from the data stream <NUM>. The decoder <NUM> may further be configured to perform the subjecting of the tree root blocks (CTUs) <NUM>, <NUM>, <NUM>, <NUM> to the recursive multi-tree sub-divisioning depending on said partitioning information. The decoder <NUM> may further be configured to determine the first measure depending on said partitioning information.

According to an embodiment, the decoder <NUM> may be configured to determine the first measure by determining, for each tree root block (CTU) <NUM>, <NUM>, <NUM>, <NUM>, the number of leaf blocks (sub-blocks) 181a-<NUM>, 182a-182d into which the respective tree root block (CTU) <NUM>, <NUM>, <NUM>, <NUM> is split. In the following, this first measure may be referenced with capital letter A. That is, A may represent, for each tree root block (CTU), the number of leaf blocks or sub-blocks (CUs), respectively.

CTUs <NUM>, <NUM>, <NUM>, <NUM> with low visual activity (i.e. few image details) are typically not sub-partitioned or are sub-partitioned into only a few relatively large sub-blocks, as shown in <FIG>, for example. Moreover, for these low-activity CTUs <NUM>, <NUM>, <NUM>, <NUM>, the spatial intra-picture prediction (and temporal inter-picture prediction, if applicable) typically works very efficiently. As a result, the prediction residuals <NUM>" in said CTUs <NUM>, <NUM>, <NUM>, <NUM> may comprise very little signal energy and, thus, can often be fully quantized to zero and, in doing so, can be exempt from transmission.

Sometimes, though, at least one sub-block in such a low-activity CTU <NUM>, <NUM>, <NUM>, <NUM> may exhibit relatively high signal variance in its prediction residual <NUM>", thus requiring a transmission of at least one (coarsely) quantized residual which is not fully zero and which is likely to cause visible blocking in the decoded picture <NUM>'.

Residual coefficient signals, which are also referred to as residual transform units (TU) in HEVC, are each associated with one sub-block. In other words, each sub-block (CU) may comprise a transform unit (TU) for performing a piecewise transformation of the prediction residual with at least one transform unit per block, i.e. per coding block or per sub-block depending on the granularity of partitioning. Accordingly, in the coding tree, for each tree root block (CTU), a number of leaf blocks (CUs) and a number of coefficient blocks (TUs) may be determined.

A coded block flag (CBF) may indicate whether a residual coefficient signal (TU) has been fully quantized to zero (CBF = <NUM>) or whether a residual coefficient signal (TU) has not been fully quantized to zero (CBF = <NUM>). The latter may also be referred to as a non-zero coded block flag, or non-zero CBF. The number of non-zero coded block flags (CBF = <NUM>) may be signaled in the bitstream for each CTU.

In the following, the number of non-zero coded block flags (CBF = <NUM>) may be referenced with capital letter B. In other words, the capital letter B may represent the number of coefficient blocks being not fully quantized to zero. According to the inventive principle, this number B of non-zero coded blocks may represent the second measure.

According to such an embodiment, the decoder <NUM> may be configured to decode the prediction residual from the data stream <NUM> in units of coefficient blocks (TUs) representing a piecewise transformation of the prediction residual with at least one coefficient block (TU) per block (CTU or CU). The decoder <NUM> may further be configured to determine the second measure B by determining, for each tree root block (CTU), the number of coefficient blocks (TUs) being not fully quantized to zero. This may be managed by counting the number of non-zero coded block flags (CBF = <NUM>) in the CTU, for example.

As a non-limiting example, CTUs <NUM>, <NUM>, <NUM>, <NUM> which are sub-partitioned into fewer than nine (i.e. A < <NUM>) sub-blocks (CUs) with, at the same time, the non-zero coding and transmission of B > <NUM> residual coefficient signals (TUs, each associated with one sub-block), may benefit most from the application of very strong deblocking or deringing post-filters. Accordingly, these CTUs may be candidate blocks for being subject to very strong deblocking or deringing.

For example, <FIG> shows a partitioning of coding block <NUM> into nine sub-blocks, i.e. the number of sub-blocks in this example is A = <NUM>. Thus, the above mentioned condition of A < <NUM> would, for example, not be met. Thus, the coding block <NUM> of <FIG> may not be subject to very strong deblocking post-filters. Accordingly, this CTU <NUM> may not be a candidate block for being subject to very strong deblocking or deringing.

Again, at both the encoder and decoder side, the case of partitioning into fewer than A sub-blocks can be identified by way of the CTU's coding tree signaled in the bit-stream, whereas the presence of B non-zero residual coefficient signals can be noticed by counting the number of non-zero coded block flags (CBFs) in the CTU, which are also signaled in the bit-stream.

As mentioned above, it may be checked whether a block (CTU) is a potential candidate block for being subject to a very strong deblocking or deringing using the deblocking or deringing filter, or whether this block shall rather be subject to a lower strength of deblocking or deringing. This corresponds to the herein described principle of a highly selective control of the application of said strong deblocking filtering. In other words, the strength of the deblocking or deringing filter may be locally varied.

This local variation of the filter strength may depend on two measures, namely a first measure A representing the number of sub-blocks into which the respective coding block is split, and on a second measure B representing the number of non-zero coded residuals. If a block fulfils these two measures A and B, then this block is a potential candidate block for being subject to strong deblocking or deringing. This may be indicated in the bit stream by means of a filter control parameter (FCP).

Thus, according to such an embodiment, the decoder <NUM> may perform the local variation of the filter strength by, for first portions of the picture (i.e. for candidate blocks), where the first and second measures A, B fulfill a predetermined criterion (e.g. A < <NUM>, B > <NUM>), reading strength information (FCP) from the data stream <NUM> indicative of a strength of the deblocking or deringing filter <NUM> to be applied at the respective portion (i.e. block). For second portions of the picture (i.e. for non-candidate blocks), where the first and second measures A, B do not fulfill the predetermined criterion (e.g. A < <NUM>, B > <NUM>), the decoder <NUM> may be configured to set the strength of the deblocking or deringing filter <NUM> to be applied at the respective portion (block) to a lower second strength which is lower than the first filter strength.

Thus, at least for the above mentioned non-limiting example, it can be summarized that.

Condition <NUM>: a desired in-loop filtering (e.g. very strong deblocking) shall be allowed in a CTU if.

In other words, if condition <NUM> is not met, said desired in-loop filtering shall be disallowed and shall, therefore, always be disabled in the affected CTU at both the encoder and decoder side. If, on the other hand, condition <NUM> is met in a CTU, the desired in-loop filtering is allowed, but this does not necessarily mean that said in-loop filtering is also enabled.

In the above described non-limiting example, the predetermined criterion, i.e. Condition <NUM>, was met when A < <NUM> and B > <NUM>. However, stated in more general terms, the predetermined criterion is fulfilled if the first measure A falls below a predetermined threshold, and if the second measure B exceeds or is equal to a second predetermined threshold.

For example, the first predetermined threshold is p with p fulfilling <NUM> < p < <NUM> for each of the tree root blocks (CTUs), i.e. A < p. Additionally or alternatively, the second predetermined threshold is q with q fulfilling -<NUM> < q < <NUM>, i.e. B z q.

In fact, as discussed above, it is highly desirable to provide a means for realizing highly selective control of the application of super-strong in-loop filters such as very strong deblocking filters. One exemplary way to provide this means is to.

Condition <NUM>: signal an in-loop filter control parameter (FCP), e.g. via transmission in a bit stream, in a CTU if.

In other words, if condition <NUM> is not met, said in-loop filter control parameter (FCP) is not signaled. If, on the other hand, condition <NUM> is met in a CTU, said filter control parameter (FCP) - e.g., an additional single-bit element - is written to the bit-stream by the encoder and read from said bit-stream by the decoder.

If the filter control parameter is present in the bit-stream (i.e., condition <NUM> is met) for a given CTU, then the value of this control parameter determines whether the decoder is to enable the desired in-loop filtering (e.g. value <NUM>) or to disable it (e.g. value <NUM>) in said CTU. In this way, the encoder can control - and signal - the desired application of, e.g. very strong deblocking.

Summarizing, the concept of the present application may suggest a selective signaling of an in-loop filter control parameter per coding block (e. , coding tree unit, CTU), to disable or attenuate the application of said in-loop filter in said coding block. The in-loop filter control parameter may only be signaled if the coding block is partitioned into fewer than A sub-blocks or if residual coefficient coding (i.e. non-zero coding) is applied in B of the sub-blocks.

<FIG> shows an encoder <NUM> which may be applied according to the concept of the present application according to the first aspect of the invention. The encoder <NUM> as depicted in <FIG> may comprise a similar internal structure as the one described above with reference to <FIG>. Thus, equal or equivalent elements or elements with equal or equivalent functionality are denoted in <FIG> and <FIG> by equal or equivalent reference numerals. However, the encoder <NUM> of <FIG> may differ from the encoder of <FIG> in that it may additionally comprise a deblocking or deringing filter <NUM> for filtering and attenuating blocking and/or ringing artefacts, wherein blocking may be regarded as a particular case of a ringing artefact. The deblocking or deringing filter <NUM> may be an in-loop filter.

The encoder <NUM> is configured for block-based encoding of picture data using a deblocking or deringing filter <NUM> as an in-loop filter. The encoder <NUM> may further be configured to encode, in a blockwise manner, a picture <NUM> into a data stream <NUM> using prediction and by coding a prediction residual into the data stream <NUM> with providing a reconstructed version of the picture in a prediction loop <NUM> of the encoder <NUM>. The prediction loop <NUM> may be a part of the prediction stage <NUM> which was already explained above with reference to <FIG>.

In said prediction loop <NUM>, the reconstruction of the picture <NUM> and applying the deblocking or deringing filter <NUM> may be simulated. Accordingly, the encoder <NUM> may be configured to apply the deblocking or deringing filter <NUM> onto the reconstructed version <NUM>' of the picture <NUM>.

In said prediction loop <NUM>, the encoder <NUM> may further try different filter strengths of the deblocking or deringing filter <NUM> in a similar fashion as explained above for the decoder side. In particular, the filter strength may be varied depending on the above described first measure A and second measure B. Accordingly, the encoder <NUM> may be configured to locally vary a strength of the deblocking or deringing filter <NUM> depending on a first measure A locally measuring a mean block size, and a second measure B locally measuring a frequency of non-zero coding of the prediction residual.

The encoder <NUM> is further configured to partition the picture <NUM> into blocks (CTUs) <NUM>, <NUM>, <NUM>, <NUM>, as described in <FIG> and <FIG> above. The encoder <NUM> is further configured to perform the encoding using the blocks <NUM>, <NUM>, <NUM>, <NUM>, wherein the first measure A is designed to locally measure a size of the blocks <NUM>, <NUM>, <NUM>, <NUM>.

In particular with reference to <FIG>, the encoder <NUM> may also split the blocks <NUM>, <NUM>, <NUM>, <NUM> into one or more sub-blocks by using a multi-tree subdivisioning, wherein the coding tree may, for instance, be a quad tree or a (generalized) binary tree.

Thus, the encoder <NUM> may be configured to perform the partitioning by subjecting each of a plurality of tree root blocks (CTUs) <NUM>, <NUM>, <NUM>, <NUM> into which the picture <NUM> is pre-partitioned to recursive multi-tree sub-divisioning so that the blocks <NUM>, <NUM>, <NUM>, <NUM> form leaf blocks (sub-blocks or CUs) of the plurality of tree-root blocks <NUM>, <NUM>, <NUM>, <NUM>. Furthermore, the encoder <NUM> may be configured to determine the first measure A and the second measure B locally for each tree root block <NUM>, <NUM>, <NUM>, <NUM>.

To do so, the encoder <NUM> may try, in the prediction loop <NUM>, one or more different types of multi-tree sub-divisioning. If the encoder <NUM> found a multi-tree sub-divisioning which works well with the respective CTU, then the encoder <NUM> selects this multi-tree sub-divisioning and adds corresponding partitioning information into the bit stream, based on the selected multi-tree sub-divisioning. Depending on said inserted partitioning information, the encoder <NUM> may determine the first measure A.

For example, as shown in <FIG>, the encoder <NUM> may select a combined quad tree - binary tree scheme for splitting the CTU <NUM> into six square sub-blocks (sub-blocks <NUM> to <NUM>) and into two rectangular sub-blocks (sub-blocs <NUM> and <NUM>). Accordingly, the encoder <NUM> may split the coding block (CTU) <NUM> into eight sub-blocks (CUs) which corresponds to a first measure A of A = <NUM>.

Stated in terms of the coding tree, the encoder <NUM> may be configured to perform the subjecting of each of the plurality of tree root blocks to the recursive multi-tree sub-divisioning based on partitioning information (e.g. quad tree / binary tree). The encoder <NUM> may insert the partitioning information into the data stream <NUM>, and the encoder <NUM> may determine the first measure A depending on the partitioning information.

The encoder <NUM> may determine the first measure A on a block-wise basis. That is, the encoder <NUM> may determine the number of sub-blocks (CUs) for each coding block (CTU) <NUM>, <NUM>, <NUM>, <NUM> separately.

Thus, in terms of the coding tree, the encoder <NUM> may be configured to determine the first measure A by determining, for each tree root block (CTU) <NUM>, <NUM>, <NUM>, <NUM>, the number of leaf blocks (CUs or sub-blocks) into which the respective tree root block (CTU) <NUM>, <NUM>, <NUM>, <NUM> is split.

As described above, with respect to the decoder <NUM>, also the encoder <NUM> may be configured to predictively code picture data using one or more prediction residual signals.

Residual signals, which are also referred to as residual transform units (TU) in HEVC, are each associated with one sub-block (CU). In other words, each sub-block (CU) may comprise a transform unit (TU) for performing a piecewise transformation of the prediction residual with at least one transform unit per block, i.e. per coding block or per sub-block depending on the granularity of partitioning. Accordingly, in the coding tree, for each tree root block (CTU), a number of leaf blocks (CUs) and a number of coefficient blocks (TUs) may be determined.

A coded block flag (CBF) may indicate whether a residual coefficient signal (TU) has been fully quantized to zero (CBF = <NUM>) or whether a residual coefficient signal (TU) has not been fully quantized to zero (CBF = <NUM>). The latter may also be referred to as a non-zero coded block flag, or non-zero CBF. The non-zero coded block flags (CBF = <NUM>) may be signaled in the bitstream for each CTU.

Accordingly, the encoder <NUM> may be configured to encode the prediction residual into the data stream <NUM> in units of coefficient blocks (TUs) representing a piecewise transformation of the prediction residual with at least one coefficient block (TU) per block (CTU or CU). The encoder <NUM> may further be configured to determine the second measure B by determining, for each tree root block (CTU), the number of coefficient blocks (TU) being not fully quantized to zero.

Again, at both the encoder and decoder side, the first measure A, i.e. the number of sub-blocks (CUs) into which a coding block (CTU) is partitioned can be identified by way of the CTU's coding tree that can be signaled in the bit-stream by the encoder <NUM>, whereas the presence of B non-zero residual coefficient signals can be noticed by counting the number of non-zero coded block flags (CBFs) in the CTU, which may also be signaled in the bit-stream by the encoder <NUM>.

In the prediction loop <NUM>, it may be checked whether a block (CTU) is a potential candidate block for being subject to a very strong deblocking or deringing using the deblocking or deringing filter, or whether this block shall rather be subject to a lower strength of deblocking or deringing. This corresponds to the herein described principle of a highly selective control of the application of said strong deblocking filtering at the encoder side. In other words, the strength of the deblocking or deringing filter may be locally varied.

This local variation of the filter strength may depend on two measures, namely a first measure A representing the number of sub-blocks into which the respective coding block is split, and on a second measure B representing the number of non-zero coded residuals. If a block fulfils these two measures A and B, then this block is a potential candidate block for being subject to strong deblocking or deringing. This may be indicated in the bit stream by the encoder <NUM> by means of a filter control parameter (FCP).

Thus, according to such an embodiment, the encoder <NUM> may perform the local variation of the filter strength by, for first portions of the picture (i.e. for candidate blocks), where the first and second measures A, B fulfill a predetermined criterion (e.g. A < <NUM>, B > <NUM>), inserting strength information (FCP) into the data stream <NUM> indicative of a strength of the deblocking or deringing filter <NUM> to be applied at the respective portion (i.e. block). For second portions of the picture (i.e. for non-candidate blocks), where the first and second measures A, B do not fulfill the predetermined criterion (e.g. A < <NUM>, B > <NUM>), the encoder <NUM> may be configured to set the strength of the deblocking or deringing filter <NUM> to be applied at the respective portion (block) to a lower second strength which is lower than the first filter strength.

For example, the first predetermined threshold is p with p fulfilling <NUM> < p < <NUM> for each of the tree root blocks (CTUs), i.e. A < p. Additionally or alternatively, the second predetermined threshold is q with q fulfilling -<NUM> < q < <NUM>, i.e. B ≥ q.

The operating principles of the encoder <NUM> and the decoder <NUM> shall be briefly summarized as follows:
The encoder <NUM> may calculate an optimal partitioning in a rate-distortion loop. Based on this calculation, the encoder <NUM> may check if the above mentioned Condition <NUM> (candidate block) is fulfilled. If Condition <NUM> is fulfilled, then the encoder <NUM> may try a strong deblocking per CTU. If the strong deblocking leads to measurable improvements (e.g. gain of SNR or SSIM) in the CTU, then the encoder <NUM> may use the strong deblocking for this CTU and signalize an in-loop filter control parameter (FCP) value, e.g. FCP = <NUM> (enabled), for said CTU in the bit stream <NUM>. Otherwise, if there is no measurable improvement, e.g. gain in SNR or SSIM, or if there is even a loss in SNR or SSIM, when applying the strong deblocking, then the encoder <NUM> may discard the result of the strong deblocking and signalize a respective in-loop filter control parameter (FCP) value, for example an in-loop filter control parameter (FCP) value of zero FCP = <NUM> (disabled), for the respective CTU. This shall signal towards the decoder <NUM> to refrain from using strong deblocking for said CTU. If the encoder <NUM> may determine that Condition <NUM> is not fulfilled, no in-loop filter control parameter (FCP) will be signalized in the bit stream <NUM> and, thus, strong deblocking will not be tried out for the respective CTUs.

The decoder <NUM> may receive the bit stream <NUM> with the previously calculated optimal partitioning of the CTU. Based on the corresponding partitioning information, the decoder <NUM> may check, for each CTU, whether Condition <NUM> is fulfilled. If Condition <NUM> is fulfilled in the respective CTU, an in-loop filter control parameter (FCP) is read from the bit stream <NUM>. If the value of the in-loop filter control parameter (FCP) is enabled (e.g. FCP = <NUM>) the strong deblocking will be exploited. If the value of the in-loop filter control parameter (FCP) is disabled (e.g. FCP = <NUM>) the strong deblocking will not be exploited. If the decoder <NUM> may determine that Condition <NUM> is not fulfilled, no in-loop filter control parameter (FCP) will be read from the bit stream <NUM> and, thus, strong deblocking will not be exploited for the respective CTUs.

<FIG> shows a schematic block diagram of a method for block-based decoding of picture data using a deblocking or deringing filter <NUM>.

In block <NUM> a picture is reconstructed, in a blockwise manner, from a data stream <NUM> using prediction and using a prediction residual coded in the data stream <NUM> to obtain a reconstructed version <NUM>' of the picture <NUM>.

In block <NUM> the deblocking or deringing filter <NUM> is applied to the reconstructed version <NUM>' of the picture <NUM>.

In block <NUM> a strength of the deblocking or deringing filter <NUM> is locally varied depending on a first measure A locally measuring a mean block size, and a second measure B locally measuring a frequency of non-zero coding of the prediction residual.

<FIG> shows a schematic block diagram of a method for block-based encoding of picture data using a deblocking or deringing filter <NUM> as an in-loop filter.

In block <NUM>, a picture <NUM> is encoded, in a blockwise manner, into a data stream <NUM> using prediction and coding a prediction residual into the data stream <NUM> with providing a reconstructed version <NUM>' of the picture <NUM> in a prediction loop <NUM>.

In the above discussed first aspect, a selective signaling of an (in-loop) filter control, e.g. by means of a filter control parameter, was suggested in order to choose between different deblocking or deringing filters to be applied on a picture. In the following, the second aspect will be discussed, wherein a deblocking filter concept is suggested. Said deblocking or deringing filter according to the second aspect may be combined with the first aspect, or it may be applied separately in encoders and/or decoders and/or methods different from the first aspect.

Conventional deblocking methods based on the signal adaptive approach applied in HEVC (see the Introduction section on page <NUM>) generally lead to subjective improvements but, sometimes, are still too weak to remove excessive blocking artifacts around large coding blocks, as mentioned earlier. In combination with the first aspect described previously, it was found beneficial to apply "superstrong" deblocking filtering with a larger filter support (e. , <NUM> spatial samples) then what is used in the state of the art (e.g., <NUM> or <NUM> samples). Given that the filter control parameter (FCP) introduced in the first aspect allows control over the filtering process on the encoder side, e.g., by means of rate/distortion testing, it can also be concluded that the application of very simple superstrong deblocking filters, with little or no signal-adaptive strength control, is sufficient (since their execution can be disabled by the encoder using the signaled FCP).

For example, if FCP = <NUM> in a CTU, traditional deblocking with decoder-side filter strength detection on the reconstructed image component may be applied for each TU sub-area of the CTU. If FCP = <NUM> in a CTU, the very strong deblocking filter described herein may be used on each TU whose width and height both equal <NUM> or more pixels. It can be extended by a decoder-side filter strength detection. A filter may remain strong enough, i.e., it may advantageously employ between <NUM> and <NUM> filter taps.

The first step in a superstrong deblocking algorithm is the derivation of left, right, top, and bottom boundary offsets for each TU satisfying the above size constraint. Specifically, given d = {left, right, top, bottom}, <MAT> where Pd and Qd are the sums of the outer and inner boundary reconstructed samples of the TU, respectively (excluding the outer four corner samples), along the direction d of length Nd (i. , the TU width resp. Unavailable Pd sums at slice or image borders are replaced by the adjacent Qd sums. Then, upon deblocking in case of FCP = <NUM> in a CTU (see also first aspect), weighted additions of offsetd may be applied, for each d, along the <NUM> inner boundary sample columns or rows perpendicular to d. This adds a linear ramp with slope offsetd + <NUM> towards the d TU boundary, reaching offsetd at the boundary, with blend-overs at the TU corners.

In a modification, the very strong deblocking may be executed before the traditional deblocking on the luma as well as chroma channels (note that, for <NUM>:<NUM>:<NUM> [spatially downsampled] chroma, the three bold values above are halved).

In summary, a conventional superstrong corrective deblocking algorithm may depend on four control values (mean offset values computed along left, right, top, and bottom block boundaries, respectively) with undisclosed blend-overs of the four control values at the four block corners.

However, these conventional deblocking algorithm may lead to blocking or ringing artefacts, particularly in areas near block corners, which may lead to a partially suboptimal subjective perception of the coding qualtiy. Given the partially suboptimal subjective performance of the state of the art, the second aspect of the invention suggests improvements by means of a corrective deblocking design dependent on eight instead of four control values.

Said inventive eight-value corrective deblocking approach extends the conventional four-value design by four additional corner values, i.e., a mean offset value offsetc for each of the c = {top-left, top-right, bottom-left, bottom-right} corners of the given block. Furthermore, given the conventional undisclosed (and suboptimal) blend-over implementation of the prior art, a detailed embodiment of an inventive blend-over algorithm for deblocking around the block corners, using pairs of adjacent control values, will be described.

<FIG> shows an example of a block <NUM> that may be processed by the deblocking filter according to the second aspect of the invention. The block <NUM> may be a subblock of a block based coding scheme. For instance, the block <NUM> may be a so-called Transform Unit (TU).

The block <NUM> may comprise a square or generally rectangular shape. The block <NUM> may comprise a block border <NUM>, which may represent the outer circumferential demarcation of the block <NUM>. The block border <NUM> may comprise a plurality of border portions into which the block border <NUM> may be subdivided.

The block <NUM> may comprise at least four corners and four edges extending between said four corners. Accordingly, the block border <NUM> may comprise four corner border portions <NUM>, <NUM>, <NUM>, <NUM> and four edge border portions <NUM>, <NUM>, <NUM>, <NUM> extending between the four corner border portions <NUM>, <NUM>, <NUM>, <NUM>.

The block <NUM> may contain a plurality of pixels representing a picture content <NUM>. The picture content inside the block <NUM> is illustrated by means of hatched lines. There may also be picture content <NUM> outside the block <NUM> which may be represented by surrounding pixels. Said surrounding picture content (surrounding pixels) <NUM> may be arranged around the block <NUM> along the respective border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

The picture content <NUM> inside the block <NUM> may yet be unfiltered, and it may thus be referred to as an unfiltered content of the block <NUM>. Applying the deblocking filter on said yet unfiltered picture content <NUM> may provide a filtered picture content, which may also be referred to as a filtered content of the block <NUM>.

Therefore, the deblocking filter may compare the yet unfiltered picture content <NUM> inside the block <NUM> with an adjacent picture content <NUM> outside the block <NUM>. This may be done at each of the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

For example, as shown in <FIG>, a yet unfiltered picture content <NUM> inside the block <NUM> which extends along the first edge border portion <NUM> may be compared with the adjacent picture content <NUM> which also extends along the first edge border portion <NUM> but outside the block <NUM>. The picture content <NUM> inside the block <NUM> may differ from the picture content <NUM> outside the block <NUM>. Thus, a dissimilarity between the picture content <NUM> inside the block <NUM> and the picture content <NUM> outside the block <NUM> may exist, which may be determined by the deblocking filter.

Processing, i.e. deblocking filtering, of the deblocking filter may be adjusted based on said dissimilarities. In other words, the deblocking filtering may be parametrized based on the determined dissimilarities between the picture content <NUM> inside the block <NUM> and the picture content <NUM> outside the block <NUM>.

Thus, according to an embodiment, a deblocking filter for filtering a block <NUM> of a picture is suggested, wherein the deblocking filter may be configured to determine, for each of at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> of a border <NUM> of the block <NUM>, a dissimilarity between an unfiltered content <NUM> of the block <NUM> and a surrounding picture content <NUM> around the block <NUM> along the respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, the eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> including four corner border portions <NUM>, <NUM>, <NUM>, <NUM>, each arranged at a corner of the block <NUM>, and four edge border portions <NUM>, <NUM>, <NUM>, <NUM>, each arranged at intermediary portions of the border <NUM> between the corners of the block <NUM>. The deblocking filter may further be configured to parametrize a deblocking filtering of the block <NUM> using the dissimilarities determined for the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> in order to obtain a filtered content of the block <NUM>.

Still with reference to <FIG>, the exemplarily illustrated picture content <NUM> inside the block <NUM> and the exemplarily illustrated picture content <NUM> outside the block <NUM> may each comprise a plurality of pixels, also referred to as samples. For instance, a plurality of first samples may be located inside the block <NUM> and a plurality of second samples may be located outside the block <NUM>.

According to an embodiment, the deblocking filter may be configured to determine, for each of the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, the dissimilarity by computing a mean difference between first samples and second samples, said first samples being located inside the block <NUM> and adjoining the respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and said second samples being located outside the block <NUM> and adjoining the respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

The plurality of first samples may be subsumed in a first sample vector Q, and the plurality of second samples may be subsumed in a second sample vector P. Since the sample vectors P, Q may extend along a respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, said vectors P, Q may also be referred to as boundary sample vectors. Each boundary sample vector P, Q may comprise a sum of samples, e.g. the first sample vector Q may comprise a sum of first samples and the second sample vector P may comprise a sum of second samples.

The above example was described by referring to an edge border portion <NUM>. However, the same holds true for corner border portions <NUM>, <NUM>, <NUM>, <NUM>, as shall be exemplarily described with reference to <FIG>.

<FIG> shows a corner boundary portion <NUM> in the left upper corner of the block <NUM>. An inner boundary sample vector Q<NUM> extends along a vertical part of the corner boundary portion <NUM> inside the block <NUM>. Adjacent to the inner boundary sample vector Q<NUM> an outer boundary sample vector P<NUM> extends along a vertical part of the corner boundary portion <NUM> outside the block <NUM>.

Furthermore, a further inner boundary sample vector Q<NUM> extends along a horizontal part of the corner boundary portion <NUM> inside the block <NUM>. Adjacent to the further inner boundary sample vector Q<NUM> a further outer boundary sample vector P<NUM> extends along a horizontal part of the corner boundary portion <NUM> outside the block <NUM>.

The inner boundary sample vectors Q<NUM> and Q<NUM> may be subsumed as a first corner sample vector Qc and the outer boundary sample vectors P<NUM> and P<NUM> may be subsumed as a second corner sample vector Pc.

According to an embodiment, the deblocking filter may be configured to determine, for each of the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, the dissimilarity by computing a difference between a first sum (vector Q) over first samples and a second sum (vector P) over second samples, said first samples being located inside the block <NUM> and adjoining the respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and said second samples being located outside the block <NUM> and adjoining the respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>.

Accordingly, a (mean) difference between the first (inner) sample vectors Q and the second (outer) sample vectors P may be computed, which corresponds to Δ = P - Q.

An offset between outer and inner sample vectors along edge border portions <NUM>, <NUM>, <NUM>, <NUM> may be computed in a conventional manner, given d = {left edge, right edge, top edge, bottom edge}: <MAT>.

According to the herein described concept, in addition to the edge border portions <NUM>, <NUM>, <NUM>, <NUM>, an offset between outer and inner sample vectors along corner border portions <NUM>, <NUM>, <NUM>, <NUM> may be computed, given c = {top left corner, top right corner, bottom left corner, bottom right corner}: <MAT> wherein the above mentioned (mean) difference Δ = P - Q is considered for corner boundary vectors Qc and Pc.

In other words, for each of the four corners c (top-left, top-right, bottom-left, and bottom-right), the corrective offset, to be used as control value in the actual deblocking processing, is the average of the difference between sets of outer (Pc) and inner (Qc) boundary reconstructed samples around the (e.g. TU) block corner (again excluding the outer four corner samples). Preferably, Nc equals the filter support length, i.e., the number of columns/rows to be deblocked on each side of a block border <NUM>, but Nc could also be a power-of-two fraction of Nd, i.e., the (TU) block width or height as in the prior art. More specifically, according to the herein described concept, Nc = <NUM> (or <NUM> in case of <NUM>:<NUM>:<NUM> chroma), while the prior art uses Nd ≥ <NUM>.

According to an embodiment, the deblocking filter may generally be configured to set widths of the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> depending on a size of the block <NUM> so that, at least for one of the eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, a width of the respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> equals a fraction of a length of the block's border <NUM> which may vary for different block sizes. For example, the larger the block, the larger the width of a respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> may be. As mentioned above, the border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> may be fractions of the length of the blocks border <NUM>. In other words, the border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> themselves may always be shorter than the length of the block's border <NUM>.

With varying widths of the border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, also the spatial locations of the above discussed boundary vectors P and Q may be varied.

<FIG> show the possible spatial locations of the Pc and Qc sums of samples used in the derivation of offsetc for the non-limiting example of c = top-left. Generally, an offsetc can be calculated from four individual vector sums P<NUM>, P<NUM>, Q<NUM>, Q<NUM>, as illustrated, with Pc = P<NUM> + P<NUM> and Qc = Q<NUM> + Q<NUM>.

According to some examples, P<NUM> and Q<NUM> may not overlap or may partially overlap with Pd and Qd, as introduced in the prior art (in which case it can simply be defined Pd = P<NUM> and Qd = Q<NUM>, see <FIG>), or P<NUM> and Q<NUM> may fully overlap with Pd and Qd (in which case it can be specified Pd = P<NUM> + P<NUM> or Pd = P<NUM> + P<NUM> + P<NUM> and Qd = Q<NUM> + Q<NUM> or Qd = Q<NUM> + Q<NUM> + Q<NUM>, see <FIG>, with P<NUM> and Q<NUM> used to calculate an offsetc at another corner). In case of <FIG>, such summation of non-overlapping vector sums reduces algorithmic complexity. Note that the vectors for P<NUM> and Q<NUM> may also be of length zero, i. , Pd = P<NUM> + P<NUM> and Qd = Q<NUM> + Q<NUM>.

Accordingly, <FIG> show possible spatial locations of border edge border portions <NUM> and corner boundary portions <NUM> and corresponding boundary sample vectors P and Q of a coding block <NUM>. In particular, in <FIG> the respective border portions <NUM>, <NUM> and the corresponding boundary sample vectors Pc, Qc (at the corners) and Pd, Qc along the edges may not overlap (<FIG>), may abut each other (<FIG>), may partially overlap (<FIG>), or may fully overlap (<FIG>) for offsetc and offsetd. As an example, the case depicted in <FIG> may be preferred for block dimensions larger than 4Nc, the cases depicted in <FIG> and <FIG> may be preferred for block dimensions between 3Nc and 4Nc, and the case depicted in <FIG> may be preferred for block dimensions smaller than 3Nc.

Thus, according to an embodiment, the deblocking filter may be configured to set widths of the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> depending on a size of the block <NUM> in generally two different ways. First, if the size of the block <NUM> is smaller than a first predetermined amount, e.g. block size < 3Nc, the four edge border portions <NUM>, <NUM>, <NUM>, <NUM> and the four corner border portions <NUM>, <NUM>, <NUM>, <NUM> may mutually overlap (see <FIG>). Second, if the size of the block <NUM> is greater than the first predetermined amount, e.g. block size > 3Nc, the four edge border portions <NUM>, <NUM>, <NUM>, <NUM> and the four corner border portions <NUM>, <NUM>, <NUM>, <NUM> may not overlap (see <FIG>).

According to more precise embodiment, the deblocking filter may be configured to set the widths of the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> depending on a size of the block <NUM> in three different ways. First, if the size of the block <NUM> is smaller than a first predetermined amount, e.g. block size < 3Nc, the four edge border portions <NUM>, <NUM>, <NUM>, <NUM> and the four corner border portions <NUM>, <NUM>, <NUM>, <NUM> may mutually overlap (see <FIG>). Second, if the size of the block <NUM> is between a first predetermined amount and a second predetermined amount, e.g. 3Nc ≤ block size ≤ 4Nc, the four edge border portions <NUM>, <NUM>, <NUM>, <NUM> and the four corner border portions <NUM>, <NUM>, <NUM>, <NUM> may mutually abut each other (see <FIG>). Third, if the size of the block <NUM> is greater than the second predetermined amount, e.g. block size >4Nc, the four edge border portions <NUM>, <NUM>, <NUM>, <NUM> and the four corner border portions <NUM>, <NUM>, <NUM>, <NUM> may be mutually spaced apart from each other (see <FIG>).

For the cases of <FIG>, it may be advantageous to modify the prior-art calculation of offsetd as follows: <MAT> where Ne ≤ Nd-<NUM>Nc is the length of the P<NUM> and Q<NUM> vectors. Ne may be a power of <NUM> for the » shift.

<FIG> may serve to give a brief introduction in the terminology of the following description. <FIG> shows the above discussed block <NUM> having a block border <NUM> that is partitioned into eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. Inside the block <NUM> rows <NUM> and columns <NUM> of samples may be arranged.

The depicted row <NUM> in <FIG> may be the second row when viewed from the block boarder <NUM>. Accordingly, said row <NUM> may have a sample offset of one row relative to the block border <NUM>. The depicted column <NUM> in <FIG> may be the third column when viewed from the block boarder <NUM>. Accordingly, said column <NUM> may have a sample offset of two columns relative to the block border <NUM>. The rows <NUM> and columns <NUM> may be subsumed as lines of samples.

The deblocking filter may comprise a filter support length which represents the number of lines <NUM>, <NUM> (i.e. rows/columns) of the block <NUM> that shall be subject to the filtering process, i.e. the number of lines <NUM>, <NUM> (i.e. rows/columns) of the block <NUM> that shall be deblocked on each side of the block border <NUM>. Said number of lines <NUM>, <NUM> (i.e. rows/columns) are counted from the outside, i.e. from the borderline <NUM> to the inside <NUM> of the block <NUM>.

Accordingly, a boundary band <NUM> (illustrated in hatched lines) may represent the number of lines <NUM>, <NUM> of samples that shall be deblocked by the deblocking filter, while the bandwidth of said boundary band <NUM> may represent the filter strength. The boundary band <NUM> may circumferentially extend around the inside <NUM> of the block <NUM>.

In the example depicted in <FIG>, the boundary band <NUM> may comprise a vertical bandwidth of five columns on each vertical portion of the borderline <NUM>, and a a horizontal bandwidth of four rows on each horizontal portion of the borderline <NUM>. Stated in more general terms, the filter support length, i.e. the bandwidth of the boundary band <NUM> may be different between vertical and horizontal portions.

Alternatively, the filter support length, i.e. the bandwidth of the boundary band <NUM>, may be equal on each border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. Thus, according to an embodiment, the deblocking filter may be configured so that the boundary band <NUM> comprises a constant circumferential width.

Furthermore, it may be possible that the filter strength, i.e. the bandwidth of the boundary band <NUM> may vary with the size of the block <NUM> to be coded. For example, the filter strength, i.e. the bandwidth of the boundary band <NUM>, may increase with increasing size of the block <NUM>. Thus, according to an embodiment, the deblocking filter may be configured to set a band width of the boundary band <NUM> monotonically increasing with respect to a size of the block <NUM>.

As shown in <FIG>, the boundary band <NUM> may be partitioned into one or more boundary regions <NUM>, <NUM>, <NUM>, which may also be referred to as processing regions. For instance, the boundary band <NUM> may comprise corner regions <NUM> and edge-centered regions <NUM>. These boundary regions <NUM>, <NUM> may be square-shaped or generally rectangular. These boundary regions <NUM>, <NUM> may be subject to the deblocking filtering. The boundary band <NUM> may further comprise an inner region <NUM> that may not be subject to the deblocking filtering.

As shown in <FIG>, the boundary regions <NUM>, <NUM> may comprise boundary portions. For instance, the corner regions <NUM> (exemplarily shown in the top left and top right corner) may comprise corner boundary portions 1041a, 1041b extending diagonally from the respective corner of the block <NUM> towards the inside (inner region) <NUM> of the block <NUM>. The exemplarily depicted two corner boundary portions 1041a, 1041b form a pair of neighboring corner boundary portions.

Two or more further boundary portions may be arranged between the pair of neighboring corner boundary portions 1041a, 1041b. For example, a middle boundary portion <NUM> may circumferentially extend between the pair of neighboring corner boundary portions 1041a, 1041b. Furthermore, sections 1061a, 1061b may circumferentially extend between the neighboring corner boundary portions 1041a, 1041b and the middle boundary portion <NUM>. For instance, a first section 1061a may be arranged between the middle boundary portion <NUM> and the first corner boundary portion 1041a, and a second section 1061b may be arranged between the middle boundary portion <NUM> and the second corner boundary portion 1041b.

Thus, according to an embodiment, the deblocking filter may be configured so that the boundary portions of the boundary band <NUM> may at least comprise, at each corner of the block <NUM>, a corner boundary portion 1041a, 1041b diagonally extending from the respective corner towards an inside <NUM> of the block <NUM>, and between each pair of neighboring corner boundary portions 1041a, 1041b of the block <NUM>, two or three boundary portions <NUM>, 1061a, 1061b.

According to a further embodiment, the deblocking filter may be configured so that the two or three boundary portions <NUM>, 1061a, 1061b between each pair of neighboring corner boundary portions 1041a, 1041b of the block <NUM> may comprise a first section 1061a circumferentially neighboring a first corner boundary portion 1041a of the respective pair, a second section 1061b circumferentially neighboring a second corner boundary portion 1041b of the respective pair, and a middle boundary portion <NUM> circumferentially between the first and second corner boundary portions of the respective pair.

In some examples, the middle boundary portion <NUM> may not be present. This may depend on the block size and on the boundary segmentation of the borderline <NUM>. In this case, the above described sections 1061a, 1061b may abut each other.

Thus, according to an embodiment, the deblocking filter may be configured to either provide two three boundary portions <NUM>, 1061a, 1061b or only two boundary portions 1061a, 1061b between each pair of neighboring corner boundary portions 1041a, 1041b. This may depend on the block size which is measured horizontally in this example, i.e. between the two corner boundary portions 1041a, 1041b. Or stated in more general terms, the block size is measured along a direction extending between the corners from which the respective pair of neighboring corner boundary portions 1041a, 1041b extends towards the inside of the block <NUM>.

In a first case, if the block size is greater than two times a width of the boundary band <NUM>, then three boundary portions may be arranged between the pair of corner boundary portions 1041a, 1041b, namely a first section 1061a circumferentially neighboring the first corner boundary portion 1041a of the respective pair, a second section 1061b circumferentially neighboring the second corner boundary portion 1041b of the respective pair, and the middle boundary portion <NUM> circumferentially extending between the first and second corner boundary portions 1041a, 1041b of the respective pair.

In a second case, if the block size is not greater than two times a width of the boundary band <NUM>, then the middle boundary portion <NUM> may not be present. Accordingly, only two boundary portions may be present, namely the first section 1061a circumferentially neighboring the first corner boundary portion 1041a of the respective pair, and the second section 1061b circumferentially neighboring the second corner boundary portion 1041b of the respective pair. However, in this case the first and second sections 1061a, 1061b abut each other.

Based on the above description, one non-limiting example of a deblocking processing of the deblocking filter being applied to a block (e.g. TU) having middle boundary portions <NUM> and corner boundary portions 1041a, 1041b shall be explained in the following.

As mentioned before, the filter characteristics of the deblocking filter may be parametrized based on the dissimilarities between the inner and outer boundary vectors P and Q in corner regions <NUM>, and in edge-centered regions <NUM> if applicable.

<FIG> shows an example of how the deblocking filter may apply the deblocking filtering process to a block <NUM>.

According to an embodiment, the deblocking filter may be configured to offset each sample within a boundary band <NUM> of the block <NUM> extending along the block border <NUM> by using an offset value (e.g. offsetc for corner regions <NUM> and offsetd for edge-centered regions <NUM>). Said offset value (offsetc and offsetd) is set so that the offset value (offsetc and offsetd) is constant for each line <NUM>, <NUM> (i.e. row/column) of samples within each boundary portion 1041a, 1041b, <NUM>, 1061a, 1061b. Or stated in more general terms, constant for each line <NUM>, <NUM> of samples being equally shaped to the block's border <NUM> and having a constant sample offset to the block's border <NUM>.

According to the herein described concept, the filter strength decreases from the borderline <NUM> of the block <NUM> to the inside <NUM> of the block <NUM>, i.e. in the direction of the arrow <NUM>. Stated differently, the filter strength is higher for lines <NUM>, <NUM> (i.e. rows/columns) being located nearer to the borderline <NUM> of the block <NUM> than for lines <NUM>, <NUM> (i.e. rows/columns) being located nearer to the inside <NUM> of the block <NUM>. Accordingly, the offset value (offsetc and offsetd) is subject to an attenuation from the border <NUM> to a middle <NUM> of the block <NUM>.

Furthermore, the offset value (offsetc and offsetd) within the respective boundary portion 1041a, 1041b, <NUM>, 1061a, 1061b is computed based on the dissimilarity determined for one or more of the border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> being circumferentially nearest to the respective boundary portion 1041a, 1041b, <NUM>, 1061a, 1061b. For example, the offset value offsetd of the samples contained in the edge-centered region <NUM>, i.e. in the middle boundary portion <NUM>, is computed based on the dissimilarity between outer vector P<NUM> and inner vector Q<NUM> (not depicted in <FIG>), while the offset value offsetc of the samples contained in the corner regions <NUM>, i.e. in the corner boundary portion 1041a and in the sections 1061a, are computed based on the dissimilarity between outer vector Pc = P<NUM> + P<NUM> and inner vector Qc = Q<NUM> + Q<NUM> (not depicted in <FIG>).

Accordingly, eight offset values may be computed based on the dissimilarities between the inner and outer boundary vectors P and Q, wherein four offset values offsetd for the four edge border portions <NUM>, <NUM>, <NUM>, <NUM> and four offset values offsetc for the four corner border portions <NUM>, <NUM>, <NUM>, <NUM> are computed. Using the four offsetc and the four offsetd control values determined as described above, an exemplary deblocking process may be constructed as follows.

As depicted in <FIG>, given a coding block <NUM> (e.g., TU) of size M×N to be subjected to the deblocking, eight processing regions (i.e. four corner regions <NUM> and four edge-centered regions <NUM>) at the inner block boundaries are defined based on the boundary segmentation of <FIG> and <FIG>. The square-shaped regions <NUM> at the block corners may preferably be of fixed size Nc×Nc each, while the edge-centered regions <NUM> may be of variable size Nc×(Nd-<NUM>Nc) or (Nd-<NUM>Nc)×Nc each with, possibly, Nd - <NUM>Nc = <NUM>. Furthermore, the pixels inside the inner rectangular region <NUM> of the coding block <NUM> may not be modified by the deblocking process.

Thus, according to an embodiment, the deblocking filter may be configured so that the edge-centered region <NUM> and, thus, the corresponding middle boundary portion <NUM> is circumferentially as wide as the block <NUM> minus two times the width of the corner regions <NUM> and, thus, the corresponding corner boundary portions 10401a, 1041b. Furthermore, if the boundary band <NUM> comprises a constant circumferential width, the middle boundary portion <NUM> is circumferentially as wide as the block <NUM> minus two times the width of the boundary band <NUM>, since the circumferential width may equal the length of vectors P<NUM>, P<NUM>, Q<NUM>, Q<NUM> in the corner regions <NUM>.

However, as mentioned above, the edge-centered regions <NUM>, and thus the middle boundary portions <NUM> may not be present at all, depending on the block size. For instance, in blocks of width or height Nd = <NUM>Nc, said edge-centered regions <NUM> may not be present along the top/bottom or left/right edges, respectively, and, thus, may not be processed. Nd < <NUM>Nc is, however, not allowed.

Still with reference to <FIG>, which shows the spatial locations of boundary block segments <NUM>, <NUM>, <NUM> processed by deblocking, it can be seen that the outer P<NUM> to P<NUM> columns/rows of pixel samples are not processed when deblocking the current coding block. Instead they are processed when deblocking adjacent, here left and upper, blocks. The Q<NUM>, Q<NUM>, and Q<NUM> line regions are part of their respective corner regions <NUM> and Q<NUM> is part of its edge-centered region <NUM>.

In the following, let Nc be the filter length which, preferably, equals <NUM> or <NUM> pixels and which may depend on the image component (luma or chroma) or block size. The deblocking processing of an edge-centered region <NUM> is straightforward. In case of a horizontal top or bottom block border, Nc rows <NUM> of pixels p(x, y) are subjected to a weighted addition of offsetd:.

In both cases, the length of each processed row/column <NUM>, <NUM> of pixels equals Nd - <NUM>Nc. These operations may be similar to those performed in the prior-art. Effectively, the closer pixel p(x, y) is to the center of the block <NUM>, the more attenuated offsetd is added. Again, in a block <NUM> having width or height Nd = <NUM>Nc, edge-centered regions <NUM> will not be present along the affected dimension according to this aspect. The deblocking applied to the corner regions <NUM>, which is the main concept of this aspect of the invention, is a bit more sophisticated:.

Alternatively, oc in the deblocking of the corner regions <NUM> could be simplified to depend only on offsetc but not on mc, as follows: <MAT>.

This yields very similar subjective quality. The remaining two corner sections are given by.

Stated in terms of the appended claims, according to an embodiment, the deblocking filter may be configured to set, for each of the corner boundary portions <NUM>, the offset value oc for samples within the respective corner boundary portion <NUM> so that the offset value oc of the samples within the respective corner boundary portion <NUM> varies from the border <NUM> of the block <NUM> towards the inside of the block <NUM> according to a weighted average.

Said weighted average is an average over a first offset value offsetc for the corner border portions <NUM> and a second offset value mc for the two corner-adjacent edge border portions <NUM>, <NUM> (see also <FIG>). In more general terms, the first offset value offsetc may be determined based on the dissimilarity determined for the respective corner border portion <NUM>, <NUM>, <NUM>, <NUM> from which the respective corner boundary portion <NUM> diagonally extends towards the inside <NUM> of the block <NUM>, while the second offset value mc may be determined based on the dissimilarities determined for the edge border portions <NUM>, <NUM>, <NUM>, <NUM> circumferentially adjacent to the respective corner border portion <NUM>, <NUM>, <NUM>, <NUM> from which the respective corner boundary portion <NUM> diagonally extends towards the inside <NUM> of the block <NUM>.

The weights (see Nc-f and f in the formula above) of the weighted average may depend on the samples' distance f from the border <NUM> of the block <NUM> in a manner so that the weighted average depends monotonically decreasingly less on the first offset value offsetc compared to the second offset value mc at increasing distance. In other words, the closer the diagonal pixel p(f, f) or p(W-<NUM>-f, f) or p(f, H-<NUM>-f) or p(W-<NUM>-f, H-<NUM>-f) is to the center <NUM> of the block <NUM>, the more mc dominates the weighted addition and the more attenuated the weighted addition is.

Referring back again to <FIG>, the corner boundary portions 1041a, 1041b may diagonally extend from the corner of the block <NUM> to the inside <NUM> of the block. A first and a second section 1061a, 1061b may be arranged between a middle boundary portion <NUM>, as explained above.

Offset values for samples contained in the first section 1061a may be interpolated between the first corner boundary portion 1041a and the middle boundary portion <NUM>. In particular, when viewed along a row <NUM>, an offset value for a sample contained in the first section 1061a may be interpolated between an offset value of the first corner boundary portion 1041a (in the same row <NUM>) and an offset value of the middle boundary portion <NUM> (in the same row <NUM>).

Offset values for samples contained in the second section 1061b may be interpolated between the second corner boundary portion 1041b and the middle boundary portion <NUM>. In particular, when viewed along a row <NUM>, an offset value of a sample contained in the second section 1061b may be interpolated between an offset value of the second corner boundary portion 1041b (in the same row <NUM>) and an offset value of the middle boundary portion <NUM> (in the same row <NUM>).

Stated in terms of the appended claims, according to an embodiment, the deblocking filter is configured to set, for the first section 1061a, the offset value for samples within the first section 1061a so that for each line (i.e. row/column) <NUM>, <NUM>, the offset value of samples within the respective line (i.e. row/column) <NUM>, <NUM> is interpolated between the offset value of the first corner boundary portion 1041a in the respective line (i.e. row/column) <NUM>, <NUM> and the offset value of the middle boundary portion <NUM> in the respective line (i.e. row/column) <NUM>, <NUM>. Furthermore, the deblocking filter is configured to set, for the second section 1061b, the offset value for samples within the second section 1061b so that for each line (i.e. row/column) <NUM>, <NUM>, the offset value of samples within the respective line (i.e. row/column) <NUM>, <NUM> is interpolated between the offset value of the second corner boundary portion 1041b in the respective line (i.e. row/column) <NUM>, <NUM> and the offset value of the middle boundary portion <NUM> in the respective line (i.e. row/column) <NUM>, <NUM>.

<FIG> shows a schematic block diagram of a method for filtering a block <NUM> of a block-based coded picture <NUM>.

In block <NUM>, for each of at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> of a border <NUM> of the block <NUM>, a dissimilarity between an unfiltered content <NUM> of the block <NUM> and a surrounding picture content <NUM> around the block <NUM> along the respective border portion <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> is determined, wherein the eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> include four corner border portions <NUM>, <NUM>, <NUM>, <NUM>, each arranged at a corner of the block <NUM>, and four edge border portions <NUM>, <NUM>, <NUM>, <NUM>, each arranged at intermediary portions of the border <NUM> between the corners of the block <NUM>.

In block <NUM> a deblocking filtering of the block <NUM> is parametrized using the dissimilarities determined for the at least eight border portions <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> in order to obtain a filtered content of the block <NUM>.

Note that, unlike prior-art filters like, e.g., those in HEVC, the inventive "superstrong" deblocking algorithm may not adapt its deblocking strength depending on the pixel samples to be deblocked and, thus, exhibits lower computational complexity than the prior art during the filter application. Of course, a deblocking strength adaptation can be achieved by means of the first aspect of the present invention, i.e., by a usage of a conditionally signaled filter control parameter allowing to select, in both the encoder and decoder, a deblocking process from a prior-art "weak" or "medium-strong" algorithm and the herein described "superstrong" algorithm. Note, furthermore, that in <FIG> not all pixel samples along a block boundary are considered in the calculation of the offsetc and offsetd control values. This represents an algorithmic complexity reduction compared with the state of the art during the filter calculation. To conclude, it can be summarized that the inventive very strong corrective deblocking approach presented in the second aspect of the invention attains better visual performance than the prior art while, at the same time, exhibiting lower computational complexity in both the calculation and application of the corrective deblocking algorithm.

Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software.

The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.

Claim 1:
Decoder (<NUM>) for block-based decoding of picture data using a deblocking or deringing filter (<NUM>), configured to
reconstruct, in a blockwise manner, a picture (<NUM>) from a data stream (<NUM>) using prediction and using a prediction residual coded in the data stream (<NUM>) to obtain a reconstructed version (<NUM>') of the picture (<NUM>),
wherein the step of reconstructing the picture (<NUM>) in a blockwise manner is done by partitioning the picture (<NUM>) into blocks (<NUM>, <NUM>, <NUM>, <NUM>; 181a-<NUM>; 182a-182d) and performing the reconstruction of the picture (<NUM>) using the blocks (<NUM>, <NUM>, <NUM>, <NUM>; 181a-<NUM>; 182a-182d),
wherein the partitioning of the picture (<NUM>) into blocks (<NUM>, <NUM>, <NUM>, <NUM>; 181a-<NUM>; 182a-182d) is performed by subjecting each of a plurality of tree root blocks (<NUM>, <NUM>, <NUM>, <NUM>) into which the picture (<NUM>) is pre-partitioned to recursive multi-tree sub-divisioning so that the blocks form leaf blocks (181a-<NUM>; 182a-182d) of the plurality of tree-root blocks (<NUM>, <NUM>, <NUM>, <NUM>), and
apply the deblocking or deringing filter (<NUM>) to the reconstructed version (<NUM>') of the picture (<NUM>), and locally vary a strength of the deblocking or deringing filter (<NUM>) depending on
a first measure (A) locally measuring for each tree root block (<NUM>, <NUM>, <NUM>, <NUM>) a mean block size, and
a second measure (B) locally measuring for each tree root block (<NUM>, <NUM>, <NUM>, <NUM>) a number of prediction residuals that are not fully quantized to zero,
wherein the decoder (<NUM>) is further configured to determine the first measure (A) by determining, for each tree root block (<NUM>, <NUM>, <NUM>, <NUM>), the number of leaf blocks (181a-<NUM>; 182a-182d) into which the respective tree root block (<NUM>, <NUM>, <NUM>, <NUM>) is split.