Patent Description:
Modern video coding standards make use of in-loop filters like Adaptive Loop Filter (ALF), Sample Adaptive Offset (SAO) and deblocking filter.

In-loop filters are located in the decoding loop of the encoder. During all the video coding stages and especially in the lossy compression performed in the quantization stage, the subjective quality of a video sequence can be reduced resulting in the appearance of blocking, ringing or blurring artifacts. In order to remove these artifacts, and increase the subjective and objective quality of the reconstructed sequence, a set of in-loop filters are used. In-loop filters in the encoder estimate the optimal filter parameters that increase the objective quality of a frame the most. These parameters are then transmitted to the decoder so that the in-loop filters of the decoder can use these parameters to optimally filter the reconstructed frame and achieve the same quality improvements reached for the reconstructed frame in the encoder.

The deblocking filter aims to remove the blocking artifacts that appear in the edge of CUs (coding units), and specifically PUs (prediction units) and TUs (transform units), as a consequence of using a block structure in the processing of every stage of the encoder.

The SAO filter aims to reduce undesirable visible artifacts such as ringing. The key idea of SAO is to reduce sample distortion by first classifying reconstructed samples into different categories, obtaining an offset for each category, and then adding the offset to each sample of the category.

The key idea of ALF is to minimize the mean square error between original pixels and decoded pixels using Wiener-based adaptive filter coefficients. ALF is located at the last processing stage of each picture and can be regarded as a tool trying to catch and fix artifacts from previous stages. The suitable filter coefficients are determined by the encoder and explicitly signalled to the decoder. That is, the ALF requires a set of parameters, i.e., the suitable filter coefficients, to be sent to the decoder. These parameters are sent in a high-level syntax structure, e.g. the Adaptation Parameter Set (APS). An APS is a parameter set that is sent in the bitstream before the Video Coding Layer (VCL) NAL (network abstraction layer) units, i.e. the slices of a picture. ALF is applied to the complete picture after reconstruction. Also, at the encoder, ALF estimation is one of the last steps in the encoding process.

In low-delay environments this causes a problem, because the encoder wants to start sending the processed parts of the picture as soon as possible, especially before finishing the encoding process of the picture. ALF cannot be used optimally in these environments, because the APS with the filter parameters estimated for the encoded picture have to be sent before the first slice of the picture.

In addition, a set of NAL units in a specified form is referred to as an access unit, AU, and the decoding of each AU results in one decoded picture. Each AU contains a set of VCL NAL units that together compose a primary coded picture. It may also be prefixed with an access unit delimiter (AUD) to aid in locating the start of the AU.

The AUD is used to separate AUs in the bitstream. It can optionally contain information about the following picture, like the allowed slice types (I, P, B).

In Versatile Video Coding, WC, several different parameter sets may be referred to by a picture: Video Parameter Set (VPS), Decoder Parameter Set (DPS), Sequence Parameter Set (SPS), multiple Picture Parameter Sets (PPS) and different types of Adaptation Parameter Sets (APS), also more than one. A decoder needs to have all the parameter sets available to be able to decode a picture.

The contribution "<NPL>, discloses one example of the concept of transmitting parameter sets including lop filter information. The concept of signalling parameters in a unit following all video coding units is e.g. disclosed in <CIT> (MICROSOFT TECHNOLOGY LICENSING LLC [US]) <NUM> August <NUM>.

Different slices of a picture may refer to different PPSs and APSs. Thus, it may be hard to determine for a decoder whether all required parameter sets are available, because it needs to parse all slice headers of the picture and decode, which parameter are referred to.

The object of the subject-matter of the present application is to provide a decoder which derives necessary parameters from access unit.

This object is achieved by the subject-matter of the claims of the present application.

In accordance with embodiments of the present invention, a video decoder comprises a decoding core (<NUM>) configured to reconstruct a decoded picture, e.g., a currently decoded picture or a following decoded picture, using motion compensated prediction and transform-based residual decoding from video coding units (<NUM>), e.g., VCL NAL unit, within an access unit, AU, of a video data stream to obtain a reconstructed version (46a) of the decoded picture; an in-loop filter (<NUM>), e.g. including ALF, configured to filter the reconstructed version of the decoded picture to obtain a version (46b) of the decoded picture to be inserted into the decoded picture buffer (<NUM>), DPB, of the video decoder; and a parametrizer configured to parametrize the in-loop filter by reading in-loop filter control information, e.g., ALF coefficients (or parameters) and ALF per CTU (coding tree unit) flags, for parametrizing the in-loop filter from parameter sets (<NUM>, <NUM>), e.g., ALF APS and ALF per CTU APS, located within the access unit, AU, of the decoded picture which follow , i.e., individually with the VCL NALUs or following all of them, along data stream order, the video coding units (<NUM>), and parametrizing the in-loop filter so as to filter the reconstructed version of the decoded picture in a manner depending on the in-loop filter control information. That is, the in-loop filter control information is derived for video coding unit, and, therefore, it is possible to start decoding before receiving all the video coding unit of a picture. Hence, the decoding delay is reduced in low-delay environments.

The in-loop filter control information comprises one or more filter coefficients for parametrizing the in-loop filter in terms of transfer function. That is, the ALF is, for instance, a FIR (finite impulse response) or IIR (infinite impulse response) filter and the filter coefficients FIR or IIR coefficients which control the filter's transfer function.

The in-loop filter control information comprises spatially selective in-loop filter control information for spatially varying the filtering of the reconstructed version of the decoded picture, e.g., the currently decoded picture or the following decoded picture, by the in-loop filter.

The parameter sets (<NUM>) comprise for each of the video coding units (<NUM>) a further predetermined parameter set which follows in data stream order the respective video coding unit (<NUM>) and comprises spatially selective in-loop filter control information for spatially varying the filtering of the reconstructed version of the decoded picture, e.g., the currently decoded picture or the following decoded picture, by the in-loop filter within a portion of the picture which is encoded into the respective video coding unit (<NUM>).

In accordance with embodiments of the present application, the parametrizer may be configured to locate the parameter sets (<NUM>, <NUM>), e.g., ALF APS and ALF per CTU APS, within the access unit, AU, of the decoded picture, e.g., the currently decoded picture or the following decoded picture, at a position which follows, i.e., individually with the VCLs or following all of them, along data stream order, the video coding units (<NUM>), in case of a predetermined indication in the video data stream assuming a first state, and at a different position within the access unit which precedes all of the video coding units (<NUM>) in case of the predetermined indication in the video data stream assuming a second state.

In accordance with embodiments of the present application, the video decoder is configured to read the predetermined indication from the video coding units (<NUM>). The predetermined indication indicates, in case of assuming the first state, the parameter sets by one or more identifier, and, in case of assuming the second state, different one or more in-loop filter control information parameter sets. The video decoder is configured to be responsive to the predetermined indication on a per access unit basis so as to perform locating differently for different access units of the video data stream in case of the predetermined indication being different for the different access units. The parametrizer is configured to reconstruct the decoded picture using the in-loop filter control information included in the previously signalled access unit, AU.

In accordance with embodiments of the present application, the video decoder is configured to, in detecting the boundaries of access unit, AU, interpret video coding units carrying the in-loop filter control information, e.g. the ALF filter data, in the form of one or more of parameter sets (<NUM>, <NUM>), e.g. suffix APS, as not starting an access unit, AU, therefrom, e.g. ignoring them in AU boundary detection and thereby detecting absence of an AU boundary, and interpret video coding units carrying the in-loop filter control information not in the form of one or more of parameter sets (<NUM>, <NUM>), e.g. prefix APS, as starting an access unit therefrom, e.g. detecting an AU boundary from such video coding units.

In accordance with embodiments of the present application, the method comprises reconstructing a decoded picture, e.g., a currently decoded picture or a following decoded picture, using motion compensated prediction and transform-based residual decoding from video coding units (<NUM>), e.g., VCL NAL unit, within an access unit, AU, of a video data stream to obtain a reconstructed version of the decoded picture; filtering the reconstructed version of the decoded picture to obtain a version of the decoded picture to be inserted into the decoded picture buffer, DPB, of the video decoder by using an in-loop filter; and parametrizing the in-loop filter by reading in-loop filter control information for parametrizing the in-loop filter from parameter sets (<NUM>, <NUM>), e.g., ALF APS and ALF per CTU APS, located within the access unit, AU, of the decoded picture which follow, along data stream order, the video coding units (<NUM>), so as to filter the reconstructed version of the decoded picture in a manner depending on the in-loop filter control information.

Preferred embodiments of the present application are described below with respect to the figures, among which:.

Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals.

In the following description, a plurality of details is set forth to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to one skilled in the art that embodiments of the present application may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present application. In addition, features of the different embodiments described hereinafter may be combined with each other, unless specifically noted otherwise.

In the following, it should be noted that individual aspects described herein can be used individually or in combination. Thus, details can be added to each of said individual aspects without adding details to another one of said aspects.

It should also be noted that the present disclosure describes, explicitly or implicitly, features usable in a video decoder (apparatus for providing a decoded representation of a video signal on the basis of an encoded representation). Thus, any of the features described herein can be used in the context of a video decoder.

Moreover, features and functionalities disclosed herein relating to a method can also be used in an apparatus (configured to perform such functionality). Furthermore, any features and functionalities disclosed herein with respect to an apparatus can also be used in a corresponding method. In other words, the methods disclosed herein can be supplemented by any of the features and functionalities described with respect to the apparatuses.

The following description of the figures starts with a presentation of a description of video encoder and video decoder of a block-based predictive codec for coding pictures of a video in order to form an example for a coding framework into which embodiments for a layered video data stream codec may be built in. The video encoder and video decoder are described with respect to <FIG>. Thereinafter the description of embodiments of the concept of the layered video data stream codec of the present application are presented along with a description as to how such concepts could be built into the video encoder and decoder of <FIG> and <FIG>, respectively, although the embodiments subsequently described, may also be used to form video encoder and video decoders not operating according to the coding framework underlying the video encoder and video decoder of <FIG> and <FIG>.

<FIG> shows a block diagram of an apparatus for predictively coding a video as an example for a video decoder where a motion compensated prediction for inter-predicted blocks according to embodiments of the present application could be implemented. That is, <FIG> shows an apparatus for predictively coding a video <NUM> composed of a sequence of pictures <NUM> into a data stream <NUM>. Block-wise predictive coding is used to this end. Further, transform-based residual coding is exemplarily used. The apparatus, or encoder, is indicated using reference sign <NUM>.

<FIG> shows a block diagram of an apparatus for predictively decoding a video as an example for a video decoder where a motion compensated prediction for inter-predicted blocks according to embodiments of the present application could be implemented. That is, <FIG> shows a corresponding decoder <NUM>, i.e. an apparatus <NUM> configured to predictively decode the video <NUM>' composed of pictures <NUM>' in picture blocks from the data stream <NUM>, also here exemplarily using transform-based residual decoding, wherein the apostrophe has been used to indicate that the pictures <NUM>' and video <NUM>', respectively, as reconstructed by decoder <NUM> deviate from pictures <NUM> originally encoded by apparatus <NUM> in terms of coding loss introduced by a quantization of the prediction residual signal. <FIG> and <FIG> exemplarily use transform based prediction residual coding, although embodiments of the present application are not restricted to this kind of prediction residual coding. This is true for other details described with respect to <FIG> and <FIG>, too, as will be outlined hereinafter.

The encoder <NUM> is configured to subject the prediction residual signal to spatial-to-spectral transformation and to encode the prediction residual signal, thus obtained, into the data stream <NUM>. Likewise, the decoder <NUM> is configured to decode the prediction residual signal from the data stream <NUM> and subject the prediction residual signal thus obtained to spectral-to-spatial transformation.

Internally, the encoder <NUM> may comprise a prediction residual signal former <NUM> which generates a prediction residual <NUM> so as to measure a deviation of a prediction signal <NUM> from the original signal, i.e. video <NUM> or a current picture <NUM>. The prediction residual signal former <NUM> may, for instance, be a subtractor which subtracts the prediction signal from the original signal, i.e. current picture <NUM>. The encoder <NUM> then further comprises a transformer <NUM> which subjects the prediction residual signal <NUM> to a spatial-to-spectral transformation to obtain a spectral-domain prediction residual signal <NUM>' which is then subject to quantization by a quantizer <NUM>, also comprised by encoder <NUM>. The thus quantized prediction residual signal <NUM>" is coded into data stream <NUM>. To this end, encoder <NUM> may optionally comprise an entropy coder <NUM> which entropy codes the prediction residual signal as transformed and quantized into data stream <NUM>. The prediction residual <NUM> is generated by a prediction stage <NUM> of encoder <NUM> on the basis of the prediction residual signal <NUM>" decoded into and decodable from, data stream <NUM>. To this end, the prediction stage <NUM> may internally comprise a dequantizer <NUM> which dequantizes prediction residual signal <NUM>" so as to gain spectral-domain prediction residual signal <NUM>‴, which corresponds to signal <NUM>' except for quantization loss, followed by an inverse transformer <NUM> which subjects the latter prediction residual signal <NUM>‴ to an inverse transformation, i.e. a spectral-to-spatial transformation, to obtain prediction residual signal <NUM>"", which corresponds to the original prediction residual signal <NUM> except for quantization loss. A combiner <NUM> of the prediction stage <NUM> then recombines, such as by addition, the prediction signal <NUM> and the prediction residual signal <NUM>"" so as to obtain a reconstructed signal 46a, i.e. a reconstruction of the original signal <NUM> (reconstructed version). Reconstructed signal 46a may correspond to signal <NUM>'.

An in-loop filter <NUM> filters the reconstructed signal 46a to obtain a version of the decoded picture, e.g., the currently decoded picture or the following decoded picture, decoded signal 46b, to be inserted into the decoded picture buffer, DPB, <NUM>.

A prediction module <NUM> of prediction stage <NUM> then generates the prediction signal <NUM> on the basis of signal 46b by using, for instance, spatial prediction, i.e. intra prediction, and/or temporal prediction, i.e. inter prediction. Details in this regard are described in the following.

The decoder <NUM> comprises a decoding core <NUM> comprising an entropy decoder <NUM>, a dequantizer <NUM>, an inverse transformer <NUM>, combiner <NUM> and a prediction module <NUM>, and an in-loop filter <NUM> and a DPB <NUM>.

Likewise, decoder <NUM> may be internally composed of components corresponding to, and interconnected in a manner corresponding to, prediction stage <NUM>. In particular, entropy decoder <NUM> of decoder <NUM> may entropy decode the quantized spectral-domain prediction residual signal <NUM>" from the data stream, whereupon dequantizer <NUM>, inverse transformer <NUM>, combiner <NUM> and prediction module <NUM>, interconnected and cooperating in the manner described above with respect to the modules of prediction stage <NUM>, recover the reconstructed signal on the basis of prediction residual signal <NUM>" so that, as shown in <FIG>, the output of combiner <NUM> results in the reconstructed signal, namely the video <NUM>'or a current picture <NUM>' thereof.

Although not specifically described above, it is readily clear that the encoder <NUM> may set some coding parameters including, for instance, prediction modes, motion parameters and the like, according to some optimization scheme such as, for instance, in a manner optimizing some rate and distortion related criterion, i.e. coding cost, and/or using some rate control. As described in more details below, encoder <NUM> and decoder <NUM> and the corresponding modules <NUM>, <NUM>, respectively, support different prediction modes such as intra-coding modes and inter-coding modes which form a kind of set or pool of primitive prediction modes based on which the predictions of picture blocks are composed in a manner described in more detail below. The granularity at which encoder and decoder switch between these prediction compositions may correspond to a subdivision of the pictures <NUM> and <NUM>', respectively, into blocks. Note that some of these blocks may be blocks being solely intra-coded and some blocks may be blocks solely being inter-coded and, optionally, even further blocks may be blocks obtained using both intra-coding and inter-coding, but details are set-out hereinafter. According to intra-coding mode, a prediction signal for a block is obtained on the basis of a spatial, already coded/decoded neighbourhood of the respective block. Several intra-coding sub-modes may exist the selection among which, quasi, represents a kind of intra prediction parameter. There may be directional or angular intra-coding sub-modes according to which the prediction signal for the respective block is filled by extrapolating the sample values of the neighbourhood along a certain direction which is specific for the respective directional intra-coding sub-mode, into the respective block. The intra-coding sub-modes may, for instance, also comprise one or more further sub-modes such as a DC coding mode, according to which the prediction signal for the respective block assigns a DC value to all samples within the respective block, and/or a planar intra-coding mode according to which the prediction signal of the respective block is approximated or determined to be a spatial distribution of sample values described by a two-dimensional linear function over the sample positions of the respective block with deriving tilt and offset of the plane defined by the two-dimensional linear function on the basis of the neighbouring samples. Compared thereto, according to inter-prediction mode, a prediction signal for a block may be obtained, for instance, by temporally predicting the block inner. For parametrization of an inter-prediction mode, motion vectors may be signalled within the data stream, the motion vectors indicating the spatial displacement of the portion of a previously coded picture of the video <NUM> at which the previously coded/decoded picture is sampled in order to obtain the prediction signal for the respective block. This means, in addition to the residual signal coding comprised by data stream <NUM>, such as the entropy-coded transform coefficient levels representing the quantized spectral-domain prediction residual signal <NUM>", data stream <NUM> may have encoded thereinto prediction related parameters for assigning to the blocks prediction modes, prediction parameters for the assigned prediction modes, such as motion parameters for inter-prediction modes, and, optionally, further parameters which control a composition of the final prediction signal for the blocks using the assigned prediction modes and prediction parameters as will be outlined in more detail below. Additionally, the data stream may comprise parameters controlling and signalling the subdivision of picture <NUM> and <NUM>', respectively, into the blocks. The decoder <NUM> uses these parameters to subdivide the picture in the same manner as the encoder did, to assign the same prediction modes and parameters to the blocks, and to perform the same prediction to result in the same prediction signal.

<FIG> shows a schematic diagram illustrating an example for a relationship between a prediction residual signal, a prediction signal and a reconstructed signal so as to illustrate possibilities of setting subdivisions of defining the prediction signal, handling the prediction residual signal and the like, respectively. That is, <FIG> illustrates the relationship between the reconstructed signal, i.e. the reconstructed picture <NUM>', on the one hand, and the combination of the prediction residual signal <NUM>‴′ as signalled in the data stream, and the prediction signal <NUM>, on the other hand. As already denoted above, the combination may be an addition. The prediction signal <NUM> is a subdivision of the picture area into blocks <NUM> of varying size, although this is merely an example. The subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of blocks, or a multi-tree subdivision of picture <NUM> into leaf blocks of varying size, such as a quadtree subdivision or the like, wherein a mixture thereof where the picture area is firstly subdivided into rows and columns of tree-root blocks which are then further subdivided in accordance with a recursive multi-tree subdivisioning to result into blocks <NUM>.

In the following each aspect of the present invention of the present application is explained.

According to one aspect of the present invention of the present application, it makes possible an encoder to start sending parts of the picture (e.g. slices) before finishing the encoding process of the whole picture, while still using slices. This is achieved by allowing an Adaptation Parameter Set (APS) to be sent after the coded slices of a picture moving per-CTU (Coding Tree Unit) ALF parameters behind the actual slice data.

<FIG> shows an illustration of a state-of-the-art encoder. First the whole picture is encoded (intra-prediction, motion estimation, residual encoding, etc.), then the ALF estimation process is started. ALF filter coefficients are written to the APS, then the slice data can be written including ALF per CTU parameters (which are interspersed with other parameters). The picture can only be sent after the ALF encoding is finished.

<FIG> shows an example of a low-delay encoder envisioned by this invention. Slices can be sent out before the picture encoding process is finished. Especially the APS carrying the coefficients is moved behind the slice data (VCL NAL units).

In this process, the encoder can send out the coded slices of the picture first, while collecting the estimated ALF parameters (filter coefficients, filter control information) and then the APS containing the ALF parameters after the coded picture. The decoder can start parsing and decoding the slices of the picture, as soon as they arrive. Since ALF is one of the last decoding steps, the ALF parameters can arrive after the coded picture to be applied after the other decoding steps.

The invention includes the following aspects:.

Typically, not only the derivation the ALF parameters (filter coefficients) is carried out towards the end of the encoding process (based reconstructed sample values), but also further ALF control information (info regarding whether a Coding Tree Unit, CTU, is filtered or not and how it is filtered) is derived at this stage. The ALF control information is carried in several syntax elements per coding_tree_unit in the slice payload, interspersed with the block splitting (e.g., as indicated in <FIG>), transform coefficients and so on. For instance, the following syntax elements might be present:.

All this ALF control information depends on the derivation of the filter parameters of ALF towards the end of the encoding process of a picture.

In one arrangement, the ALF control information is signalled in a separate loop over the CTUs of a slice at the end of the respective slice payload so that an encoder can finalize the first part of the slice payload (transform coefficients, block structure, etc.) before ALF is carried out. This is illustrated in <FIG> and <FIG>.

As indicated in <FIG>, a video coding unit (VCL NAL unit) <NUM> comprises a slice header, slice data <NUM> and a portion (ALF per CUT APS) <NUM>, and one or more parameter set (ALF coefficients) <NUM> is separately signalled, i.e., as a suffix APS (in a non-VCL-NAL unit). That is, each video coding unit <NUM> is continuously arithmetically coded along the data stream order across the data to the end of the portion.

<FIG> shows that the portion <NUM> is interspersed with the slice data <NUM>. That is, ALF per CUT APS is interspersed with the block and one or more parameter set <NUM> is separately signalled as a suffix APS.

In another arrangement, the slice header would indicate that the ALF control information are not indicated in the syntax elements within the coded slice payload, i.e. in the above described CTU loop, but that the ALF control information is included into the suffix APS, i.e. in a separate loop over all CTUs in the respective suffix APS, e.g. through the referred to APS being of a suffix APS type.

In an embodiment, the slice header would indicate that the ALF control information are not indicated in the syntax elements within the coded slice payload, i.e. in the above described CTU loop, but that the ALF control information is included into a new type of suffix APS, which is different from the suffix APS that carries the ALF coefficients, i.e. in a separate loop over all CTUs in the respective suffix APS, e.g. through the referred to APS being of a suffix APS type. The per-CTU data can optionally be CABAC encoded. This embodiment is illustrated in <FIG>.

As indicated in <FIG>, data units are signalled in a data stream order, a video coding unit (VCL NAL unit) <NUM> including a slice header and a slice data <NUM>, a parameter set <NUM> (Suffix ALF CTU-data APS: non-VCL NAL unit), a further video coding unit <NUM>, a further parameter set <NUM>, and a parameter set (filter control information) <NUM> (Suffix ALF coefficient APS: non-VCLNAL unit). That is, contrary to the data stream indicated in <FIG> and <FIG>, the filter coefficients are not necessary to be signalled following every video coding unit. In other words, the filter coefficients may be sent collectively for more than one video coding unit <NUM> following them in bitstream order or be further used by further video coding units following the filter coefficients in bitstream order.

In another embodiment, the slice header that refers to a suffix APS and all CTUs are inferred to having the adaptive loop filter applied with the filter parameters signalled in the suffix APS and default values for the ALF control information.

In the following, to another aspect of the present invention of the present application, i.e., a method for easier access to a list of all parameter sets that are referred to in the picture is described.

According to this aspect of the present invention of the present application, a decoder can easily determine, if all necessary parameter sets are available before starting to decode.

An example syntax is shown in <FIG>, i.e., in a predetermined unit, i.e., AUD, a plurality of identifiers <NUM> is included. For example, an identifier of VPS "aud_vps_id", an identifier of DPS "aud_dps_id", an identifier of SPS "aud_sps_id", an identifier of PPS "aud_pps_id", etc..

APSs are referred to by each slice of a picture. When combining bitstreams, different APSs may need to be rewritten and/or combined.

To avoid rewriting of slice headers, the APS IDs are signalled in the Access Unit delimiter instead of the slice header. So, in case of changes, the slice header does not have to be rewritten. Rewriting the Access Unit delimiter is a much easier operation.

In another embodiment, the APS IDs are only sent in the AUD conditioned on another syntax element. If the syntax elements indicate, that the APS IDs are not present in the AUD, the APS IDs are present in the slice header. An example syntax is shown in <FIG>. That is, as depicted in <FIG>, the AUD includes a flag <NUM>, e.g., a syntax "aps_ids_in_aud_enabled_flag".

<FIG> show a schematic illustration of examples indicating the relationship between the predetermined coding parameters and the AU according to the above mentioned embodiments shown in <FIG>.

As depicted in <FIG>, the plurality of parameter sets <NUM> comprises one or more first predetermined parameter sets <NUM>, e.g. including APA and PPS, and one or more of second predetermined parameter sets <NUM>, e.g., including SPS, DPS and VPS. The second predetermined parameter sets <NUM> belongs to a higher hierarchy level than the first predetermined parameter sets <NUM>. As shown in <FIG>, the AU comprises a plurality of slice data, e.g., VCL <NUM> to VCL n, and the first and second predetermined parameter sets are contained by the predetermined parameter sets <NUM>, e.g., the parameter set for VCL <NUM>.

The plurality of parameter sets <NUM> is stored in the AUD of the AU and signalled to the decoder.

In case a flag <NUM> is included in the AUD as indicated in <FIG>, the flag <NUM> indicative <NUM> of whether either predetermined identifiers of the identifiers <NUM> which refer to specific predetermined parameter sets 126b are present in the predetermined unit <NUM>, or the predetermined identifiers which refer to the specific predetermined parameter sets 126b are present in the one or more video coding units <NUM>. That is, the flag <NUM> is indicative (depicted as arrow in <FIG>) whether the APS 126b is in the AUD or VLC.

As depicted in <FIG>, the first predetermined parameter sets <NUM> comprises a third predetermined parameter sets 126a, e.g., PPS, which are referred to by identifiers in the one or more video coding units <NUM>, e.g., AU, and fourth predetermined parameter sets 126b, e.g., APS, which are referred to by identifiers <NUM> (as shown in <FIG> and <FIG>) present in the predetermined unit <NUM>, e.g., AUD, but are neither referred to by any of the identifiers in the one or more video coding units <NUM> (AU), nor by any of the predetermined parameter sets.

Currently the AUD indicates whether the following slices are of type I, B, or P. In most systems this feature is not very useful as I pictures do not necessarily mean there is a Random Access Point. Prioritization of AUs if some need to be dropped can be typically done by other means, e.g. parsing the temporal ID, parsing whether they are discardable pictures (not referenced by any other) and so on.

Instead of indicating the picture type, the NAL unit type could be indicated as well as the fact whether they are discardable pictures, etc. In addition, in the multi-layer case, the properties might be more difficult to be described:.

Therefore, in an embodiment depicted in <FIG>, the AUD indicates whether the information applies to a single layer or to all layers. That is, in the AUD flags refer to information regarding the AU properties as indicated below, for instance:.

<FIG> depicts an indication in the AUD, i.e., the video coding type of the video coding units is indicated by describing a random access property of a multiple pictures. That is, a syntax "random access info_present flag" indicates a random access property of the picture, e.g., as indicated in <FIG> "all_pics_in_au_random_access_flag", specified by an overall NAL unit type (e.g. IDR, CRA, etc.) to be used for all VCL NAL units in the access unit, instead of the one specifies in the NAL unit header.

An example syntax is shown in <FIG> according to an embodiment of the present invention.

In this example, the parameter set <NUM>, i.e., implementation "layer_specific_aud_flag" indicated whether the information in the AUD applies to all layers, "dependent_aud_flag" indicated whether the AUD starts a new global access unit, or only a "layer-access unit". In a dependent AUD, inheritance from the base layer AUD is indicated by "aud_inheritance_flag".

The inventive data stream can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the internet.

Depending on certain implementation requirements, embodiments of the application can be implemented in hardware or in software.

Generally, embodiments of the present application can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.

Other embodiments comprise a computer program for performing one of the methods described herein, stored on a machine readable carrier.

The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example via the internet.

Claim 1:
Video decoder comprising:
a decoding core (<NUM>) configured to reconstruct a decoded picture from video coding units (<NUM>) within an access unit, AU, of a video data stream to obtain a reconstructed version (46a) of the decoded picture, wherein the decoding core (<NUM>) is configured to start decoding parts of a picture before receiving all parts of the picture;
an in-loop filter (<NUM>) configured to filter the reconstructed version of the decoded picture to obtain a filtered version (46b) of the decoded picture; and
a parametrizer configured to parametrize the in-loop filter by:
reading one or more filter coefficients for parametrizing the in-loop filter in terms of transfer function from a predetermined parameter set (<NUM>) which follows, along data stream order, all of the video coding units (<NUM>) and which is located within the access unit, AU, of the decoded picture;
reading spatially selective in-loop filter control information for spatially varying the filtering of the reconstructed version of the decoded picture by the in-loop filter from, for each of the video coding units (<NUM>), a further predetermined parameter set (<NUM>) which follows in data stream order the respective video coding unit (<NUM>) and which is located within the access unit, AU, of the decoded picture and which comprises spatially selective in-loop filter control information for spatially varying the filtering of the reconstructed version of the decoded picture by the in-loop filter within a portion of the picture which is encoded into the respective video coding unit (<NUM>); and
parametrizing the in-loop filter (<NUM>) so as to filter the reconstructed version (46a) of the decoded picture in a manner depending on the one or more filter coefficients and the spatially selective in-loop filter control information.