Patent Description:
Video codecs specifications such as the HEVC base specification [<NUM>] and its extensions such as the multi-view extension [<NUM>] [<NUM>] usually inherit mechanisms to signal the layer to which each coded chunk of video data belongs to in a layered coding scenario. The signaling mechanism needs to be readily available, cp. the layer identifier in den NAL unit header (nuh_layer_id) in HEVC as given in the table in <FIG>.

With its finite size in terms of bits, the respective HEVC syntax element allows for <NUM> values to be used to identify scalable layers, views and/or depth. For multi-view video using the current HEVC NAL unit header syntax, this for example implies a constraint of <NUM> views or <NUM> view plus depth combinations as maximum. While this is sufficient for many common stereo view scenarios and other applications, huge camera arrays with more than <NUM> views for applications like light field imaging [<NUM>], [<NUM>] or holographic displays [<NUM>] require an extendable mechanism for signaling a larger space of layer identifiers.

Furthermore, structuring the space of layer identifiers by grouping certain layers or views with respect to their coding dependencies or spatial relation to each other may be of benefit. Further, providing a compact representation of the coding dependencies in the structured layer identifier space is also of benefit as deriving such information from already available information within the coded video bitstream may require intolerable computational resources.

JCT3V-D0211 proposed an extension mechanism for layer identifiers in increase the number of supported layers in MV-HEVC and 3D-HEVC. The bit length of an extension layer-ID field is mentioned using layer_id_ext_len.

Thus, it is the object of the present application to provide a concept for the signaling of at least one characteristic for layers in connection with multi-layered coding, which is more efficient in terms of, for example, reduced side information overhead and/or improved backward compatibility. This object is achieved by the subject matter of the pending independent claims.

One aspect of the present application is concerned with the signaling of at least one characteristic for layers of a multi-layered video signal such as, for example, for each layer the indication of dependent layers to which the respective layer directly relates via inter-layer prediction, or the signaling of the afore-mentioned second inter-dependency syntax structure. In accordance with this aspect, a maximum syntax element is signaled within the multi-layered video signal to indicate a maximally used value of an extension layer-ID field of the packets of the multi-layered video signal, the scope of the maximum syntax element being, for example, a predetermined portion of the multi-layered video signal extending, for example, across several portions of the multi-layered video signal. Accordingly, according to this aspect it is feasible for devices such as decoders or network elements receiving the multi-layered video signal to gain, for a relatively large predetermined portion of the multi-layered video signal, knowledge about the actually consumed portion of the possible domain of possible values signalizable by the extension layer-ID field and the at least one characteristic does not need to be signaled for each combination of base layer-ID field value and extension layer-ID field value, but rather it is sufficient to signal the at least one characteristic for a maximum number of layers determined based on the maximum assumed value. Accordingly, the at least one characteristic does not need to be transmitted/signalized for layers with a layer ID, the extension layer-ID of which does not occur within the predetermined portion of the multi-layered video signal. Beyond this, in accordance with a further embodiment, the knowledge of the maximally assumed value may be used to reduce the side information overhead for signaling the layer-ID of each portion, i.e. for reducing the bits needed to signal the extension layer-ID field within the multi-layered video signal's packets.

One of the aspects of the present application is concerned with the signalization of the inter-layer dependencies between layers of a multi-layered data stream. According to this aspect, a good compromise between a too intensive restriction of the potential diversity of inter-layer dependencies on the one hand and a too complex signaling of the inter-layer dependencies on the other hand has been found by describing the inter-layer dependencies by way of a first inter-dependency syntax structure indicating inter-dependencies between pairs of different values representable by a base layer-ID and a second inter-dependency syntax structure indicating inter-dependencies between pairs of different values representable by an extension layer-ID, the base layer ID and extension layer ID indexing the layers the portions of the multi-layer data stream are associated with. In accordance with this concept, emphasis may be shifted between increased diversity of the signalizable inter-layer dependencies on the one hand and reduced side-information overhead for signaling the inter-layer dependencies on the other hand: for example, calling the sets of layers having a common base-layer ID, respectively, "clusters", the same second inter-dependency syntax structure may be used to regulate the inter-dependencies within all clusters and between all clusters related to each other via the first inter-dependency syntax structure, separately. Alternatively, two instantiations of the second inter-dependency syntax structure may be used to describe the inter-dependencies of the layers within the clusters on the one hand and between the layers of different clusters, on the other hand. Irrespective of the emphasis placed towards increased diversity or reduced side information overhead, the inter-dependency signaling concept results in keeping the signaling overhead low.

One of the aspects of the present application is concerned with the signaling of the layer ID which each of the packets of a multi-layered video signal is associated with. In particular, this aspect achieves an efficient way of signaling this layer association, nevertheless maintaining the backward compatibility with codecs according to which a certain value of the base layer-ID field is restricted to be non-extendable such as base layer-ID value <NUM> in the base layer-ID field. Instead of circumventing this restriction specifically with respect to this non-extendable base layer-ID value, the layer-ID of portions of the multi-layer data stream is signaled in an extendable manner by sub-dividing the base layer-ID field into a first sub-field and a second sub-field: whenever the first sub-field of the base layer-ID field fulfills a predetermined criterion, an extension layer-ID field is provided, and if the first sub-field of the base layer-ID field does not fulfill the predetermined criterion, the extension layer-ID field is omitted. The aforementioned non-extendable base layer-ID value is "hidden" within the group of base layer-ID values for which the first sub-field of the base layer-ID field does not fulfill the predetermined criterion, and accordingly this non-extendable base layer-ID value is not handled separately, but rendered part of the former group. Rather, if the first sub-field of the base layer-ID field fulfills the predetermined criterion, an extension value is derived from the extension layer-ID field signaled within the multi-layer data stream such that same lies within a first subset of a domain of extension values, and if the first sub-field of the base layer-ID field does not fulfill the predetermined criterion, this extension value is set to a value disjoint to the first subset of the domain of extension values. The layer which a respective portion is associated with is then indexed using the extension value as well as the cluster value which is derived from a second sub-field of the base layer-ID field. All in all, no signaling efficiency has been lost despite the maintenance of the backward compatibility.

The just outlined aspects may be combined or used individually and preferred implementations of the aspects outlined above are the subject of the dependent claims and are discussed later with respect to the figures, among which:.

Before describing certain embodiments of the present application, an exemplary multi-layered video codec environment is described with respect to <FIG>, in which the below-outlined embodiment and aspects may advantageously be used. Although this overview of an exemplary multi-layered video codec environment is not to be understood as restricting the embodiment outlined further below, the details and functionalities exemplarily provided with respect to the following figures shall be treated as describing a reservoir of possible extensions of the embodiments further outlined below so as to result in more detailed embodiments. In any case, the illustrative multi-layered video codec environment presented hereinafter renders the understanding of advantages of the below-outlined embodiment easier.

<FIG> shows illustratively a multi-layered video encoder <NUM> which receives at its input a video material <NUM>. For illustrative purposes only, the video material <NUM> is illustrated as a multi-view video signal conveying video data for each of a plurality of views such as texture and depth for each picture of the video. Generally speaking, the video material <NUM> may represent a multi-dimensional sampling of one scene. One dimension might be time t. That is, the video material <NUM> temporally samples this scene, thereby resulting in one picture per time instant. Moreover, this scene may have been captured from different viewpoints, thereby resulting in one video per view or in one picture per view for each time instant. Further, the video material <NUM> may provide for each view, or each subset of the views, in addition to the spatial sampling of the texture of the scene, i.e. the spatial sampling of the colors of the scene, a depth map of the scene - for the respective view. <FIG>, for example, differentiates between depth maps, which are illustrated using hatching, and texture maps or images, which are drawn non-hatched. However, it should be noted that none of the embodiments outlined further below are restricted to such multi-view materials. Rather, the illustration in <FIG> has only been provided for the sake of alleviating the description brought forward below.

The multi-layered video encoder <NUM> is configured to encode the video material <NUM> into a multi-layered data stream or video signal <NUM>. In particular, multi-layered video encoder <NUM> codes the video material into the multi-layer data stream <NUM> at different layers, corresponding to different levels of information amount, using inter-layer prediction. This means the following. The multi-layered video encoder <NUM> generates the multi-layered data stream such that same comprises a plurality of packets <NUM>, each of which is associated with one of different layers. The multi-layered video encoder <NUM> encodes, for example, into packets <NUM> of a certain base layer some base level of information amount of the video material <NUM> such as, for example, merely the texture of one view or the like. The association of the packets <NUM> to any of the various layers is, for example, derivable from a layer identification syntax element structure <NUM> within each packet <NUM>, wherein the layer identification syntax element structure <NUM> may, for example, be contained within a header <NUM> of the respective packet <NUM>. For example, for illustration purposes, numbers <NUM>, <NUM> and <NUM> are shown to be inscribed into packets <NUM>, thereby illustrating layers with layer-ID <NUM>, <NUM> and <NUM>, respectively. Among these layers, the layer with layer-ID <NUM> may, for example, form the lowest or base layer providing the base level of information amount. Packets <NUM> of other layers may increase the information amount available on the packets <NUM> of the base layer only in terms of one or more or various information types. For example, packets <NUM> of a certain layer may have a further view encoded therein in addition to the views already coded within the base or reference layer's packets <NUM>. Alternatively, packets <NUM> of a certain layer may have depth information on a scene of the video material <NUM>, such as depth information of a view, the texture of which has already been encoded into the packets of the base or - more generally - reference layer which may any lower layer. Likewise, surface reflectance information may be coded into the packets of a further layer, i.e. a spatial sampling of the scene with respect to the surface reflectance of the objects within that scene irrespective of the illumination circumstances. And even further, alpha blending information may be coded into the packets of a further layer, i.e. a spatial sampling of the transparency of the scene points corresponding to the individual picture samples. Packets <NUM> of a certain layer may also add at a certain color component information or may increase the spatial resolution, i.e. provide spatial resolution refinement. Similarly, packets <NUM> of a certain layer may simply provide an SNR resolution refinement, i.e. increase the signal to noise ratio of the coded video material. In order to avoid redundancy within the multi-layered data stream <NUM> as far as the various layers are concerned, inter-layer prediction is used by multi-layered video encoder <NUM>: that is, packets <NUM> of a certain layer have encoded therein a prediction residual with respect to an inter-layer prediction as obtained from one or more other layers, called reference layers with respect to the former layer. The inter-layer prediction may be applied to different entities such as syntax elements and parameters describing the video material <NUM>. For example, the prediction residual conveyed by packets <NUM> of a certain layer may correct the inter-layer prediction as obtained from the one or more reference layers in the spatial domain on a per pixel basis. Alternatively, the multi-layered video encoder <NUM> may, for example, use transform residual coding and the inter-layer prediction correction may take place in the transform domain, such as DCT domain on a per transform coefficient basis, for example. Alternatively or additionally, the multi-layered video encoder <NUM> may be of a hybrid video codec type using spatial and/or temporal prediction and the inter-layer prediction may, for example, additionally or alternatively pertain to a refinement of motion vectors. Even further, the multi-layered video encoder <NUM> may use hierarchical multi-tree subdivision in order to subdivide the pictures of video material <NUM> into coding blocks in units of which different prediction modes are applied, such as spatial and temporal prediction, transform blocks in units of which the aforementioned transform is performed on the prediction residual, and/or other blocks in units of which certain coding parameters are set for coding the video material <NUM> and instead of signaling subdivision information for a further layer anew, the subdivisioning may either be adopted completely from any of the one or more base layers, or a refinement with respect to that subdivision information may be signaled.

By way of the inter-layer dependencies among the layers, the layers of multi-layer video signal <NUM> are interrelated with each other via branches of a tree <NUM>, the nodes of which are formed by the aforementioned layers. In order to convey information about the inter-layer dependencies among the layers, i.e. render information on the inter-dependency tree <NUM> available at the decoding side, the multi-layered video signal <NUM> may have information thereon coded thereinto.

In <FIG>, for example, it is illustrated that multi-layered video encoder <NUM> intersperses into the video data stream <NUM> high-level syntax packets <NUM> which comprise or convey information <NUM>. Information <NUM> may comprise an inter-dependency syntax structure so as to describe the inter-layer dependencies in tree <NUM>.

However, the information on the tree <NUM> may, alternatively, be known or reconstructible by default at encoder and decoder. Accordingly, additionally or alternatively, the layer-related information <NUM> may comprise information related to some layer-ID extension mechanism. In particular, while for many applications a moderate number of layers is sufficient in order to form the multi-layered video signal <NUM>, some other applications may suffer from a too small number of layers signalizable by the layer identification syntax element structure <NUM>. In other words, constructing the layer identification syntax element structure <NUM> such that same would also accommodate the huge number of layers for those applications which necessitate such a high number of layers, would disadvantageously result in a high side information overhead when considering the overall multiplicity of applications, and construing the layer identification syntax element <NUM> so that same merely accommodates the moderate number of layers would exclude the other applications from being sufficiently supported by the multi-layered video codec underlying data stream <NUM>. Accordingly, some extension mechanism may be used in order to be able to switch between modes of the data stream <NUM>, where the layer identification syntax element structure <NUM> accommodates merely the lower moderate number of layers, and a mode where the layer identification syntax element structure even accommodates the high number of layers and information <NUM> may additionally or alternatively participate in the switching between these modes.

While the scope of validity of the high level syntax packet <NUM> may encompass the whole multi-layered video signal <NUM>, it is also possible that the predetermined portions to which packets <NUM> belong may be smaller, such as, for example, they may relate to chunks into which the multi-layered video signal <NUM> is temporally segmented, each chunk being a sequence of pictures. For example, using DASH or another adapted streaming protocol, multi-layered video encoder <NUM> may change the number of layers coded into bitstream <NUM> in units of the just mentioned predetermined portions, the inter-layer dependency in the form of tree structure <NUM> and/or the switching between extension or non-extension with respect to the layer identification syntax element structure <NUM>. In any case, periodically transmitting the high level syntax packets <NUM> enables recipients to, on a random access basis, be able to have multiple random access time instances to start decoding multi-layered video signal <NUM> inbetween.

<FIG> also shows exemplarily devices for receiving the multi-layered data stream. An optional network element <NUM>, for example, receives the multi-layered video data stream <NUM> and processes same in order to forward it to a recipient of the multi-layered video data stream <NUM>. Accordingly, <FIG> also shows a multi-layered video decoder <NUM> configured to decode the multi-layered video data stream <NUM>. Both entities, i.e. network element <NUM> and multi-layered video decoder <NUM> form examples for devices for processing the multi-layered data stream <NUM>.

The network element <NUM> does not need to be able to decode the multi-layered data stream <NUM> using the inter-layer prediction between the various layers thereof. Despite this, both network element <NUM> and multi-layered video decoder <NUM>, i.e. the recipient, need to be informed of the packets <NUM> association with the various layers and the inter-layer dependencies thereamong as defined by tree <NUM>. Network element <NUM> discards, for example, packets <NUM> of data stream <NUM> associated with layers where the additional information amount provided is, for example, preliminarily not needed within the further link between network element <NUM> and multi-layered video decoder <NUM>, respectively, for example, due to bitrate shortcomings within the link, the recipient's inability to reproduce the additional information amount or the like. Similarly, multi-layered video decoder <NUM> may also decide to discard some of the packets <NUM> of certain layers responsive to external parameters such as a currently available computation power, a reproduction device's display performance such as spatial resolution, maximum number of input views or the like. That is, the device <NUM>/<NUM> is able to read the layer identification syntax element structure <NUM> of packets <NUM> and, if present, to derive the inter-layer dependencies among the layers from information <NUM>, and/or to switch between the extension mode or non-extension mode with respect to the layer identification syntax element structure <NUM> responsive to the information 214and/or to read other characteristics with respect to the layers from information <NUM>.

Beyond that, the multi-layered video decoder <NUM> is able to reconstruct the video material <NUM> from the inbound data stream <NUM> up to a certain level by collecting and using the information of packets <NUM> associated with that layer and all layers to which this layer is interrelated by the inter-prediction dependencies (see the tree structure <NUM>). That is, as outlined above multi-layered video decoder <NUM> may use packets <NUM> of a certain layer as a prediction residual with respect to an inter-layer prediction which the multi-layered video decoder <NUM> derives from the packets <NUM> of the one or more reference layers of that layer. In this regard, multi-layered video decoder <NUM> may, as outlined above, be a decoder using transform residual decoding, hybrid video decoding, hierarchical multi-tree subdivisioning and/or other coding concepts.

For the sake of completeness only, <FIG> shows that multi-layered video encoder <NUM> and data stream <NUM> may exemplarily be provided such that packets <NUM> are included within data stream <NUM> in such a manner that packets <NUM> belonging to different time instances or pictures are not interleaved with each other so as to form, per time instant, consecutive access units <NUM>, each collecting the packets <NUM> belonging to the respective time instant of that access unit <NUM> so that packets <NUM> belonging to any other time instant are within the data stream <NUM> either before or subsequent to this access unit <NUM>. However, it should be noted that this restriction has been chosen merely for illustration purposes and a more relaxed arrangement of the packets <NUM> within data stream <NUM> may alternatively be chosen.

In the following, a possibility of signaling the inter-layer dependencies according to tree structure <NUM> is described in more detail. As mentioned above, this signaling may be comprised by the information <NUM>. In accordance with this possibility, the layers are grouped into clusters. Within the data stream, the inter-layer dependencies among the clusters on the one hand and the layers within the clusters on the other hand are separately signaled. Insofar, the description brought forward below represents a possibility of implementing the information <NUM> of <FIG> in order to signal the inter-layer dependencies <NUM>, but as already noted above, the details described hereinafter with respect to <FIG> should not be restricted to the details set out in <FIG>. Rather, <FIG> should be seen as a possible implementation reservoir for the description with respect to <FIG>.

<FIG> illustrates a network device <NUM>, which may be the network element <NUM> or the multi-layered video decoder <NUM> of <FIG>, and a multi-layered data stream <NUM> same is configured to process, such as data stream <NUM> of <FIG>.

<FIG> illustrates multi-layer data stream <NUM> as having coded thereinto a video material at different layers identified by layer-ID IID. Each layer corresponds to a different level of information amount. Possibilities in this regard have already been discussed with respect to <FIG>, but for the sake of an easier understanding one could think of each layer adding a certain view to data stream <NUM>. However, this is not meant to be restrictive for the description of <FIG>. Each layer may, alternatively, correspond to a combination of different measures for the information amount, such as, for example, a number of views and spatial resolution or the like.

The multi-layer data stream <NUM> is thus composed of a plurality of packets <NUM> which may, exemplarily, correspond to packets <NUM> of <FIG>. Such packets <NUM> (or <NUM>) may be substreams allowing, for example, wavefront parallel processing of the individual pictures coded into video signal <NUM>, which substreams may, in turn, be composed of smaller units such as NAL units, slices or the like. However, it is submitted herewith that packets <NUM> or packets <NUM> of <FIG> and <FIG> may also be NAL units, slices or other units.

Each packet <NUM> is associated with one of the different layers and in order to reduce the bit consumption of the multi-layer data stream <NUM>, inter-layer prediction is used so that each packet <NUM> merely adds a "residual" to packets of a certain subset of lower layers of the data stream <NUM>. As shown at <NUM>, the "lower layers" are, for each layer, illustrated by small dots.

In accordance with the embodiment shown in the <FIG>, the inter-layer prediction dependencies may be restricted as described in more detail below and illustrated in the figure using continuous lines <NUM>. In particular, the layer-ID is formed by a concatenation of two values as described in the following. In particular, the network device, which receives the multi-layer data stream <NUM>, reads, per packet <NUM>, a layer-ID syntax structure indicated using reference sign <NUM> in <FIG>, but here composed of a base layer-ID field <NUM> and, on a conditional basis, - conditionally depending on the base layer-ID field or a high-level syntax element switching on and off extension mechanism, for example - an extension layer-ID field <NUM>. Exemplarily, merely packets <NUM> having both fields are shown in the figure. But, as described above, one or more values of the base layer-ID field <NUM> may signal the absence of field <NUM> for the respective packet <NUM>. For example, the whole extension functionality could be switchable on/off in the data stream via an extension flag in the data stream, and possible conditions for necessitating or not necessitating the extension layer-ID field could be that, as described above, the base layer-ID field is zero, is or has a certain bit set or not set, or assumes a certain value within a sub-portion of field <NUM>, or base layer-ID field having a value greater than, or smaller than some value. Further examples are set out below.

Based on the layer-ID syntax structure <NUM> and <NUM>, the network device <NUM> derives the layer-ID identifying the layer the respective packet <NUM> is associated with, i.e. IID. Different possibilities will be described below. The network device <NUM> also reads, from the multi-layered data stream <NUM>, a first inter-dependency syntax structure containing, for example, the above-identified direct_dependency_flags, and a second inter-dependency syntax structure containing, for example, the below denoted direct_ext_dependency_flag and/or general_direct_ext_dependency_flags. The first inter-dependency syntax structure indicates, in a binary manner, inter-dependencies between pairs of different values representable by the base layer-ID field <NUM>, whereas the second inter-dependency syntax structure indicates, in the binary manner, inter-dependencies between pairs of different values representable by the extension layer-ID field. Based on both inter-dependency syntax structures, the network device <NUM> then forms an inter-layer dependency matrix revealing inter-layer prediction dependencies between different layers, such as the one depicted at <NUM> in the figure. The way the derivation may be done is illustrated in the following using a pseudo code using for-next loops involving the first and second inter-dependency syntax structures and is explained with respect to the following figures, too. It should be noted, however, that signaled inter-layer prediction dependencies need not to be actually used in the data stream. Rather, the signaling of possible inter-layer prediction dependencies is for instructing the decoder or other network devices to take the necessary steps that the interrelated packets are available in the appropriate order, i.e. the packets referred to by other packets according to the inter-layer prediction dependencies prior to the referring packets.

As will get clear from the description below, the construction of the inter-layer dependency matrix <NUM> may be performed such that the second inter-dependency syntax structure is applied to all instances where the inter-layer dependency matrix <NUM> relates to inter-layer prediction dependencies between layers of layer-ID having been derived from, and thus being associated with, the same value within the base layer-ID field <NUM>. In accordance with another embodiment also described below in more detail, the second inter-dependency syntax structure is read from and is transmitted within the bitstream <NUM> several times such as, for example, for each possible value of the base layer-ID field <NUM> individually or for subsets of possible values of the base layer-ID field <NUM>, while associating the various instantiations of the second inter-dependency syntax structure to the possible values of the base layer-ID field <NUM> using indexing, for example, such as using included_nuh_layer_id in the below presented example. In even other embodiments the structure is generalized in that all possible inter-layer prediction dependencies are allowed and describable by way of the second inter-dependency syntax structure, namely by transmitting same per pair of different values of the base layer-ID field for which the first inter-dependency syntax structure indicates inter-dependency existence.

In other words, and as further illustrated in <FIG>, <FIG> presented an example for signaling the inter-layer dependencies according to which separate inter-dependency syntax structures <NUM> and <NUM> were used to describe the inter-layer dependencies in an hierarchical manner with the interface between the scope of both inter-dependency syntax structures <NUM> and <NUM> coinciding with the interface between the scope of the base layer-ID and extension layer-ID provided by fields <NUM> and <NUM> within each packet of the data stream. Base layer-ID and extension layer-ID as provided by fields <NUM> and <NUM> uniquely define the layer-ID of the respective packet by which fields <NUM> and <NUM> are comprised. The set of all representable layer-IDs representable by the combination of extension layer-ID and base layer-ID are indicated within circle <NUM> by dots <NUM>. That is, each dot <NUM> corresponds to a different couple of base layer-ID and extension layer-ID. For example, the layer-ID may be the concatenation of base-layer ID and extension layer-ID. Using the base layer-ID as provided by field <NUM> only, the complete set <NUM> of layer-IDs <NUM> are subdivided into disjoint sets <NUM> of layer-IDs called clusters <NUM> in the following, wherein all layer-IDs belonging to a certain cluster <NUM> have the same base layer-ID. As described previously with respect to <FIG>, the layers associated with layer-IDs <NUM> are linked to each other in a tree-like manner due to inter-layer prediction with these inter-layer dependencies being illustrated in <FIG> using dashed lines between dots <NUM>. For the ease of understanding, only a subset of the actual inter-layer dependencies are illustrated in <FIG>.

In any case, the linking between two clusters <NUM> by way of an inter-layer dependency between a layer of the first cluster <NUM> and a layer of the second cluster <NUM> of this pair is indicated by way of the first inter-dependency syntax structure <NUM>. In other words, the first inter-dependency syntax structure <NUM> describes, coarsely or cluster-wise, the inter-dependencies between layers. In <FIG>, these inter-dependencies are illustrated using continuous lines between clusters <NUM>. All pairs of a first and a second cluster <NUM> wherein at least one layer of the first cluster is linked to one cluster in the second cluster are interlinked and so indicated in the first inter-dependency syntax structure <NUM>. The second inter-dependency syntax structure <NUM> then clarifies which of the layers of pairs of clusters <NUM> indicated as being interlinked by the first inter-dependency syntax structure <NUM>, are actually related to each other by inter-layer prediction. That is, the second inter-dependency syntax structure <NUM> clarifies the fine-granular inter-dependencies. The second inter-dependency syntax structure <NUM>, however, also defines the inter-dependencies among the layers within each cluster <NUM> individually, that is, the intra-cluster dependencies between layers. In <FIG>, for example, six clusters <NUM> are shown, thereby resulting in <NUM> possible pairs of clusters <NUM> for which the second inter-dependency syntax structure <NUM> regulates the fine-granular inter-dependencies between layers of distinct clusters <NUM>, plus six clusters for which the second inter-dependency syntax structure <NUM> may regulate the inter-dependencies internally. As was outlined above and as will be further outlined below, there may accordingly be <NUM> to <NUM> instantiations <NUM> of the second inter-dependency syntax structure at the maximum, namely one per cluster individually and one per interlinked pair of clusters <NUM>. Obviously, the second inter-dependency syntax structure <NUM> does not need to be signaled for pairs of clusters <NUM> signaled to be non-linked by the first inter-dependency syntax structure <NUM>, thereby saving valuable side information bitrate. It should be noted that in case of using the second inter-dependency syntax structure <NUM> so as to describe the inter-layer dependencies of layers of different pairs of clusters <NUM>, the numbers of layers per cluster <NUM> should be equal for all clusters <NUM>. If the layer-ID is described using the concatenation of base layer-ID and extension layer-ID, this is the case. However, it should be mentioned that theoretically the number of layers of the clusters <NUM> could vary among the clusters <NUM>. In that case, individual instantiations <NUM> of the second inter-dependency syntax structure <NUM> would, for example, be signaled within the data stream for each interlinked pair of clusters <NUM> and at least one instantiation <NUM> would be transmitted for each cluster size.

<FIG>, for example, illustrated the case where the layer-ID was obtained from base layer-ID and extension layer-ID by using the base layer-ID as the most significant digit and the extension layer-ID as the less significant digit. <FIG> also illustrated the exemplary case where one instantiation <NUM> was used to describe the inter-layer dependencies of layers within one cluster and another instantiation <NUM> was used to describe the dependencies between layers of different clusters. For the sake of completeness, it is noted that the matrix <NUM> of <FIG> has as many lines and as many columns as layer-IDs. Merely the lower half below the diagonal is filled because any layer may merely be dependent on any previous, i.e. hierarchically lower, layer by way of inter-layer prediction. In the example of <FIG>, the column number corresponds to the layer-ID of the layer which, by use of inter-layer prediction, depends on further layers, i.e. base layers, with these base layers indicated by, for example, binary ones, while binary zeroes indicate layers not participating in inter-layer predicting the respective layer, the layer-ID of which corresponds to the current column. Insofar, in the embodiment of <FIG>, the second inter-dependency syntax structure <NUM> more or less describes sub-matrices of matrix <NUM>.

As was already described above, the first and second inter-dependency syntax structures <NUM> and <NUM> may be comprised by information <NUM> within high-level packets <NUM> (compare <FIG>). <FIG> illustrates an example, where the first inter-dependency syntax structure <NUM> reveals inter-dependencies among the layer clusters as depicted at <NUM>. For example, a cluster with base layer-ID <NUM> is dependent on clusters with base layer-ID <NUM> and <NUM>.

A first instantiation <NUM> of the second inter-dependency syntax structure is also present in the data stream and regulates the intra cluster dependencies among the layers depicted in <FIG> in form of a sub-matrix <NUM>. Further, in accordance with the example of <FIG>, the data stream also comprises an instantiation <NUM> of the second inter-dependency syntax structure <NUM> which regulates the layer-wise inter-dependencies of layers of different clusters. In particular, the second instantiation may describe the dependencies between layers of different clusters via a sub-matrix <NUM> having one row per enhancement layer-ID of the referenced cluster and one column per enhancement layer-ID of the referencing cluster.

In the example of <FIG>, sub-matrix <NUM> is placed at each position where matrix <NUM> indicates an inter-dependency between clusters, i.e. where a <NUM> is positioned, and which clusters are of the same base layer-ID, i.e. cluster lying on the diagonal of matrix <NUM>, and the sub-matrix <NUM> is placed where matrix <NUM> indicates by a "<NUM>" an inter-dependency between clusters of different base layer-ID. The result is shown at <NUM>.

It should be noted that the description of the inter-layer dependencies via a matrix such as matrix <NUM> is merely one example for describing the inter-layer dependencies. Other descriptions may be used as well. The way matrices <NUM> to <NUM> are coded by the first and second inter-dependency syntax structures may by as follows: the first inter-dependency syntax structure <NUM> may signal a binary value for each coefficient of matrix <NUM> beneath and including the diagonal. The Instantiation <NUM> of the second inter-dependency syntax structure <NUM> indicating matrix <NUM>, may signal a binary value for each coefficient of matrix <NUM> beneath and excluding the diagonal. The Instantiation <NUM> of the second inter-dependency syntax structure <NUM> indicating matrix <NUM> may signal a binary value for all coefficients of matrix <NUM>.

After having described possibilities as to how to signal the inter-layer dependencies, more detailed implementations are presented hereinafter exemplarily as an extension of the HEVC standard.

In particular, the layer identifier clustering and cluster dependency signaling may be built into an existing codec as follows.

Two syntax elements nuh_layer_id and layer_id_ext may be used to group the coded layers within the video bitstream into so called clusters based on properties such as spatial relation to each other, coding dependencies or others. The structuring of layer identifiers into cluster may allow for clusters with equally structured coding dependencies within the individual clusters, i.e. the coding dependencies within all or a subset of the defined cluster are the same. Signaling the dependencies within a cluster as an additional set of dependency flags and combining them with existing dependency signaling (cp. direct_dependency_flag in the VPS extension of HEVC extensions) to determine the reference layers of a particular layer may be done as depicted in <FIG>.

direct_ext_dependency_flag[ i ][ j ] equal to <NUM> specifies that the layer with extension index j is not a direct reference layer for the layer with extension index i within the same layer cluster. direct_dependency_flag[ i ][ j ] equal to <NUM> specifies that the layer with extension index j may be a direct reference layer for the layer with extension index i within the same cluster. When direct_ext_dependency_flag[ i ][ j ] is not present for i and j in the range of <NUM> to ( <NUM> << layer_id_ext_len ) - <NUM>, it is inferred to be equal to <NUM>.

An alternative syntax may limit the loop over the direct_ext_depencency_flag syntax element with max_layer_id_ext to avoid signaling coding dependencies to unused layer identifiers.

A more flexible signaling of the coding dependencies within and between clusters may be allowed as follows:.

Note that for c) and d) a cluster can depend on itself, when there are dependencies between layers within the cluster.

a) to d) allow to structure the space of layer identifiers when the property space used for structuring, e.g. spatial position of the camera, is not uniformly used. One example is a two-dimensional camera array, which spatial density is not constant or which is not rectangular. The following gives syntax examples for signaling the definition of cluster dependency sets in multiple ways according to the above.

a) <FIG> gives an exemplary embodiment in terms of video parameter set extension syntax that defines the coding dependencies within clusters on a per cluster basis.

direct_dependency_flag[ i ][ j ] equal to <NUM> specifies that the layer or cluster with index j is not a direct reference layer or reference cluster for the layer or cluster with index i. direct_dependency_flag[ i ][ j ] equal to <NUM> specifies that the layer or cluster with index j may be a direct reference layer or reference cluster for the layer or reference cluster with index i. When direct_dependency_flag[ i ][ j ] is not present for i and j in the range of <NUM> to vps_max_layers_minus1, it is inferred to be equal to <NUM>.

general_direct_ext_dependency_flag[ i ][ j] equal to <NUM> specifies that the layer with extension index j is not a direct reference layer for the layer with extension index i within the same layer cluster. general_direct_dependency_flag[ i ][ j ] equal to <NUM> specifies that the layer with extension index j may be a direct reference layer for the layer with extension index i within the same cluster. When general_direct_ext_dependency_flag[ i ][ j ] is not present for i and j in the range of <NUM> to max_layer_id_ext, it is inferred to be equal to <NUM>.

direct_ext_dependency_flag[ i ][ j ][ k] equal to <NUM> specifies that the layer with extension index k is not a direct reference layer for the layer with extension index j within the i-th layer cluster. direct_ext_dependency_flag[ i ][ j ][ k ] equal to <NUM> specifies that the layer with extension index k may be a direct reference layer for the layer with extension index j within the i-th layer cluster. When direct_ext_dependency_flag[ i ][ j ][ k ] is not present for i, j and k in the range of <NUM> to max_layer_id_ext, it is inferred to be equal to <NUM>.

b) <FIG> gives an exemplary embodiment in terms of syntax in the video parameter set extension that defines the coding dependencies within clusters through signaling of a number of cluster dependency sets (number_of_cluster_dependency_sets), coding dependencies (direct_ext_dependency_flag) within each cluster dependency set and the applying clusters (exemplary identified through a positive flag for their respective nuh_layer_id by the given syntax element included_nuh_layer_id).

c) <FIG> gives an exemplary embodiment for hierarchical signaling of dependencies in a way that first dependencies between clusters are signaled and second only dependencies of layers within two dependent clusters are signaled.

NOTE - If layer_id_ext_len is greater than <NUM>, direct_dependency_flag[ i ][ j ] signalsdependencies between clusters, otherwise ( layer_id_ext_len is equal to <NUM>), direct_dependency_flag[ i ][ j ] signals dependencies for layers.

direct_ext_dependency_flag[ i ][ j ][ k ][ l ] equal to <NUM> specifies that the I-th layer in the j-th cluster is not a direct reference layer for the k-th layer in the i-th cluster. direct_ext_dependency_flag[ i ][ j ][ k ][ l ] equal to <NUM> specifies that the I-th layer in the j-th cluster may be direct reference layer for the k-th layer in the i-th cluster. When not present direct_ext_dependency_flag[ i ][ j ][ k ][ l ] is inferred to be equal to <NUM>.

d) <FIG> gives an exemplary embodiment for combination of b) and c).

NOTE - If layer_id_ext_len is greater than <NUM>, direct_dependency_flag[ i ][ j ] signals dependencies between clusters, otherwise ( layer_id_ext_len is equal to <NUM>), direct_dependency_flag[ i ][ j ] signals dependencies for layers.

direct_ext_dependency_flag[ n ][ k ][ l ] equal to <NUM> specifies that the l-th layer in the j-th cluster is not a direct reference layer for the k-th layer in the i-th cluster when dependency_set_applies_flag[ n ][ i ][ j ] is equal to <NUM>. direct_ext_dependency_flag[ n ][ k ][ l ] equal to <NUM> specifies that the I-th layer in the j-th cluster may be a direct reference layer for the k-th layer in the i-th cluster when dependency_set_applies_flag[ n ][ i ][ j ] is equal to <NUM>. dependency_set_applies_flag[ n ][ i ][ j ] equal to <NUM> specifies that dependencies between layers in the i-th cluster and layers in the j-th cluster are not specified by direct_ext_dependency_flag[ n ][ k ][ l ]. dependency_set_applies_flag[ n ][ i ][ j ] equal to <NUM> specifies that dependencies between layers in the i-th cluster and layers in the j-th cluster are specified by direct_ext_dependency_flag[ n ][ k ][ l ]. When not present dependency_set_applies_flag[ n ][ i ][ j ] is inferred to be equal to <NUM>. When dependency_set_applies_flag[ n ][ i ][ j ] is equal to <NUM> for a particular combination of i and j and all n in the range of <NUM> to ( number_of_cluster_dependency_sets - <NUM> ), inclusive, no layer in the j-th cluster is a direct reference layer of any layer in the i-th cluster.

Thus, in the above examples of implementing the cluster-based inter-dependency signaling of <FIG> into an HEVC extension, the packets <NUM> of <FIG> were slices or slice segments and packets <NUM> were, for example parameter sets such as video, picture or sequence parameter sets. In accordance with the syntax example of <FIG>, the first inter-dependency syntax structure <NUM> is signaled using the syntax element direct_dependency_flag for each pair of clusters i, j, with i ≤ j, wherein vps_max_layers_minus1 is the maximum number of base layer-IDs minus <NUM>. The syntax elements thus reveal the coarse matrix <NUM> of <FIG>, for example. A syntax element, called unique_cluster_dependencies_flag, <NUM> distinguishes between two options: either one instantiation of the second inter-dependency syntax structure <NUM> is submitted and applied to all base layer-IDs, or one instantiation is transmitted for each base layer-ID i. In the first case, flags general_direct_ext_dependency_flag build up a sub-matrix according to <NUM> for each base layer-ID commonly, and in the second case, flags direct_ext_dependency_flag build up a sub-matrix according to <NUM> for each base layer-ID separately. unique_cluster_dependencies_flag switches between both options. Accordingly, in the case of <FIG>, if unique_cluster_dependencies_flag equals <NUM>, the resulting sub-matrix <NUM> is placed within matrix <NUM> at each position corresponding to equal referencing and referenced cluster ID, where a <NUM> is indicated by direct_dependency_flag. The remaining positions, where the first inter-dependency syntax structure <NUM> indicate an interdependency between clusters of different cluster-IDs, may be filled using predetermined sub-matrices, i.e. ones known by all participating devices such as encoder, network element and multi-layer video decoder, by default. In the case of unique_cluster_dependencies_flag being equal to <NUM>, for each position where the first inter-dependency syntax structure <NUM> indicates a <NUM> at positions of equal referenced and referencing cluster-ID, separate sub-matrix for this very cluster-ID is signaled and positioned at the respective position in matrix <NUM>. Accordingly, max_layer_id_ext corresponds to the number of layers within each of the vps_max_layers_minus1 clusters.

In the case of <FIG>, the first inter-dependency syntax structure <NUM> is transmitted in the same manner as in the example of <FIG>: for each cluster-ID one flag per cluster ID being equal to or smaller than the respective cluster-ID. A flag cluster_dependency_sets_present_flag <NUM> switches between the first option of <FIG> using syntax elements general_direct_ext_dependency_flag as already outlined above, and a different mode according to which the number of instantiations of the second inter-dependency syntax structure <NUM> submitted in the data stream using syntax elements direct_ext_dependency_flag is explicitly signaled using a syntax element called number-of-cluster-dependency-sets. These instantiations are then associated with the cluster-IDs for which these instantiations shall apply, by explicitly listing the cluster-IDs for which each instantiation shall apply by using of syntax elements included_nuh_layer_id <NUM>. By this measure, the instantiations <NUM> of the second inter-dependency syntax structure <NUM> are not submitted for each cluster-ID. Rather, merely a smaller number of instantiations of the second inter-dependency syntax structure <NUM> are submitted in case of cluster_dependency_sets_present_flag being equal to <NUM>, associating each instantiation to the cluster-ID which same is associated with. Again, as far as inter-dependent pairs of different clusters are concerned, default rules may be applied for filling matrix <NUM> at such positions (below the cluster diagonal).

In accordance with the example of <FIG>, the signaling of the second inter-dependency syntax structure is exemplarily signaled in an interleaved manner with respect to the first inter-dependency syntax structure, but this interleaving may be left out in accordance with an alternative embodiment. In any case, in the embodiment of <FIG>, a sub-matrix <NUM> or <NUM>, i.e. an instantiation <NUM> of the second inter-dependency syntax structure <NUM>, is submitted for each pair of (equal or unequal) cluster-IDs for which the first inter-dependency syntax structure <NUM> indicates inter-cluster dependency. As is derivable from the syntax example, the transmission of sub-matrices <NUM> consumes more bits direct_ext_dependency_flag than sub-matrices <NUM>. This is derivable from the conditional clause <NUM> and conditional clause <NUM>, respectively. In other words, in transmitting instantiations of the second inter-dependency syntax structure <NUM> for cluster-internal inter-dependency regulations, where the referencing cluster-ID indicated by counter i equals the referenced cluster-ID indicated by j, direct_ext_dependency_flag is merely transmitted for combinations of the enhancement layer-ID for the reference and referencing layer for which the enhancement layer-ID of the referenced layer, here I, is smaller than the enhancement layer-ID of the enhancement layer-ID of the referenced layer, here k. That is, of sub-matrix <NUM>, direct_ext_dependency_flag is merely transmitted for the positions below the diagonal of the sub-matrix <NUM>. In the other case, i.e. for instantiations of the second inter-dependency syntax structure <NUM> for inter-dependent cluster pairs i ≠ j, the complete sub-matrix is transmitted, i.e. a flag direct_ext_dependency_flag for each position of sub-matrix <NUM>.

The embodiment of <FIG> combines the special features of the embodiments of <FIG> and <FIG>: complete sub-matrices, i.e. instantiations of the second inter-dependency syntax structure <NUM> are submitted, namely number_of_cluster_dependency_sets in number, and then each of these instantiations are associated with the grid sites of matrix <NUM>, where the respective instantiation of the second syntax structure <NUM> shall apply (among those for which the first syntax structure <NUM> indicates the presence of cluster-independency (compare <NUM>) by a one, for example.

In even other words, as described with respect to <FIG>, a plurality of instantiations <NUM> of the second inter-dependency structure <NUM> may be read from the multi-layer data stream, namley for pairs (j,k) of values <NUM><k≤j≤n<N, with N being the number of values representable by the base layer-ID. n may set to be equal to N, but as also described herein, an explicit signaling of the actually used cluster cardinality may be used so as to restrict the transmission overhead. The pairs (j,k) are traversed (see for loops over i and j at <NUM> in <FIG>) and reading the second inter-dependency structure for is performed or suppresses for pair (j,k) conditionally depending on the first inter-dependency syntax structure indicating a presence or non-presence of interdependency between this pair (see "if direct_dependency_flag[i,j]" in <FIG>, where i,j correspond to j and k, respectively), wherein the instantiation <NUM> of the second inter-dependency structure <NUM> read for pairs (j,k) with j=k, indicates which layers among those having base layer-ID j are inter-layer prediction dependent on other layers among those having base layer-ID j, and the instantiation <NUM> of the second inter-dependency structure <NUM> read for a pair of different values (j,k) with j>k, indicates which layers among those having base layer-ID j are inter-layer prediction dependent on layers among those having base layer-ID k. Similar to <FIG>, however, at may also be that one instantiation of the second inter-dependency structure <NUM> is read from the multi-layer data stream, the one instantiation of the second inter-dependency structure <NUM> indicating commonly for each cluster value pair (j,j), which layers among those having base layer-ID j are inter-layer prediction dependent on other layers among those having base layer-ID j, as it is the case in <FIG> if unique_cluster_dependencies_flag equals <NUM>, or commonly for pairs (j,k) with j unequal k, for which the first inter-dependency syntax structure (<NUM>) indicates a presence of interdependency between the respective pair, which layers among those having base layer-ID j are inter-layer prediction dependent on layers among those having base layer-ID k, respectively. Denoting the number of values representable by the extension layer-ID <NUM> as M, the second inter-dependency syntax structure may comprises one binary value for each pair (p,q) of extension values <NUM>< q<p≤m<M, in order to indicate which layers among those having base layer-ID i are inter-layer prediction dependent on other layers among those having base layer-ID i, and one binary value for each pair of extension values <NUM><p,q≤m<M, in order to indicate which layer with extension value p and having base layer-ID j is inter-layer prediction dependent on layer with extension value q and having base layer-ID k. Again, as described elsewhere in the specification, m may set to be equal to M, but as also described herein, an explicit signaling of the actually used extension cardinality may be used so as to restrict the transmission overhead. Further, as shown in <FIG> and <FIG>, an index syntax structure <NUM> may be used so as to associate instantiations of the second inter-dependency structure to the individual pairs (j,k) of base layer-ID values. Here, <FIG> reveals that the association syntax structure may comprise a flag for each pair (j,k) and the parsing of the association syntax structure may involve skipping pairs (j,k) of values <NUM><k≤j≤n<N for which the first inter-dependency syntax structure indicates the non-presence of interdependency (independency) between pair (j,k).

That is, what has been described in the above section could be denoted as a "layer identifier clustering and cluster dependency signaling". In this regard, the device may be a video decoder. The video decoder, thus operating, would be able to select packets of the bitstream identified by (the extended) layerID for decoding. The device could, however, alternatively be a network element which would be able to discard packets of a bitstream that are identified by (the extended) layerID based on external factors such as network utilization, knowledge about decoders, etc..

Naturally, above outlined concept would also impact an existing encoder of a codec to be improved by the concept: The encoder such as the one of <FIG>, would be configured to encode into a multi-layered data stream <NUM> a video material at different layers, corresponding to different levels of information amount, using inter-layer prediction, the multi-layer data stream comprising a plurality of packets <NUM>, each of which is associated with one of the different layers, each layer being indexed by a base layer ID <NUM> or a base layer ID <NUM> and an extension layer ID <NUM>, and to insert, into the multi-layered data stream, the first inter-dependency syntax structure <NUM> indicating interdependencies between pairs of different values representable by the base layer-ID <NUM>, and a second inter-dependency syntax structure <NUM> indicating inter-dependencies between pairs of different values representable by the extension layer-ID <NUM>, with setting the first and second inter-dependency syntax structures such that, based on the first and second inter-dependency syntax structures, an inter-layer dependency description <NUM> revealing possible inter-layer prediction dependencies between the different layers is constructible.

The details described next concern a possibility to signal layer-related information in an efficient manner, wherein such layer-related information may, for example, include information concerning inter-layer dependencies as outlined with respect to <FIG>. As a starting point, the situation often presented above is used, according to which the layer identification syntax element structure <NUM> comprised by each packet <NUM> (compare <FIG>) is composed of a base layer-ID field <NUM> and an extension layer-ID field <NUM> as depicted in <FIG> shows a device <NUM> for processing a multi-layered data stream such as a network element <NUM> or a multi-layered video decoder <NUM> as shown in <FIG>. The device <NUM> is shown as comprising a receiver <NUM> configured to receive a multi-layered video signal composed of a sequence of packets, each of which comprises a layer identification syntax element.

The multi-layered video signal <NUM> (corresponding to <NUM> in <FIG>) received by receiver <NUM> is composed of a sequence of packets <NUM> (corresponding to <NUM> in <FIG>), each of which comprises a layer identification syntax element structure (corresponding to <NUM> in <FIG>) comprising at least a base layer-ID field <NUM> and, unconditionally or conditionally depending on the value of a high-level syntax element or the base layer-ID field <NUM>, an extension layer-ID field <NUM>.

The packets <NUM> may, for example, be NAL units, slices, tiles, pictures or other packets into which the multi-layered video signal <NUM> is subdivided. A layer identification extender <NUM> also comprised by device <NUM> reads, for a predetermined portion of the multi-layered video signal comprising a plurality of packets <NUM>, such as portion <NUM>, a maximum syntax element from the multi-layered video signal, indicating a maximally assumed value of the extension layer-ID field <NUM> of the packets <NUM> comprised by the predetermined portion <NUM> of the multi-layered video signal. The predetermined portion <NUM> may, for example, be a coded video sequence, a chunk, a group of pictures or the like. The maximum syntax element <NUM> may be contained within a special packet of portion <NUM> such as, for example, a VPS NAL unit. For each of the packets within the predetermined portion <NUM>, the extender <NUM> determines a layer ID for the respective packet based on the layer-ID syntax structure composed of <NUM> and, conditionally, <NUM>. For example, a concatenation of both values of both syntax elements <NUM> and <NUM> may result in the layer ID.

The extender <NUM> may derive a bit length of the extension layer-ID field using the maximum syntax element as is illustrated above. Alternatively, an extra syntax element may be used to this end. Even alternatively, the number of bits for representing the extension layer-ID field <NUM> may be fixed by default.

Extender <NUM> then determines the maximum number of layers within the portion <NUM> based on the maximum assumed value. For example, extender <NUM> also reads a syntax element from the data stream <NUM> indicating for portion <NUM> the maximum assumed value for the base layer-ID field <NUM> of the packets <NUM> within portion <NUM> and combines both maximum assumed values for determining the maximum number of layers.

The extender <NUM> determines at least one characteristic for each of the maximum number of layers by iteratively parsing a layer characterizing syntax portion of the multi-layered video signal a number of times equal to the maximum number of layers. Advantageously, the data stream <NUM> does not need to signal the at least one characteristic for each possible value, i.e. not for the whole cardinality of the extension layer-ID field <NUM>, but merely for the actually used sub-portion out of this cardinality. Additionally, a maximum syntax element <NUM> may even be used so as to derive of the number of representation bits of the extension layer-ID field <NUM> within the data stream <NUM> as just-described. The "characteristic" may, for example, be the inter-layer prediction to other layers or the like.

In other words, with respect to <FIG> it has been described that it is possible to transmit within the data stream <NUM>/<NUM> an explicit signalization as to how many out of the representable states of the extension layer-ID field are actually assumed when considering all packets <NUM>/<NUM> within the predetermined portion <NUM> of interest. For example, let the extension layer-ID field <NUM> be a field of N bits, then <NUM>N extension layer-IDs <NUM> would be representable by the extension layer-ID field <NUM>. The length of field <NUM> may be fixed or may be signaled and set within the data stream by way of a certain high-level syntax element indicated using dashed lines <NUM> in <FIG>. For some reason, however, not all of the available extension layer-IDs may have been used within the predetermined portion <NUM> of interest. In <FIG>, for example, it is illustrated that merely <NUM> representable extension layer-IDs are used, namely by way of hatching.

In accordance with <FIG>, a maximum syntax element <NUM> indicates this number of actually used/assumed values of the extension layer-ID field <NUM>. Owing to this syntax element <NUM>, it is feasible to insert layer-specific information merely with respect to the actually assumed number of extension layer-IDs into the data stream rather than for the complete number of representable values of extension layer-ID field <NUM>. Thus, as illustrated at <NUM> in <FIG>, a device parsing the multi-layered video signal may, for example, parse a certain layer characterizing syntax portion <NUM> of the multi-layered video signal, merely a number of times equal to the maximum number of layers as indicated by syntax element <NUM> rather than the complete numbers of actually available/representable values of the extension layer-ID field. The side information amount consumed with the data stream may, accordingly, kept lower. For example, in the examples of <FIG>, it would be feasible to set max_layer_id_ext to be equal to the maximally assumed number of extension states as indicated by syntax element <NUM> rather than setting this value to be equal to the maximum number of representable values of the extension layer-ID field <NUM>. Thus, the "characterizing syntax portion" <NUM> may be embodied by the previously presented flag direct_ext_dependency_flag.

As already outlined above, the existence/signaling of the bit length of field <NUM> is optional. In any case, the granularity/fidelity at which syntax element <NUM> indicates the number of actually used/active extension layer-IDs is increased compared to the more coarse setting of the number of available/representable values of the extension layer-ID field <NUM> as set by syntax element <NUM>. Preferably the fidelity or granularity at which syntax element <NUM> indicates the number of actually assumed values of the extension layer-ID field <NUM> is fine enough in order to indicate the exact number of actually assumed values. Alternatively, a fidelity/granularity somewhere between one and the fidelity/granularity of the signalization of the maximum number of available/representable values of field <NUM> as provided by syntax element <NUM> would be feasible as well. In other words, the maximum syntax element <NUM> may indicates the maximally assumed value of the extension layer-ID field <NUM> in units smaller than (n-<NUM>)<NUM> with n being the bit length of the extension layer-ID field, or may even be one.

Thus, in accordance with <FIG> and <FIG> it has been described that a device, such as a network element or a multi-layered video decoder, may comprise a receiver <NUM> and an extender <NUM> wherein the receiver <NUM> is configured to receive a multi-layered video signal <NUM> composed of a sequence of packets <NUM> each of which comprises a layer identification syntax element structure composed of a base layer-ID field and, possibly merely on a conditional basis, an extension layer-ID field <NUM>, wherein the layer identification extender <NUM> is configured to read, for a predetermined portion <NUM> of the multi-layered video signal comprising a plurality of packets <NUM> out of the sequence of packets <NUM>, a maximum syntax element <NUM> from the multi-layered video signal <NUM>, the maximum syntax element <NUM> indicating a maximally assumed value of the extension layer-ID field <NUM> among the packets <NUM> comprised by the predetermined portion <NUM>. The extender <NUM> determines, for each of the packets <NUM> within the predetermined portion <NUM>, the layer-ID for the respective packet <NUM> based on the layer identification syntax element structure such as by concatenating base layer-ID and extension layer-ID of field <NUM> and <NUM> as outlined above. The extender <NUM> also determines a maximum number of layers within the predetermined portion <NUM> based on the maximally assumed value and determines at least one characteristic for each of the maximum number of layers by iteratively parsing a layer characterizing syntax portion <NUM> a number of times equal to the maximum number of layers. The "maximum number of layers" may denote the maximum number of layers per cluster when combining the embodiment of <FIG> and <FIG> with the embodiment outlined above with respect to <FIG>,in which case the determination of the "maximum number of layers" is for example directly adopting the maximally assumed value as the "maximum number of layers", and the "at least one characteristic" may be the number of flags indicating the inter-layer dependencies within the second inter-dependency syntax structure. Another example of a characteristic, however, may also be a signalization within the data stream of corresponding coding parameters set for the individual layer-IDs. In that case, the "maximum number of layers" may be determined to be equal to the number of used or representable base layer-IDs times the maximally assumed value, i.e. the number of actually used/assumed extension layer-IDs. Other examples are feasible as well. In any case, the transmission of the actually assumed value by way of syntax element <NUM> enables to save valuable side information bit rate.

The concept of <FIG> and <FIG> may, as already outlined above, be combined with the concept outlined above with respect to <FIG> or may be used isolatedly without using the concept previously described.

Similarly to the above description brought forward with respect to <FIG>, however, the following description reveals a possibility how to build the concept of <FIG> and <FIG> into an existing video codec such as an extension of HEVC.

In HEVC, a layer identifier variable, referred to as LayerlD, may be derived from the explicitly signaled layer identifier in the header of video and meta data packets, e.g. nuh_layer_id and additional information signaled in the each chunk of video or meta data packets, forming a new variable LayerId of the specific data packet to be used for its identification.

The additional information signaled according to <FIG> and <FIG> may involve an additional syntax element, cp. layer_id_ext, e.g. within some part of the header or payload of the video or meta data packet, given that the specific video or meta data packet belongs to an enhancement layer, thereby keeping compatibility with base layer only legacy devices. A default value can be assumed for layer_id_ext when the video or meta data packet belongs to the base layer but is processed by an enhancement layer processing capable device.

<FIG> gives an example for a slice-segment header that signals the additional information as the syntax element layer_id_ext in the slice header that is contained in a video data packet of enhancement layer (c. nuh_layer_id > <NUM>) of a HEVC coded video.

A mathematical operation is used in order to combine nuh_layer_id and layer_id_ext into LayerId that allows a larger identifier space and is unique for a unique combination of two values of the nuh_layer_id and layer_id_ext. The mathematical operation can for example be using the bits of nuh_layer_id as MSB and the layer_id_ext as LSB of the LayerId variable or vice versa, e.g. as following.

The variable LayerId is determined as follows, where LengthOfExtension is the length of the layer_id_ext syntax elements in terms of bits.

The concept of <FIG> and <FIG> may include explicit signaling of the amount of bits that is used to transmit the additional information (cp LengthOfExtension above) in some part of the header or payload of the video or meta data packet in order to make efficient use of the transmitted amount of data. <FIG> gives exemplary syntax and semantics of the LengthOfExtension value signaling in meta data packets such as the HEVC Video Parameter Set extension syntax.

layer-id-ext-len indicates the number of bits used for extending the LayerId range.

The concept of <FIG> and <FIG> includes explicit signaling of the maximum value of the layer_id_ext syntax element used in the coded video bitstream, e.g. max_layer_id_ext. Based thereon, the lowest possible amount of bits required to transmit the additional information (cp LengthOfExtension above) in some part of the header or payload of the video or meta data packet may be derived in order to make efficient use of the transmitted amount of data. <FIG> gives exemplary syntax and semantics of the maximum value of the layer_id_ext syntax element value signaling in meta data packets such as the HEVC Video Parameter Set extension syntax.

max_layer_id_ext indicates the maximum value of layer_id_ext syntax element in any slice header within the coded video sequence.

Two exemplary variants of the semantics of the layer_id_ext syntax element depending on the above extensions of the invention are given in the following.

layer_id_ext is used in combination with nuh_layer_id to identify the layer. The syntax element layer_id_ext is coded using layer_id_ext_len bits. When not present, its value is inferred to <NUM>.

layer_id_ext is used in combination with nuh_layer_id to identify the layer. The syntax element layer_id_ext is coded Ceil( Log2( max_layer_id_ext )) bits. When not present, its value is inferred to <NUM>.

Layer identifier of parameter set or SEI messages of the HEVC video codec standard can be extended given in <FIG> without parsing dependencies.

In a layered coding scenario, signaling that is related to the layer identifier or the maximum number thereof, is adjusted to cover the extended layer identifier space (cp. value range of LayerId) instead of the regular explicit layer identifier space (cp. value range of nuh_layer_id), as exemplarily given with the syntax table for the video parameter set extension in <FIG>, where the variable VpsMaxNumLayers is exemplarily given as follows.

The variable VpsMaxNumLayers specifying the maximum number of layer in the extended layer range is set equal to ( vps_max_layers_minus1 ) * <NUM> layer_id_ext_len. +<NUM>
or
The variable VpsMaxNumLayers specifying the maximum number of layer in the extended layer range is set equal to ( vps_max_layers_minus1 ) * max_layer_id_ext.

layer_id_in_nalu[ i ] specifies the value of the Layerld value associated with VCL NAL units of the i-th layer. For i in a range from <NUM> to VpsMaxNumLayers - <NUM>, inclusive, when not present, the value of layer_id_in_nalu[ i ] is inferred to be equal to i.

When i is greater than <NUM>, layer_id_in_nalu[ i ] shall be greater than layer_id_in_nalu[ i - <NUM> ]. For i in a range from <NUM> to vps_max_layers_minus1, inclusive, the variable LayerIdInVps[ layer_id_in_nalu[ i ] ] is set equal to i.

dimension_id[ i ][ j ] specifies the identifier of the j-th present scalability dimension type of the i-th layer. The number of bits used for the representation of dimension_id[ i ][ j ] is dimension_id_len_minus1[ j ] + <NUM> bits. When dimension_id[ i ][ j ] is not present for j in the range of <NUM> to NumScalabilityTypes - <NUM>, inclusive, dimension_id[ i ][ j ] is inferred to be equal to ( ( layer_id_in_nalu[ i ] & ( (<NUM> << dimBitOffset[ j + <NUM> ] ) - <NUM>) ) >> dimBitOffset[ j ] ).

The variable ScalabilityId[ i ][ smIdx ] specifying the identifier of the smldx-th scalability dimension type of the i-th layer, the variable ViewId[ layer_id_in_nalu[ i ] ] specifying the view identifier of the i-th layer and the variable ViewScalExtLayerFlag specifying whether the i-th layer is a view scalability extension layer are derived as follows:
<IMG>.

Naturally, above outlined concept would also impact an existing encoder of a codec to be improved by the concept: The encoder such as the one of <FIG>, would be configured to encode the video signal into the multi-layered video signal <NUM> composed of a sequence of packets <NUM> each of which comprises a layer identification syntax element structure composed of a base layer-ID field <NUM> and an extension layer-ID field <NUM>, and to insert, for a predetermined portion <NUM> of the multi-layered video signal comprising a plurality of packets, the maximum syntax element <NUM> into the multi-layered video signal, indicating the maximally assumed value of the extension layer-ID field <NUM> of the packets comprised by the predetermined portion <NUM> of the multi-layered video signal, determine a maximum number of layers within the predetermined portion of the multi-layered video signal based on the maximally assumed value, and signal at least one characteristic for each of the maximum number of layers in the multi-layered video signal <NUM> by iteratively writing a layer characterizing syntax portion <NUM> of the multi-layered video signal a number of times equal to the maximum number of layers. The encoder would, for example, delay the output of the stream <NUM> till having finished the coding of portion <NUM>.

The following figures deal with a concept of signaling the layer ID of the packets contained in the multi-layer data stream and a possibility of achieving an efficient of signaling this layer-ID using the extension mechanism without compromising backward compatibility in the case of a necessity which, due to any reasoning, requests that a certain value of the base layer ID, such as <NUM>, is excluded from the possibility of having appended thereto an extension layer-ID. The latter necessity results in the following asymmetry: for almost all possible values of the base layer-ID, appending an extension layer-ID is feasible. This results in the following asymmetry: for almost all values of the base layer-ID, the appending of an extension layer-ID is feasible, thereby resulting in the number of these almost all values of base layer-ID times the number of representable values of the extension layer-ID representable layer-ID values. The restricted base layer-ID value, however, excludes the possibility of appending any extension layer-ID. Accordingly, either this asymmetry is accepted, or the forbidden value of the base layer-ID is not used in case of using the extension mechanism. The solution set out below overcomes this problem in a more efficient way.

This time, the description of this next concept is immediately outlined with respect to a possible extension of the HEVC standard where the just outlined restriction exactly applies. However, the concept outlined below is also transferrable to other codecs which, for example, impose similar restrictions onto any layer indication extension mechanism.

A restriction of HEVC is, as outlined above, that the base layer cannot be extended. Hence to comply with the HEVC specification the layer id extension value layer_id_ext cannot be signaled for layers with nuh_layer_id equal to <NUM> and therefore the cluster associated with nuh_layer_id equal to <NUM> can only include the base layer itself. Such an unequal number of layers in different clusters might be undesirable.

The issue can be resolved by the below outlined concept, which derives clusters and layer id extension values within the clusters in a different way than described above. Since it is not possible to have multiple layers for nuh_layer_id equal to <NUM>, the restriction to not have multiple layers is extended to other nuh_layer_id values not equal to <NUM>. Layers belonging to these restricted nuh_layer_id values are then combined with layers from unrestricted nuh_layer_id values to form clusters with an equal number of layers.

The basic idea is that nuh_layer_id is split into two values by, for example, bit masking. One value is used to determine the cluster. This way different nuh_layer_id values can belong to the same cluster. The other value is utilized to determine the presence of the layer_id_ext value (hence to determine if multiple layers can belong to the nuh_layer_id value) and to determine an additional offset to the layer_id_ext_value in LayerId derivation.

An example for an embodiment of the basic idea is given in the following and illustrated in <FIG>, thus, provides an example for the relationships between layer_id_ext, nuh_layer_id and LayerId for the exemplary case of max_layer_id_ext equal to <NUM> (LengthOfExtension equal to <NUM>).

In the example only, the five most significant bits of the six bit nuh_layer_id value are used to determine the cluster, hence two nuh_layer_id values with different least significant bits belong to one cluster.

If the least significant bit is equal to <NUM>, no layer_id_ext values are signaled for the nuh_layer_id and only one layer belongs to the nuh_layer_id. By definition this layer is the layer with layer_id_ext equal to <NUM> within the cluster specified by the five most significant bits. Otherwise, if the least significant bit is equal to <NUM>, multiple layers belong to the nuh_layer_id value. For each layer a layer_id_ext value is signaled explicitly and each layer belongs to the cluster specified by the five most significant bits.

The syntax table of <FIG> might be used for the embodiment. It shows a header of a packet <NUM>, i.e. a slice segment header - or, to be more precise, a fragment thereof.

Layer_id_ext is not present when the least significant bit of nuh_layer_id is equal to <NUM>. When it is not present, it is inferred to be equal to <NUM>. The variable LayerId is determined as follows, where LengthOfExtension is the length of the layer_id_ext syntax elements in terms of bits.

In another embodiment a variable add_layer_id_ext_len is signaled in the bitstream, e.g. in the VPS. (Please note that the previously described embodiment is a special case of this embodiment with add_layer_id_ext_len equal to <NUM>). The ( <NUM> - add_layer_id_ext_len ) most significant bits of the six bit nuh_layer_id syntax element are used determine the cluster. The least add_layer_id_ext_len significant bits specify whether layer_id_ext is inferred. Furthermore they specify an offset additional to the layer_id_ext value. The syntax tables of <FIG> are used to demonstrate the embodiment:
In the embodiment layer_id_ext is only present when the add_layer_id_ext_len least significant bits of nuh_layer_id are equal to <NUM>. When layer_id_ext is not present it is inferred to be equal to <NUM>. The variable LayerId is determined as follows, where LengthOfExtension is the length of the layer_id_ext syntax elements in terms of bits.

Thus, in accordance with the concept outlined above with respect to <FIG> a device, such as a network element of multi-layered video decoder such as those depicted in <FIG>, configured to process a multi-layer data stream into which a video material is coded at different layers, corresponding to different levels of information amount, using inter-layer prediction, would do the following for each of the packets comprised by the multi-layer data stream, each packet being associated with one of different layers, as already described above.

In particular, as shown in <FIG>, the device reads the base layer-ID field <NUM> from the multi-layer data stream or, to be more precise, from the current packet. The base layer-ID field comprises a first sub-field 308a and a second sub-field 308b. For example, the first sub-field may be formed by one or more least significant bits of field <NUM>, while the second sub-field 308b is formed by the other most significant bits of field <NUM>. As indicated in the example of <FIG>, a higher-level syntax element, called add_layer_id_ext_len <NUM> may define the length of sub-field 308a.

After reading the base layer-ID field <NUM> in step <NUM>, the device checks whether the sub-field 308a fulfills a predetermined criterion in step <NUM>. For example, the criterion is fulfilled if the sub-field is one or is an "all-ones sequences". In case of the predetermined criterion being fulfilled, the device reads an extension layer-ID field <NUM> from the data stream or, to be more precise, from the current packet, in step <NUM> and derives in step <NUM> an extension value using the extension layer-ID field so that the extension value lies within a first subset of a domain of extension values. The derivation in step <NUM> may correspond to a mere copying of the bit representation within field <NUM>. It may be a requirement of the data stream that the value within extension layer-ID field <NUM> does not assume a certain value such as <NUM>. It is the latter value which is reserved for the case that the base layer-ID field's <NUM> sub-field 308a does not fulfill the criterion as will be outlined further below.

After step <NUM>, the device derives a cluster value from the second subfield 308b by mapping from a domain of possible values of the second sub-field 308b onto a domain of cluster values. The layer which the current packet is associated with, i.e. the one comprising fields <NUM> and <NUM> is then indexed using the cluster and the extension value derived in steps <NUM> and <NUM> in step <NUM>. Cluster and extension values are, for example, simply concatenated to each other using, for example, the cluster value for the most significant digit. The derivation of the cluster value <NUM> on the basis of sub-field 308b may, for example, also consist of a direct usage of the bits of sub-field 308b as a binary representation of the cluster value.

However, if the check in step <NUM> results in the predetermined criterion not being met, the device refrains from reading an extension layer-ID field from the multi-layer data stream for the current packet. Rather, the device sets the extension value onto a value disjoint to the first subset of the domain of the extension values in step <NUM>, such as by setting the extension value to zero, and derives the cluster value from the second sub-field 308b by mapping from the domain of possible values of the second sub-field 308b onto the domain of cluster values. The derivations in step <NUM> and <NUM> may be performed using the same mapping. Then, the layer-ID of the current packet is determined from the cluster and extension values as set and derived in steps <NUM> and <NUM>, wherein the determination of the layer-ID may again be performed in the same manner as if the cluster and extension values were derived in steps <NUM> and <NUM>.

Relating the description of <FIG> to the embodiment of <FIG>, the first sub-field 308a may thus be <NUM> bit long and in particular be the LSB of field <NUM>. The predetermined criterion may be fulfilled if this LSB 308a is won and the aforementioned domain of extension values spanned by extension layer-ID field <NUM> would be {<NUM>, <NUM>, <NUM>, <NUM>} as, in case of <FIG>, layer_id_ext, i.e. field <NUM>, is <NUM> bits long. Accordingly, the first subset of the domain of extension values, i.e. the codomain of the derivation in step <NUM>, would be {<NUM>, <NUM>, <NUM>}. In particular, the derivation in step <NUM> is, in the example of <FIG>, embodied by way of a unitary matrix applied to the extension layer-ID field. Alternatively, some other computation may be used, such as adding the value of the first sub-field 308a to the result of the unitary matrix application. Accordingly, in step <NUM> the extension value is set to a value disjoint to {<NUM>, <NUM>, <NUM>}, namely <NUM>. The mapping used in the derivations <NUM> and <NUM> may be the same, but may be different alternatively. The mapping may in particular be bijective.

The determination of the index in <NUM> may use the cluster value to set a more significant digit and the extension value to set a lower significant digit as illustrated in <FIG>. Thus, as illustrated in <FIG>, the layer_id_ext field, i.e. the extension layer-ID field <NUM>, is merely present in case of the LSB of nuh_layer_id, i.e. base layer-ID field <NUM>, being <NUM>. Otherwise, the layer_id_ext field is inferred to be zero. Accordingly, the mapping of layer-IDs to the combinations of layer_id_ext and nuh_layer_id is completely symmetric and there is no unused bit representation. The inferred value of layer_id_ext may be exploited in the data stream by transmitting layer_id_ext such that the bits spanned in the data stream for the transmission of layer_id_ext is reduced so to account for the fact that layer_id_ext in fact merely assumes three different values {<NUM>, <NUM>, <NUM>}. For example, a binary unary code may be used to map the three elements of {<NUM>, <NUM>, <NUM>} onto binary strings which are then, for example, encoded into the data stream, and decoded therefrom, using entropy coding/decoding.

Thus, inter alias, it has been described with respect to the latter concept, a device configured to process a multi-layered data stream into which a video material is coded at different layers, corresponding to different levels of information amount, using inter-layer prediction, the multi-layer data stream comprising a plurality of packets, each of which is associated with one of the different layers, the device configured to, for each of the packets of the multi-layered data stream, read a base layer-ID field from the multi-layered data stream, the base layer-ID field comprising a first sub-field and a second sub-field; check as to whether the first sub-field of the base layer-ID field fulfills a predetermined criterion, and.

wherein the device indexes the layer which the respective packet is associated with using the cluster and extension values. It should be noted that the predetermined criterion may be read from the multi-layer data stream. As described above, "first sub-field" may be one bit long and, in particular, LSB; "predetermined criterion" may be "if one", "domain of extension values" may be, for example, {<NUM>,<NUM>,<NUM>,<NUM>}, "first subset" may be {<NUM>,<NUM>,<NUM>}, "using the extension layer-ID field" may be a unitary matrix applied to extension layer-ID field, alternatively: adding the value of the first sub-field to the result of the unitary matrix application, "set the extension value to a value disjoint to" may be a setting "to <NUM>"; "mapping from the domain of possible values of the first sub-field onto the domain of cluster values" may be the same for both cases, but may different alternatively; the mapping may be bijective, and "index the layer which the respective packet is associated with using the cluster and extension values"may involve using, for example, the cluster value for setting a ,more significant digit and the extension value to set a lower significant digit.

The latter concept directly translates into an encoder configured to encode into a multi-layered data stream <NUM> a video material <NUM> at different layers, corresponding to different levels of information amount, using inter-layer prediction, the multi-layer data stream comprising a plurality of packets <NUM>, each of which is associated with one of the different layers, wherein the layer which the respective packet is associated with, is uniquely determined by a cluster and extension value, the encoder configured to,for each for the packets <NUM> of the multi-layered data stream, if the extension value is within a first subset of a domain of extension values, insert a base layer-ID field <NUM> into the multi-layered data stream, the base layer-ID field comprising a first sub-field 308a and a second sub-field 308b, with using the cluster value to set the second sub-field and setting the first sub-field so as to fulfill a predetermined criterion; and insert an extension layer-ID into the multi-layered data stream, with using the extension value to set the extension layer-ID; and, if the extension value equals a value disjoint to the first subset of the domain of extension values, refrain from inserting an extension layer ID field and insert <NUM> a base layer-ID field <NUM> into the multi-layered data stream, the base layer-ID field comprising a first sub-field 308a and a second sub-field 308b, with using the cluster value to set the second sub-field and setting the first sub-field so as to not fulfill the predetermined criterion. That is, the encoder generates the data stream so that the decoder may derive the correct layer-ID in the manner outlined above and uses to this end inverse mappings compared to those outlined above with respect to the decoding or receiving side.

Summarizing above description, same provided an extension mechanism for the layer identifier of enhancement layer data, the grouping of layer identifiers into clusters and an efficient signaling of coding dependencies within clusters.

The details set out above may be apply to the data of enhancement layers in a layered coding scenario, while data of the base layer is kept unchanged. The base layer of the layered coding scenario may be identified through a layer identifier (cp. nuh_layer_id) in the common header syntax of base and enhancement layer data packets.

In the above exmples, the respective device at the recipients side may be a video decoder. The video decoder, thus operating, would be able to select packets of the bitstream identified by (the extended) layerID for decoding. The device could, however, alternatively be a network element which would be able to discard packets of a bitstream that are identified by (the extended) layerID based on external factors such as network utilization, knowledge about decoders, etc..

In all of the above embodiments, the multi-layered data stream may have the video material coded there into at the different layers using inter-layer prediction such that any layer inter-layer predicted from another layer adds one or more of further views; depth information; surface reflectance information; color component information; spatial resolution refinement; and SNR resolution refinement.

In the following, additional embodiments and aspects of the invention will be described which can be used individually or in combination with any of the features and functionalities and details described herein.

According to a first aspect, a device may have: a receiver <NUM> configured to receive a multi-layered video signal <NUM> composed of a sequence of packets <NUM> each of which comprises a layer identification syntax element structure composed of a base layer-ID field <NUM> and an extension layer-ID field <NUM>; a layer identification extender <NUM> configured to read, for a predetermined portion <NUM> of the multi-layered video signal comprising a plurality of packets, a maximum syntax element <NUM> from the multi-layered video signal, indicating a maximally assumed value of the extension layer-ID field <NUM> of the packets comprised by the predetermined portion <NUM> of the multi-layered video signal, for each of the packets within the predetermined portion of the multi-layered video signal, determine the layer ID for the respective packet based on the layer identification syntax element structure, determine a maximum number of layers within the predetermined portion of the multi-layered video signal based on the maximally assumed value, and determine at least one characteristic for each of the maximum number of layers by iteratively parsing a layer characterizing syntax portion <NUM> of the multi-layered video signal a number of times equal to the maximum number of layers.

According to a second aspect when referring back to the first aspect, the multi-layered video signal may have the video material coded thereinto at the different layers using inter-layer prediction such that any layer inter-layer predicted from another layer adds one or more of further views; depth information; alpha blending information; color component information; spatial resolution refinement; and SNR resolution refinement.

According to a third aspect when referring back to the first or second aspect, the packets <NUM> may be one of the group consisting of NAL units, slices, tiles and pictures, and the predetermined portion <NUM> may be one of the group consisting of a coded video sequence, a chunk, or a group of pictures.

According to a fourth aspect when referring back to any of the first to third aspects, the device may further be configured to derive a bit length of the extension layer-ID field using the maximally assumed value.

According to a fifth aspect when referring back to any of the first to fourth aspects, the device may further be configured to derive a bit length of the extension layer-ID field by rounding up the logarithm to base <NUM> of the maximally assumed value.

According to a sixth aspect when referring back to any of the first to fifth aspects, the characteristic may relate to an inter-layer prediction dependency of the respective layer to any other of the maximum number of layers.

According to a seventh aspect when referring back to any of the first to sixth aspects, the device may be configured to obtain a bit length of the extension layer-ID field <NUM> from an explicit signaling in the bit stream, wherein the maximum syntax element <NUM> indicates the maximally assumed value of the extension layer-ID field <NUM> in units smaller than the bit length of the extension layer-ID field minus one <NUM> to the power of two.

According to an eighth aspect when referring back to any of the first to sixth aspects, the device may be configured to obtain a bit length of the extension layer-ID field <NUM> from an explicit signaling in the bit stream, wherein the maximum syntax element <NUM> indicates the maximally assumed value of the extension layer-ID field <NUM> in units of one.

According to a ninth aspect when referring back to any of the first to sixth aspects, the device may be configured to determine the layer ID for the respective packet based on the layer identification syntax element structure by concatenating the base layer-ID field and extension layer-ID field.

According to a tenth aspect when referring back to any of the first to ninth aspects, the device may be a video decoder configured to decode the multi-layered video signal using inter-layer prediction using the characteristic determined for each layer, or a network element configured to discard packets of the multi-layered video signal on the basis of the characteristic determined for each layer.

According to an eleventh aspect, a method may have the steps of: receiving a multi-layered video signal <NUM> composed of a sequence of packets <NUM> each of which comprises a layer identification syntax element structure composed of a base layer-ID field <NUM> and an extension layer-ID field <NUM>; reading, for a predetermined portion <NUM> of the multi-layered video signal comprising a plurality of packets, a maximum syntax element <NUM> from the multi-layered video signal, indicating a maximally assumed value of the extension layer-ID field <NUM> of the packets comprised by the predetermined portion <NUM> of the multi-layered video signal, for each of the packets within the predetermined portion of the multi-layered video signal, determining the layer ID for the respective packet based on the layer identification syntax element structure, determining a maximum number of layers within the predetermined portion of the multi-layered video signal based on the maximally assumed value, and determining at least one characteristic for each of the maximum number of layers by iteratively parsing a layer characterizing syntax portion <NUM> of the multi-layered video signal a number of times equal to the maximum number of layers.

According to a twelfth aspect, an encoder for encoding a video signal into a multi-layered video signal <NUM> composed of a sequence of packets <NUM> each of which comprises a layer identification syntax element structure composed of a base layer-ID field <NUM> and an extension layer-ID field <NUM>, may be configured to insert, for a predetermined portion <NUM> of the multi-layered video signal comprising a plurality of packets, a maximum syntax element <NUM> into the multi-layered video signal, indicating a maximally assumed value of the extension layer-ID field <NUM> of the packets comprised by the predetermined portion <NUM> of the multi-layered video signal, determine a maximum number of layers within the predetermined portion of the multi-layered video signal based on the maximally assumed value, and signal at least one characteristic for each of the maximum number of layers in the multi-layered video signal <NUM> by iteratively writing a layer characterizing syntax portion <NUM> of the multi-layered video signal a number of times equal to the maximum number of layers.

According to a thirteenth aspect, a method for encoding a video signal into a multi-layered video signal <NUM> composed of a sequence of packets <NUM> each of which comprises a layer identification syntax element structure composed of a base layer-ID field <NUM> and an extension layer-ID field <NUM>, may have the steps of: inserting, for a predetermined portion <NUM> of the multi-layered video signal comprising a plurality of packets, a maximum syntax element <NUM> into the multi-layered video signal, indicating a maximally assumed value of the extension layer-ID field <NUM> of the packets comprised by the predetermined portion <NUM> of the multi-layered video signal, determining a maximum number of layers within the predetermined portion of the multi-layered video signal based on the maximally assumed value, and signaling at least one characteristic for each of the maximum number of layers in the multi-layered video signal <NUM> by iteratively writing a layer characterizing syntax portion <NUM> of the multi-layered video signal a number of times equal to the maximum number of layers.

According to a fourteenth aspect, a computer program may have a program code configured to perform, when running on a computer, a method according to the eleventh or thirteenth aspect.

The inventive encoded signals mentioned above can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Claim 1:
Device comprising:
a receiver (<NUM>) configured to receive a multi-layered video signal (<NUM>) composed of a sequence of packets (<NUM>) each of which comprises a layer identification syntax element structure composed of a base layer-ID field (<NUM>) and an extension layer-ID field (<NUM>);
a layer identification extender (<NUM>) configured to
read, for a predetermined portion (<NUM>) of the multi-layered video signal comprising a plurality of packets, a maximum syntax element (<NUM>) from the multi-layered video signal, indicating a maximally assumed value of the extension layer-ID field (<NUM>) of the packets comprised by the predetermined portion (<NUM>) of the multi-layered video signal,
for each of the packets within the predetermined portion of the multi-layered video signal, determine the layer ID for the respective packet based on the layer identification syntax element structure,
determine a maximum number of layers within the predetermined portion of the multi-layered video signal based on the maximally assumed value, and
determine at least one characteristic for each of the maximum number of layers by iteratively parsing a layer characterizing syntax portion (<NUM>) of the multi-layered video signal a number of times equal to the maximum number of layers,
wherein the at least one characteristic signals coding parameters which are set for each of the maximum number of layers.