Patent Description:
Another embodiment according to the invention is related to a method for providing at least two output audio signals on the basis of an encoded representation.

Another embodiment according to the present invention is related to a computer program for performing the method.

Generally, some embodiments according to the invention are related to a combined residual and parametric coding.

In recent years, demand for storage and transmission of audio content has been steadily increasing. Moreover, the quality requirements for the storage and transmission of audio contents have also been increasing steadily. Accordingly, the concepts for the encoding and decoding of audio content have been enhanced. For example, the so-called "advanced audio coding" (AAC) has been developed, which is described, for example, in the international standard ISO/IEC <NUM>-<NUM>: <NUM>.

Moreover, some spatial extensions have been created, like, for example, the so-called "MPEG surround" concept, which is described, for example, in the international standard ISO/IEC <NUM>-<NUM>:<NUM>. Moreover additional improvements for the encoding and decoding of a spatial information of audio signals are described in the international standard ISO/IEC <NUM>-<NUM>:<NUM>, which relates to the so-called spatial audio object coding. Moreover, a flexible (switchable) audio encoding/decoding concept, which provides the possibility to encode both general audio signals and speech signals with good coding efficiency and to handle multi-channel audio signals is defined in the international standard ISO/IEC <NUM>-<NUM>:<NUM>, which describes the so-called "unified speech and audio coding" concept. <CIT> discloses a parametric stereo upmix apparatus using a downmix signal, a residual signal an a decorrelated signal.

However, there is a desire to provide an even more advanced concept for an efficient decoding of multi-channel audio signals.

According to the invention there is provided a multi-channel audio decoder for providing at least two output signals on the basis of an encoded representation as set forth in claim <NUM>. There are also provided a method as set forth in claim <NUM> and a computer program as set forth in claim <NUM>. The invention is based on the finding that output audio signals can be obtained on the basis of an encoded representation in a very efficient way if a weight describing a contribution of the decorrelated signal to the weighted combination of a downmix signal, a decorrelated signal and a residual signal is adjusted in dependence on the residual signal. Accordingly, by adjusting the weight describing the contribution of the decorrelated signal in the weighted combination in dependence on the residual signal, it is possible to blend (or fade) between a parametric coding (or a mainly parametric coding) and a residual coding (or mostly residual coding) without transmitting an additional control information. Moreover it has been found out, that the residual signal, which is included in the encoded representation, is a good indication for the weight describing the contribution of the decorrelated signal in the weighted combination, since it is typically preferable to put a (comparatively) higher weight on the decorrelated signal if the residual signal is (comparatively) weak (or insufficient for a reconstruction of the desired energy) and to put a (comparatively) smaller weight on the decorrelated signal if the residual signal is (comparatively) strong (or sufficient to reconstruct the desired energy). Accordingly, the concept mentioned above allows for a gradual transition between a parametric coding (wherein, for example, desired energy characteristics and/or correlation characteristics are signaled by parameters and reconstructed by adding a decorrelated signal) and a residual coding (wherein the residual signal is used to reconstruct to output audio signals - in some cases even the waveform of the output audio signals - on the basis of a downmix signal). Accordingly, it is possible to adapt the technique for the reconstruction, and also the quality of the reconstruction, to the decoded signals without having additional signaling overhead. By determining the weight describing the contribution of the decorrelated signal in the weighted combination both in dependence on the residual signal and the dependence on the decorrelated signal, the weight can be well-adjusted to the signal characteristics, such that a good quality of reconstruction of the at least two output audio signals on the basis of the encoded representation (in particular, on the basis of the downmix signal, the decorrelated signal and the residual signal) can be achieved.

In a preferred embodiment, the multi-channel audio decoder is configured to obtain upmix parameters on the basis of the encoded representation and to determine the weight describing the contribution of the decorrelated signal in the weighted combination in dependence on the upmix parameters. By considering the upmix parameters, it is possible to reconstruct desired characteristics of the output audio signals (like, for example a desired correlation between the output audio signals, and/or desired energy characteristics of the output audio signals) to take a desired value.

According to the invention, the multi-channel audio decoder is configured to determine the weight describing the contribution of the decorrelated signal in the weighted combination such that the weight of the decorrelated signal decreases with increasing energy of the one or more residual signals. This mechanism allows to adjust the precision of the reconstruction of the at least two output audio signals in dependence on the energy of the residual signal. If the energy of the residual signals is comparatively high, the weight of the contribution of the decorreiated signal is comparatively small, such that the decorrelated signal does no longer detrimentally affect a high quality of the reproduction which is caused by using the residual signal. In contrast, if the energy of the residual signal is comparatively low, or even zero, a high weight is given to the decorrelated signal, such that the decorrelated signal can efficiently bring the characteristics of the output audio signals to desired values.

In a preferred embodiment, the multi-channel audio decoder is configured to determine the weight describing the contribution of the decorrelated signal in the weighted combination such that a maximum weight, which is determined by a decorrelated signal upmix parameter, is associated to the decorrelated signal if an energy of the residual signal is zero, and such that a zero weight is associated to the decorrelated signal if an energy of the residual signal weighted using a residual signal weighting coefficient is larger than or equal to an energy of the decorrelated signal, weighted with the decorrelated signal upmix parameter. This embodiment is based on the finding that the desired energy, which should be added to the downmix signal, is determined by the energy of the decorrelated signal, weighted with the decorrelated signal upmix parameter. Accordingly, it is concluded, that it is no longer necessary to add the decorrelated signal if the energy of the residual signal, weighted with the residual signal weighting coefficient, is larger than or equal to said energy of the decorrelated signal, weighted with the decorrelated signal upmix parameter. In other words, the decorrelated signal is no longer used for providing the at least two output audio signals if it is judged that the residual signal carries sufficient energy (for example, sufficient in order to reach a sufficient total energy).

In a preferred embodiment, the multi-channel audio decoder is configured to compute a weighted energy value of the decorrelated signal, weighted in dependence on one or more decorrelated signal upmix parameters, and to compute a weighted energy value of the residual signal, weighted using one or more residual signal upmix parameters (which may be equal to the residual signal weighting coefficients mentioned above), to determine a factor in dependence on the weighted energy value of the decorrelated signal and the weighted energy value of the residual signal, and to obtain a weight describing the contribution of the decorrelated signal to (at least) one of the audio output signals on the basis of the factor. It has been found, that this procedure is well suited for an efficient computation of the weight describing the contribution of the decorrelated signal to one or more output audio signals.

In a preferred embodiment, the multi-channel audio decoder is configured to multiply the factor with a decorrelated signal upmix parameter, to obtain the weight describing the contribution of the decorrelated signal to (at least) one of the output audio signals. By using such procedure, it is possible to consider both one or more parameters describing desired signal characteristics of the at least two output audio signals (which is described by the decorrelated signal upmix parameter) and the relationship between the energy of decorrelated signal and the energy of the residual signal, in order to determine the weight describing the contribution of the decorrelated signal in the weighted combination. Thus, there is both the possibility for blending (or fading) between a parametric coding (or predominantly parametric coding) and a residual coding (or a predominantly residual coding) while still considering the desired characteristics of the output audio signals (which are reflected by the decorrelated signal upmix parameter).

In a preferred embodiment, the multi-channel audio decoder is configured to compute the energy of the decorrelated signal, weighted using the decorrelated signal upmix parameters, over a plurality of upmix channels and time slots, to obtain the weighted energy value of the decorrelated signal. Accordingly, it is possible to avoid strong variations of the weighted energy value of the decorrelated signal. Thus, a stable adjustment of the multi-channel audio decoder is achieved.

Similarly, the multi-channel audio decoder is configured to compute the energy of the residual signal, weighted using residual signal upmix parameters, over a plurality of upmix channels and time slots, to obtain the weighted energy value of the residual signal. Accordingly, a stable adjustment of the multi-channel audio decoder is achieved, since strong variations of the weighted energy value of the residual signal are avoided.

However, the averaging period may be chosen short enough to allow for a dynamic adjustment of the weighting.

In a preferred embodiment, the multi-channel audio decoder is configured to compute the factor in dependence on a difference between the weighted energy value of the decorrelated signal and the weighted energy value of the residual signal. A computation, which "compares" the weighted energy value of the decorrelated signal and the weighted energy value of the residual signal allows to supplement the residual signal (or the weighted version of the residual signal) using the (weighted version of the) decorrelated signal, wherein the weight describing the contribution of the decorrelated signal is adjusted to the needs for the provision of the at least two audio channel signals.

In a preferred embodiment, the multi-channel audio decoder is configured to compute the factor in dependence on a ratio between a difference between the weighted energy value of the decorrelated signal and the weighted energy value of the residual signal, and the weighted energy value of the decorrelated signal. It has been found, that the computation of the factor in dependence on this ratio brings a long particular good results. Moreover, it should be noted, that the ratio describes which portion of the total energy of the decorrelated signal (weighted using the decorrelated signal upmix parameter) is necessary in the presence of the residual signal in order to achieve a good hearing impression (or equivalently, to have substantially the same signal energy in the output audio signals when compared to the case in which there is no residual signal).

In a preferred embodiment, the multi-channel audio decoder is configured to determine weights describing contributions of the decorrelated signal to two or more output audio signals. In this case, the multi-channel audio decoder is configured to determine a contribution of the decorrelated signal to a first output audio signal on the basis of the weighted energy value of the decorrelated signal and a first-channel decorrelated signal upmix parameter. Moreover, the multi-channel audio decoder is configured to determine a contribution of the decorrelated signal to a second output audio channel on the basis of the weighted energy value of the decorrelated signal and a second-channel decorrelated signal upmix parameter. Accordingly, two output audio signals can be provided with moderate effort and good audio quality, wherein the differences between the two output audio signals are considered by usage of a first-channel decorrelated signal upmix parameter and a second-channel decorrelated signal upmix parameter.

In a preferred embodiment, the multi-channel audio decoder is configured to disable a contribution of the decorrelated signal to the weighted combination if a residual energy exceeds a decorrelator energy (i.e. an energy of the decorrelated signal, or of a weighted version thereof). Accordingly, it is possible to switch to a pure residual coding, without the usage of the decorrelated signal, if the residual signal carries sufficient energy, if the residual energy exceeds the decorrelator energy.

In a preferred embodiment, the audio decoder is configured to band-wisely determine the weight describing the contribution of the decorrelated signal in the weighted combination in dependence on a band wise determination of a weighted energy value of the residual signal. Accordingly, it is possible to flexibly decide, without an additional signaling overhead, in which frequency bands a refinement of the at least two output audio signals should be based (or should be predominantly based) on a parametric coding, and in which frequency bands the refinement of the at least two output audio signals should based (or should be predominantly based) on a residual coding. Thus, it can be flexibly decided in which frequency bands a wave form reconstruction (or at least a partial wave from reconstruction) should be performed by using (at least predominantly) the residual coding while keeping the weight of the decorrelated signal comparatively small. Thus, it is possible to obtain a good audio quality by selectively applying the parametric coding (which is mainly based on the provision of a decorrelated signal) and the residual coding (which is mainly based on the provision of a residual signal).

In a preferred embodiment, the audio decoder is configured to determine the weight describing the contribution of the decorrelated signal in a weighted combination for each frame of the output audio signals. Accordingly, a fine timing resolution can be obtained, which allows to flexibly switch between a parametric coding (or predominantly parametric coding) and the residual coding (or predominantly residual coding) between subsequent frames. Accordingly, the audio decoding can be adjusted to the characteristics of the audio signal with a good time resolution.

Embodiments according the invention will subsequently be described taking reference to the enclosed figures, in which.

<FIG> shows a block schematic diagram of a multi-channel audio encoder <NUM> for providing an encoded representation of a multi-channel signal.

The multi-channel audio encoder <NUM> is configured to receive a multi-channel audio signal <NUM> and to provide, on the basis theirs, an encoded representation <NUM> of the multi-channel audio signal <NUM>. The multi-channel audio encoder <NUM> comprises a processor (or processing device) <NUM>, which is configured to receive the multi-channel audio signal and to obtain a downmix signal <NUM> on the basis of the multi-channel audio signal <NUM>. The processor <NUM> is further configured to provide parameters <NUM> describing dependencies between the channels of the multi-channel audio signal <NUM>. Moreover, the processor <NUM> is configured to provide a residual signal <NUM>. Furthermore, the multi-channel audio encoder comprises a residual signal processing <NUM>, which is configured to vary an amount of residual signal included into the encoded representation <NUM> in dependence on the multi-channel audio signal <NUM>.

However, it should be noted, that it is not necessary that the multi-channel audio decoder comprises a separate processor <NUM> and a separate residual signal processing <NUM>. Rather, it is sufficient if the multi-channel audio encoder is somehow configured to perform the functionality of the processor <NUM> and of the residual signal processing <NUM>.

Regarding the functionality of the multi-channel audio encoder <NUM>, it can be noted that the channel signals of the multi-channel audio signal <NUM> are typically encoded using a multi-channel encoding, wherein the encoded representation <NUM> typically comprises (in an encoded form) the downmix signal <NUM>, the parameters <NUM> describing dependencies between channels (or channel signals) of the multi-channel audio signal <NUM> and the residual signal <NUM>. The downmix signal <NUM> may, for example, be based on a combination (for example, linear combination) of the channel signals of the multi-channel audio signal. However a signal downmix signal <NUM> may provided on the basis of a plurality of channel signals of the multi-channel audio signal. However, alternatively, two or more downmix signal may be associated with a larger number (typically larger than the number of downmix signals) of channel signals of the multi-channel audio signal <NUM>. The parameters <NUM> may describe dependencies (for example, a correlation, a covariance, a level relationship or the like) between channels (or channel signals) of the multi-channel audio signal <NUM>. Accordingly, the parameters <NUM> serve the purpose to derive a reconstructed version of the channel signals of the multi-channel audio signal <NUM> on the basis of the downmix signal <NUM> at the side of an audio decoder. For this purpose, the parameters <NUM> describe desired characteristics (for example, individual characteristics or relative characteristics) of the channel signals of the multi-channel audio signal, such that an audio encoder, which uses a parametric decoding, can reconstruct channel signals on the basis of the one or more downmix signals <NUM>.

In addition, the multi-channel audio decoder <NUM> provides the residual signal <NUM>, which typically represents signal components that, according to the expectation or estimation of the multi-channel audio encoder, cannot be reconstructed by an audio decoder (for example, by an audio decoder following a certain processing rule) on the basis of the downmix signal <NUM> and the parameters <NUM>. Accordingly, the residual signal <NUM> can typically be considered as a refinement signal, which allows for a wave from reconstruction, or at least for a partial wave from reconstruction, at the side of an audio decoder.

However, the multi-channel audio encoder <NUM> is configured to vary an amount of residual signal included into the encoded representation <NUM> in dependence on the multi-channel audio signal <NUM>. In other words, the multi-channel audio encoder may, for example, decide about the intensity (or the energy) of the residual signal <NUM> which is included into the encoded representation <NUM>. Additionally or alternatively, the multi-channel audio encoder <NUM> may decide, for which frequency bands and/or for how many frequency bands the residual signal is included into the encoded representation <NUM>. By varying the "amount" of residual signal <NUM> included into the encoded representation <NUM> in dependence on the multi-channel audio signal (and/or in dependence on an available bitrate), the multi-channel audio encoder <NUM> can flexibly determine with which accuracy the channel signals of the multi-channel audio signal <NUM> can be reconstructed at the side of an audio decoder on the basis of the encoded representation <NUM>. Thus, the accuracy with which the channel signals of the multi-channel audio signal <NUM> can be reconstructed, can be adapted to a psychoacoustic relevance of different signal portions of the channel signals of the multi-channel audio signal <NUM> (like, for example, temporal portions, frequency portions and/or time/frequency portions). Thus, signal portions of high psychoacoustic relevance (like, for example, tonal signal portions or signal portions comprising transient events can be encoded with particularly high resolution by including a "large amount" of the residual signal <NUM> into the encoded representation. For example, it can be achieved that a residual signal with a comparatively high energy is included in the encoded representation <NUM> for signal portions of high psychoacoustic relevance. Moreover, it can be achieved that a residual signal of high energy is included in the encoded representation <NUM> if the downmix signal <NUM> comprises a "poor quality", for example, if there is a substantial cancellation of signal components when combining the channel signals of the multi-channel audio signal <NUM> into the downmix signal <NUM>. In other words, the multi-channel audio decoder <NUM> can selectively embed a "larger amount" of residual signal (for example, a residual signal having a comparatively high energy) into the encoded representation <NUM> for signal portions of the multi-channel audio signal <NUM> for which the provision of a comparatively large amount of the residual signal brings along a significant improvement of the reconstructed channel signals (reconstructed at the side of an audio decoder).

Accordingly, the variation of the amount of residual signal included in the encoded representation in dependence on the multi-channel audio signal <NUM> allows to adapt the encoded representation <NUM> (for example, the residual signal <NUM>, which is included into the encoded representation in an encoded form) of the multi-channel audio signal <NUM>, such that a good trade off between bitrate efficiency and audio quality of the reconstructed multi-channel audio signal (reconstructed at the side of an audio decoder) can be achieved.

It should be noted, that the multi-channel audio encoder <NUM> can be optionally improved in many different ways. For example the multi-channel audio encoder may be configured to vary a bandwidth of the residual signal <NUM> (which is included into the encoded representation) in dependence on the multi-channel audio signal <NUM>. Accordingly, the amount of residual signal included into the encoded representation <NUM> may be adapted to perceptually most important frequency bands.

Optionally, the multi-channel audio decoder may be configured to select frequency bands for which the residual signal <NUM> is included into the encoded representation <NUM> in dependence on the multi-channel audio signal <NUM>. Accordingly, the encoded representation <NUM> (more precisely, the amount of residual signal included into the encoded representation <NUM>) may be adapted to the multi-channel audio signal, for example, to the perceptually most important frequency bands of the multi-channel audio signal <NUM>.

Optionally, the multi-channel audio encoder may be configured to including the residual signal <NUM> into the encoded representation for frequency bands for which the multi-channel audio signal is tonal. In addition, the multi-channel audio encoder may be configured to not include the residual signal <NUM> into the encoded representation <NUM> for frequency bands in which the multi-channel audio signal is non-tonal (unless any other specific condition is fulfilled which causes an inclusion of the residual signal into the encoded representation for a specific frequency band). Thus, the residual signal may be selectively included into the encoded representation for perceptually important tonal frequency bands.

Optionally, the multi-channel audio encoder <NUM> may be configured to selectively include the residual signal into the encoded representation for time portions and/or for frequency bands in which the formation of the downmix signal results in a cancellation of signal components of the multi-channel audio signal. For example, the multi-channel audio encoder may be configured to detect a cancellation of signal components of the multi-channel audio signal <NUM> in the downmix signal <NUM>, and to activate the provision of the residual signal <NUM> (for example, the inclusion of the residual signal <NUM> into the encoded representation <NUM>) in response to the result of the detection. Accordingly, if the downmixing (or any other typically linear combination) of channel signals of the multi-channel audio signal <NUM> into the downmix signal <NUM> results in a cancellation of signal components of the multi-channel audio signal <NUM> (which may be caused, for example, by signal components of different channel signals which are phase-shifted by <NUM> degrees), the residual signal <NUM>, which helps to overcome the detrimental effect of this cancellation when reconstructing the multi-channel audio signal <NUM> in an audio decoder, will be included into the encoded representation <NUM>. For example, the residual signal <NUM> may be selectively included in the encoded representation <NUM> for frequency bands for which there is such a cancellation.

Optionally, the multi-channel audio encoder may be configured to compute the residual signal using a linear combination of at least two channel signals of the multi-channel audio signal and in dependence on upmix coefficients to be used at the side of a multi-channel audio decoder. Such a computation of a residual signal is efficient and allows for a simple reconstruction of the channel signals at the side of an audio decoder.

Optionally, the multi-channel audio encoder may be configured to encode the upmix coefficients using the parameter <NUM> describing dependencies between the channels of the multi-channel audio signal, or to derive the upmix coefficients from the parameters describing dependencies between the channels of the multi-channel audio signal. Accordingly, the parameters <NUM> (which may, for example, be intra-channel level difference parameters, intra-channel correlation parameters, or the like) may be used both for the parametric coding (encoding or decoding) and for the residual signal-assisted coding (encoding or decoding). Thus, the usage of the residual signal <NUM> does not bring along an additional signaling overhead. Rather, the parameters <NUM>, which are used for the parametric coding (encoding/decoding) anyway, are re-used also for the residual coding (encoding/decoding). Thus high coding efficiency can be achieved.

Optionally, the multi-channel audio decoder may be configured to time-variantly determine the amount of residual signal included into the encoded representation using a psychoacoustic model. Accordingly, the encoding precision can be adapted to psychoacoustic characteristics of the signal, which typically results in a good bitrate efficiency.

However, it should be noted, that the multi-channel audio encoder can optionally be supplemented by any of the features or functionalities described herein (both in the description and in the claims). Moreover, the multi-channel audio encoder can also be adapted in parallel with the audio decoder described herein, to cooperate with the audio decoder.

<FIG> shows a block schematic diagram of a multi-channel audio decoder <NUM> according to an embodiment of the present invention.

The multi-channel audio decoder <NUM> is configured to receive an encoded representation <NUM> and to provide, on the basis thereof, at least two output audio signals <NUM>, <NUM>. The multi-channel audio decoder <NUM> may, for example, comprise a weighting combiner <NUM>, which is configured to perform a weighted combination of a downmix signal <NUM>, a decorrelated signal <NUM> and a residual signal <NUM>, to obtain (at least) one of the output signals, for example, the first output audio signal <NUM>. it should be noted here, that the downmix signal <NUM>, the decorrelated signal <NUM> and the residual signal <NUM> are derived from the encoded representation <NUM>, wherein the encoded representation <NUM> may carry an encoded representation of the downmix signal <NUM> and an encoded representation of the residual signal <NUM>. Moreover, the decorrelated signal <NUM> may, for example, be derived from the downmix signal <NUM> or may be derived using additional information included in the encoded representation <NUM>. However, the decorrelated signal may also be provided without any dedicated information from the encoded representation <NUM>.

According to an example useful for understanding the invention, the multi-channel audio decoder <NUM> is also configured to determine a weight describing a contribution of the decorrelated signal <NUM> in the weighted combination in dependence on the residual signal <NUM>. For example, the multi-channel audio decoder <NUM> may comprise a weight determinator <NUM>, which is configured to determine a weight <NUM> describing the contribution of the decorrelated signal <NUM> in the weighted combination (for example, the contribution of the decorrelated signal <NUM> to the first output audio signal <NUM>) on the basis of the residual signal <NUM>.

Regarding the functionality of the multi-channel audio decoder <NUM>, it should be noted, that the contribution of the decorrelated signal <NUM> to the weighted combination, and consequently to the first output audio signal <NUM>, is adjusted in a flexible (for example, temporally variable and frequency-dependent) manner in dependence on the residual signal <NUM>, without additional signaling overhead. Accordingly, the amount of decorrelated signal <NUM>, which is included into the first output audio signal <NUM>, is adapted in dependence on the amount of residual signal <NUM> which is included into the first output audio signal <NUM>, such that a good quality of the first output audio signal <NUM> is achieved. Accordingly, it is possible to obtain an appropriate weighting of the decorrelated signal <NUM> under any circumstances and without an additional signaling overhead. Thus, using the multi-channel audio decoder <NUM>, a good quality of the decoded output audio signal <NUM> can be achieved with moderate bitrate. A precision of the reconstruction can be flexibly adjusted by an audio encoder, wherein the audio encoder can determine an amount of residual signal <NUM> which is included in the encoded representation <NUM> (for example, how big the energy of the residual signal <NUM> included in the encoded representation <NUM> is, or to how many frequency bands the residual signal <NUM> included in the encoded representation <NUM> relates), and the multi-channel audio decoder <NUM> can react accordingly and adjust the weighting of the decorrelated signal <NUM> to fit the amount of residual signal <NUM> included in the encoded representation <NUM>. Consequently, if there is a large amount of residual signal <NUM> included in the encoded representation <NUM> (for example, for a specific frequency band, or for specific temporal portion), the weighted combination <NUM> may predominantly (or exclusively) consider the residual signal <NUM> while giving little weight (or no weight) to the decorrelated signal <NUM>. In contrast, if there is only a smaller amount of a residual signal <NUM> included in the encoded representation <NUM>, the weighted combination <NUM> may predominantly (or exclusively) consider the decorrelated signal <NUM> but only to a comparatively small degree (or not at all) the residual signal <NUM> in addition to the downmix signal <NUM>. Thus, the multi-channel audio decoder <NUM> can flexible cooperate with an appropriate multi-channel audio encoder and adjust the weighted combination <NUM> to achieve the best possible audio quality under any circumstances (irrespective of whether a smaller amount or a larger amount of residual signal <NUM> is included in the encoded representation <NUM>).

It should be noted, that the second output audio signal <NUM> may be generated in a similar manner. However, it is not necessary to apply the same mechanisms to the second output audio signal <NUM>, for example, if there are different quality requirements with respect to the second output audio signal.

According to the invention, the multi-channel audio decoder is configured to determine the weight <NUM> describing the contribution of the decorrelated signal <NUM> in the weighted combination in dependence on the decorrelated signal <NUM>. In other words, the weight <NUM> is dependent both on the residual signal <NUM> and the decorrelated signal <NUM>. Accordingly, the weight <NUM> is even better adapted to a currently decoded audio signal without additional signaling overhead.

As optional improvement, the multi-channel audio decoder may be configured to obtain upmix parameters on the basis of the encoded representation <NUM> and to determine the weight <NUM> describing the contribution of the decorrelated signal in the weighted combination in dependence on the upmix parameters. Accordingly, the weight <NUM> may be additionally dependent on the upmix parameters, such that an even better adaptation of the weight <NUM> can be achieved.

As another optional improvement, the multi-channel audio decoder may be configured to determine the weight describing the contribution of the decorrelated signal in the weighted combination such that the weight of the decorrelated signal decreases with increasing energy of the residual signal. Accordingly, a blending or fading can be performed between a decoding which is predominantly based on the decorrelated signal <NUM> (in addition to a downmix signal <NUM>) and a decoding which is predominantly based on the residual signal <NUM> (in addition to a downmix signal <NUM>).

As another optional improvement, the multi-channel audio decoder <NUM> may be configured to determine the weight <NUM> such that a maximum weight, which is determined by a decorrelated signal upmix parameter (which may be included in, or derived from, the encoded representation <NUM>) is associated to the decorrelated signal <NUM> if an energy of the residual signal <NUM> is zero, and that such that a zero weight is associated to the decorrelated signal <NUM> if an energy of the residual signal <NUM>, weighted with the residual signal weighting coefficient (or a residual signal upmix parameter), is larger than or equal to an energy of the decorrelated signal <NUM>, weighted with the decorrelated signal upmix parameter. Accordingly, it is possible to completely blend (or fade) between a decoding based on the decorrelated signal <NUM> and a decoding based on the residual signal <NUM>. if the residual signal <NUM> is judged to be strong enough (for example, when the energy of the weighted residual signal is equal to or larger than the energy of the weighted decorrelated signal <NUM>), the weighted combination may fully rely on the residual signal <NUM> to refine the downmix signal <NUM> while leaving the decorrelated signal <NUM> out of consideration. In this case, a particularly good (at least partial) wave form reconstruction at the side of the multi-channel audio decoder <NUM> can be performed, since the consideration of the decorrelated signal <NUM> typically prevents a particularly good wave form reconstruction while the usage of the residual signal <NUM> typically allows for a good wave form reconstruction.

In another optional improvement, the multi-channel audio decoder <NUM> may be configured to compute a weighted energy value of a decorrelated signal, weighted in dependence on one or more decorrelated signal upmix parameters, and to compute a weighted energy value of the residual signal, weighted using one or more residual signal upmix parameters. In this case, the multi-channel audio decoder may be configured to determine a factor in dependence on the weighted energy value of the decorrelated signal and the weighted energy value of the residual signal and to obtain a weight describing the contribution of the decorrelated signal <NUM> to one of the output audio signals (for example, the first output audio signal <NUM>) on the basis of the factor. Thus, the weight determination <NUM> may provide particularly well-adapted weighting values <NUM>.

In an optional improvement, the multi-channel audio decoder <NUM> (or the weight determinator <NUM> thereof) may be configured to multiply the factor with the decorrelated signal upmix parameter (which may be included in the encoded representation <NUM>, or derived from the encoded representation <NUM>), to obtain the weight (or weighting value) <NUM> describing the contribution of the decorrelated signal <NUM> to one of the output audio signals (for example the first output audio signal <NUM>).

In an optional improvement, the multi-channel audio decoder (or the weight determinator <NUM> thereof) may be configured to compute the energy of the decorrelated signal <NUM>, weighted using decorrelated signal upmix parameters (which may be included in the encoded representation <NUM>, or which may be derived from the encoded representation <NUM>), over a plurality of upmix channels and time slots, to obtain the weighted energy value of the decorrelated signal.

As a further optional improvement, the multi-channel audio decoder <NUM> may be configured to compute the energy of the residual signal <NUM>, weighted using residual signal upmix parameters (which may be included in the encoded representation <NUM> or which may be derived from the encoded representation <NUM>) over a plurality of upmix channels and time slots, to obtain the weighted energy value of the residual signal.

As another optional improvement, the multi-channel audio decoder <NUM> (or the weight determinator <NUM> thereof) may be configured to compute the factor mentioned above in dependence on a difference between the weighted energy value of the decorrelated signal and the weighted energy value of the residual signal. It has been found, that such computation is an efficient solution to determine the weighting values <NUM>.

As an optional improvement, the multi-channel audio decoder may be configured to compute the factor in dependence on a ratio between a difference between the weighted energy value of the decorrelated signal <NUM> and the weighted energy value of the residual signal <NUM>, and the weighted energy value of the decorrelated signal <NUM>. It has been found, that such a computation for the factor brings along good results for blending between a predominantly decorrelation signal based refinement of the downmix signal <NUM> and a predominantly residual signal based refinement of the downmix signal <NUM>.

As an optional improvement, the multi-channel audio decoder <NUM> may be configured to determine weights describing contributions of the decorrelated signals to two or more output audio signals, like, for example, the first output audio signal <NUM> and the second output audio signal <NUM>. in this case, the multi-channel audio decoder may be configured to determine a contribution of the decorrelated signal <NUM> to the first output audio signal <NUM> on the basis of the weighted energy value of the decorrelated signal <NUM> and a first-channel decorrelated signal upmix parameter. Moreover, the multi-channel audio decoder may be configured to determine a contribution of the decorrelated signal <NUM> to the second output audio signal <NUM> on the basis of the weighted energy value of the decorrelated signal <NUM> and a second-channel decorrelated signal upmix parameter. in other words, different decorrelated signal upmix parameters may be used for providing the first output audio signal <NUM> and the second output audio signal <NUM>. However, the same weighted energy value of the decorrelated signal may be used for determining the contribution of the decorrelated signal to the first output audio signal <NUM> and the contribution of the decorrelated signal to the second output audio signal <NUM>. Thus, an efficient adjustment is possible, wherein nevertheless different characteristics of the two output audio signals <NUM>, <NUM> can be considered by different decorrelated signal upmix parameters.

As an optional improvement, the multi-channel audio decoder <NUM> may be configured to disable a contribution of the decorrelated signal <NUM> to the weighted combination if a residual energy (for example, an energy of the residual signal <NUM> or of a weighted version of the residual signal <NUM>) exceeds a decorrelated energy (for example, an energy of the decorrelated signal <NUM> or of a weighted version of the decorrelated signal <NUM>). As a further optional improvement, the audio decoder may be configured to band-wisely determine the weight <NUM> describing a contribution of the decorrelated signal <NUM> in the weighted combination in dependence on a band-wise determination of a weighted energy value of the residual signal. Accordingly a fine-tuned adjustment of the multi-channel audio decoder <NUM> to the signals to be decoded can be performed.

In another optional improvement, the audio decoder may be configured to determine the weight describing a contribution of the decorrelated signal in the weighted combination for each frame of the output audio signal <NUM>, <NUM>. Accordingly, a good temporal resolution can be achieved.

In a further optional improvement, the determination of the weighting value <NUM> may be performed in accordance with some of the equations provided below.

Moreover, it should be noted, that the multi-channel audio decoder <NUM> can be supplemented by any of the features or functionalities described herein, also with respect to other embodiments.

<FIG> shows a block schematic diagram of a multi-channel audio decoder <NUM> according to an embodiment of the invention. The multi-channel audio decoder <NUM> is configured to receive an encoded representation <NUM> and to provide, on the basis thereof, two or more output audio signals <NUM>, <NUM>. The encoded representation <NUM> may, for example, comprise an encoded representation of a downmix signal, an encoded representation of one or more spatial parameters and an encoded representation of a residual signal. The multi-channel audio decoder <NUM> is configured to obtain (at least) one of the output audio signals, for example, a first output audio signal <NUM> and/or a second output audio signal <NUM>, on the basis of the encoded representation of the downmix signal, a plurality of encoded spatial parameters and an encoded representation of the residual signal.

According to an example useful for understanding the invention, the multi-channel audio decoder <NUM> is configured to blend between a parametric coding and a residual coding in dependence on the residual signal (which is included, in an encoded form, in the encoded representation <NUM>). In other words, the multi-channel audio decoder <NUM> may blend between a decoding mode in which the provision of the output audio signals <NUM>, <NUM> is performed on the basis of the downmix signal and using spatial parameters which describe a desired relationship between the output audio signals <NUM>, <NUM> (for example, a desired inter-channel level difference or a desired inter-channel correlation of the output audio signals <NUM>, <NUM>), and a decoding mode in which the output audio signals <NUM>, <NUM> are reconstructed on the basis of the downmix signal using the residual signal. Thus, the intensity (for example, energy) of the residual signal, which is included in the encoded representation <NUM>, may determine whether the decoding is mostly (or exclusively) based on the spatial parameters (in addition to the downmix signal) or whether the decoding is mostly (or exclusively) based on the residual signal (in addition to the downmix signal), or whether an intermediate state is taken in which both the spatial parameters and the residual signal affect the refinement of the downmix signal, to derive the output audio signals <NUM>, <NUM> from the downmix signal.

Moreover, the multi-channel audio decoder <NUM> allows for a decoding which is well-adapted to the current audio content without high signaling overhead by blending between the parametric coding, (in which, typically, a comparatively high weight is given to a decorrelated signal when providing the output audio signals <NUM>, <NUM>) and a residual coding (in which, typically, a comparatively small weight is given to a decorrelated signal) in dependence on the residual signal.

Moreover, it should be noted, that the multi-channel audio decoder <NUM> is based on similar considerations as the multi-channel audio decoder <NUM> and that optional improvements described above with respect to the multi-channel audio decoder <NUM> can also be applied to the multi-channel audio decoder <NUM>.

<FIG> shows a flow chart of a method <NUM> for providing an encoded representation of a multi-channel audio signal.

The method <NUM> comprises a step <NUM> of obtaining a downmix signal on the basis of a multi-channel audio signal. The method <NUM> also comprises a step <NUM> of providing parameters describing dependencies between the channels of the multi-channel audio signal. For example, inter-channel-level-difference parameters and/or inter-channel correlation parameters (or covariance parameters) may be provided, which describe dependencies between channels of the multi-channel audio signal. The method <NUM> also comprises a step <NUM> of providing a residual signal. Moreover, the method comprises a step <NUM> of a varying an amount of residual signal included into the encoded representation in dependence on the multi-channel audio signal.

It should be noted, that the method <NUM> is based on the same considerations as the audio encoder <NUM> according to <FIG>. Moreover, the method <NUM> can be supplemented by any of the features and functionalities described herein with respect to the inventive apparatuses.

<FIG> shows a flow chart of a method <NUM> for providing at least two output audio signals on the basis of an encoded representation. According to an example useful for understanding the invention, the method <NUM> comprises determining <NUM> a weight describing a contribution of a decorrelated signal in a weighted combination in dependence on a residual signal. The method <NUM> also comprises performing <NUM> a weighted combination of a downmix signal, a decorrelated signal and a residual signal, to obtain one of the output audio signals.

It should be noted, that the method <NUM> can be supplemented by any of the features and functionalities described herein with respect to the inventive apparatuses.

<FIG> shows a flow chart of a method <NUM> for providing at least two output audio signals on the basis of an encoded representation. The method <NUM> comprises obtaining <NUM> one of the output audio signals on the basis of an encoded representation of a downmix signal, a plurality of encoded spatial parameters and an encoded representation of a residual signal. According to an example useful for understanding the invention, obtaining <NUM> one of the output audio signals comprises performing <NUM> a blending between a parametric coding and a residual coding in dependence on the residual signal.

In the following, some general considerations and some further embodiments will be described.

Embodiments according to the invention are based on the idea that, instead of using a fixed residual bandwidth, a decoder (for example, a multi-channel audio decoder) detects the amount of transmitted residual signal by measuring its energy band-wise for each frame (or, generally, at least for a plurality of frequency ranges and/or for a plurality of temporal portions). Depending on the transmitted spatial parameters, a decorrelated output is added where residual energy "is missing", to achieve a required (or desired) amount of output energy and decorrelation. This allows a variable residual bandwidth as well as band pass-style residual signals. For example, it is possible to only use residual coding for tonal bands. To be able to use the simplified downmix for parametric coding as well as for wave form-preserving coding (which is also designated as residual coding), a residual signal for the simplified downmix is defined herein.

In the following, some considerations regarding the calculation of the residual signal and regarding the construction of channel signals of a multi-channel audio signal will be described.

In unified-speech- and audio-coding (USAC), there is no residual signal defined when a so-called "simplified downmix" is used. Thus, no partially waveform preserving coding is possible. However, in the following, a method for a calculating a residual signal for the so-called "simplified downmix" will be described.

"Simplified downmix" weights d<NUM>, d<NUM> are calculated per scale factor band, whereas parametric upmix coefficients ud1, ud2 are calculated per parameter band. Thus, coefficients wr1, wr2, for calculating the residual signal cannot be directly computed from the spatial parameters (as it is the case for a classic MPEG surround), but may need to be determined scale factor band-wise from the down- and upmix coefficients.

With L, R being the input channels and D being the downmix channel, a residual signal res should fulfill the following properties: <MAT> <MAT> <MAT>.

This is achieved by calculating the residual as <MAT> using the downmix weights <MAT> <MAT>.

The residual upmix coefficients ur,<NUM>, ur,<NUM> used by the decoder are preferably chosen in a way to ensure robust decoding. Since the simplified downmix has asymmetric properties (as opposed to MPEG Surround with fixed weights) an upmix depending on the spatial parameters is applied, e.g. using the following upmix coefficients: <MAT> <MAT>.

Another option is to define the residual upmix coefficients to be orthogonal to the downmix signal's upmix coefficients, so that: <MAT> in other words, an audio decoder may obtain the downmix signal D using a linear combination of a left channel signal L (first channel signal) and a right channel signal R (second channel signal). Similarly, the residual signal res is obtained using a linear combination of the left channel L and the right channel signal R (or, generally, of a first channel signal and a second channel signal of the multi-channel audio signal).

It can be seen, for example, in Equations (<NUM>) and (<NUM>), the downmix weights wr,<NUM> and wr,<NUM> for obtaining the residual signal res can be obtained when the simplified downmix weights d<NUM>, d<NUM>, the parametric upmix coefficients ud,<NUM> and ud,<NUM> and the residuai upmix coefficients ur,<NUM> and ur,<NUM> are determined. Moreover it can be seen, that ur,<NUM> and ur,<NUM> can be derived from ud,<NUM> and ud,<NUM> using equations (<NUM>) and (<NUM>) or equation (<NUM>). The simplified downmix weights d<NUM> and d<NUM>, as well as the parametric upmix coefficients ud,<NUM> and ud,<NUM> can be obtained in the usual manner.

In the following, some details regarding the encoding process will be described. The encoding may, for example, be performed by the multi-channel audio encoder <NUM> or by any other appropriate means or computer programs.

Preferably, the amount of a residual that is transmitted is determined by a psychoacoustic model of the encoder (for example, multi-channel audio encoder), depending on the audio signal (for example, depending on the channel signals of the multi-channel audio signal <NUM>) and an available bitrate. The transmitted residual signal can, for example, be used for partial wave form preservation or to avoid signal cancellation caused by the used downmixing method (for example, the downmixing method described by equation (<NUM>) above).

In the following, it is described how a partial wave form preservation can be achieved. For example, the calculated residual (for example, the residual res according to equation (<NUM>)) is transmitted full-band or band-limited to provide partial wave form preservation within the residual bandwidth. Residual parts, which are detected as perceptually irrelevant by the psychoacoustic model may, for example, be quantized to zero (for example, when providing the encoded representation <NUM> on the basis of the residual signal <NUM>). This includes, but is not limited to, reducing the transmitted residual bandwidth at runtime (which may be considered as varying an amount of residual signal which is included into the encoded representation). This system may also allow band-pass-style deletion of residual signal parts, as missing signal energy will be reconstructed by the decoder (for example, by the multi-channel audio decoder <NUM> or the multi-channel audio decoder <NUM>). Thus, for example, residual coding may be only applied to tonal components of the signal, preserving their phase-relations, whereas background noise can be parametrically coded to reduce the residual bitrate. In other words, the residual signal <NUM> may only be included into the encoded representation <NUM> (for example, by the residual signal processing <NUM>) for frequency bands and/or temporal portions for which the multi-channel audio signal <NUM> (or at least one of the channel signals of the multi-channel audio signal <NUM>) are found to be tonal. In contrast, the residual signal <NUM> may not be included into the encoded representation <NUM> for frequency bands and/or temporal portions for which the multi-channel audio signal <NUM> (or at least one or more channel signals of the multi-channel audio signal <NUM>) are identified as being noise-like. Thus, an amount of residual signal included into the encoded representation is varied in dependence on the multi-channel audio signal.

In the following, it will be described how a signal cancellation in the downmix can be prevented (or compensated).

For low bitrate applications, parametric coding (which predominantly or exclusively relies on the parameters <NUM>, describing dependencies between channels of the multi-channel audio signal) instead of wave form preserving coding (which, for example, predominantly relies on the residual signal <NUM>, in addition to the downmix signal <NUM>) is applied. Here, the residual signal <NUM> is only used to compensate for signal cancellations in the downmix <NUM>, to minimize the bit usage of the residual. As long as no signal cancellations in the downmix <NUM> are detected, the system runs in parametric mode using decorrelators (at the side of the audio decoder). When signal cancellations occur, for example, for phasing tonal signals, a residual signal <NUM> is transmitted for the impaired signal parts (for example, frequency bands and/or temporal portions). Thus, the signal energy can be restored by the decoder.

In the decoder (for example, in the multi-channel audio decoder <NUM> or in the multi-channel audio decoder <NUM>), the transmitted downmix and residual signals (for example, downmix signal <NUM> or residual signal <NUM>) are decoded by a core decoder and fed into an MPEG surround decoder together with the decoded MPEG surround payload. Residual upmix coefficients for the classic MPS downmix are unchanged, and residual upmix coefficient for the simplified downmix are defined in equations (<NUM>) and (<NUM>) and/or (<NUM>). Additionally, decorrelator outputs and its weighting coefficients are calculated, as for parametric decoding. The residual signal and the decorrelator outputs are weighted and both mixed to the output signal. Therefore, weighting factors are determined by measuring the energies of the residual and decorrelator signals.

In other words, residual upmix factors (or coefficients) may be determined by measuring the energies of the residual and decorrelated signals.

For example, the downmix signal <NUM> is provided on the basis of the encoded representation <NUM>, and the decorrelated signal <NUM> is derived from the downmix signal <NUM> or generated on the basis of parameters included in the encoded representation <NUM> (or otherwise). The residual upmix coefficients may, for example be derived from the parametric upmix coefficients ud,<NUM> and ud,<NUM> in accordance with equations (<NUM>) and (<NUM>) by the decoder, wherein the parametric upmix coefficients ud,<NUM> ud,<NUM> may be obtained on the basis of the encoded representation <NUM>, for example, directly or by deriving them from spatial data included in the encoded representation <NUM> (for example, from inter-channel correlation coefficients and inter-channel level difference coefficients, or from inter-object correlation coefficients and inter-object level differences).

Upmixing coefficients for the decorrelator output (or outputs) may be obtained as for conventional MPEG surround decoding. However, weighting factors for weighting the decorrelator output (or decorrelator outputs) may be determined on the basis of the energies of the residual signal (and possibly also on the basis of the energies of the decorrelator signal or signals) such that a weight describing a contribution of the decorrelated signal in the weighted combination is determined in dependence on the residual signal.

In the following, an example implementation will be described taking reference to <FIG>. However, it should be noted, that the concept described herein can also be applied in the multi-channel audio decoders <NUM> or <NUM> according to <FIG> and <FIG>.

<FIG> shows a block schematic diagram (or flow diagram) of a decoder (for example, of a multi-channel audio decoder). The decoder according to <FIG> is designated with <NUM> in its entirety. The decoder <NUM> is configured to receive a bit stream <NUM> and to provide, on the basis thereof, a first output channel signal <NUM> and a second output channel signal <NUM>. The decoder <NUM> comprises a core decoder <NUM>, which is configured to receive the bit stream <NUM> and to provide, on the basis thereof, a downmix signal <NUM>, a residual signal <NUM> and spatial data <NUM>. For example, the core decoder <NUM> may provide, as the downmix signal, a time domain representation or transform domain representation (for example, frequency domain representation, MDCT domain representation, QMF domain representation) of the downmix signal represented by the bit stream <NUM>. Similarly, the core decoder <NUM> may provide a time domain representation or transform domain representation of the residual signal <NUM>, which is represented by the bit stream <NUM>. Moreover, the core decoder <NUM> may provide one or more spatial parameters <NUM>, like, for example, one or more inter-channel-correlation parameter, inter-channel-level difference parameters, or the like.

The decoder <NUM> also comprises a decorrelator <NUM>, which is configured to provide a decorrelated signal <NUM> on the basis of the downmix signal <NUM>. Any of the known decorrelation concepts may be used by the decorrelator <NUM>. Moreover, the decoder <NUM> also comprises an upmix coefficient calculator <NUM>, which is configured to receive spatial data <NUM> and to provide upmix parameters (for example, upmix parameters udmx,<NUM>, udmx,<NUM>, udec,<NUM> and udec,<NUM>). Moreover, the decoder <NUM> comprises an upmixer <NUM>, which is configured to apply the upmix parameters <NUM> (also designated as upmix coefficients) which are provided by the upmix coefficient calculator <NUM> on the basis of the spatial data <NUM>. For example, the upmixer <NUM> may scale the downmix signal <NUM> using two downmix-signal upmix coefficients (for example the udmx,<NUM>, udmx,<NUM>), to obtain two upmixed versions <NUM>, <NUM> of the downmix signal <NUM>. Moreover, the upmixer <NUM> is also configured to apply one or more upmix parameters (for example two upmix parameters) to the decorrelated signal <NUM> provided by the decorrelator <NUM>, to obtain a first upmixed (scaled) version <NUM> and a second upmixed (scaled) version <NUM> of the decorrelated signal <NUM>. Moreover, the upmixer <NUM> is configured to apply one or more upmix coefficients (for example, two upmix coefficients) to the residual signal <NUM>, to obtain a first upmixed (scaled) version <NUM> and a second upmixed (scaled) version <NUM> of the residual signal <NUM>.

The decoder <NUM> also comprises a weight calculator <NUM>, which is configured to measure energies of the upmixed (scaled) versions <NUM>, <NUM> of the decorrelated signal <NUM> and of the upmixed (scaled) version <NUM>, <NUM> of the residual signal <NUM>. Moreover, the weight calculator <NUM> is configured to provide one or more weighting values <NUM> to a weighter <NUM>. The weighter <NUM> is configured to obtain a first upmixed (scaled) and weighted version <NUM> of the decorrelated signal <NUM>, a second upmixed (scaled) and a weighted version <NUM> of the decorrelated signal <NUM>, a first upmixed (scaled) and weighted version <NUM> of the residual signal <NUM> and a second upmixed (scaled) and weighted version <NUM> of the residual signal <NUM> using one or more weighting values <NUM> provided by the weight calculator <NUM>. The decoder also comprises a first adder <NUM>, which is configured to add up the first upmixed (scaled) version <NUM> of the downmix signal <NUM>, the first upmixed (scaled) and weighted version <NUM> of the decorrelated signal <NUM> and the first upmixed (scaled) and weighted version <NUM> of the residual signal <NUM>, to obtain the first output channel signal <NUM>. Moreover, the decoder comprises a second adder <NUM>, which is configured to add up the second upmixed version <NUM> of the downmix signal <NUM>, the second upmixed (scaled) and weighted version <NUM> of the decorrelated signal <NUM> and the second upmixed (scaled) and weighted version <NUM> of the residual signal <NUM>, to obtain the second output channel signal <NUM>.

However, it should be noted, that it is not necessary that the weighter <NUM> weights all of the signals <NUM>, <NUM>, <NUM>, <NUM>. For example, in some embodiments it may be sufficient to weight only the signals <NUM>, <NUM>, while leaving the signals <NUM>, <NUM> unaffected (such that, effectively, the signals <NUM>, <NUM> are directly applied to the adders <NUM>, <NUM>. Alternatively, however, the weighting of the residual signals <NUM>, <NUM> may be varied over time. For example, the residual signals may be faded in or faded out. For example, the weighting (or the weighting factors) of the decorrelated signals may be smoothened over time, and the residual signals may be faded in or faded out correspondingly.

Moreover, it should be noted, that the weighting, which is performed by the weighter <NUM> and the upmixing, which is applied by the upmixer <NUM>, may also be performed as a combined operation, wherein the weight calculation may be performed directly using the decorrelated signal <NUM> and the residual signal <NUM>.

In the following, some further details regarding the functionality of the decoder <NUM> will be described.

A combined residual and parametric coding mode may, for example, be signaled in a semi-backwards compatible way, for example, by signaling a residual bandwidth of one parameter band in the bit stream. Thus, a legacy decoder will still pass and decode the bit stream by switching to parametric decoding above the first parameter band. Legacy bit streams using a residual bandwidth of one would not contain residual energy above the first parameter band, leading to a parametric decoding in the proposed new decoder.

However, within a 3D audio codec system, the combined residual and parametric coding may be used in combination with other core decoder tools like a quad channel element, enabling the decoder to explicitly detect legacy bit streams and decode them in regular band-limited residual coding mode. An actual residual bandwidth is preferably not explicitly signaled, as it is determined by the decoder at run time. The calculation of the upmix coefficients is set to parametric mode instead of a residual coding mode. The energies of the weighted decorrelator output Edec and weighted residual signal Eres are calculated per hybrid band hb over all time slots ts and upmix channels ch for each frame: <MAT> <MAT>.

Here, udec designates a decorrelated signal upmix parameter for a frequency band hb, for a time slot ts and for an upmix channel ch, <MAT> designates a sum over upmix channels, and <MAT> designates a sum over time slots. xdec designates a value (for example, a complex transform domain value) of the decorrelated signal for a frequency band hb, for a time slot ts and for an upmix channel ch.

The residual signal (for example, the upmixed residual signal <NUM> or the upmixed residual signal <NUM>) is added to output channels (for example, to output channels <NUM>, <NUM>) with a weight of one. The decorrelator signal (for example the upmixed decorrelator signal <NUM> or the upmixed decorellator signal <NUM>) may be weighted with a factor r (for example by the weighter <NUM>) that is calculated as <MAT> wherein Edec(hb) represents a weighted energy value of the decorrelated signal xdec for a frequency band hb, and wherein Eres(hb) represents a weighted energy value of the residual signal xres for a frequency band hb.

If no residual (for example, no residual signal <NUM>) has been transmitted, for example, if Eres = <NUM>, r (the factor which may be applied by the weighter <NUM>, and which may be considered as a weighting value <NUM>) becomes <NUM>, which is equivalent to a purely parametric decoding. If the residual energy (for example, the energy of the upmixed residual signal <NUM> and/or of the upmixed residual signal <NUM>) exceeds the decorrelator energy (for example, the energy of the upmixed decorrelated signal <NUM> or of the upmixed decorrelated signal <NUM>), for example, if Eres > Edec, the factor r may be set to zero, thus disabling the decorrelator and enabling partially wave form preserving decoding (which may be considered as residual coding). In the upmixing process, the weighted decorrelator output (for example, signals <NUM> and <NUM>) and the residual signal (for example, signals <NUM>, <NUM> or signals <NUM>, <NUM>) are both added to the output channels (for example, signals <NUM>, <NUM>).

In conclusion, this leads to an upmix rule in matrix form <MAT> wherein ch1 represents one or more time domain samples or transform domain samples of a first output audio signal, wherein ch2 represents one or more time domain samples or transform domain samples of a second output audio signal, wherein xdmx represents one or more time domain samples or transform domain samples of a downmix signal, wherein xdec represents one or more time domain samples or transform domain samples of a decorrelated signal, wherein xres represents one or more time domain samples or transform domain samples of a residual signal, wherein udmx,<NUM> represents a downmix signal upmix parameter for the first output audio signal, wherein udmx,<NUM> represents a downmix signal upmix parameter for the second output audio signal, wherein udec,<NUM> represents a decorrelated signal upmix parameter for the first output audio signal, wherein udec,<NUM> represents a decorrelated signal upmix parameter for the second output audio signal, wherein max represents a maximum operator, and wherein r represents a factor describing a weighting of the decorrelated signal in dependence on the residual signal.

The upmix coefficients Udmx,<NUM>, Udmx,<NUM>, Udec,<NUM>,, Udec,<NUM> are calculated as for the MPS two-one-two (<NUM>-<NUM>-<NUM>) parametric mode. For details, reference is made to the above referenced standard of the MPEG surround concept.

To summarize, an embodiment according to the invention creates a concept to provide output channel signals on the basis of a downmix signal, a residual signal and spatial data, wherein a weighting of the decorrelated signal is flexibly adjusted without any significant signaling overhead.

in some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

The inventive encoded audio signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

In the following, an example will be described taking reference to <FIG>, which shows a block schematic diagram of a so-called Hybrid Residual Decoder.

The Hybrid Residual Decoder <NUM> according to <FIG> is very similar to the Decoder <NUM> according to <FIG>, such that reference is made to the above explanations. However, in the Hybrid Residual Decoder <NUM>, an additional weighting (in addition to the application of the upmix parameters) is only applied to the upmixed decorrelated signals (which correspond to the signals <NUM>,<NUM> in the decoder <NUM>), but not to the upmixed residual signals (which correspond to the signals <NUM>, <NUM> in the decoder <NUM>). Thus, the weighter in the Hybrid Residual Decoder <NUM> is somewhat simpler than the weighter in the decoder <NUM>, but is well in agreement, for example, with the weighting according to equation (<NUM>).

In the following, the combined Parametric and Residual Decoding (Hybrid Residual Coding) according to <FIG> will be explained in some more detail.

However, firstly, an overview will be provided.

In addition to using either decorrelator-based mono-to-stereo upmixing or residual coding as described in ISO/IEC <NUM>-<NUM>, subclause <NUM>. <NUM>, Hybrid Residual Coding allows a signal dependent combination of both modes. Residual signal and decorrelator output are blended together, using time and frequency dependent weighting factors depending on the signal energies and the spatial parameters, as illustrated in <FIG>.

In the following, the decoding process will be described.

Hybrid Residual Coding mode is indicated by the syntax elements bsResidualCoding == <NUM> and bsResidualBands == <NUM> in Mps212Config(). In other words, the usage of the Hybrid Residual coding may be signaled using a bitstream element of the encoded representation. The calculation of mix-matrix M2 is performed as if bsResidualCoding == <NUM>, following the calculation in ISO/IEC <NUM>-<NUM>, subclause <NUM>. The matrix <MAT> for the decorrelator based part is defined as <MAT>.

The upmixing process is split up into Downmix, decorrelator output and residual. The upmixed Downmix udmx is calculated using: <MAT>.

The upmixed decorrelator output udec is calculated using: <MAT>.

The upmixed residual signal ures is calculated using: <MAT>.

The energies of the upmixed residual signal Eres and of the upmixed decorrelator output Edec are calculated per hybrid. band as sum over both output channels ch and all timeslots ts and of one frame as: <MAT> <MAT>.

The upmixed decorrelator output is weighted using a weighting factor rdec calculated for each hybrid band per frame as: <MAT> with ε a small number to prevent division by zero (for example, ε = le-<NUM>, or <NUM><ε<=1e-<NUM>). However, in some embodiments, ε may be set to zero (replacing " Eres < ε " by " Eres = <NUM>").

All three upmix signals are added to form the decoded output signal.

To conclude, embodiments according to the invention create a combined residual and parametric coding.

The present invention creates a method for a signal dependent combination of parametric and residual coding for joint stereo coding, which is based on the USAC unified stereo tool. Instead of using a fixed residual bandwidth, the amount of transmitted residual is determined signal dependently by an encoder, time and frequency variant. On decoder side, the required amount of decorrelation between the output channels is generated by mixing residual signal and decorrelator output. Thus, a corresponding audio coding/decoding system is able to blend between fully parametric coding and wave form preserving residual coding at run time, depending on the encoded signal.

Embodiments according to the invention outperform conventional solutions. For example, in USAC, an MPEG surround two-one-two (<NUM>-<NUM>-<NUM>) system is used for parametric stereo coding, or unified stereo, transmitting a band-limited or full-bandwidth residual signal for partial wave form preservation. If a band-limited residual is transmitted, parametric upmixing with the use of decorrelators is applied above the residual bandwidth. The drawback of this method is, that the residual bandwidth is set to a fixed value at the encoder initialization.

In contrast, embodiments according to the invention allow for a signal dependent adaptation of the residual bandwidth or switching to parametric coding. Moreover, if the downmixing process in parametric coding mode produces signal cancellations for ill-conditioned phase relations, embodiments according to the invention allow to reconstruct missing signal parts (for example, by providing an appropriate residual signal). It should be noted, that the simplified downmix method produces less signal cancellations than the classic MPS downmix for parametric coding. However, while the conventional simplified downmix cannot be used for partial wave form preservation, since no residual signal is defined in USAC, embodiments according to the invention allow for a wave form reconstruction (for example, a selective partial wave form reconstruction for signal portions in which partial wave form reconstruction appears to be important).

Claim 1:
A multi-channel audio decoder (<NUM>; <NUM>; <NUM>; <NUM>) for providing at least two output audio signals (<NUM>, <NUM>; <NUM>, <NUM>; <NUM>, <NUM>) on the basis of an encoded representation (<NUM>; <NUM>; <NUM>),
wherein the multi-channel audio decoder is configured to perform a weighted combination (<NUM>; <NUM>, <NUM>, <NUM>) of a downmix signal (<NUM>; <NUM>, <NUM>), a decorrelated signal (<NUM>; <NUM>,<NUM>) and a residual signal (<NUM>; <NUM>, <NUM>; res), to obtain one of the output audio signals (<NUM>,<NUM>; <NUM>, <NUM>),
wherein the multi-channel audio decoder is configured to determine a weight (<NUM>; r; rdec) describing a contribution of the decorrelated signal in the weighted combination in dependence on both the residual signal and the decorrelated signal,
wherein the downmix signal, the decorrelated signal and the residual signal are derived from the encoded representation;
wherein the multi-channel audio decoder is configured to determine the weight (<NUM>; r; rdec) describing the contribution of the decorrelated signal in the weighted combination such that the weight of the decorrelated signal decreases with increasing energy of the residual signal.