Patent Description:
In conventional multi-channel audio coding, possible coding schemes include discrete multi-channel coding or parametric coding such as MPEG Surround. The scheme used depends on the bandwidth of the audio system. Parametric coding methods are known to be scalable and efficient in terms of listening quality, which makes them particularly attractive in low bitrate applications. In high bitrate applications, the discrete multi-channel coding is often used. The existing distribution or processing formats and the associated coding techniques may be improved from the point of view of their bandwidth efficiency, especially in applications with a bitrate in between the low bitrate and the high bitrate.

<CIT>) relates to a hybrid coding method wherein a hybrid audio signal is formed from at least one downmixed spectral component and at least one unmixed spectral component. The method presented in that application may increase the capacity of an application having a certain bitrate, but further improvements may be needed to further increase the efficiency of an audio processing system.

As a general background, reference can also be made to the ATSC standard "<NPL>).

The document "<NPL> summarizes the results of the standardization process by describing the underlying ideas and providing an overview of the MPEG Surround technology.

Example embodiments will now be described with reference to the accompanying drawings, on which:.

All the figures are schematic and generally only show parts which are necessary in order to elucidate the disclosure, whereas other parts may be omitted or merely suggested. Unless otherwise indicated, like reference numerals refer to like parts in different figures.

<FIG> and <FIG> correspond to embodiments of the invention. The other figures do not form part of the invention but are useful for its understanding.

As used herein, an audio signal may be a pure audio signal, an audio part of an audiovisual signal or multimedia signal or any of these in combination with metadata.

As used herein, downmixing of a plurality of signals means combining the plurality of signals, for example by forming linear combinations, such that a lower number of signals is obtained. The reverse operation to downmixing is referred to as upmixing that is, performing an operation on a lower number of signals to obtain a higher number of signals.

According to a first aspect, example embodiments propose methods, devices and computer program products, for reconstructing a multi-channel audio signal based on an input signal. The proposed methods, devices and computer program products may generally have the same features and advantages.

According to example embodiments, a decoder for a multi-channel audio processing system for reconstructing M encoded channels, wherein M > <NUM>, is provided. The decoder comprises a first receiving stage configured to receive N waveform-coded downmix signals comprising spectral coefficients corresponding to frequencies between a first and a second cross-over frequency, wherein <NUM> <N<M.

The decoder further comprises a second receiving stage configured to receive M waveform-coded signals comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency, each of the M waveform-coded signals corresponding to a respective one of the M encoded channels.

The decoder further comprises a downmix stage downstreams of the second receiving stage configured to downmix the M waveform-coded signals into N downmix signals comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency.

The decoder further comprises a first combining stage downstreams of the first receiving stage and the downmix stage configured to combine each of the N downmix signals received by the first receiving stage with a corresponding one of the N downmix signals from the downmix stage into N combined downmix signals.

The decoder further comprises a high frequency reconstructing stage downstreams of the first combining stage configured to extend each of the N combined downmix signals from the combining stage to a frequency range above the second cross-over frequency by performing high frequency reconstruction.

The decoder further comprising an upmix stage downstreams of the high frequency reconstructing stage configured to perform a parametric upmix of the N frequency extended signals from the high frequency reconstructing stage into M upmix signals comprising spectral coefficients corresponding to frequencies above the first cross-over frequency, each of the M upmix signals corresponding to one of the M encoded channels.

The decoder further comprises a second combining stage downstreams of the upmix stage and the second receiving stage configured to combine the M upmix signals from the upmix stage with the M waveform-coded signals received by the second receiving stage.

The M waveform-coded signals are purely waveform-coded signals with no parametric signals mixed in, i.e. they are a non-downmixed discrete representation of the processed multi-channel audio signal. An advantage of having the lower frequencies represented in these waveform-coded signals may be that the human ear is more sensitive to the part of the audio signal having low frequencies. By coding this part with a better quality, the overall impression of the decoded audio may increase.

An advantage of having at least two downmix signals is that this embodiment provides an increased dimensionality of the downmix signals compared to systems with only one downmix channel. According to this embodiment, a better decoded audio quality may thus be provided which may outweigh the gain in bitrate provided by a one downmix signal system.

An advantage of using hybrid coding comprising parametric downmix and discrete multi-channel coding is that this may improve the quality of the decoded audio signal for certain bit rates compared to using a conventional parametric coding approach, i.e. MPEG Surround with HE-AAC. At bitrates around <NUM> kilobits per second (kbps), the conventional parametric coding model may saturate, i.e. the quality of the decoded audio signal is limited by the shortcomings of the parametric model and not by lack of bits for coding. Consequently, for bitrates from around <NUM> kbps, it may be more beneficial to use bits on discretely waveform-coding lower frequencies. At the same time, the hybrid approach of using a parametric downmix and discrete multi-channel coding is that this may improve the quality of the decoded audio for certain bitrates, for example at or below <NUM> kbps, compared to using an approach where all bits are used on waveform-coding lower frequencies and using spectral band replication (SBR) for the remaining frequencies.

An advantage of having N waveform-coded downmix signals that only comprises spectral data corresponding to frequencies between the first cross-over frequency and a second cross-over frequency is that the required bit transmission rate for the audio signal processing system may be decreased. Alternatively, the bits saved by having a band pass filtered downmix signal may be used on waveform-coding lower frequencies, for example the sample frequency for those frequencies may be higher or the first cross-over frequency may be increased.

Since, as mentioned above, the human ear is more sensitive to the part of the audio signal having low frequencies, high frequencies, as the part of the audio signal having frequencies above the second cross-over frequency, may be recreated by high frequency reconstruction without reducing the perceived audio quality of the decoded audio signal.

A further advantage with the present embodiment may be that since the parametric upmix performed in the upmix stage only operates on spectral coefficients corresponding to frequencies above the first cross-over frequency, the complexity of the upmix is reduced.

According to another embodiment, the combining performed in the first combining stage, wherein each of the N waveform-coded downmix signals comprising spectral coefficients corresponding to frequencies between a first and a second cross-over frequency are combined with a corresponding one of the N downmix signals comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency into N combined downmix, is performed in a frequency domain.

An advantage of this embodiment may be that the M waveform-coded signals and the N waveform-coded downmix signals can be coded by a waveform coder using overlapping windowed transforms with independent windowing for the M waveform-coded signals and the N waveform-coded downmix signals, respectively, and still be decodable by the decoder.

According to another embodiment, extending each of the N combined downmix signals to a frequency range above the second cross-over frequency in the high frequency reconstructing stage is performed in a frequency domain.

According to a further embodiment, the combining performed in the second combining step, i.e. the combining of the M upmix signals comprising spectral coefficients corresponding to frequencies above the first cross-over frequency with the M waveform-coded signals comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency, is performed in a frequency domain. As mentioned above, an advantage of combining the signals in the QMF domain is that independent windowing of the overlapping windowed transforms used to code the signals in the MDCT domain may be used.

According to another embodiment, the performed parametric upmix of the N frequency extended combined downmix signals into M upmix signals at the upmix stage is performed in a frequency domain.

According to yet another embodiment, downmixing the M waveform-coded signals into N downmix signals comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency is performed in a frequency domain.

According to an embodiment, the frequency domain is a Quadrature Mirror Filters, QMF, domain.

According to another embodiment, the downmixing performed in the downmixing stage, wherein the M waveform-coded signals is downmixed into N downmix signals comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency, is performed in the time domain.

According to yet another embodiment, the first cross-over frequency depends on a bit transmission rate of the multi-channel audio processing system. This may result in that the available bandwidth is utilized to improve quality of the decoded audio signal since the part of the audio signal having frequencies below the first cross-over frequency is purely waveform-coded.

According to another embodiment, extending each of the N combined downmix signals to a frequency range above the second cross-over frequency by performing high frequency reconstruction at the high frequency reconstructions stage are performed using high frequency reconstruction parameters. The high frequency reconstruction parameters may be received by the decoder, for example at the receiving stage and then sent to a high frequency reconstruction stage. The high frequency reconstruction may for example comprise performing spectral band replication, SBR.

According to another embodiment, the parametric upmix in the upmixing stage is done with use of upmix parameters. The upmix parameters are received by the encoder, for example at the receiving stage and sent to the upmixing stage. A decorrelated version of the N frequency extended combined downmix signals is generated and the N frequency extended combined downmix signals and the decorrelated version of the N frequency extended combined downmix signals are subjected to a matrix operation. The parameters of the matrix operation are given by the upmix parameters.

According to another embodiment, the received N waveform-coded downmix signals in the first receiving stage and the received M waveform-coded signals in the second receiving stage are coded using overlapping windowed transforms with independent windowing for the N waveform-coded downmix signals and the M waveform-coded signals, respectively.

An advantage of this may be that this allows for an improved coding quality and thus an improved quality of the decoded multi-channel audio signal. For example, if a transient is detected in the higher frequency bands at a certain point in time, the waveform coder may code this particular time frame with a shorter window sequence while for the lower frequency band, the default window sequence may be kept.

According to embodiments, the decoder may comprise a third receiving stage configured to receive a further waveform-coded signal comprising spectral coefficients corresponding to a subset of the frequencies above the first cross-over frequency. The decoder may further comprise an interleaving stage downstream of the upmix stage. The interleaving stage may be configured to interleave the further waveform-coded signal with one of the M upmix signals. The third receiving stage may further be configured to receive a plurality of further waveform-coded signals and the interleaving stage may further be configured to interleave the plurality of further waveform-coded signal with a plurality of the M upmix signals.

This is advantageous in that certain parts of the frequency range above the first cross-over frequency which are difficult to reconstruct parametrically from the downmix signals may be provided in a waveform-coded form for interleaving with the parametrically reconstructed upmix signals.

In one exemplary embodiment, the interleaving is performed by adding the further waveform-coded signal with one of the M upmix signals. According to another exemplary embodiment, the step of interleaving the further waveform-coded signal with one of the M upmix signals comprises replacing one of the M upmix signals with the further waveform-coded signal in the subset of the frequencies above the first cross-over frequency corresponding to the spectral coefficients of the further waveform-coded signal.

According to exemplary embodiments, the decoder may further be configured to receive a control signal, for example by the third receiving stage. The control signal may indicate how to interleave the further waveform-coded signal with one of the M upmix signals, wherein the step of interleaving the further waveform-coded signal with one of the M upmix signals is based on the control signal. Specifically, the control signal may indicate a frequency range and a time range, such as one or more time/frequency tiles in a QMF domain, for which the further waveform-coded signal is to be interleaved with one of the M upmix signals. Accordingly, Interleaving may occur in time and frequency within one channel.

An advantage of this is that time ranges and frequency ranges can be selected which do not suffer from aliasing or start-up/fade-out problems of the overlapping windowed transform used to code the waveform-coded signals.

The encoder does not form part of the invention but is useful for its understanding.

According to a second aspect, example embodiments propose methods, devices and computer program products for encoding a multi-channel audio signal based on an input signal.

The proposed methods, devices and computer program products may generally have the same features and advantages.

Advantages regarding features and setups as presented in the overview of the decoder above may generally be valid for the corresponding features and setups for the encoder.

According to the example embodiments, an encoder for a multi-channel audio processing system for encoding M channels, wherein M > <NUM>, is provided.

The encoder comprises a receiving stage configured to receive M signals corresponding to the M channels to be encoded.

The encoder further comprises first waveform-coding stage configured to receive the M signals from the receiving stage and to generate M waveform-coded signals by individually waveform-coding the M signals for a frequency range corresponding to frequencies up to a first cross-over frequency, whereby the M waveform-coded signals comprise spectral coefficients corresponding to frequencies up to the first cross-over frequency.

The encoder further comprises a downmixing stage configured to receive the M signals from the receiving stage and to downmix the M signals into N downmix signals, wherein <NUM> <N<M.

The encoder further comprises high frequency reconstruction encoding stage configured to receive the N downmix signals from the downmixing stage and to subject the N downmix signals to high frequency reconstruction encoding, whereby the high frequency reconstruction encoding stage is configured to extract high frequency reconstruction parameters which enable high frequency reconstruction of the N downmix signals above a second cross-over frequency.

The encoder further comprises a parametric encoding stage configured to receive the M signals from the receiving stage and the N downmix signals from the downmixing stage, and to subject the M signals to parametric encoding for the frequency range corresponding to frequencies above the first cross-over frequency, whereby the parametric encoding stage is configured to extract upmix parameters which enable upmixing of the N downmix signals into M reconstructed signals corresponding to the M channels for the frequency range above the first cross-over frequency.

The encoder further comprises a second waveform-coding stage configured to receive the N downmix signals from the downmixing stage and to generate N waveform-coded downmix signals by waveform-coding the N downmix signals for a frequency range corresponding to frequencies between the first and the second cross-over frequency, whereby the N waveform-coded downmix signals comprise spectral coefficients corresponding to frequencies between the first cross-over frequency and the second cross-over frequency.

According to an embodiment, subjecting the N downmix signals to high frequency reconstruction encoding in the high frequency reconstruction encoding stage is performed in a frequency domain, preferably a Quadrature Mirror Filters, QMF, domain.

According to a further embodiment, subjecting the M signals to parametric encoding in the parametric encoding stage is performed in a frequency domain, preferably a Quadrature Mirror Filters, QMF, domain.

According to yet another embodiment, generating M waveform-coded signals by individually waveform-coding the M signals in the first waveform-coding stage comprises applying an overlapping windowed transform to the M signals, wherein different overlapping window sequences are used for at least two of the M signals.

According to embodiments, the encoder may further comprise a third wave-form encoding stage configured to generate a further waveform-coded signal by waveform-coding one of the M signals for a frequency range corresponding to a subset of the frequency range above the first cross-over frequency.

According to embodiments, the encoder may comprise a control signal generating stage. The control signal generating stage is configured to generate a control signal indicating how to interleave the further waveform-coded signal with a parametric reconstruction of one of the M signals in a decoder. For example, the control signal may indicate a frequency range and a time range for which the further waveform-coded signal is to be interleaved with one of the M upmix signals.

<FIG> is a generalized block diagram of a decoder <NUM> in a multi-channel audio processing system for reconstructing M encoded channels. The decoder <NUM> comprises three conceptual parts <NUM>, <NUM>, <NUM> that will be explained in greater detail in conjunction with <FIG> below. In first conceptual part <NUM>, the decoder receives N waveform-coded downmix signals and M waveform-coded signals representing the multi-channel audio signal to be decoded, wherein <NUM> <N<M. In the illustrated example, N is set to <NUM>. In the second conceptual part <NUM>, the M waveform-coded signals are downmixed and combined with the N waveform-coded downmix signals. High frequency reconstruction (HFR) is then performed for the combined downmix signals. In the third conceptual part <NUM>, the high frequency reconstructed signals are upmixed, and the M waveform-coded signals are combined with the upmix signals to reconstruct M encoded channels.

In the exemplary embodiment described in conjunction with <FIG>, the reconstruction of an encoded <NUM> surround sound is described. It may be noted that the low frequency effect signal is not mentioned in the described embodiment or in the drawings. This does not mean that any low frequency effects are neglected. The low frequency effects (Lfe) are added to the reconstructed <NUM> channels in any suitable way well known by a person skilled in the art. It may also be noted that the described decoder is equally well suited for other types of encoded surround sound such as <NUM> or <NUM> surround sound.

<FIG> illustrates the first conceptual part <NUM> of the decoder <NUM> in <FIG>. The decoder comprises two receiving stages <NUM>, <NUM>. In the first receiving stage <NUM>, a bit-stream <NUM> is decoded and dequantized into two waveform-coded downmix signals 208a-b. Each of the two waveform-coded downmix signals 208a-b comprises spectral coefficients corresponding to frequencies between a first cross-over frequency ky and a second cross-over frequency kx.

In the second receiving stage <NUM>, the bit-stream <NUM> is decoded and dequantized into five waveform-coded signals 210a-e. Each of the five waveform-coded downmix signals 208a-e comprises spectral coefficients corresponding to frequencies up to the first cross-over frequency kx.

By way of example, the signals 210a-e comprises two channel pair elements and one single channel element for the centre. The channel pair elements may for example be a combination of the left front and left surround signal and a combination of the right front and the right surround signal. A further example is a combination of the left front and the right front signals and a combination of the left surround and right surround signal. These channel pair elements may for example be coded in a sum-and-difference format. All five signals 210a-e may be coded using overlapping windowed transforms with independent windowing and still be decodable by the decoder. This may allow for an improved coding quality and thus an improved quality of the decoded signal.

By way of example, the first cross-over frequency ky is <NUM>. By way of example, the second cross-over frequency kx lies within the range of is <NUM>-<NUM>. It should be noted that the first cross-over frequency ky can vary, even on an individual signal basis, i.e. the decoder can detect that a signal component in a specific output signal may not be faithfully reproduced by the stereo downmix signals 208a-b and can for that particular time instance increase the bandwidth, i.e. the first cross-over frequency ky, of the relevant waveform coded signal, i.e. 210a-e, to do proper wavefrom coding of the signal component.

As will be described later on in this description, the remaining stages of the decoder <NUM> typically operates in the Quadrature Mirror Filters (QMF) domain. For this reason, each of the signals 208a-b, 210a-e received by the first and second receiving stage <NUM>, <NUM>, which are received in a modified discrete cosine transform (MDCT) form, are transformed into the time domain by applying an inverse MDCT <NUM>. Each signal is then transformed back to the frequency domain by applying a QMF transform <NUM>.

In <FIG>, the five waveform-coded signals <NUM> are downmixed to two downmix signals <NUM>, <NUM> comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency ky at a downmix stage <NUM>. These downmix signals <NUM>, <NUM> may be formed by performing a downmix on the low pass multi-channel signals 210a-e using the same downmixing scheme as was used in an encoder to create the two downmix signals 208a-b shown in <FIG>.

The two new downmix signals <NUM>, <NUM> are then combined in a first combing stage <NUM>, <NUM> with the corresponding downmix signal 208a-b to form a combined downmix signals 302a-b. Each of the combined downmix signals 302a-b thus comprises spectral coefficients corresponding to frequencies up to the first cross-over frequency ky originating from the downmix signals <NUM>, <NUM> and spectral coefficients corresponding to frequencies between the first cross-over frequency ky and the second cross-over frequency kx originating from the two waveform-coded downmix signals 208a-b received in the first receiving stage <NUM> (shown in <FIG>).

The decoder further comprises a high frequency reconstruction (HFR) stage <NUM>. The HFR stage is configured to extend each of the two combined downmix signals 302a-b from the combining stage to a frequency range above the second cross-over frequency kx by performing high frequency reconstruction. The performed high frequency reconstruction may according to some embodiments comprise performing spectral band replication, SBR. The high frequency reconstruction may be done by using high frequency reconstruction parameters which may be received by the HFR stage <NUM> in any suitable way.

The output from the high frequency reconstruction stage <NUM> is two signals 304a-b comprising the downmix signals 208a-b with the HFR extension <NUM>, <NUM> applied. As described above, the HFR stage <NUM> is performing high frequency reconstruction based on the frequencies present in the input signal 210a-e from the second receiving stage <NUM> (shown in <FIG>) combined with the two downmix signals 208a-b. Somewhat simplified, the HFR range <NUM>, <NUM> comprises parts of the spectral coefficients from the downmix signals <NUM>, <NUM> that has been copied up to the HFR range <NUM>, <NUM>. Consequently, parts of the five waveform-coded signals 210a-e will appear in the HFR range <NUM>, <NUM> of the output <NUM> from the HFR stage <NUM>.

It should be noted that the downmixing at the downmixing stage <NUM> and the combining in the first combining stage <NUM>, <NUM> prior to the high frequency reconstruction stage <NUM>, can be done in the time-domain, i.e. after each signal has transformed into the time domain by applying an inverse modified discrete cosine transform (MDCT) <NUM> (shown in <FIG>). However, given that the waveform-coded signals 210a-e and the waveform-coded downmix signals 208a-b can be coded by a waveform coder using overlapping windowed transforms with independent windowing, the signals 210a-e and 208a-b may not be seamlessly combined in a time domain. Thus, a better controlled scenario is attained if at least the combining in the first combining stage <NUM>, <NUM> is done in the QMF domain.

<FIG> illustrates the third and final conceptual part <NUM> of the decoder <NUM>. The output <NUM> from the HFR stage <NUM> constitutes the input to an upmix stage <NUM>. The upmix stage <NUM> creates a five signal output 404a-e by performing parametric upmix on the frequency extended signals 304a-b. Each of the five upmix signals 404a-e corresponds to one of the five encoded channels in the encoded <NUM> surround sound for frequencies above the first cross-over frequency ky. According to an exemplary parametric upmix procedure, the upmix stage <NUM> first receives parametric mixing parameters. The upmix stage <NUM> further generates decorrelated versions of the two frequency extended combined downmix signals 304a-b. The upmix stage <NUM> further subjects the two frequency extended combined downmix signals 304a-b and the decorrelated versions of the two frequency extended combined downmix signals 304a-b to a matrix operation, wherein the parameters of the matrix operation are given by the upmix parameters. Alternatively, any other parametric upmixing procedure known in the art may be applied. Applicable parametric upmixing procedures are described for example in <NPL>).

The output 404a-e from the upmix stage <NUM> does thus not comprising frequencies below the first cross-over frequency ky. The remaining spectral coefficients corresponding to frequencies up to the first cross-over frequency ky exists in the five waveform-coded signals 210a-e that has been delayed by a delay stage <NUM> to match the timing of the upmix signals <NUM>.

The decoder <NUM> further comprises a second combining stage <NUM>, <NUM>. The second combining stage <NUM>, <NUM> is configured to combine the five upmix signals 404a-e with the five waveform-coded signals 210a-e which was received by the second receiving stage <NUM> (shown in <FIG>).

It may be noted that any present Lfe signal may be added as a separate signal to the resulting combined signal <NUM>. Each of the signals <NUM> is then transformed to the time domain by applying an inverse QMF transform <NUM>. The output from the inverse QMF transform <NUM> is thus the fully decoded <NUM> channel audio signal.

<FIG> illustrates a decoding system <NUM>' being a modification of the decoding system <NUM> of <FIG>. The decoding system <NUM>' has conceptual parts <NUM>', <NUM>', and <NUM>' corresponding to the conceptual parts <NUM>, <NUM>, and <NUM> of <FIG>. The difference between the decoding system <NUM>' of <FIG> and the decoding system of <FIG> is that there is a third receiving stage <NUM> in the conceptual part <NUM>' and an interleaving stage <NUM> in the third conceptual part <NUM>'.

The third receiving stage <NUM> is configured to receive a further waveform-coded signal. The further waveform-coded signal comprises spectral coefficients corresponding to a subset of the frequencies above the first cross-over frequency. The further waveform-coded signal may be transformed into the time domain by applying an inverse MDCT <NUM>. It may then be transformed back to the frequency domain by applying a QMF transform <NUM>.

It is to be understood that the further waveform-coded signal may be received as a separate signal. However, the further waveform-coded signal may also form part of one or more of the five waveform-coded signals 210a-e. In other words, the further waveform-coded signal may be jointly coded with one or more of the five waveform-coded signals 201a-e, for instance using the same MDCT transform. If so, the third receiving stage <NUM> corresponds to the second receiving stage, i.e. the further waveform-coded signal is received together with the five waveform-coded signals 210a-e via the second receiving stage <NUM>.

<FIG> illustrates the third conceptual part <NUM>' of the decoder <NUM>' of <FIG> in more detail. The further waveform-coded signal <NUM> is input to the third conceptual part <NUM>' in addition to the high frequency extended downmix-signals 304a-b and the five waveform-coded signals 210a-e. In the illustrated example, the further waveform-coded signal <NUM> corresponds to the third channel of the five channels. The further waveform-coded signal <NUM> further comprises spectral coefficients corresponding to a frequency interval starting from the first cross-over frequency ky. However, the form of the subset of the frequency range above the first cross-over frequency covered by the further waveform-coded signal <NUM> may of course vary in different embodiments. It is also to be noted that a plurality of waveform-coded signals 710a-e may be received, wherein the different waveform-coded signals may correspond to different output channels. The subset of the frequency range covered by the plurality of further waveform-coded signals 710a-e may vary between different ones of the plurality of further waveform-coded signals 710a-e.

The further waveform-coded signal <NUM> may be delayed by a delay stage <NUM> to match the timing of the upmix signals <NUM> being output from the upmix stage <NUM>. The upmix signals <NUM> and the further waveform-coded signal <NUM> are then input to an interleave stage <NUM>. The interleave stage <NUM> interleaves, i.e., combines the upmix signals <NUM> with the further waveform-coded signal <NUM> to generate an interleaved signal <NUM>. In the present example, the interleaving stage <NUM> thus interleaves the third upmix signal 404c with the further waveform-coded signal <NUM>. The interleaving may be performed by adding the two signals together. However, typically, the interleaving is performed by replacing the upmix signals <NUM> with the further waveform-coded signal <NUM> in the frequency range and time range where the signals overlap.

The interleaved signal <NUM> is then input to the second combining stage, <NUM>, <NUM>, where it is combined with the waveform-coded signals 201a-e to generate an output signal <NUM> in the same manner as described with reference to <FIG>. It is to be noted that the order of the interleave stage <NUM> and the second combining stage <NUM>, <NUM> may be reversed so that the combining is performed before the interleaving.

Also, in the situation where the further waveform-coded signal <NUM> forms part of one or more of the five waveform-coded signals 210a-e, the second combining stage <NUM>, <NUM>, and the interleave stage <NUM> may be combined into a single stage. Specifically, such a combined stage would use the spectral content of the five waveform-coded signals 210a-e for frequencies up to the first cross-over frequency ky. For frequencies above the first cross-over frequency, the combined stage would use the upmix signals <NUM> interleaved with the further waveform-coded signal <NUM>.

The interleave stage <NUM> may operate under the control of a control signal. For this purpose the decoder <NUM>' may receive, for example via the third receiving stage <NUM>, a control signal which indicates how to interleave the further waveform-coded signal with one of the M upmix signals. For example, the control signal may indicate the frequency range and the time range for which the further waveform-coded signal <NUM> is to be interleaved with one of the upmix signals <NUM>. For instance, the frequency range and the time range may be expressed in terms of time/frequency tiles for which the interleaving is to be made. The time/frequency tiles may be time/frequency tiles with respect to the time/frequency grid of the QMF domain where the interleaving takes place.

The control signal may use vectors, such as binary vectors, to indicate the time/frequency tiles for which interleaving are to be made. Specifically, there may be a first vector relating to a frequency direction, indicating the frequencies for which interleaving is to be performed. The indication may for example be made by indicating a logic one for the corresponding frequency interval in the first vector. There may also be a second vector relating to a time direction, indicating the time intervals for which interleaving are to be performed. The indication may for example be made by indicating a logic one for the corresponding time interval in the second vector. For this purpose, a time frame is typically divided into a plurality of time slots, such that the time indication may be made on a sub-frame basis. By intersecting the first and the second vectors, a time/frequency matrix may be constructed. For example, the time/frequency matrix may be a binary matrix comprising a logic one for each time/frequency tile for which the first and the second vectors indicate a logic one. The interleave stage <NUM> may then use the time/frequency matrix upon performing interleaving, for instance such that one or more of the upmix signals <NUM> are replaced by the further wave-form coded signal <NUM> for the time/frequency tiles being indicated, such as by a logic one, in the time/frequency matrix.

It is noted that the vectors may use other schemes than a binary scheme to indicate the time/frequency tiles for which interleaving are to be made. For example, the vectors could indicate by means of a first value such as a zero that no interleaving is to be made, and by second value that interleaving is to be made with respect to a certain channel identified by the second value.

<FIG> shows by way of example a generalized block diagram of an encoding system <NUM> for a multi-channel audio processing system for encoding M channels in accordance with an embodiment.

In the exemplary embodiment described in <FIG>, the encoding of a <NUM> surround sound is described. Thus, in the illustrated example, M is set to five. It may be noted that the low frequency effect signal is not mentioned in the described embodiment or in the drawings. This does not mean that any low frequency effects are neglected. The low frequency effects (Lfe) are added to the bitstream <NUM> in any suitable way well known by a person skilled in the art. It may also be noted that the described encoder is equally well suited for encoding other types of surround sound such as <NUM> or <NUM> surround sound. In the encoder <NUM>, five signals <NUM>, <NUM> are received at a receiving stage (not shown). The encoder <NUM> comprises a first waveform-coding stage <NUM> configured to receive the five signals <NUM>, <NUM> from the receiving stage and to generate five waveform-coded signals <NUM> by individually waveform-coding the five signals <NUM>, <NUM>. The waveform-coding stage <NUM> may for example subject each of the five received signals <NUM>, <NUM> to a MDCT transform. As discussed with respect to the decoder, the encoder may choose to encode each of the five received signals <NUM>, <NUM> using a MDCT transform with independent windowing. This may allow for an improved coding quality and thus an improved quality of the decoded signal.

The five waveform-coded signals <NUM> are waveform-coded for a frequency range corresponding to frequencies up to a first cross-over frequency. Thus, the five waveform-coded signals <NUM> comprise spectral coefficients corresponding to frequencies up to the first cross-over frequency. This may be achieved by subjecting each of the five waveform-coded signals <NUM> to a low pass filter. The five waveform-coded signals <NUM> are then quantized <NUM> according to a psychoacoustic model. The psychoacoustic model are configure to as accurate as possible, considering the available bit rate in the multi-channel audio processing system, reproducing the encoded signals as perceived by a listener when decoded on a decoder side of the system.

As discussed above, the encoder <NUM> performs hybrid coding comprising discrete multi-channel coding and parametric coding. The discrete multi-channel coding is performed by in the waveform-coding stage <NUM> on each of the input signals <NUM>, <NUM> for frequencies up to the first cross-over frequency as described above. The parametric coding is performed to be able to, on a decoder side, reconstruct the five input signals <NUM>, <NUM> from N downmix signals for frequencies above the first cross-over frequency. In the illustrated example in <FIG>, N is set to <NUM>. The downmixing of the five input signals <NUM>, <NUM> is performed in a downmixing stage <NUM>. The downmixing stage <NUM> advantageously operates in a QMF domain. Therefore, prior to being input to the downmixing stage <NUM>, the five signals <NUM>, <NUM> are transformed to a QMF domain by a QMF analysis stage <NUM>. The downmixing stage performs a linear downmixing operation on the five signals <NUM>, <NUM> and outputs two downmix signal <NUM>, <NUM>.

These two downmix signals <NUM>, <NUM> are received by a second waveform-coding stage <NUM> after they have been transformed back to the time domain by being subjected to an inverse QMF transform <NUM>. The second waveform-coding stage <NUM> is generating two waveform-coded downmix signals by waveform-coding the two downmix signals <NUM>, <NUM> for a frequency range corresponding to frequencies between the first and the second cross-over frequency. The waveform-coding stage <NUM> may for example subject each of the two downmix signals to a MDCT transform. The two waveform-coded downmix signals thus comprise spectral coefficients corresponding to frequencies between the first cross-over frequency and the second cross-over frequency. The two waveform-coded downmix signals are then quantized <NUM> according to the psychoacoustic model.

To be able to reconstruct the frequencies above the second cross-over frequency on a decoder side, high frequency reconstruction, HFR, parameters <NUM> are extracted from the two downmix signals <NUM>, <NUM>. These parameters are extracted at a HFR encoding stage <NUM>.

To be able to reconstruct the five signals from the two downmix signals <NUM>, <NUM> on a decoder side, the five input signals <NUM>, <NUM> are received by the parametric encoding stage <NUM>. The five signals <NUM>, <NUM> are subjected to parametric encoding for the frequency range corresponding to frequencies above the first cross-over frequency. The parametric encoding stage <NUM> is then configured to extract upmix parameters <NUM> which enable upmixing of the two downmix signals <NUM>, <NUM> into five reconstructed signals corresponding to the five input signals <NUM>, <NUM> (i.e. the five channels in the encoded <NUM> surround sound) for the frequency range above the first cross-over frequency. It may be noted that the upmix parameters <NUM> is only extracted for frequencies above the first cross-over frequency. This may reduce the complexity of the parametric encoding stage <NUM>, and the bitrate of the corresponding parametric data.

It may be noted that the downmixing <NUM> can be accomplished in the time domain. In that case the QMF analysis stage <NUM> should be positioned downstreams the downmixing stage <NUM> prior to the HFR encoding stage <NUM> since the HRF encoding stage <NUM> typically operates in the QMF domain. In this case, the inverse QMF stage <NUM> can be omitted.

The encoder <NUM> further comprises a bitstream generating stage, i.e. bitstream multiplexer, <NUM>. According to the exemplary embodiment of the encoder <NUM>, the bitstream generating stage is configured to receive the five encoded and quantized signal <NUM>, the two parameters signals <NUM>, <NUM> and the two encoded and quantized downmix signals <NUM>. These are converted into a bitstream <NUM> by the bitstream generating stage <NUM>, to further be distributed in the multi-channel audio system.

In the described multi-channel audio system, a maximum available bit rate often exists, for example when streaming audio over the internet. Since the characteristics of each time frame of the input signals <NUM>, <NUM> differs, the exact same allocation of bits between the five waveform-coded signals <NUM> and the two downmix waveform-coded signals <NUM> may not be used. Furthermore, each individual signal <NUM> and <NUM> may need more or less allocated bits such that the signals can be reconstructed according to the psychoacoustic model. According to an exemplary embodiment, the first and the second waveform-coding stage <NUM>, <NUM> share a common bit reservoir. The available bits per encoded frame are first distributed between the first and the second waveform-encoding stage <NUM>, <NUM> depending on the characteristics of the signals to be encoded and the present psychoacoustic model. The bits are then distributed between the individual signals <NUM>, <NUM> as described above. The number of bits used for the high frequency reconstruction parameters <NUM> and the upmix parameters <NUM> are of course taken in account when distributing the available bits. Care is taken to adjust the psychoacoustic model for the first and the second waveform-coding stage <NUM>, <NUM> for a perceptually smooth transition around the first cross-over frequency with respect to the number of bits allocated at the particular time frame.

<FIG> illustrates an alternative embodiment of an encoding system <NUM>. The difference between the encoding system <NUM> of <FIG> and the encoding system <NUM> of <FIG> is that the encoder <NUM> is arranged to generate a further waveform-coded signal by waveform-coding one or more of the input signals <NUM>, <NUM> for a frequency range corresponding to a subset of the frequency range above the first cross-over frequency.

For this purpose, the encoder <NUM> comprises an interleave detecting stage <NUM>. The interleave detecting stage <NUM> is configured to identify parts of the input signals <NUM>, <NUM> that are not well reconstructed by the parametric reconstruction as encoded by the parametric encoding stage <NUM> and the high frequency reconstruction encoding stage <NUM>. For example, the interleave detection stage <NUM> may compare the input signals <NUM>, <NUM>, to a parametric reconstruction of the input signal <NUM>, <NUM> as defined by the parametric encoding stage <NUM> and the high frequency reconstruction encoding stage <NUM>. Based on the comparison, the interleave detecting stage <NUM> may identify a subset <NUM> of the frequency range above the first cross-over frequency which is to be waveform-coded. The interleave detecting stage <NUM> may also identify the time range during which the identified subset <NUM> of the frequency range above the first cross-over frequency is to be waveform-coded. The identified frequency and time subsets <NUM>, <NUM> may be input to the first waveform encoding stage <NUM>. Based on the received frequency and time subsets <NUM> and <NUM>, the first waveform encoding stage <NUM> generates a further waveform-coded signal <NUM> by waveform-coding one or more of the input signals <NUM>, <NUM> for the time and frequency ranges identified by the subsets <NUM>, <NUM>. The further waveform-coded signal <NUM> may then be encoded and quantized by stage <NUM> and added to the bit-stream <NUM>.

The interleave detecting stage <NUM> may further comprise a control signal generating stage. The control signal generating stage is configured to generate a control signal <NUM> indicating how to interleave the further waveform-coded signal with a parametric reconstruction of one of the input signals <NUM>, <NUM> in a decoder. For example, the control signal may indicate a frequency range and a time range for which the further waveform-coded signal is to be interleaved with a parametric reconstruction as described with reference to <FIG>. The control signal may be added to the bitstream <NUM>.

Further embodiments of the present disclosure will become apparent to a person skilled in the art after studying the description above. Even though the present description and drawings disclose embodiments and examples, the deisclosure is not restricted to these specific examples. Numerous modifications and variations can be made without departing from the scope of the invention, which is defined by the accompanying claims. Any reference signs appearing in the claims are not to be understood as limiting their scope.

Additionally, variations to the disclosed embodiments can be understood and effected by the skilled person in practicing the disclosure, from a study of the drawings, the disclosure, and the appended claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage.

Claim 1:
A method for reconstructing a multi-channel audio signal in a decoder of a multi-channel audio processing system, comprising the steps of:
receiving N frequency extended downmix signals (<NUM>) comprising spectral coefficients corresponding to frequencies between a first and second cross-over frequency, ky, kx, and above the second cross-over frequency kx, wherein kx > ky;
generating M upmix signals (<NUM>) comprising spectral coefficients corresponding to frequencies above the first cross-over frequency ky by parametrically upmixing the N downmix signals into M upmix signals, and wherein the M upmix signals (<NUM>) do not comprise frequencies below the first cross-over frequency ky, each of the M upmix signals corresponding to a respective one of M encoded channels, wherein <NUM><N<M;
receiving a first waveform-coded signal (<NUM>) comprising spectral coefficients corresponding to a subset of frequencies above the first cross-over frequency ky;
receiving M second waveform-coded signals (<NUM>) comprising spectral coefficients corresponding to frequencies up to the first cross-over frequency, each of the M second waveform-coded signals (<NUM>) corresponding to a respective one of the M encoded channels; and
interleaving the first waveform-coded signal (<NUM>) with one of the M upmix signals such that an interleaved version of said one of the M upmix signals is obtained, wherein the interleaving comprises adding the first waveform-coded signal with one of the M upmix signals or replacing one of the M upmix signals with the first waveform-coded signal in the subset of the frequencies above the first cross-over frequency corresponding to the spectral coefficients of the first waveform-coded signal; and
combining the M second waveform-coded signals with the M upmix signals before the interleaving, after the interleaving, or in a combined step with the interleaving to generate an M-channel output signal (<NUM>).