Patent Description:
Acoustic echo cancellation is desirable for a wide range of applications. For instance, it facilitates human-machine interaction with far-field sound capture and barge-in functionality and enables full-duplex voice communication. In order to perform AEC, typically a microphone signal or plurality of microphone signals and AEC reference signal or a AEC reference signal comprising one, two or more channels are used. Generally, all of the methods in the literature use the loudspeaker driving signals as reference signal as shown by <FIG>.

<FIG> shows an audio processing path <NUM> as well as an acoustic echo cancellation path <NUM>.

The audio processing path <NUM> comprises an audio processor <NUM> as well as one or more loudspeakers <NUM>. The loudspeakers <NUM> may be formed by a conventional multi-speaker setup (<NUM> or <NUM>, etc.) or by a soundbar having at least one, preferable two or three or more transducers.

The audio processor <NUM> receives a multichannel audio (e.g., a <NUM> or <NUM> signal) and processes same, such that a surround sound can be reproduced by use of the loudspeaker <NUM>. For example, the audio processor <NUM> is configured to process the multichannel audio signal MS to obtain spatial components SC via which the sound bar <NUM> is controlled.

The echo cancellation path <NUM> comprises an acoustic echo cancellation unit <NUM> which is configured to calculate echo cancellation components based on an input signal IS and a reference signal RS. As input signal IS a microphone signal from one or more microphones (cf. reference numeral <NUM>) is used. Typically, the spatial components SC are used as reference signal RS. Therefore, the echo cancellation unit <NUM> comprise an input for the reference signal RS which is connected to the audio processor <NUM> and for the one or more microphones <NUM>. The echo cancellation parameters are output to the backend <NUM>.

A significant AEC performance degradation may be expected when a large number of loudspeakers <NUM> are used and/or there exists a high correlation between the loudspeaker driving signals as is the usual case for playback of multichannel audio with soundbars. The correlation between the loudspeaker driving signals can be reduced by applying decorrelation methods but this comes at a cost of reproduction fidelity and is therefore undesirable for some applications. Therefore, there is the need for an improved approach.

Document <CIT> discloses an echo cancelling apparatus, comprising a controller configured to perform steps comprising: receiving a single combined reference signal comprising a combination of a first channel of an audio signal for reproduction by a first transducer and a second channel of the audio signal for reproduction by a second transducer; pre-whitening the single combined reference signal in a frequency domain; and generating an acoustic echo cancellation signal based, on the pre-whitened single combination reference signal.

It is an objective of the present invention to provide a concept for echo cancellation having an improved tradeoff for echo cancellation quality and the field of applications. Especially, it is an objective to provide an echo cancellation concept applicable for soundbars.

This objective is solved by the subject-matter of the independent claims.

Embodiments of the present invention provide an acoustic echo cancellation unit according to claim <NUM>, comprising: an audio processor configured to receive a multichannel audio signal and comprising: a first stage which is configured to process the multichannel audio signal to obtain a first set of spatial audio components; and a second stage which is configured to process the first set of spatial audio components to obtain a second set of spatial audio components; an echo cancellation processor configured to perform echo cancellation by use of the first set of spatial audio components or a deviated version of the first set of spatial audio components as reference signal.

According to further embodiments, the echo cancellation processor comprises a spatial component combiner configured to process the first set of spatial audio components to obtain the deviated version of the first set of spatial components to be used as reference signal.

Embodiments of the present invention are based on the finding, that it is beneficial to apply a two stage process for sound processing, wherein the output signal/spatial component signal of the first stage is preferred to be used as reference signal for the acoustic echo cancellation (when compared to the second stage). Here, the first spatial component signal which can be further processed to obtain the final spatial components and which is more suitable for acoustic echo cancellation than the final spatial components. For example, the interim signal can comprise component signals associated with the room directions to be rendered as 3D sound scene by the soundbar. According to a preferred embodiment, the first spatial components are further processed before being used as reference signal. Therefore, also a deviated version of the first set of spatial components may be used. This deviated version is obtained by use of a spatial component combiner, e.g., by performing a linear processing or another processing.

Expressed in other words, this means that the first spatial components are accessed and processed with a spatial component combiner to obtain the reference signal. The resulting reference signal are used for - AEC. The use of the intermediate signals, i.e., the first spatial components in conjunction with a spatial component combiner enables to obtain a proper AEC reference signal or proper AEC reference signal what makes the AEC applicable in practice.

According to further embodiments, there is provided an acoustic echo cancellation unit, wherein the spatial component combiner is configured to output at least one reference signal or a reference signal comprising one, two or more channels. According to an embodiment, the spatial component combiner is configured to perform a linear combination of the first set of spatial components to obtain the reference signal and/or to apply a time-invariant downmix matrix to the first set of spatial components to obtain the reference signal. For Example the spatial component combiner may be configured to perform its processing based on the following formula
<MAT>
wherein Ri is the ith reference signal, βki are the weights, Ck is the kth spatial component signal. Here, each reference signal is obtained from a subset of the first set of spatial components. In addition, each spatial component channel can only be used in maximal one reference signal channel.

According to embodiments, the spatial component combiner is configured to reduce the number of spatial components to obtain at least one reference signal.

Regarding the echo cancellation processor, it should be noted that typically, the echo cancellation is based on the microphone signal received via a microphone input. According to further embodiments, there is provided the acoustic echo cancellation unit, wherein the echo cancellation processor is configured to perform the echo cancellation using a single or multichannel adaptive filter or a single or multichannel adaptive filter which is configurable based on a comparison between the reference signal and a microphone signal.

According to the invention, there is provided an acoustic echo cancellation unit wherein the first stage is configured to perform a non-linear processing or a time-varying processing or a highly time-varying processing to obtain the first set of spatial audio components. According to another embodiment, the second stage is configured to output the second set of spatial audio components to a playback device or soundbar. Note, that according to further embodiments, there is provided an acoustic echo cancellation, wherein the set of spatial audio components enable to directly control one or more transducers of a playback device or to control one or more transducers of a playback device or soundbar by use of one or more amplifiers. For example, there is provided an acoustic echo cancellation unit, wherein the second stage is configured to perform a linear processing.

Another embodiment provides a playback device or soundbar comprising an acoustic echo cancellation unit.

According to a further embodiment a method for acoustic echo cancellation according to claim <NUM> is provided.

According to further embodiments, this method may be performed by use of a computer.

Embodiments of the present invention will subsequently be discussed referring to the enclosed figures, wherein.

Below, embodiments of the present invention will subsequently be discussed referring to the enclosed figures, wherein identical reference numerals are provided to objects having identical or similar function, so that a description thereof is interchangeable and mutually applicable.

<FIG> shows a sound cancellation unit <NUM> comprising the two paths <NUM> and <NUM>. Within the path <NUM> a multichannel audio signal MS is processed so as to enable the playback of the multichannel audio signal MS by use of a sound reproduction device <NUM>, e.g., a soundbar. Here, the processing as described in context of the prior art is subdivided into two stages <NUM> and <NUM>. The first stage <NUM> processes the multichannel audio signal MS to obtain an interim signal which is marked by the reference numeral SC_1. This interim signal SC_1 represents a first set of spatial component signals. This interim signal SC_1 is then further processed by use of the second stage <NUM> to obtain the second set of spatial component signals SC_2 enabling to derive the loudspeaker <NUM> directly. Directly driving means that no further spatial processing is used. , the transducers of the soundbar <NUM> may be driven by the second set of spatial components (e.g., after amplifying the spatial components).

For example, the first stage <NUM> performs non-linear signal processing and/or (highly) time-varying processing. The second processing stage <NUM> may mainly perform linear time-invariant processing steps. Background for the subdividing into the two stages is that the processing steps performed by the second stage <NUM>, e.g., the linear time-invariant processing steps can have a negative influence to the suitability for an echo cancellation.

The echo cancellation path <NUM> performs the echo cancellation based on the first set of spatial components SC_1. For this, the echo cancellation unit <NUM> receives the signal SC_1 from the first stage of audio processing <NUM>. The echo cancellation unit <NUM> comprises at least an echo cancellation processor <NUM> performing the echo cancellation based on the reference signal RS. According to embodiments, the first set of spatial components SC_1may be used as reference signal RS. According to a further (preferred) embodiment, a deviated version of the first set of spatial components SC_1 may be used as reference signal RS. Therefore, the echo cancellation unit <NUM> may optionally comprise a processor <NUM> (e.g., a so-called combiner). This combiner <NUM> performs a processing, for example a linear processing to obtain the reference signal RS based on the first set of spatial components SC_1.

According to the invention the echo cancellation performed by the entity <NUM> uses beside the reference signal RS another input signal received via one or more microphones (not shown) as will be discussed below.

Before discussing enhanced embodiments, the improved concept of echo cancellation will be discussed with respect to its method steps.

<FIG> shows the method steps of the echo cancellation method <NUM>. Within the basic implementation, the acoustic echo cancellation <NUM> comprises the three basic steps <NUM>, <NUM> and <NUM>.

The steps <NUM> and <NUM> represent the audio processing performed within the first stages.

Within the first stage <NUM>, the multichannel audio signal (cf. reference numeral MS within <FIG>) to obtain the first set of spatial components SC_1 which is then further processed by the second stage <NUM> to obtain the second set of spatial components SC_2. In parallel to this second stage processing <NUM> plus <NUM>, the echo cancellation processing is performed. This echo cancellation processing uses as input the signal SC_1, so that the first step of the echo cancellation is performed subsequently to the step <NUM>. Here, the basic step of the echo cancellation is marked by the reference numeral <NUM> performing the echo cancellation based on the signal SC_1 used as reference signal. As discussed above, this process step <NUM> may use additional signals, e.g., a microphone signal as input signal. Optionally, another processing step <NUM> may be arranged before the step <NUM>. This step <NUM> enables the processing of the first set of component signals SC_1 to obtain the reference signal RS.

Below, with respect to <FIG>, optional elements of the echo cancellation processing will be discussed.

<FIG> shows an echo cancellation unit <NUM>' comprising the audio processor <NUM> with the two stages <NUM> and <NUM> as well as the echo cancellation unit <NUM>' comprising the two main stages <NUM> and <NUM> (cf. <FIG>) as well as one or more microphones <NUM> and a backend processing <NUM>.

Regarding the microphones <NUM> and the loudspeakers <NUM> it should be noted that same may be combined in one common or a plurality of common housings.

Regarding the backend processing <NUM> it should be noted that same may be used as human-machine interface, e.g., by use of voice recognition/far-field sound capturing or for applications like full-duplex communication.

Before, the functionality of the entire system <NUM>' will be discussed.

Due to problems encountered by the state-of-the-art methods, we propose to use intermediate signals of the soundbar processing instead of the signals driving the loudspeakers as the AEC reference signal. The general approach is illustrated in <FIG>. The audio processor <NUM> receives a multichannel audio signal <NUM> e.g. in a <NUM> surround format or a <NUM> + <NUM> immersive format. The audio input signals <NUM> are processed by a first processing block <NUM> to generate a set of first spatial component signals SC_1. The resulting set of spatial component signals SC_1 are not suitable for direct playback via the loudspeakers <NUM> of the soundbar (or any other multi-loudspeaker playback systems), but are the basis for further processing by the audio processing stage <NUM>. The first spatial component signals SC_1 are input into a second audio processing block <NUM> in which a second set of spatial component SC_2 signals are generated. Typically, these component signals SC_2 are then reproduced by the loudspeakers <NUM> of the soundbar, i.e., they represent the loudspeaker playback signals.

In order to generate a suitable reference signal RS for the MC-AEC <NUM>, the first set of spatial component signals SC_1 are further processed by the spatial component combiner <NUM>. Typically, the spatial component combiner <NUM> determines the AEC reference signal RS by a linear combination of the first set of spatial component signals. In some embodiments, certain AEC reference signal RS may also correspond to one of the spatial component signals SC_1 without any further modification. In typical embodiments, the number of AEC reference signal RS is smaller than the number of spatial component signals SC_1, i.e., the spatial component combiner <NUM> reduces the number of signals. One advantage is that the configuration and the computational complexity of the MC-AEC <NUM> is not directly dependent on the number of loudspeakers <NUM>. The reduction of the computational complexity is especially relevant if the number of loudspeakers <NUM> included in the soundbar is significantly larger compared to the number of AEC reference signal RS. Another advantage is that the statistical properties of the loudspeaker driving signals are often not suitable for using them directly as AEC reference signal RS due to high correlations between the different loudspeaker channels, whereas the AEC reference signal RS derived from the first set of spatial component signals usually have properties that are better suited for adaptive filtering by the MC-AEC <NUM>.

According to embodiments, the echo paths to be modeled by the MC-AEC <NUM> based on the AEC reference signal RS are preferably (only) slowly time-varying and linear. Therefore, it is important to appropriately distribute different processing steps between the first and the second processing blocks <NUM> and <NUM> of the audio processor <NUM>/of the soundbar processing chain <NUM>. For example, any non-linear or highly time-varying processing steps should be applied in the first processor <NUM>, while the second processing block should contain mainly linear time-invariant processing steps <NUM>.

In some embodiments the first processor generates spatial component signals SC_1 associated with the left, right, center, low-frequency, top and rear portion of a 3D sound scene as rendered with the soundbar <NUM>. A suitable implementation of the spatial component combiner <NUM> would generate AEC reference signal <NUM> as a linear combination of the first set of spatial components, e.g., by applying a time-invariant downmix matrix to the spatial components SC_1 as shown in the Equation <NUM>: <MAT> where Ri, is the i'th AEC reference signal, βki, are the weights, Ck is the k'th spatial component signal. For example let the left, right, center, low-frequency, top and rear portions be represented by C<NUM>, C<NUM>, C<NUM>, C<NUM>, C<NUM> and C<NUM>, respectively. If a <NUM> channel AEC reference signal is desired the spatial component combiner could combine the components by applying the following weights: <MAT> <MAT>.

In this case the <NUM> channel AEC reference signal may be obtained as follows: <MAT> <MAT>.

The processing performed by the echo cancellation unit <NUM>' can be described as follows:.

As illustrated this step is performed by the entity <NUM>.

According to a further embodiment, an additional step 5a (subsequent to the step <NUM> before the step <NUM>) may be performed: process first set of spatial components SC_1 with a spatial component combiner <NUM> resulting in the number of AEC reference signal RS being smaller than the number of the spatial components SC_1.

An encoded audio signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Although in above embodiments, the audio processor <NUM> has been described as having just the first stage <NUM> and the second stage <NUM>, it should be noted that same may have additional stages, e.g. an amplification stage at the output of <NUM>, an input stage at the input of <NUM> and/or a stage between <NUM> and <NUM>.

Claim 1:
An acoustic echo cancellation unit, comprising:
an audio processor (<NUM>) configured to receive a multichannel audio signal (MS) and characterised by comprising:
a first stage (<NUM>) which is configured to process the multichannel audio signal (MS) to obtain a first set (SC_1) of spatial audio components representing an interim signal, wherein the first stage (<NUM>) is configured to perform a non-linear processing or a time-varying processing, or a highly time-varying processing, to obtain the first set (SC_1) of spatial audio components; and
a second stage (<NUM>) which is configured to process the first set (SC_1) of spatial audio components to obtain a second set (SC_ <NUM>) of spatial audio components representing loudspeaker playback signals;
an echo cancellation processor (<NUM>) configured to perform echo cancellation by use of the first set (SC_1) of spatial audio components or a deviated version of the first set (SC_1) of spatial audio components as reference signal (RS) and by use of at least one received microphone signal; wherein the deviated version is obtained by use of a spatial component combiner, by performing a linear processing.