Patent Description:
A telecommunications device such as an audio conferencing system generally includes both a loudspeaker and a microphone. The two parties in a communication may be referred to as the near end party and the far end party. The near end party is proximal to a first telecommunications device, and the far end party is at a different location than the near end party and communicates using a second telecommunications device via a wired or wireless telecommunications network. The microphone of the near end device captures not only the speech of the near end party, but may also capture the speech of the far end party that has been output from the loudspeaker at the near end. The output from the loudspeaker that is captured by the microphone is generally referred to as echo. The near end telecommunications device generally includes an echo management system for reducing the echo prior to transmitting the audio captured at the near end to the far end.

The echo management system generally includes an echo cancellation system followed by an echo suppression system. The echo cancellation system is a linear system and typically may include an adaptive filter. The echo remaining in the audio signal after echo cancellation is generally referred to as the "echo residual". The echo suppression system is a nonlinear system that applies additional attenuation to the audio signal in order to reduce the echo residual. The attenuation performed by the echo suppression system is referred to as non-linear in that it may apply different gains to different frequencies or frequency bands.

Telecommunications devices generally implement approximately an aggregate <NUM> dB of echo reduction based on the contributions of the physical echo return loss (e.g., due to the distance between the loudspeaker and the microphone), the echo cancellation and the echo suppression.

<CIT> discloses a spectral domain, non-linear echo cancellation method in a hands-free device.

<NPL>, discloses two alternative modified versions of the post-filter to reduce the effects of the distortion of the near-end speech, which guarantees the quality of speech communications. The proposed post-filters are seamlessly combined with the subband-based AEC system with quite small computational burden.

One issue with existing echo management systems is that laptop computers are increasingly used for telecommunications, such as audioconferencing or videoconferencing. In a laptop computer, the loudspeaker and microphone are in close proximity, making it difficult to achieve the target of <NUM> dB for echo reduction. Given the above, there is a need to improve the echo suppression system in order to reduce the echo residual.

According to the invention, there is provided a method for providing a computer-implemented method of audio processing as defined by claim <NUM>, a non-transitory computer readable medium storing a computer program as defined by claim <NUM>, and an apparatus for audio processing as defined by claim <NUM>.

The following detailed description and accompanying drawings provide a further understanding of the nature and advantages of various implementations.

Described herein are techniques related to echo suppression. In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be evident, however, to one skilled in the art that the present disclosure as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

In the following description, various methods, processes and procedures are detailed. Although particular steps may be described in a certain order, such order is mainly for convenience and clarity. A particular step may be repeated more than once, may occur before or after other steps (even if those steps are otherwise described in another order), and may occur in parallel with other steps. A second step is required to follow a first step only when the first step must be completed before the second step is begun. Such a situation will be specifically pointed out when not clear from the context.

In this document, the terms "and", "or" and "and/or" are used. Such terms are to be read as having an inclusive meaning. For example, "A and B" may mean at least the following: "both A and B", "at least both A and B". As another example, "A or B" may mean at least the following: "at least A", "at least B", "both A and B", "at least both A and B". As another example, "A and/or B" may mean at least the following: "A and B", "A or B". When an exclusive-or is intended, such will be specifically noted (e.g., "either A or B", "at most one of A and B").

This document describes various processing functions that are associated with structures such as blocks, elements, components, circuits, etc. In general, these structures may be implemented by a processor that is controlled by one or more computer programs.

A brief overview describing various embodiments for echo management is as follows. The system first estimates the echo residual power in each band of the input signal by modeling the power gain of the echo canceller for each band as a Gaussian distributed random variable, the mean and variance of which depend on the loopback power and a gain profile estimation. Next, the system estimates the statistical gain profile for each band and for each different input level. The system uses the statistical gain profile to quantize the input level into several segments. Next, the system models the wideband echo residual as a Gaussian mixture model, and calculates the likelihood of local talk. Finally, depending on the likelihood of local talk, the system applies different maximum suppressions to the bands of the input signal. Further details are provided below.

<FIG> is a block diagram of an echo management system <NUM>. The echo management system <NUM> generally removes echo from the audio captured by the near end device prior to transmission to the far end device. For example, when the audio from the far end is received by the near end device and is outputted by the loudspeaker of the near end device, the "echo" corresponds to this output from the loudspeaker that is captured by the microphone of the near end device. The echo management system <NUM> includes an analysis filter bank <NUM>, an echo canceller <NUM>, an echo suppressor <NUM>, and a synthesis filter bank <NUM>.

The analysis filter bank <NUM> receives a reference signal <NUM> (also denoted as x(t)) and a captured signal <NUM> (also denoted as y(t)), performs a transform, and generates a transformed reference signal <NUM> (also denoted as X(k, t')) and a transformed captured signal <NUM> (also denoted as Y(k, t')). The reference signal <NUM> corresponds to the signal received from the far end device that is output by the loudspeaker of the near end device. The captured signal <NUM> corresponds to the audio captured by the microphone of the near end device, which may include local talk (e.g., the speech of a person associated with the near end device) as well as the echo of the output of the loudspeaker (e.g., the speech of a person associated with the far end device).

The transform generally corresponds to transforming an input signal from one domain to another domain (e.g., a time domain to frequency domain transform). The analysis filter bank <NUM> performs filtering and decimation of the reference signal <NUM> and the captured signal <NUM> to generate the transformed reference signal <NUM> and the transformed captured signal <NUM>. For example, the analysis filter bank <NUM> performs filtering and decimation of x(t) to generate X(k, t'), where k indicates the sub-band filter index and t' indicates the new time variable after decimation. Similarly, the analysis filter bank <NUM> performs filtering and decimation of y(t) to generate Y(k, t').

The parameter k may be adjusted as desired to adjust the fineness of the transform bins for a given input signal bandwidth. For example, common bandwidths of the reference signal <NUM> (or the captured signal <NUM>) may be <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc., with corresponding sampling rates of <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc. For a transform bin bandwidth of <NUM>, k may be <NUM> bins for a <NUM> signal, <NUM> bins for a <NUM> signal, etc. when performing the transform (e.g., a fast Fourier transform).

The echo canceller <NUM> receives the transformed reference signal <NUM> and the transformed captured signal <NUM>, performs echo cancellation, and generates an echo residual signal <NUM> (also denoted as V(k, t')). In general, echo cancellation refers to applying a linear filtering to Y(k, t') based on X(k, t') to generate V(k, t'). The echo canceller <NUM> typically implements an adaptive filter. Further details of the echo canceller <NUM> are provided with reference to <FIG>. The echo residual signal <NUM> is referred to as the "echo residual signal" because it results from applying echo cancellation to the transformed captured signal <NUM>; the remaining echo in the signal after echo cancellation, and prior to echo suppression, is "residual echo".

The echo suppressor <NUM> receives the transformed reference signal <NUM> and the echo residual signal <NUM>, performs echo suppression, and generates a modified echo residual signal <NUM> (also denoted as Ṽ(k, t')). In general, echo suppression refers to applying a non-linear attenuation to V(k, t') based on X(k, t') to generate the modified echo residual signal Ṽ(k, t') as a suppressed signal. Further details of the echo suppressor <NUM> are provided with reference to <FIG>.

The synthesis filter bank <NUM> receives the modified echo residual signal <NUM>, performs an inverse transform, and generates a modified captured signal <NUM> (also denoted as ṽ(t), which is a time domain signal). The inverse transform generally corresponds to an inverse of the transform performed by the analysis filter bank <NUM>. The modified captured signal <NUM> may then be transmitted from the near end device to the far end device.

For the echo management system <NUM>, the echo can be modeled as applying a linear filter to the playback signal, for example using Equation (<NUM>): <MAT>.

In Equation (<NUM>), x̂(t) corresponds to the echo to be canceled out, h(t) corresponds to a model of the speaker-room-microphone impulse response (of the room in which the near end device with the speaker and microphone is located), x(t) corresponds to the reference signal <NUM>, and n(t) corresponds to noise.

When there is no uplink voice (e.g., no local talk at the near end, and the microphone captures only the echo of the loudspeaker output; uplink voice may also be referred to as near end voice), we have Equation (<NUM>): <MAT>.

In Equations (<NUM>) and (<NUM>), h(t) may be estimated by various techniques, including a least mean squares (LMS) process, a normalized least mean squares (NLMS) process, a recursive least squares (RLS) process, etc..

Due to the acoustic nature, a typical room impulse response has more than thousands taps, and the adaptive filter thus will require a large amount of computation resources. This is highly undesirable for laptop and desktop computers, and the filter bank <NUM> is used to implement a sub-band adaptive filter to reduce the computation.

<FIG> is a block diagram of the echo canceller <NUM> (see <FIG>). The echo canceller <NUM> includes a plurality of adaptive filters (e.g., k adaptive filters) corresponding to the number of analysis bins. Two adaptive filters <NUM> and <NUM> are shown. The adaptive filter <NUM> receives X(<NUM>, t') and Y(<NUM>, t'), applies an adaptive filter H<NUM>, and generates V(<NUM>, t'). The adaptive filter <NUM> receives X(k, t') and Y(k, t'), applies an adaptive filter Hk, and generates V(k, t'). The other adaptive filters in the echo canceller <NUM> that are not shown generate the remaining k - <NUM> components. X, Y, and V respectively correspond to the transformed reference signal <NUM>, the transformed captured signal <NUM>, and the echo residual signal <NUM> (see <FIG>), with the signals corresponding to the respective bins being denoted with a suffix corresponding to the bin (e.g., "<NUM>-<NUM>" for the <NUM>-th bin of the reference signal <NUM>, "<NUM>-k" for the k-th bin, etc.). The filter response H corresponds to the room impulse response h(t) in the transform domain.

Although the sub-band adaptive filter implemented by the filter bank <NUM> can greatly reduce the complexity of the algorithm, the performance of the echo canceller <NUM> is degraded because of the spectrum overlap between filters. A closed-loop sub-band adaptive filter is proposed to alleviate the problem, but due to the fact that complexity is largely increased, the open-loop sub-band adaptive filter <FIG> is still a commonly adopted technique for acoustic echo cancelation.

<FIG> is a block diagram of the echo suppressor <NUM> (see <FIG>). As mentioned above, an open-loop sub-band adaptive filter is a practical (but not an adequate) solution to cancel echo to the amount of <NUM> dB to enable a good experience of voice communication. Even though most of the time during a voice call, full duplex communication (e.g., double talk, when both ends are talking) happens rarely, and when it happens, none of the active speakers will expect to understand the other party's speech; instead, their intention is merely to interrupt. Based on this, a nonlinear echo suppression can be applied after echo cancellation to further suppress echo but to allow the uplink voice of the near end party to interrupt the far end party.

Having that in mind, during a multi-end call with three participants, the third party (who is not the active two talkers in the duplex scenario) will have a bad experience if the suppressor lets through too much of the echo residual, even though the person being interrupted might not. Thus, a good suppressor shall suppress as much as possible of the echo residual while passing through as much as possible of the uplink voice. This goal is only possible, for a linear operation, by an adaptive filter such as the echo canceller <NUM>.

To address these issues and to provide additional echo management, the echo suppressor <NUM> uses the intrinsic statistics of the echo residual to perform echo suppression. The echo suppressor <NUM> includes a banding component <NUM>, a power gain profile estimator <NUM>, an echo residual estimator <NUM>, a gain calculator <NUM>, and a signal combiner <NUM>.

The banding component <NUM> receives the transformed reference signal <NUM> (also denoted as X(k, t')) and the echo residual signal <NUM> (also denoted as V(k, t')), performs frequency banding, and generates a banded reference signal <NUM> (also denoted as <IMG>(b, t')) and a banded echo residual signal <NUM> (also denoted as <IMG>(b, t')). In general, "banding" refers to grouping or aggregating multiple sub-bands or frequency bins of a signal to form a band, where the resulting number of bands is less than the original number of sub-bands or frequency bins. For example, the banding may be performed by summing multiple sub-bands or frequency bins to form a band, and may include additional operations such as multiplication, absolute value calculation, magnitude calculation, etc. The banding component <NUM> may perform banding of the echo residual signal <NUM> to generate the banded echo residual signal <NUM> according to Equation (<NUM>):
<MAT>.

The banding component <NUM> may perform banding of the transformed reference signal <NUM> to generate the banded reference signal <NUM> according to Equation (<NUM>): <MAT>.

In Equations (<NUM>) and (<NUM>), B(b) corresponds to a set of bins k that are associated with a band b, and the output of the banding corresponds to the signal power for each particular band. The banding may be equivalent rectangular bandwidth (ERB) banding, which corresponds to a psychoacoustic model of human hearing and associates larger bandwidths with lower frequencies. For example, the number of bins k may be <NUM> bins, the number of bands b may be <NUM>, and each band b is associated with a number of the bins k, where more bins are associated with the lower frequency bands than with the higher frequency bands.

The power gain profile estimator <NUM> receives the banded reference signal <NUM> and the banded echo residual signal <NUM>, estimates a power gain profile, and generates an estimated power gain profile <NUM> and a loopback power <NUM> (also denoted as <IMG>(b, t'); see <FIG>). The estimated power gain profile <NUM> corresponds to an estimate of the power gain profile of an acoustic echo canceller such as the echo canceller <NUM> (see <FIG>). The power gain profile estimator <NUM> may also receive a control signal <NUM> that indicates the presence of double talk. Double talk refers to the situation of when both the far end party and the near end party are speaking at the same time (e.g., where the far end speech is indicated according to the reference signal <NUM> and the near end speech is indicated by the captured signal <NUM>, which also includes the echo of the far end speech outputted by the loudspeaker and captured by the microphone at the near end).

When the echo canceller <NUM> reaches steady state, and when there is no uplink voice (e.g., indicated by no double talk according to the control signal <NUM>, and the presence of echo according to the reference signal <NUM>), we will have the echo residual signal <NUM> (also denoted as V(k, t')) as represented according to Equation (<NUM>): <MAT>.

In Equation (<NUM>), Ĥ(k, t) - H(k, t) is the error response of the adaptive filter not matching the real speaker-room-mic response. Notice that because Ĥ(k, t') depends on the input X(k, t), H̃(b, t') will also depend on the input. And because the adaptive filter Ĥ(k, t) has multiple taps, the current residual power <IMG>(b, t') might not only depend on the current input power, but also on previous input power. In other words, echo residual V(k, t') will not be a linear time invariant (LTI) system output.

For simplicity and practical implementation, we model the power of the residual according to Equation (<NUM>):
<MAT>.

Additional details of the power gain profile estimator <NUM> are provided in <FIG>.

<FIG> is a block diagram of a loopback power calculator <NUM>. The loopback power calculator <NUM> may be a component of the power gain profile estimator <NUM> (see <FIG>). The loopback power calculator <NUM> generally calculates the loopback power <NUM> (also denoted as <IMG>(b, t'); see also <FIG>) of the banded reference signal <NUM> (see <FIG>; also denoted as <IMG>(b, t')). In general, the loopback power <NUM> corresponds to a weighted combination of historical values of the banded reference signal <NUM>, as more fully detailed below. The loopback power calculator <NUM> includes a dominant adaptive filter tap finder <NUM>, a memory <NUM>, multipliers <NUM> and <NUM>, and an adder <NUM>.

The dominant adaptive filter tap finder <NUM> receives filter coefficients <NUM> (corresponding to the filter coefficients of the adaptive filters Hk in the echo canceller <NUM> of <FIG>), determines the two dominant taps, and provides the weights w<NUM> and w<NUM> of the dominant taps to the multipliers <NUM> and <NUM>. A dominant tap is a tap that provides a greater weight contribution to the adaptive filter than another tap. The dominant adaptive filter tap finder <NUM> uses the filter coefficients <NUM> corresponding to Hk across all the sub-bands. The weights w<NUM> and w<NUM> may be relative weights.

The memory <NUM> stores a history of the banded reference signal <IMG>(b, t'). The memory <NUM> has several memory elements, to store the current values of the banded reference signal <IMG>(b, t') and one or more previous values of the banded reference signal <IMG>(b, t' - <NUM>), <IMG>(b, t' - <NUM>),. , <IMG>(b, t' - n). As a specific example, <FIG> shows the memory <NUM> containing <NUM> memory elements.

The multiplier <NUM> multiplies one of the historical values of the banded reference signal <NUM> stored in the memory <NUM> by the weight w<NUM>, and the multiplier <NUM> multiplies another of the historical values of the banded reference signal <NUM> stored in the memory <NUM> by the weight w<NUM>. The adder <NUM> adds the results of the multipliers <NUM> and <NUM>. As a result, the loopback power <IMG>(b, t') may be calculated according to Equation (<NUM>): <MAT>.

In Equation (<NUM>), n<NUM> and n<NUM> are the delay indices respectively corresponding to the dominant taps w<NUM> and w<NUM>. In the example shown in <FIG>, n<NUM> is <NUM> and n<NUM> is <NUM>.

When the input power is larger in a particular band (as corresponding to the loopback power <IMG>(b, t')), H̃(b, t') also tends to be larger in absolute value. This relationship is further detailed in <FIG>.

<FIG> is a histogram of the relationship between the loopback power <IMG>(b, t') (also referred to as the loopback power <NUM> in <FIG>) and the error (echo residual) transfer function H̃(b, t') of the adaptive filter. As can be seen from <FIG>, when the input power is larger in a particular band (as corresponding to the loopback power <IMG>(b, t') in a particular band), H̃(b, t') also tends to be larger in absolute value. On the other hand, when the band contains mainly noise, H̃(b, t') tends to be small as the noise will not be cancelled.

Returning to <FIG>, the power gain profile estimator <NUM> uses the relationship of <FIG> as a histogram to estimate a gain profile for each band. The estimated power gain profile <NUM> may be represented as the mean µ(b, t') and variance σ<NUM>(b, t') of H̃(b, t'). (The mean and variance are denoted as µb and <MAT> in the following paragraphs, for simplicity. ) In summary, the power gain profile estimator <NUM> uses the statistics of the echo residual signal (e.g., the echo residual signal <NUM>) in combination with the history of the input signal (e.g., the reference signal <NUM>) to estimate the power gain profile.

The power gain profile estimator <NUM> uses the control signal <NUM> (e.g., the double talk indicator) to update the gain profile. Specifically, the power gain profile estimator <NUM> updates the gain profile only when it has confidence that there is no uplink (near end) voice, as indicated by the control signal <NUM>. The control signal <NUM> may be generated in various ways, including by a double talk detector or using a stochastic method. An example of the stochastic method is to update the estimation at a random time once the reference signal has exceeded a first threshold for a duration longer than a second threshold.

The echo residual estimator <NUM> receives the banded echo residual signal <NUM> (also denoted as <IMG>(b, t')), the loopback power <NUM> (also denoted as <IMG>(b, t')) and the estimated power gain profile <NUM> (the mean and variance of H̃(b, t'), denoted as µb and <MAT>), and calculates an estimated echo residual power <NUM> (also denoted as <IMG>(b, t')). The echo residual estimator <NUM> may calculate the estimate echo residual power <NUM> based on a combination of the estimated power gain profile <NUM> and the loopback power <NUM>. The echo residual estimator <NUM> may calculate the estimate echo residual power <NUM> using a minimum operator applied to a combination of the banded echo residual signal <NUM>, the loopback power <NUM> and the estimated power gain profile <NUM>. The echo residual estimator <NUM> may calculate the estimated echo residual power <NUM> according to Equation (<NUM>):
<MAT>.

In Equation (<NUM>), β is a factor determined by tuning. The minimum operator ensures that the estimated echo residual power <NUM> will never exceed V(b, t'), which is a composite of the echo residual power and the local talk power. The estimated echo residual power <NUM> may be modeled as a Gaussian random variable with its centroid at <IMG>(b, t'). In summary, the echo residual estimator <NUM> calculates the estimated echo residual power <NUM> according to the loopback power <NUM> for each band, by modeling the power gain of the echo canceller <NUM> (see <FIG>) for each band as a Gaussian random variable parameterized by the estimated power gain profile <NUM>.

The gain calculator <NUM> receives the banded echo residual signal <NUM> and the estimated echo residual power <NUM>, and calculates banded gains <NUM> (also denoted as <IMG>(b, t')). One goal of calculating the banded gains <NUM> is to apply, to each particular band, different amounts of suppression for the half duplex scenario versus the full duplex scenario. (The half duplex scenario describes when only one party is talking, e.g. either the near end or the far end. The full duplex scenario describes when both parties are talking, e.g. double talk. ) In other words, calculating the banded gains <NUM> includes selectively calculating the plurality of banded gains based on a likelihood of local talk in the echo residual signal. As part of calculating the banded gains <NUM>, the gain calculator <NUM> calculates a feature to steer the gain calculation between the two scenarios.

Due to the nature of room noise, electronic noise, and randomness of the voice signal, it is assumed that the estimated echo residual power <NUM> in dB has a Gaussian distribution with mean dB{<IMG>(b, t')} and variance <MAT>, which corresponds to the estimated power gain profile <NUM> calculated by the power gain profile estimator <NUM>. It is also assumed that the estimated echo residual power <NUM> in dB has no cross-correlation (with the first assumption, independent) between bands. Let Ec be the event when there is no uplink voice; the probability density function for echo residual power in dB a particular band is given by Equation (<NUM>): <MAT>.

In Equation (<NUM>), x is the estimated echo residual power <NUM> in dB, and <IMG>(b, t') corresponds to the centroid of the estimated echo residual power <NUM>.

<FIG> is a graph that illustrates the probability density function Pr(x| Ec) of Equation (<NUM>). In <FIG>, the x-axis corresponds to x and the y-axis corresponds to Pr(x| Ec).

By modeling the wideband echo residual as a Gaussian mixture model, the log likelihood L of no uplink voice across all bands b is given by Equation (<NUM>):
<MAT>.

In Equation (<NUM>), the log likelihood L is the sum over all bands b of the exponential component
<MAT>
from Equation (<NUM>). In other words, the likelihood of local talk in the echo residual signal <NUM> is a log likelihood based on a mean and a variance of the estimated power gain profile summed over a plurality of frequency bands. In summary, the log likelihood L is calculated by modeling the echo residual signal <NUM> as a Gaussian mixture model applied over all the bands b.

When there is no uplink voice (e.g., no local talk captured by the device), L(Ec) should be low; otherwise L(E) is high, and the log likelihood L of Equation (<NUM>) will be the signal to steer the gain calculation.

The log likelihood L of Equation (<NUM>) is used to calculate a global maximum suppression gain across all bands Gmax according to Equation (<NUM>): <MAT>.

In Equation (<NUM>), G<NUM> is the gain in dB when there is uplink voice, and G<NUM> is the predefined gain in dB when there is no uplink voice. Lth is a pre-defined threshold, which can be obtained for each device during initial setup via prior knowledge of whether local talk presents or not.

In other words, calculating the banded gains <NUM> includes calculating a global maximum suppression gain Gmax across the plurality of frequency bands. The global maximum suppression gain is based on a first gain G<NUM> corresponding to a presence of the local talk, a second gain G<NUM> corresponding to an absence of the local talk, and the likelihood L of the local talk.

Across each band, the gain calculator <NUM> may calculate the gain for each band <IMG>(b, t') using one or more gain calculation processes. Example gain calculation processes that are suitable include the gain calculation processes described in <CIT>, <CIT>, and <CIT>, which are incorporated herein by reference.

The signal combiner <NUM> receives the echo residual signal <NUM> (also denoted as V(k, t')) and the banded gains <NUM> (also denoted as <IMG>(b, t')) and generates a modified echo residual signal <NUM> (also denoted as Ṽ(k, t')). The gain calculated for a given band b is applied to all of the sub-bands k that belong to that given ERB band. The signal combiner <NUM> may generate the modified echo residual signal <NUM> according to Equation (<NUM>): <MAT>.

In other words, the echo residual signal <NUM> has a number of frequency bins k, where a given banded gain <IMG>(b, t') of the banded gains <NUM> corresponds to a given frequency bin (e.g., each band b may be associated with a number of bins k), and generating the modified echo residual signal <NUM> includes, for each of the bins k of the echo residual signal <NUM>, applying a corresponding banded gain <IMG>(b, t') to generate the modified echo residual signal <NUM>.

<FIG> is a mobile device architecture <NUM> for implementing the features and processes described herein, according to an embodiment. The architecture <NUM> may be implemented in any electronic device, including but not limited to: a desktop computer, consumer audio/visual (AV) equipment, radio broadcast equipment, mobile devices (e.g., smartphone, tablet computer, laptop computer, wearable device), etc.. In the example embodiment shown, the architecture <NUM> is for a laptop computer and includes processor(s) <NUM>, peripherals interface <NUM>, audio subsystem <NUM>, loudspeakers <NUM>, microphone <NUM>, sensors <NUM> (e.g., accelerometers, gyros, barometer, magnetometer, camera), location processor <NUM> (e.g., GNSS receiver), wireless communications subsystems <NUM> (e.g., Wi-Fi, Bluetooth, cellular) and I/O subsystem(s) <NUM>, which includes touch controller <NUM> and other input controllers <NUM>, touch surface <NUM> and other input/control devices <NUM>. Other architectures with more or fewer components can also be used to implement the disclosed embodiments.

Memory interface <NUM> is coupled to processors <NUM>, peripherals interface <NUM> and memory <NUM> (e.g., flash, RAM, ROM). Memory <NUM> stores computer program instructions and data, including but not limited to: operating system instructions <NUM>, communication instructions <NUM>, GUI instructions <NUM>, sensor processing instructions <NUM>, phone instructions <NUM>, electronic messaging instructions <NUM>, web browsing instructions <NUM>, audio processing instructions <NUM>, GNSS/navigation instructions <NUM> and applications/data <NUM>. Audio processing instructions <NUM> include instructions for performing the audio processing described herein.

<FIG> is a flowchart of a method <NUM> of audio processing. The method <NUM> may be performed by a device (e.g., a laptop computer, a mobile telephone, etc.) with the components of the architecture <NUM> of <FIG>, to implement the functionality of the echo management system <NUM> (see <FIG>), the echo canceller <NUM> (see <FIG>), the echo suppressor <NUM> (see <FIG>), the loopback power calculator <NUM> (see <FIG>), etc., for example by executing one or more computer programs.

At <NUM>, a reference signal and an echo residual signal are received. For example, the echo suppressor <NUM> (see <FIG>) may receive the transformed reference signal <NUM> and the echo residual signal <NUM>. The transformed reference signal <NUM> corresponds to the reference signal <NUM>, as can be seen in <FIG>.

At <NUM>, a banded reference signal and a banded echo residual signal are generated by performing frequency banding on the reference signal and the echo residual signal. For example, the banding component <NUM> (see <FIG>) may perform banding on the transformed reference signal <NUM> and the echo residual signal <NUM> to generate the banded reference signal <NUM> and the banded echo residual signal <NUM>, respectively.

At <NUM>, an estimated power gain profile and a loopback power are calculated based on the banded reference signal and the banded echo residual signal. The estimated power gain profile <NUM> corresponds to an estimate of the power gain profile of an acoustic echo canceller such as the echo canceller <NUM> (see <FIG>). For example, the power gain profile estimator <NUM> (see <FIG>) may calculate the estimated power gain profile <NUM> based on the banded reference signal <NUM> and the banded echo residual signal <NUM>. The loopback power calculator <NUM> (see <FIG>) may calculate the loopback power <NUM> based on the banded reference signal <NUM>. The estimated power gain profile <NUM> may indicate a long-term amount of echo cancellation as calculated by the loopback power calculator <NUM>, e.g. by using the history of the banded reference signal <NUM> stored in the memory <NUM>.

At <NUM>, an estimated echo residual power is calculated based on the banded echo residual signal, the loopback power and the estimated power gain profile. For example, the echo residual estimator <NUM> (see <FIG>) may calculate the estimated echo residual power <NUM> based on the banded echo residual signal <NUM>, the loopback power <NUM> and the estimated power gain profile <NUM>.

At <NUM>, a plurality of banded gains are calculated based on the banded echo residual signal and the estimated echo residual power. For example, the gain calculator <NUM> (see <FIG>) may calculate the banded gains <NUM> based on the banded echo residual signal <NUM> and the estimated echo residual power <NUM>.

At <NUM>, a modified echo residual signal is generated by applying the plurality of banded gains to the echo residual signal. For example, the signal combiner <NUM> (see <FIG>) may generate the modified echo residual signal <NUM> by applying the banded gains <NUM> to the echo residual signal <NUM>.

The method <NUM> may include additional steps corresponding to the other functionalities of the echo management system <NUM>, the echo canceller <NUM>, the echo suppressor <NUM>, the loopback power calculator <NUM>, etc. as described herein. For example, the reference signal may be outputted by a loudspeaker, such as the loudspeakers <NUM> (see <FIG>). As another example, near end audio may be captured by a microphone (e.g., the microphone <NUM> of <FIG>), a near end audio signal may be generated, and the echo residual signal may be generated by performing echo cancellation on the near end audio signal (e.g., by the echo canceller <NUM> of <FIG>). The near end audio may include local talk (e.g., speech generated by the near end person), echo (e.g., the far end speech outputted by the loudspeaker at the near end), etc. As another example, the modified echo residual may be transmitted by the near end device to a far end device, e.g. as part of a teleconference, a videoconference, etc..

An embodiment may be implemented in hardware, executable modules stored on a computer readable medium, or a combination of both (e.g., programmable logic arrays). Unless otherwise specified, the steps executed by embodiments need not inherently be related to any particular computer or other apparatus, although they may be in certain embodiments. In particular, various general-purpose machines may be used with programs written in accordance with the teachings herein, or it may be more convenient to construct more specialized apparatus (e.g., integrated circuits) to perform the required method steps. Thus, embodiments may be implemented in one or more computer programs executing on one or more programmable computer systems each comprising at least one processor, at least one data storage system (including volatile and non-volatile memory and/or storage elements), at least one input device or port, and at least one output device or port. Program code is applied to input data to perform the functions described herein and generate output information. The output information is applied to one or more output devices, in known fashion.

Each such computer program is preferably stored on or downloaded to a storage media or device (e.g., solid state memory or media, or magnetic or optical media) readable by a general or special purpose programmable computer, for configuring and operating the computer when the storage media or device is read by the computer system to perform the procedures described herein. The inventive system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer system to operate in a specific and predefined manner to perform the functions described herein. (Software per se and intangible or transitory signals are excluded to the extent that they are unpatentable subject matter.

Aspects of the systems described herein may be implemented in an appropriate computer-based sound processing network environment for processing digital or digitized audio files. Portions of the adaptive audio system may include one or more networks that comprise any desired number of individual machines, including one or more routers (not shown) that serve to buffer and route the data transmitted among the computers. Such a network may be built on various different network protocols, and may be the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), or any combination thereof.

One or more of the components, blocks, processes or other functional components may be implemented through a computer program that controls execution of a processor-based computing device of the system. It should also be noted that the various functions disclosed herein may be described using any number of combinations of hardware, firmware, and/or as data and/or instructions embodied in various machine-readable or computer-readable media, in terms of their behavioral, register transfer, logic component, and/or other characteristics. Computer-readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, physical (non-transitory), non-volatile storage media in various forms, such as optical, magnetic or semiconductor storage media.

The above description illustrates various embodiments of the present disclosure along with examples of how aspects of the present disclosure may be implemented. The above examples and embodiments should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of the present disclosure as defined by the following claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations and equivalents will be evident to those skilled in the art and may be employed without departing from the scope of the disclosure as defined by the claims.

<CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>.

Claim 1:
A computer-implemented method of audio processing, the method comprising:
receiving (<NUM>) a reference signal (<NUM>) and an echo residual signal (<NUM>);
generating (<NUM>) a banded reference signal (<NUM>) and a banded echo residual signal (<NUM>) by performing frequency banding on the reference signal and the echo residual signal, respectively;
calculating (<NUM>) an estimated power gain profile (<NUM>) of an acoustic echo canceller (<NUM>) and a loopback power (<NUM>), wherein the loopback power corresponds to a weighted combination of historical power values of the banded reference signal, the estimated power gain profile is calculated by using statistics of the banded echo residual signal in combination with the historical power values of the banded reference signal;
calculating (<NUM>) an estimated echo residual power (<NUM>) for each band based on the banded echo residual signal, the loopback power and the estimated power gain profile;
calculating (<NUM>) a plurality of banded gains (<NUM>) based on the banded echo residual signal and the estimated echo residual power; and
generating (<NUM>) a modified echo residual signal (<NUM>) by applying the plurality of banded gains to the echo residual signal.