Audio signal processing

An estimated system gain spectrum of an acoustic system is generated, and updated in real-time to respond to changes in the acoustic system. Peak gains in the estimated system gain spectrum are tracked as the estimated system gain spectrum is updated. Based on the tracking, at least one frequency at which the estimated system gain spectrum is currently exhibiting a peak gain is identified. Based on the identification of the at least one frequency, an audio equalizer is controlled to apply, to a first speech containing signal to be played out via an audio output device of the audio device and/or to a second speech containing signal received via an audio input device of the audio device, an equalization filter to reduce the level of that signal at the identified frequency. The equalization filter is applied continuously throughout intervals of both speech activity and speech inactivity in that signal.

RELATED APPLICATION

This application claims priority under 35 U.S.C. §119(b) to GB Application No. 1518004.5 titled “System Gain Equalization Filtering” and filed on Oct. 12, 2015, the entire disclosure of which is incorporated by reference herein.

BACKGROUND

Communication systems allow users to communicate with each other over a network. The network may be, for example, the Internet or public switched telephone network (PSTN). Audio signals can be transmitted between nodes of the network, to thereby allow users to transmit and receive audio data (such as speech data) to each other in a communication session over the communication system.

A user device may have an audio output device such a speaker or set of speakers for outputting audio signals to near end user. The user may enter into a communication session with another user, such as a private call (with just two users in the call) or a conference call (with more than two users in the call). The audio signals may be received over the network from a far end user during a call. The user device may also have audio an input device such as a microphone or array of microphones that can be used to receive audio signals such as speech from a user. The user's speech is received at the microphone, processed and is then transmitted over a network to the other users in the call.

As well as the audio signals from the user, the microphone may also receive other audio signals, such as background noise and echo, which are unwanted and which may disturb the audio signals received from the user. For example, in a call, the near end user's microphone signal received at the far end device via the network may be outputted via the far end user's loudspeakers. This is turn may be picked up by the far end microphone, and transmitted back to the near end device, so that the near end user's own microphone signal is played out of their loudspeakers. This is an example of an acoustic loop, which can lead to acoustic feedback when the system gain is high. Acoustic loops, whereby a microphone signal is outputted by a loudspeaker in the vicinity of the microphone itself, and received by the microphone, can arise in other contexts, such as an acoustic system with a single audio device. That is, other types of acoustic system are prone to acoustic feedback also.

SUMMARY

Various aspects of the present subject matter are directed to reducing acoustic feedback in an acoustic system comprising at least one audio device.

An estimated system gain spectrum of the acoustic system is generated, and updated in real-time to respond to changes in the acoustic system. Peak gains in the estimated system gain spectrum are tracked as the estimated system gain spectrum is updated in real-time. Based on the tracking, at least one frequency at which the estimated system gain spectrum is currently exhibiting a peak gain is identified. Based on the identification of the at least one frequency, an audio equalizer is controlled to apply, to a first speech containing signal (i.e. a first audio signal having a speech component) to be played out via an audio output device of the audio device and/or to a second speech containing signal (i.e. a second audio signal having a speech component) received via an audio input device of the audio device, an equalization filter to reduce the level of the speech containing signal at the identified frequency, i.e. in a portion of the spectrum of that signal that includes the identified frequency. The equalization filter is applied continuously throughout intervals of both speech activity and speech inactivity in the speech containing signal.

DETAILED DESCRIPTION OF EMBODIMENTS

An effect that can arise in an acoustic system is “howling”. Howling arises from acoustic feedback in the system. It can be caused by a number of factors and arises when system gain is high.

In the following described embodiments of the subject matter, a technique is described, in which an estimate of the system gain spectrum is updated in real time. A number N (one or more) of peaks in the system gain spectrum are tracked in real time, and the tracking is used to adapt that number N of equalization filters in real time. Each of the N equalization filters is applied by a respective audio equalizer, to reduce the level of at least one speech containing signal in parts of the spectrum where the system gain is high. That is, at a respective frequency matching the current frequency of a respective one of the N peaks. As one of the N peaks moves in the frequency spectrum, or becomes superseded by a new higher peak, the corresponding equalizer filter is adapted in real time accordingly to accommodate the movement or the new peak.

A speech containing signal means an audio having a speech component during at least some intervals (speech intervals of the audio signal). Note, the term “speech signal” is used herein as shorthand for “speech containing signal”. That is, the terms are equivalent.

In other words, the equalizers are applied specifically to reduce the peaks of the system gain spectrum: it is the parts of the spectrum with the highest gain that will determine the robustness to howling of a certain combination of end-points, so it is those spectral regions that are identified and directly targeted. The aim of the described embodiments is not one of completely flattening the gain spectrum, as this may lead to artificial sounding audio —the level of the speech signal across its spectrum is changed as little as possible, i.e. only enough as is needed to provide robustness to howling, as this can improve perceptual quality.

In applying equalization filters for this particular purpose, the most critical frequencies of the spectrum, corresponding to the highest peaks, are identified and equalization filters applied with a spectral gain dip at those frequencies. An example of a suitable equalizer filter is shown inFIG. 4A, with a dip at frequency fc which matches the frequency of one of the N peaks in the gain spectrum. The width of each dip (Δf,FIG. 4) is also determined from the system gain spectrum. Multiple equalizers may be cascaded, such that any desired number of such filters can be applied that will each take care of one of that same number of peaks in the system gain spectrum. No part of the spectrum is amplified by the equalizers —each equalizer only attenuates the signal to which it is applied, to reduce its level. Each dip is only as deep and as wide as is needed to provide the robustness to howling so as to minimize the perceived impact on the signal. The depth and/or the width of each peak is adapted in real time as appropriate, to accommodate changes in the estimated system gain spectrum.

Previous solutions to the issue of howling include adjusting the aggressiveness of a noise suppression method that is applied to an audio signal in each frequency band. In this method, a variable gain is applied in each frequency band to reduce a noise component in the signal relative to a speech component. The variable gain in each band is lower limited, and its respective lower limited is adjusted based on the estimated system gain to prevent howling. This has the effect of lowering the system gain in bands prone to howling, but only during intervals of speech inactivity in the signal within each band; during intervals of speech activity within each band, the system gain estimate has no effect on the noise suppression as the gains are above their respective lower limits.

By contrast, herein the equalization filter(s) is applied in a signal chain of an acoustic system to at least one speech signal continuously throughout intervals of both speech activity and speech inactivity in that speech signal. Also, while the noise reduction approach operates with a frequency resolution matching the frequency bands, the equalization filters can also be designed with a dip gain center frequency placed in-between spectral bins, and/or have an arbitrarily narrow or broad spectral gain dip.

The at least one speech containing signal to which the equalization filter(s) is applied may be a speech containing signal to be played out via an audio output device of the acoustic system and/or a speech containing signal received via an audio input device of the acoustic system.

The equalizer(s) that apply the equalization filter(s) are dedicated equalizer(s) i.e. dedicated to reducing acoustic feedback in the acoustic system, so as to provide robustness to howling in the acoustic system, in contrast to techniques which incorporate howling robustness techniques into some other signal processing applied in the signal chain, such as noise cancellation. The equalization filter(s) is applied independently from any noise cancellation (any noise suppression and/or any noise cancellation) applied in the signal chain and/or the acoustic system, to either or both of those speech containing signals; that is separately from any noise cancellation applied to either or both of those speech containing signals (whichever one of both the equalization filter(s) is applied to), anywhere else in the signal chain and/or the acoustic system.

Where noise suppression is applied to either or both of the speech containing signals, it may be applied independently of the estimated system gain spectrum e.g. if noise suppression is applied to the speech signal, a constant lower gain limit may be used in each frequency band that has no dependence on the estimated system gain spectrum. That is, relying on the equalization filter(s) rather than any noise cancellation to provide robustness to howling.

Other existing systems run single-sided measurements of each device, and equalize the playout based on an offline, pre-measurement of an impulse response, and thus cannot respond to changes in an acoustic system account for changes. Moreover, these may not take into account the impulse response of the room (where changes may occur, e.g. due to a portable audio device being moved), or the impulse response of the microphone, or take into account if any shaping is done in the driver, that differs from when the device characteristics were being measured.

Others alternative solutions may rely on a more computationally expensive linear filter based echo canceller to constantly subtract an estimate of the howling from the microphone signal. This approach is mostly suited for high-end devices with fast CPUs and without too much non-linear distortion from the loudspeaker and microphone.

Before describing the particular embodiments of the present subject matter, a context in which the subject matter can usefully be applied will now be described with reference toFIG. 1, which illustrates a communication system100.

A first user102of the communication system (User A/near end user) operates a user device104. The user device104is a computer device, which may for example may be a desktop or laptop computer device, mobile phone (e.g. smartphone), tablet computing device, wearable computing device (headset, smartwatch etc.), television (e.g. smart TV) or other wall-mounted device (e.g. a video conferencing device), set-top box, gaming console etc.

The user device104comprises a processor108, formed of one or more processing units (e.g. central processing unit (CPU)), such as a single or multi-core processor. The processor108is configured to execute code such as a communication client109for communicating over the communication system100. The client109may for be a stand-alone communication client application that runs directly on the processor108, or plugin to another application such as a Web browser etc. that is run on the processor108in an execution environment provided by the other application.

The code109allows the user device104to engage in calls and other communication sessions (e.g. instant messaging communication sessions) over the communication system100. The user device104can communicate over the communication system100via a network106, which may be, for example, the Internet or other packet-based network, or the Public Switched Telephone Network ss (PSTN). The user device104can transmit data to, and receive data from, the network106over the link110.

FIG. 1also shows a remote node with which the user device104can communicate over the communication system100. In the example shown inFIG. 1, the remote node is a second user device114which is usable by a second user112(User B/“far end” user) and which comprises a processor116which can execute code (e.g. a communication client) in order to communicate over the communication network106in the same way that the user device104communicates over the communications network106in the communication system100. a desktop or laptop computer device, mobile phone (e.g. smartphone), tablet computing device, wearable computing device (headset, smartwatch etc.), television (e.g. smart TV) or other wall-mounted device (e.g. a video conferencing device), set-top box, gaming console etc.

The user device114can transmit data to, and receive data from, the network106over the link118. Therefore User A102and User B112can communicate with each other audibly over the communications network106, whereby the user devices104,112and their surroundings constitute an acoustic system.

FIG. 2illustrates the user device104at the near end speaker in more detail. In particular,FIG. 2illustrates a microphone202receiving a speech signal201from the user102. The microphone202can be a single microphone or a microphone array comprising a plurality of microphones and optionally including a beamformer. As is known, a beamformer receives audio signals from the microphones in a microphone array and processes them in an attempt to improve the signal in a wanted direction in comparison to signals perceived to be coming from unwanted directions. This involves applying a higher gain in a desired direction.

Signals from the microphone202(whether with or without a beamformer) are applied to a signal processing stage208, via an audio interface206of the device104. The signal processing stage208includes a plurality of signal processing blocks, each of which can be implemented in hardware or software or a combination thereof as is deemed appropriate. The blocks can include, for example, an echo canceller block210, an equalizer block218, and one or more other signal processing blocks, such as digital signal processing (DSP) block(s)212, for example a digital gain block or background noise attenuation block, such as noise suppression or noise cancellation. Blocks201,212,216and218(see below) represent functionality implemented by the client software109when executed on the processor108in this example.

After signal processing, the signals input by the user102and picked up by the microphone202are transmitted for communicating with the far end user112.

At least one loudspeaker204is provided to provide audio signals205intended for the user102. Such signals can come from the far end user112to be output to the user102. The audio signals205can be processed before being emitted by the loudspeaker by signal processing logic (e.g. circuitry and/or software processing) and for the sake of convenience the loudspeaker is shown connected to signal processing stage208via the audio interface206inFIG. 2.

The audio interface206represents the hardware, such as a soundcard206a, and software of the user device104, such as sound card drivers206bexecuted on the processor108, that cooperate to allow the microphone202and loudspeaker204to perform their described functions. In some case the soundcard206aand/or drivers206bmay perform additional signal processing, such as equalization or dynamic range compression, which may be outside of the control of the client109.

The signal processing stage208further includes a system gain estimation block216. As discussed in more detail later, block216estimates a system gain spectrum215of the acoustic system. The system gain spectrum215denotes an estimate of the system gain as a function of frequency (as a discrete or continuous function). That is, the estimated system gains at different frequencies. For a discrete function, a respective estimated system gain is generated for each of a plurality of frequency bands in an audio spectrum; for a continuous function, the system gain is estimated as a continuous function over the audio spectrum.

Real time tracking of changes in the system gain for different frequencies is used to continuously adjust the tuning of the equalization block218. This tracking functionality is represented by tracking block220of the signal processing stage208, and is described in further detail below.

Howling is a symptom of having feedback with a system gain higher than 1 somewhere in the frequency spectrum. By reducing the system gain at frequencies at or near this limit, howling can be stopped or prevented.

Sometimes a resonating frequency in the loudspeaker, microphone or physical echo path will be much larger than average and will be what is limiting the robustness to howling. Resonance can also occur elsewhere in the signal processing chain, for example in DSP block(s)208, in the audio interface206(particularly in low cost soundcards), or at the far end, e.g. in the far end echo path or far end device114.

The system gain is estimated by taking into consideration the blocks involved in system processing (including the echo canceller210and other DSP block(s)212when present), and in particular, uses information from the echo path estimated in the echo canceller block210which provides information about the room in which the near end device104is located. The shape of the spectrum is usually dominated by the echo path, as the transfer function of the echo path includes the transfer function of the loudspeaker where resonating frequencies often occur. InFIG. 2, the estimated echo path is denoted by arrow211, and is in the form of a model of the echo path.

The system gain spectrum can either be single-sided, or can take into account all other endpoints as well, such as the far end device114, using a feedback mechanism whereby information214about acoustic conditions and/or signal processing at the far end device is received via the network106(far-end feedback).

The acoustic echo canceller generates an estimate of its echo path, in the form of an estimated echo path magnitude spectrum211, which is a gain spectrum of the echo path of the echo canceller210. That is, the gain of the echo path of the echo canceller210as a function of frequency. The echo path estimate is generated by comparing a reference signal to the signal to which the echo cancellation is to be applied. The term “echo path” when applied to an echo canceller means the signal path from the point in a signal chain at which the echo canceller takes its reference signal to the point at which it applies echo cancellation in the signal chain (see below). This includes the “physical echo path”, i.e. the acoustic channel (acoustic path) from the loudspeaker204to microphone202, and signal processing applied in the echo path.

The estimated system gain spectrum215is generated by combining the estimated echo path magnitude spectrum211with magnitude spectra, modelling all other digital scaling or shaping performed in the client109by DSP block(s)212, denoted by arrow213inFIGS. 2 and 3. The far-end feedback214is also accounted for in the system gain estimate215when received. The far-end feedback214may for example comprise a local system gain spectrum of the far end system, which models the effect of the far-end signal processing performed at the far-end device114. A component of the far end system gain spectrum may also model a contribution from the far end echo path.

The echo path of the acoustic echo canceller210may include the audio interface206, such that audio signal processing applied by the audio interface206is accounted for in the estimated echo path211, as in the examples described below. In this case, the audio interface processing is included in the estimated gain spectrum211“automatically” as a component of the echo estimated path211.

In other cases, the audio interface206may not be included in the echo path. For example, where a so-called “loopback” signal conveying the output of the audio interface206is available from the audio interface206, the echo path may be estimated using the loopback as a reference so as to exclude processing of signals supplied to the loudspeaker204in the audio interface206. The availability of a loopback signal may depend on the operating system of the device104. In this case, the magnitude spectrum of the audio interface206may be computed and combined with the echo path estimate211, client transfer function213and (where applicable) far-end feedback214, to include processing by the audio interface206in the estimated system gain spectrum215explicitly.

When conducting hands-free calls between two or more devices, the risk of howling depends (among other things) on the gain of the system and the performance of the echo cancellers applied on each endpoint. The gain of the system is often frequency dependent, due to non-flat spectral shape of the electro-acoustic units and of the echo paths. In the worst case, resonating frequencies coincide between the two endpoints, and as a result howling easily builds up whenever the echo cancellers are not perfectly cancelling the echoes.

The issue can be more pronounced for suppression based linear echo cancellers, which also happens to be the otherwise best suited AEC (Acoustic Echo Cancellation) design for a low-end device. Howling can be heard as noise that builds up to speech levels. It is often narrow-banded, but can also be more broad-banded. It all depends on the system gain spectrum, and the type of echo cancellers applied.

FIG. 3shows a function block diagram, in which functional blocks represent functionality implemented by the near end user device104to reduce acoustic feedback, and thereby prevent howling.FIG. 3shows the echo canceller block210, equalizer block218, system gain estimation block216and peak tracking block220connected in an exemplary signal processing chain, for the purposes of illustration.

The reference audio signal used by the acoustic echo canceller210is a first speech containing signal, denoted x(t), which is a speech containing signal received from the far end device114via the network106, which is outputted via the near end loudspeaker204(far end speech signal). In particular, the signal x(t) is a version of the far end speech signal to which one or more equalization filters have been applied by the equalizer block218(as described below), but which has not been supplied to the audio interface206. That is, the reference signal x(t) is taken at a point in the signal chain after the equalizer block218but before any processing by the audio interface206.

After equalization, the far end speech signal x(t) is supplied to the audio interface206for outputting via the loudspeaker204. The resulting output from the loudspeaker204is denoted by the arrow labelled205.

The acoustic echo canceller210also receives a microphone signal captured by the near end microphone202, denoted y(t). The echo canceller210applied an echo cancellation process to the microphone signal y(t), based on the reference x(t). The signals x(t) and y(t) are digital audio signals, formed of a plurality of digital samples.

The microphone signal y(t) has an echo component caused by the microphone202picking up part of the loudspeaker output205. During intervals of near end speech activity (i.e. when the near end user102is speaking), the microphone signal y(t) also has a speech component, i.e. the user's speech signal201. The microphone signal y(t) is received by an echo attenuation block210bof the echo canceller201, via the audio interface206.

The acoustic path from the loudspeaker204to the microphone202, plus the portions of the signal chain from the point in the signal chain at which the reference signal x(t) is taken by the echo canceller210to the point in the signal chain at which the microphone signal y(t) is received by the echo canceller210, constitute the echo path of the echo canceller210.

An echo path estimation block210aof the echo canceller210compares the received microphone signal y(t) with the reference signal x(t) in order to generate the estimate of the echo path211. The echo path estimate211models the acoustic path from the loudspeaker204to the microphone202plus any signal processing applied in the echo path, for example by the audio interface206, to the far end speech signal x(t) and/or the near end microphone signal y(t).

An echo attenuator block210bof the echo canceller210applies echo attenuation to the microphone signal y(t) based on the echo path estimate211. This reduces the level of the echo component in the microphone signal y(t) relative to the speech component therein. For example, the echo attenuator block210bmay apply echo subtraction, echo suppression or a combination of both. As is known in the art, echo subtraction refers to a form of echo cancellation where the echo path estimate211is used to generate an estimate of the echo component that is subtracted from the microphone signal y(t). Echo suppression refers to a form of echo cancellation, in which the echo path estimate211is used to determine respective gains in different frequency bands that are applied to the microphone signal y(t) to attenuate the echo component relative to the speech component.

The microphone signal y(t), to which the echo cancellation has been applied, is transmitted to the far end device114via the network106. In some cases, it may be subject to additional processing by the near end client109such as noise suppression, digital gain and/or packetization.

The system gain estimation block218combines the echo path estimate211, the local client magnitude response(s)213and far-end feedback214to estimate the system gain spectrum215in the manner described above. During a call between the near end user102and the far end user112the system gain estimation block216continuously updates the estimated system gain spectrum215in real time.

Theoretically, a perfect echo canceller would eliminate the echo component from the microphone signal y(t) entirely. Thus, theoretically, echo cancellation at both the near and far ends could prevent howling in itself. However, in practice, imperfections in the echo cancellation process, caused for example by non-linarites in the echo path, leave some residual echo in y(t). Indeed, an “imperfect” echo canceller may be desirable sometimes, as it uses fewer processing resources, or because overly-aggressive echo cancellation can cause a reduction in perceptual quality.

Thus, due to imperfections, even with the echo canceller210and similar echo cancellation at the far end, a system may in some circumstances still be prone to howling. In particular, in a call between the near end user102and the far end user112, residual echo transmitted to the far end may be outputted by the far end loudspeaker, picked up by the far end microphone, and transmitted back to the near end device104for outputting via the loudspeaker204, potentially causing howling.

In order to prevent residual echo in the microphone signal y(t) causing howling, the peak tracking block220tracks in real time a number N of the highest peaks (local maxima) in the system gain spectrum215as it is updated in real time, and the equalizer block218applies N equalizer filters, each having a gain dip centred at a different one of those N peaks.

Here, the intention is not one of completely flattening the spectrum. As the spectrum contains the contribution from the echo path between the loudspeaker(s) to the microphone and includes the effect of the microphone, flattening the spectrum completely would not necessarily make the shaping of the spectrum flat at the ears of the listener, potentially leading to an unnatural sound and thus a decrease in perceptual quality.

Accordingly, during intervals in which the equalization is applied, only the N (≧1) most dominating (i.e. the N highest) peaks are suppressed, by applying N equalizer dips at the corresponding frequencies respectively, to reduce the impact on the signal. Each gain dip is only as deep and as wide as is necessary to prevent howling. That is, the signal is modified as little as possible by the equalizers.

The number N (that is, the number of equalizer dips applied to the signal) may be determined dynamically, so that only those peaks that are high enough to risk howling are attenuated. For example, any peaks above a gain threshold may be identified and only those peaks attenuated. For as long as there are no gain peaks above the threshold, no equalization is applied.

Where a gain threshold is used, in some cases the level of each peak above the gain threshold is reduced to a level that substantially matches (i.e. that matches or approximately matches) the gain threshold.

As another example, the level of the N highest peaks may be reduced to a level that substantially matches the level of the N+1th highest peak, so that the N+1 highest peaks have substantially the same level once the equalization has been applied.

Each equalizer filter may for example be a bi-quad filter having a transfer function as defined in equation 1:

To handle multiple peaks (N>1), N such equalizers are cascaded until a sufficient number of system gain peaks have been suppressed. That is, N such equalizers can be applied to x(t) in series. The term “z” in equation 1 is defined as:
z=r*exp(j*ω),

That is, H(z) describes the system in the frequency domain.

Here, b0, b1, b2, a2 and a3 are parameters of the transfer function for the bi-quad filter. That is, each equalization filter is defined by a set of five equalizer filter coefficients —so 5N equalizer coefficients in total. The filter coefficients of each equalizer filter is generated based on the tracking block220, and updated in real time as the system gain spectrum215changes, e.g. as the N peaks move across the frequency spectrum or become superseded by other peaks of increasing magnitude.

FIG. 3shows two such equalizers218a,218bof equalizer block218, which are connected in series. Each applied a respective equalization filter according to a respective equalizer parameter set306a,306bgenerated and updated in real time tracking block220. However, this is purely exemplary and more or fewer individual equalizer can be applied as appropriate to the current state of the system gain spectrum. That is, the number of equalizers can be changed dynamically based on the tracking.

FIGS. 4A and 4Bshow the magnitude and phase spectra for one such equalization filter, which applies a gain dip having a depth of −6 dB gain at a frequency fc (center frequency). The dip has a width at −3 dB gain of Δf, as shown. In this example, fc is ⅛ times the sample rate and the width Δf 1/32 times the sample rate of the microphone signal y(t).

The sample rate of the signal y(t) in Hz is defined as the number of samples per second. Note thatFIGS. 4A and 4Bshow the normalized frequency with the sample rate normalized to 2, and thus the Nyquist frequency normalized to 1. Hence, fc is shown at a normalized frequency of 0.25. Also note the y-axis inFIGS. 4Ause logarithmic scales.

Equation 1 is just an example of one suitable filter type, and the filter coefficients can be found e.g. by using bi-linear transformation filter design of a parametric description of the desired filter. However, different types of filter can be used—both parametric and non-parametric.

The above-described equalizer filters are of dip gain type, and are cascaded (i.e. the equalizer filters are applied in series), wherein for each of the equalizers the output is a weight between the input and the filtered output from the equalizer. This allows the effect of the equalization filter to be milder. Any delay in the filter can be accounted for by synchronization.

Another possibility is the application of pass-band type equalizer filters, applied in parallel, whose outputs are scaled and added together. As example is shown in the functional block diagram ofFIG. 5, which shows three equalizers218a,218b,218cconnected in parallel (this number is purely exemplary), each configured to apply in parallel a respective band pass filter514a,514b,514cto a respective copy of the far end audio signal. The band pass-filtered versions are scaled, as denoted by respective scaling functions616a,516b,516cof each equalizer218a,218b,218c, and the scaled outputs combined, as denoted by summing function512, to generate x(t). By reducing the overall level of the output of, say, the second equalizer218b, the level of x(t) around the middle of the spectrum can be reduced. As will be apparent, having a greater number of parallel equalizers, each applying a narrower band pass provides a greater level of control.

In the example ofFIG. 3, the equalizer block218is applied on the playout side (i.e. to the received signal x(t), before it is played out via the loudspeaker204) and before the echo path (that is the reference signal is taken after equalization has been applied).

Alternatively, the equalizer block218can be applied in the echo path; that is, after the AEC takes its reference signal copy from the loudspeaker signal and before the echo is cancelled in the microphone signal y(t). In this case, the estimated echo path may be corrected as soon as the equalizer settings are modified, to immediately expect an echo path change from the change of the applied equalizer.

Applying the equalizer218on the playout side can potentially improve the playout signal x(t) when the equalizer reduces218the gain at resonating frequencies of the loudspeaker. This is also the case for resonating frequencies introduced by the room, as even though equalization is based on what is recorded by the microphone (due to the dependence of the system gain215on the echo path estimate211), it is likely that the place of the listeners ears will be affected by the same resonances. However, this is not essential, and in general the equalizer218can be applied anywhere in the signal chain. For example all or part of the equalization can be applied at the near end device104to the microphone signal y(t) prior to transmission, in or after the echo path, or at the far end device112.

As indicated, wherever it is applied in the signal chain, the equalization is applied continuously. In this context, “continuously” means over an interval of time, for the whole of the interval. For example, for the duration of a call or for part of a call. This interval can include both interval(s) of speech activity —i.e. when the near end user102is speaking in the case of the microphone signal y(t); when the far end user112is speaking in the case of the output signal x(t) —and interval(s) of speech inactivity —i.e. when the when the near end user102is not speaking for y(t); when the far end user112is not speaking for x(t). The interval of time may for example be the interval for which at least one peak in the system gain spectrum215remains above the gain threshold.

The real time signal processing described above may be performed on a per frame basis. Frames can, for example, be between 5 and 20 milliseconds in length and for the purpose of noise suppression be divided into spectral bins, for example, between 32 and 256 bins per frame. Each bin contains information about a signal component at a certain frequency, or in a certain frequency band. For dealing with wideband signals, the frequency range from 0 to 8 kHz is processed, divided into 64 or 32 frequency bands of equal width. It is not necessary that the bands are of equal width —they could for example be adjusted to better reflect the critical bands of the human hearing such as done by the Bark scale.

For speech in particular, each frame may be processed in real time and each frame receives an updated estimate of system gain for each frequency bin from system gain block218. Thus each bin is processed using an estimate of system gain specific to that frame and the frequency of that bin. However, this is not essential and other types of real time processing are within the scope of this disclosure.

In this context, “real time” means that there is no perceptible delay in the equalizer218reacting to a change in the (actual) system gain. For example, the delay between a change in the actual system gain, for example caused by a movement of the device104that causes a change in the physical echo path, and the equalizer reacting may be about 20 milliseconds or less. This can be achieved by updating the estimated system gain spectrum215and the settings of the equalizer218every frame, though that is not essential. For example, in some cases, the equalizer settings may only be updated in response to a detection of a significant change in the system gain —this means that no updates may occur in an interval of more than, say, 20 ms if there are only small (negligible) changes in the system gain spectrum (e.g. changes in gain and/or peak frequency below respective thresholds) in that interval, but as soon as a substantially change occurs (e.g. above the threshold(s)) the equalizer filter(s) respond within, say, 20 ms or less.

A first aspect of the present subject matter is directed to a method of reducing acoustic feedback in an acoustic system comprising at least one audio device, the method comprising: generating an estimated system gain spectrum of the acoustic system, wherein the estimated system gain spectrum is updated in real-time to respond to changes in the acoustic system; tracking peak gains in the estimated system gain spectrum as the estimated system gain spectrum is updated in real-time; identifying based on the tracking at least one frequency at which the estimated system gain spectrum is currently exhibiting a peak gain; and based on said identification of the at least one frequency, controlling an audio equalizer to apply, to a first speech containing signal to be played out via an audio output device of the audio device and/or to a second speech containing signal received via an audio input device of the audio device, an equalization filter to reduce the level of that speech containing signal in a portion of its spectrum that includes the identified frequency, wherein the equalization filter is applied continuously throughout intervals of both speech activity and speech inactivity in that speech containing signal.

In embodiments, the estimated system gain spectrum may be generated by comparing the first speech containing signal with the second speech containing signal.

The method may comprise applying, to at least one of the speech containing signals, an echo cancellation process to attenuate an echo component of that signal created by an echo path in the acoustic system, whereby the equalization filter causes a reduction in the level of any residual echo component remaining in the first and/or the second signal in said portion of the spectrum.

Note this does not mean that the equalization has to be applied to the output of the echo canceller (though this is not excluded) —the equalizer can be applied before the echo canceller, and still result in a reduction in the level of the residual echo in the portion of the spectrum, relative to what its level would be in that portion of the spectrum without the equalizer. For example, echo cancellation can be applied to the microphone signal, and the equalizer can be applied to the signal before it is outputted by the loudspeaker, before or in the echo path of the echo canceller.

The estimated system gain spectrum may comprise an estimate of the echo path generated by the echo cancellation process.

The method may further comprise: receiving information about at least one signal processing process applied to at least one of the speech containing signals, and/or receiving information about acoustic conditions at another device, wherein the first speech containing signal is received at the audio device from the other device; wherein the received information may be used to generate the estimated system gain spectrum.

The at least one signal processing process may be applied at the audio device, or the other audio device, for example.

A plurality of audio equalizers may be applied in parallel, wherein each equalizer may apply a respective band pass equalizer filter to a version of the first and/or second signal; wherein the band pass-filtered versions may be scaled and combined to provide the speech signal having the reduced level in said portion of the spectrum, wherein said reduction may be achieved by adapting the scaling based on the tracking.

The method may be implemented by the audio device, for example by code executed on a processor of the audio device.

The equalizer may be a parametric equalizer, and the step of controlling the audio equalizer may comprise adjusting at least one parameter of the parametric equalizer based on the tracking to reduce the level of the speech containing signal in said portion of the spectrum.

That is, the controlling step may comprise performing parametric control of a pre-designed equalization filter(s). Note the scaling and the filtering can be applied in any order, or the scaling can be incorporated into the band pass filter itself.

The step of controlling the audio equalizer may comprise: generating at least one equalization parameter for reducing the level of an audio signal at the identified frequency, wherein the equalizer may apply the equalizer filter to the speech containing signal according to the equalization parameter, and thereby reduce the level of the speech containing signal in that portion of the spectrum.

The equalizer may be controlled based on the peak gain at the identified frequency and/or a width of a peak in the estimated system gain spectrum at the identified frequency.

The equalizer filter may have at least one dip, which may be centred on the identified frequency.

For example, the dip may have a depth that is determined based on the peak gain at the identified frequency and/or a width that is determined based on the width of the peak at the identified frequency.

The equalizer filter may be a bi-quad filter.

The identifying step may comprise identifying based on the tracking a plural number of the highest peak gains currently exhibited by the system gain spectrum, and the respective frequency of each of those peak gains; wherein that number of equalizer filters may be applied to the speech containing signal to reduce the level of the speech containing signal at those frequencies, each by a respective audio equalizer.

For example, N equalizers may reduce the level of the speech containing signal at each of those frequencies such that the reduced levels substantially match the level of the (N+1)th highest gain peak in the system gain spectrum.

Alternatively or in addition, the method may comprise identifying all of the peak gain(s) in the system gain spectrum that are current above a gain threshold, and the frequency of each of those peak gain(s); wherein a respective equalizer filter may be applied, by a respective audio equalizer, to the first and/or second speech containing signal to reduce the level of that speech containing signal, for each identified frequency, in a respective portion of the spectrum that includes that frequency.

The equalizer filters may reduce the level of the speech containing signal in each of the portions of the spectrum such that the reduced level(s) substantially match the gain threshold.

The equalization filter may be applied independently of any noise cancellation applied to the first and/or the second speech containing signal.

For example, noise cancellation may be applied to the first and/or the second noise signal independently of the equalizer filter and independently of the estimated system gain spectrum.

A second aspect of the present subject matter is directed to an audio signal processing device for use in an acoustic system, the device comprising: an audio output device; an audio input device; one or more processors; a memory accessible to the one or more processors, the memory configured to hold executable audio signal processing code, wherein the audio signal processing code is configured when executed by the one or more processors to cause operations of: generating an estimated system gain spectrum of the acoustic system, wherein the estimated system gain spectrum is updated in real-time to respond to changes in the acoustic system; tracking peak gains in the estimated system gain spectrum as the estimated system gain spectrum is updated in real-time; identifying based on the tracking at least one frequency at which the estimated system gain spectrum is currently exhibiting a peak gain; and based on said identification of the at least one frequency, controlling an audio equalizer to apply, to a first speech containing signal to be played out via the audio output device of and/or to a second speech containing signal received via the audio input device, an equalization filter to reduce the level of that speech containing signal in a portion of its spectrum that includes the identified frequency, wherein the equalization filter is applied continuously throughout intervals of both speech activity and speech inactivity in that speech containing signal.

In embodiments of the second aspect, the code may be further configured to implement any of the features of embodiments of the first aspect.

According to a third aspect of the present subject matter, a computer program product comprises executable code stored on a computer readable storage medium and configured when executed to implement any method of device/system functionality disclosed herein.

Generally, any of the functions described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), or a combination of these implementations. The terms “block”, “module,” “functionality,” “component” and “logic” as used herein —such as the functional blocks ofFIGS. 2 and 3—generally represent software, firmware, hardware, or a combination thereof. In the case of a software implementation, the block, module, functionality, or logic represents program code that performs specified tasks when executed on a processor (e.g. CPU or CPUs). The program code can be stored in one or more computer readable memory devices. The features of the techniques described below are platform-independent, meaning that the techniques may be implemented on a variety of commercial computing platforms having a variety of processors.

For example, the user devices may also include an entity (e.g. software) that causes hardware of the user devices to perform operations, e.g., processors functional blocks, and so on. For example, the user devices may include a computer-readable medium that may be configured to maintain instructions that cause the user devices, and more particularly the operating system and associated hardware of the user devices to perform operations. Thus, the instructions function to configure the operating system and associated hardware to perform the operations and in this way result in transformation of the operating system and associated hardware to perform functions. The instructions may be provided by the computer-readable medium to the user devices through a variety of different configurations.

One such configuration of a computer-readable medium is signal bearing medium and thus is configured to transmit the instructions (e.g. as a carrier wave) to the computing device, such as via a network. The computer-readable medium may also be configured as a computer-readable storage medium and thus is not a signal bearing medium. Examples of a computer-readable storage medium include a random-access memory (RAM), read-only memory (ROM), an optical disc, flash memory, hard disk memory, and other memory devices that may us magnetic, optical, and other techniques to store instructions and other data.