System and method for blind subband acoustic echo cancellation postfiltering

Systems and methods for blind subband acoustic echo cancellation postfiltering in a communication device are provided. In exemplary embodiments, an acoustic signal is received via a microphone of the communication device. Acoustic echo cancellation (AEC) is applied to the acoustic signal to obtain an AEC masked signal. Because residual echo may still exist in the AEC masked signal, blind subband AEC postfiltering on the AEC masked signal may be performed to obtain an echo-free acoustic signal. The echo-free signal may then be output.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention relates generally to audio processing and more particularly to post acoustic echo cancellation filtering in an audio system.

2. Description of Related Art

Conventionally, when audio from a far-end environment is presented through a loudspeaker of a communication device, a far-end audio signal may be picked up by microphones or other audio sensors of the communication device. As such, the far-end audio signal may be sent back to the far-end environment resulting in an echo to a far-end listener. In order to reduce or eliminate this echo, an acoustic echo canceller may be utilized.

However, there may be some residual echo remaining after acoustic echo cancellation is performed. This is a result of the fact that there is some limit to the amount of echo that can be subtracted out from an acoustic signal picked up by the microphones or audio sensors. Typically, what is left is still audible. In order to reduce or eliminate the residual echo, a non-linear processing or postfiltering process may be utilized after the acoustic echo cancellation operation. Conventionally, these non-linear or postfiltering processes require knowledge of the far-end audio signal that is leaking back through the microphones or audio sensors.

SUMMARY OF THE INVENTION

Embodiments of the present invention overcome or substantially alleviate prior problems associated with reducing residual echo post acoustic echo cancellation processing. In exemplary embodiments, an acoustic signal is received by a microphone of the communication device.

Because a loudspeaker may provide audio that may be picked up by the microphone, the acoustic signals may include loudspeaker leakage. As such, acoustic echo cancellation (AEC) is applied to the acoustic signal to obtain an AEC masked signal.

Because residual echo may still exist in the AEC masked signal, blind subband AEC postfiltering on the AEC masked signal may be performed to obtain an echo-free acoustic signal. In exemplary embodiments, the AEC masked signal is processed through a noise suppression system and a blind subband AEC postfilter (BSAP) system. The noise suppression system may provide a noise estimate for each subband of the AEC masked signal which may be utilized by the BSAP system.

The exemplary BSAP system is configured to render the residual echo inaudible without knowledge of the far-end signal. In one embodiment, the BSAP system may comprise an echo dominance estimate module configured to determine an echo dominance estimate for each subband of the acoustic signal. The echo dominance estimate is then provided to an echo-free noise estimate module, which is configured to determine an echo-free noise estimate for each subband of the acoustic signal. A combined echo/noise mask generator may then generate a combined echo/noise mask for each subband of the acoustic signal based on the echo dominance estimate, echo-free noise estimate, and a noise suppression mask gain.

The combined echo/noise mask is then applied to each subband of the acoustic signal and the echo-free signal may then be output.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present invention provides exemplary systems and methods for providing acoustic echo cancellation (AEC) postfiltering. Exemplary embodiments perform the AEC postfiltering based on frequency subbands and without knowledge of a far-end signal. As such, the AEC postfiltering may be referred to as blind subband AEC postfiltering.

Exemplary embodiments are configured to reduce and/or minimize effects of loudspeaker signal leakage back to microphones in a way that the far-end environment does not perceive an echo. Embodiments of the present invention can operate after AEC filtering. In one example, an AEC filter is employed that does not require knowledge of a far-end signal being played through a loudspeaker (e.g., strength and magnitude), only a direction the far-end signal is coming from. Those skilled in the art will appreciate that various embodiments are not tied to any AEC filtration system or any AEC algorithm. While the following description will focus on a two microphone system, alternative embodiments may utilize any number of microphones in a microphone array or a single microphone.

Embodiments of the present invention may be practiced on any device that is configured to receive audio such as, but not limited to, cellular phones, phone handsets, headsets, and conferencing systems. While some embodiments of the present invention will be described in reference to operation on a speakerphone, the present invention may be practiced on any audio device.

Referring toFIG. 1, an environment in which embodiments of the present invention may be practiced is shown. A user in a near-end environment100acts as an acoustic source102to a communication device104. The exemplary communication device104comprises two microphones: a primary microphone106relative to the acoustic source102and a secondary microphone108located a distance away from the primary microphone106. In some embodiments, the primary and secondary microphones106and108comprise omni-directional microphones. It should also be noted that embodiments of the present invention may be applied in both wideband and narrowband applications so long as a distance between the primary and secondary microphones106and108is not larger than a speed of sound divided by a sample rate.

While the microphones106and108receive sound (i.e., acoustic signals) from the acoustic source102, the microphones106and108also pick up noise110in the near-end environment100. Although the noise110is shown coming from a single location inFIG. 1, the noise110may comprise any sounds from one or more locations different than the acoustic source102, and may include reverberations, echoes, and distractors. The noise110may be stationary, non-stationary, and/or a combination of both stationary and non-stationary noise.

Some embodiments of the present invention utilize level differences (e.g., energy differences) between the acoustic signals received by the two microphones106and108. Because the primary microphone106is much closer to the acoustic source102than the secondary microphone108, the intensity level is higher for the primary microphone106resulting in a larger energy level during a speech/voice segment, for example.

The level difference may then be used to discriminate speech and noise in the time-frequency domain. Further embodiments may use a combination of energy level differences and time delays to discriminate speech. Based on binaural cue decoding, speech signal extraction or speech enhancement may be performed.

An acoustic signal comprising speech from a far-end environment112may be received via a communication network114by the communication device104. The received acoustic signal may then be provided to the near-end environment100via a loudspeaker116associated with the communication device104. The audio output through the loudspeaker116may leak back into (i.e., be picked up by) the primary and/or secondary microphone106and108. This leakage may result in an echo at the far-end environment112.

Referring now toFIG. 2, the exemplary communication device104is shown in more detail. In exemplary embodiments, the communication device104is an audio receiving device that comprises a receiver200, a processor202, the primary microphone106, the secondary microphone108, an audio processing system204, and an output device206. The communication device104may comprise more or less components necessary for communication device104operations.

The exemplary receiver200is an acoustic sensor configured to receive a far-end signal from the network114. In some embodiments, the receiver200may comprise an antenna device. The received far-end signal may then be forwarded to the audio processing system204.

The audio processing system204is configured to receive the acoustic signals from the acoustic source102via the primary and secondary microphones106and108(e.g., primary and secondary acoustic sensors) and process the acoustic signals. As previously discussed, the primary and secondary microphones106and108, respectively, are spaced a distance apart in order to allow for an energy level differences between them. After reception by the microphones106and108, the acoustic signals may be converted into electric signals (i.e., a primary electric signal and a secondary electric signal). The electric signals may themselves be converted by an analog-to-digital converter (not shown) into digital signals for processing in accordance with some embodiments. In order to differentiate the acoustic signals, the acoustic signal received by the primary microphone106is herein referred to as the primary acoustic signal, while the acoustic signal received by the secondary microphone108is herein referred to as the secondary acoustic signal. It should be noted that embodiments of the present invention may be practiced utilizing a plurality of microphones.

The output device206is any device which provides an audio output to a listener (e.g., the acoustic source102). For example, the output device206may comprise the loudspeaker116, an earpiece of a headset, or handset on the communication device104.

FIG. 3is a detailed block diagram of the exemplary audio processing system204, according to one embodiment of the present invention. In exemplary embodiments, the audio processing engine204is embodied within a memory device. The exemplary audio processing system204provides acoustic echo cancellation (AEC), noise suppression, and AEC postfiltering. As a result, an acoustic signal sent from the communication device104comprises noise suppression as well as reduced or eliminated echo from loudspeaker leakage.

In operation, the acoustic signals received from the primary and secondary microphones106and108are converted to electric signals and processed through a frequency analysis module302. In one embodiment, the frequency analysis module302takes the acoustic signals and mimics the frequency analysis of the cochlea (i.e., cochlear domain) simulated by a filter bank. In one example, the frequency analysis module302separates the acoustic signals into frequency bands or subbands. Alternatively, other filters such as short-time Fourier transform (STFT), Fast Fourier Transform, Fast Cochlea transform, sub-band filter banks, modulated complex lapped transforms, cochlear models, a gamma-tone filter bank, wavelets, or any generalized spectral analysis filter/method, can be used for the frequency analysis and synthesis. Because most sounds (e.g., acoustic signals) are complex and comprise more than one frequency, a sub-band analysis on the acoustic signal may be performed to determine what individual frequencies are present in the acoustic signal during a frame (e.g., a predetermined period of time). According to one embodiment, the frame is 5-10 ms long. Alternative embodiments may utilize other frame lengths.

After frequency analysis, the signals are forwarded to an acoustic echo, cancellation (AEC) engine304. In exemplary embodiments, the AEC engine304comprises a subtractive AEC engine304. The AEC engine304is configured to reduce echo resulting from loudspeaker leakage back to the primary and secondary microphones106and108. More details regarding the operation of the AEC engine304may be found in co-pending U.S. patent application Ser. No. 12/004,899 filed Dec. 21, 2007 and entitled “System and Method for 2-Channel and 3-Channel Acoustic Echo Cancellation,” which is incorporated by reference.

The results of the AEC engine304may be provided to a noise suppression system306which incorporates AEC engine304results with noise suppression. More details on exemplary noise suppression systems306may be found in co-pending U.S. patent application Ser. No. 11/825,563 filed Jul. 6, 2007 and entitled “System and Method for Adaptive Intelligent Noise Suppression,” U.S. patent application Ser. No. 11/343,524, filed Jan. 30, 2006 and entitled “System and Method for Utilizing Inter-Microphone Level Differences for Speech Enhancement,” and U.S. patent application Ser. No. 11/699,732 filed Jan. 29, 2007 and entitled “System And Method For Utilizing Omni-Directional Microphones For Speech Enhancement,” all of which are incorporated by reference.

In some embodiments, the results of the AEC engine304(i.e., AEC masked signal) may comprise residual echo. As such, exemplary embodiments utilize a blind subband AEC postfilter (BSAP) system308to process an output signal from the AEC engine304. In exemplary embodiments, the BSAP system308is configured to calculate time and frequency varying gain values that will render residual echo from a subtractive AEC engine304inaudible. The operations of the noise suppression system306in combination with the BSAP system308will be discussed in more detail in connection withFIG. 4below.

The results of the AEC engine304, the noise suppression system306, and the BSAP system308may then be combined in a masking module310. Accordingly in exemplary embodiments, gain masks may be applied to an associated frequency band of the primary acoustic signal in the masking module310.

Next, the post-AEC frequency bands are converted back into time domain from the cochlea domain. The conversion may comprise taking the post-AEC frequency bands and adding together phase shifted signals of the cochlea channels in a frequency synthesis module312. Once conversion is completed, the synthesized acoustic signal may be output (e.g., forwarded to the communication network114and sent to the far-end environment112).

It should be noted that the system architecture of the audio processing system204ofFIG. 3is exemplary. Alternative embodiments may comprise more components, fewer components, or equivalent components and still be within the scope of embodiments of the present invention.

Referring now toFIG. 4, a block diagram of the noise suppression system306and the BSAP system308in operation is shown. The noise suppression system306may comprise, at a minimum, a noise estimate module402and a noise suppression mask generator404. Various embodiments of an exemplary noise suppression system306, exemplary noise estimate module402, and the exemplary noise suppression mask generator404are further discussed in U.S. patent application Ser. No. 11/825,563 filed Jul. 6, 2007 and entitled “System and Method for Adaptive Intelligent Noise Suppression.”

The exemplary BSAP system308(also referred to herein as BSAP engine308) is configured to determine gain values to apply to the results of the AEC engine304in order to render residual echo inaudible. In exemplary embodiments, these gain values may be less than 1 (e.g., less than 0 dB). That is, a multiplicative gain operation is applied for each subband. As such, at any given point in time, the BSAP engine308may multiply the acoustic signal in a subband by some number between 1 (i.e., no suppression) and 0 (i.e., complete suppression). In decibels, a gain close to 0 may be a very large negative dB gain. The BSAP system308may comprise an echo dominance estimate module406, an echo-free noise estimate module408, and a combined echo/noise mask generator410.

In some embodiments, switches412and414may be provided. The switch412allows calculations from, the BSAP engine308to be fed into the noise suppression mask generator404instead of results from the noise estimate module402. As such an “echo-free” noise estimate is used by the noise suppression mask generator404. The output is then provided to the combined echo/noise mask generator410. However, if a BSAP engine308is not utilized, the switches412and414may be closed and the noise suppression mask generator404will utilize the results of the noise estimate module402and provide the noise suppression mask to the masking module310. It should be noted that the use of these switches412and414allow embodiments of the present invention to be incorporated with any noise suppression system.

In operation, a strength of the BSAP engine308attenuation is desired to be proportional to a degree of dominance of the residual echo over other components of the acoustic signal (e.g., speech component) within a limit. In one embodiment, a measure of this dominance may be a ratio of an echo power to a power of all other near-end acoustic signal components (e.g., noise and speech components). This ratio may be defined as an echo to near-end signal ratio (ENR). Generally, a higher ENR indicates a more dominant and audible residual echo where a higher amount of attenuation is applied, whereas a lower ENR may indicate no applied attenuation is needed (e.g., because the residual echo is already inaudible).

The exemplary echo dominance estimate module406is configured to estimate the ENR in each frequency subband and time-frame. In exemplary embodiments, the echo dominance estimate module406compares inputs and outputs of the AEC engine304and attempts to exploit a relationship represented by

ENR=γ-1β-γ,(1)
where γ may be defined as a ratio of an input to output total power of the AEC engine304, and β may be defined as a ratio of an input to output echo power of the AEC engine304.

In exemplary embodiments, γ may be directly observed by the echo dominance estimate module406while β may be estimated. That is, the echo dominance estimate module406may determine an average β. As a result, the ENR may be inferred from a measurement of γ. This echo dominance estimate may then be provided to the combined echo/noise mask generator410.

In exemplary embodiments, the ENR may be estimated for each subband, as well as globally from a sum of all subbands for each time frame. This global dominance metric, ENRglob, may be used in combination with frequency dependent ENR estimates by downstream modules (e.g., echo-free noise estimate module408and combined echo/noise mask generator410) for robustness.

The output of the echo dominance estimate module406is also provided to the echo-free noise estimate module408. The echo-free noise estimate module408is configured to compute an estimate of the near-end noise power spectrum (e.g., time and frequency dependent portion of the acoustic signal that is not from the acoustic source102). In exemplary embodiments, the echo-free noise estimate module408refines the noise estimate received from the noise suppression system306(e.g., noise estimate module402), which may be corrupted by echo power.

The results from the echo dominance estimate module406may then be used by the mask generator (e.g., noise suppression mask generator404and/or combined echo/noise mask generator410) to determine how the echo may be masked by the noise110. This information then allows the mask generator to limit an amount of suppression applied and reduce near-end signal distortion.

In some embodiments, the noise suppression system306may also benefit from using the echo-free noise estimate instead of the noise estimate from the noise estimate module402(e.g., when the noise suppression system306is designed to adapt to the noise110but not echo levels).

An exemplary method for preventing echo leakage into the noise estimate may be to freeze the noise power estimate when echo is strong. A determination of when to freeze may be based on the estimated ENR derived by the echo dominance estimate module406. If the ENR in a subband is above a threshold, ENRdom, then a noise power estimate in that subband may be frozen. Furthermore, if the ENRglobis above ENRdom, then noise power estimates in all subbands may be frozen.

The exemplary combined echo/noise mask generator410is configured to generate an echo gain mask designed to render the residual echo inaudible. This echo gain mask may be combined with a noise suppression gain mask provided by the noise suppression system306to provide a final output gain mask for signal modification and reconstruction by the masking module310. The echo gain mask may be produced, in one embodiment, by mapping an ENR-gain relationship through Equation (1) to describe the gain as a function of γ. For a Wiener filter, the relationship may be represented by

While these gains may be computed for each subband, a global gain value may also be derived from the estimated ENRglobvia a global γ computed from a sum of subbands. In one embodiment, the echo gain mask in each subband may be derived by taking a minimum of the subband gain and the global gain.

The combined echo/noise mask generator410may also refine the echo gain mask in each subband to reduce near-end noise distortion. In one embodiment, the combined echo/noise mask generator410takes into account the near-end noise level (from the echo-free noise estimate) and the noise suppression gain mask. The echo gain mask may be limited such that a total output of power is not more than a certain amount (e.g., 6 dB) below an output noise power that may be produced by applying the noise suppression gain to the noise110. This process may reduce perception of output noise modulation correlated with the echo, while still ensuring the echo remains inaudible.

The combined echo/noise mask generator410may then combine the noise suppression gain mask with the echo gain mask. In one embodiment, this combination may comprise selecting a minimum between the two gains (i.e., the noise suppression gain mask and the echo gain mask) in each subband.

Referring now toFIG. 5, a flowchart500of an exemplary method for providing blind subband AEC postfiltering (BSAP) is provided. In step502, the acoustic signal from a near-end environment100are received by the communication device104. In one embodiment, acoustic signals may be received by the primary and secondary microphones106and108. If audio from the far-end environment112is being output through the loudspeaker116, then audio from the loudspeaker116may leak back to the primary and secondary microphones106and108. This may result in an echo being provided back to a listener at the far-end environment112.

The acoustic signals are then converted to electric signals and processed through the frequency analysis module302to obtain frequency subbands in step504. In one embodiment, a primary (microphone channel) acoustic signal and a secondary (microphone channel) acoustic signal may be analyzed by the frequency analysis module302. In one embodiment, the frequency analysis module302takes the acoustic signals and mimics the frequency analysis of a cochlea (i.e., cochlear domain) simulated by a filter bank. The result comprises frequency subbands.

In step506, subtractive AEC is performed on the acoustic signal. In accordance with one embodiment, a null coefficient may be determined for each subband. In some embodiments, this complex null coefficient may be continuously adapted to minimize residual echo. The null coefficient may then be applied to the secondary acoustic signal to generate a coefficient-modified acoustic signal. The coefficient-modified acoustic signal is then subtracted from the primary acoustic signal. The result comprises post-AEC frequency bands.

However, these post-AEC frequency bands may comprise residual echo. As such, noise suppression and BSAP may be performed in step508. Step508will be discussed in more detail in connection withFIG. 6below.

The resulting frequency bands may then be output in step510. In accordance with exemplary embodiments, the resulting frequency bands are converted back into time domain from the cochlea domain. The conversion may comprise taking the resulting frequency bands and adding together phase shifted signals of the cochlea channels in the frequency synthesis module312. Once conversion is completed, the synthesized acoustic signal may be output (e.g., forwarded to the communication network114and sent to the far-end environment112).

Referring now toFIG. 6, a flowchart of the exemplary method for performing noise suppression and blind subband AEC postfiltering (i.e., step508) is shown. In step602, a noise estimate is determined. In exemplary embodiments, the noise estimate module402determines the noise estimate for each subband.

In step604, the echo dominance estimate is calculated. In exemplary embodiments, the echo dominance estimate module406receives the inputs and outputs to the AEC engine304. Using these inputs and outputs, a comparison is made to determine the echo to near-end signal ratio (ENR) for each frequency subband and time-frame.

An echo-free noise estimate is then determined in step606. In exemplary embodiments, the echo-free noise estimate module408derives the echo-free noise estimate for each frequency subband. In exemplary embodiments, the echo-free noise estimate module408is configured to compute an estimate of the near-end noise power spectrum (e.g., time and frequency dependent portion of the acoustic signal that is not from the acoustic source102). The echo-free noise estimate module408may refine the noise estimate received from the noise suppression system306(e.g., noise estimate module402), which may be corrupted by echo power.

In step608, a noise suppression mask may be generated. In exemplary embodiments, the echo-free noise estimate is provided to the noise suppression mask generator404. Using the echo-free noise estimate, the noise suppression mask may be generated by the noise suppression mask generator404.

In step610, a combined echo/noise suppression mask is generated. In exemplary embodiments, the combined echo/noise suppression mask generator410generates the combined echo/noise suppression mask utilizing the echo dominance estimates, echo-free noise estimates, and noise suppression masks. This echo gain mask may be combined with a noise suppression gain mask provided by the noise suppression system306to provide a final output gain mask for signal modification and reconstruction by the masking module310.

The above-described modules can be comprised of instructions that are stored on storage media. The instructions can be retrieved and executed by the processor202. Some examples of instructions include software, program code, and firmware. Some examples of storage media comprise memory devices and integrated circuits. The instructions are operational when executed by the processor202to direct the processor202to operate in accordance with embodiments of the present invention. Those skilled in the art are familiar with instructions, processor(s), and storage media.

The present invention is described above with reference to exemplary embodiments. It will be apparent to those skilled in the art that various modifications may be made and other embodiments can be used without departing from the broader scope of the present invention. For example, embodiments of the present invention may be applied to any system (e.g., non speech enhancement system) utilizing AEC. Therefore, these and other variations upon the exemplary embodiments are intended to be covered by the present invention.