Abstract:
In an example, time and frequency domain speech enhancement is implemented on a platform having a programmable device, such a PC or a smartphone running an OS. Echo cancellation is done first in time domain to cancel a dominant portion of the echo. Residual echo is cancelled jointly with noise reduction during a subsequent frequency domain stage. The time domain block uses a dual band, shorter length Adaptive Filter for faster convergence. Non-linear residual echo is cancelled based on an echo estimate and an error signal from the adaptive filters. A controller locates regions that had residual echo suppressed and which do not have speech and injects comfort noise. The controller can be full-duplex and operate non-linearly. An AGC selectively amplifies the frequency bins, based on the Gain function used by the residual echo and noise canceller.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This applications claims priority from U.S. Provisional Application No. 61/697,682, entitled “SYSTEMS AND METHODS OF ECHO &amp; NOISE CANCELLATION IN VOICE COMMUNICATION”, which was filed on Sep. 6, 2012, and is hereby incorporated by reference in its entirety herein. 
    
    
     BACKGROUND 
     The present invention generally relates to improving quality of voice communication and more particularly to echo and noise cancellation in packet-based voice communication systems. 
     Most VoIP vendors have of goal of to provide a generic VoIP solution for heterogeneous platforms, including platforms such as PCs and mobile platforms. However, variation in platform requirements and characteristics make high performance and platform-generic and speech enhancement a difficult problem. For example, variation in echo path pure delay, hardware non-linearity, and negative ERL, due to situations such as bad acoustic coupling, clock drift and so on pose difficulties. Full duplex voice communication presents difficulties as well. Still other considerations are computation and power efficiency, and maintaining stable performance and quality in a multitasking environment, in which there may be variable computation resource availability. 
     SUMMARY 
     The following discloses methods and systems of echo cancellation that may find application across a wide variety of platforms. In one aspect, the proposed echo cancellation system uses dual band, shorter length time domain Adaptive Filter (ADF) followed by a frequency domain speech enhancement system. The ADF works on two bands with appropriate de-correlation filter to speed up the convergence rate. The frequency domain speech enhancement system includes a Residual Echo and Noise Cancellation System (RENC), a Non-linear Processor (NLP) controller and a Frequency domain Automatic Gain Controller (FAGC). 
     In an aspect, the residual echo from longer reverberation and non-linearity is suppressed further jointly with noise cancellation. It has been found that a large part of the residual echo is correlated with acoustic echo estimate from the ADF. Canceling the residual echo as part of noise cancellation has been found to produce better results than using a spectral subtraction method with platform specific tunable gain parameters for individual frequency bins. 
     In one example implementation, a modified Wiener Filter is used to cancel both residual echo and noise jointly. In another example, a modified Minimum Mean-Square Error Log Spectral Amplitude (MMSE-LSA) cancels residual echo and noise together. In these examples, since residual echo is canceled simultaneously with noise, additional complexity specifically for the residual echo cancellation is reduced. 
     In some examples, the FAGC uses the frequency domain gain function obtained from the residual echo canceller to produce a Voice Activity Decision (VAD). The FAGC amplifies only speech frequency bins, so that the FAGC does not boost a noise signal embedded with the speech and provides better voice quality. 
     The NLP Controller locates sample regions that have only residual echo (and not speech). These regions are processed by an Acoustic Echo Suppressor (AES), which replaces the signal in these regions with comfort noise. In an example, to identify the residual echo alone region, NLP controller uses correlation between inputs including error and microphone signal, error energy, microphone signal energy, and long term average of reference signal amplitude, as described below. In the example, the NLP controller activates non-linear processing on based on a plurality of decision parameters, and further based on a set of pre-defined validation conditions. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  depicts a system context in which methods and systems according to the disclosure can be practiced; 
         FIG. 2  depicts an example architecture of an echo cancellation system according to the disclosure; 
         FIG. 3  depicts an example architecture of a Residual Echo and Noise Canceller (RENC) according to the disclosure; 
         FIG. 4  depicts an example architecture of a gain estimation block; 
         FIG. 5  depicts an example architecture of a Frequency domain Automatic Gain Controller (FAGC) according to the disclosure; 
         FIG. 6  depicts an example flow for a controller of a Non-Linear Processor (NLP) used in the example echo cancellation architecture; 
         FIG. 7  depicts an example of NLP decision logic for a Non-Linear Processor (NLP) used in the example echo cancellation architecture; 
         FIG. 8  depicts an ensemble average of ERLE without nearend party; 
         FIG. 9  depicts ensemble average of ERLE with nearend party active; 
         FIG. 10  depicts FAGC input and output signals and global gain for a tone signal; 
         FIG. 11  depicts FAGC input and output signal power level for a tone signal; 
         FIG. 12  depicts FAGC input and output signals and global gain for a speech signal; 
         FIG. 13  depicts FAGC input and output signal power level for a speech signal; 
         FIG. 14  depicts NLP decisions on an Echo Suppressor (ES) input signal; and 
         FIG. 15  depicts ES output (AES input) and AES output signals. 
     
    
    
     DETAILED DESCRIPTION 
     This disclosure includes sections relating to an example high level architecture of speech enhancement system, details of an example Residual Echo and Noise Cancellation (RENC) system, details of an example Automatic Gain Controller (FAGC), details of a proposed NLP controller and performance examples of the proposed speech enhancement system for real-time captured test signals. 
       FIG. 1  depicts a system context in which methods and systems according to the disclosure can be practiced.  FIG. 1  depicts a situation in which devices (device  20  and device  45 ) support voice communication over packet networks (e.g., network  40 ), and in a particular example, where devices support Voice over Internet Protocol (VoIP). User satisfaction with voice communication is degraded greatly by echo. To provide context, echo can be viewed as a situation in which a far-end signal ( 13 ) from a far-end device  45  is being played from a speaker at a near end ( 12 ) device  20  (this signal can include voice from a person at device  45 , noise  16 , and echo derived from near-end  12  speech played out at a speaker of device  45 . A microphone  23  at near end device  20  samples the available audio energy, including picks up the far-end signal, encodes some part of the far-end signal and returns it to the far-end device  45  (such as in voice packets  34  and  35 ), which produces audio through a speaker, including noise and the echoed far-end signal picked up at near end  12 . note that near-end and far-end here are simply conventions which would change based on perspective; in a full-duplex conversation, they are interchangeable. 
     By further explanation, device  20  (and device  45 ) may include a display  22 , a speaker  24 , a non-volatile storage  25 , a volatile memory hierarchy  26 , and one or more processors  27 . These components can execute an echo cancellation system according to these disclosures. 
     Overview of Echo Cancellation System 
     A high level architecture of an example echo and noise cancellation system is shown in  FIG. 2 . The input signals to Acoustic Echo Canceller (AEC)  102  are the microphone signal d(n) and the farend signal x(n) being played out through speaker (signals having n as an argument are digital versions of a time-domain signal. The system contains a Band Pass Filter(BPF)  107 , Band Splitters  113 , De-Correlation Filters (DCFs)  129 ,  131 , Adaptive Filters (ADFs)  123 ,  125 , Band Mixers  115 ,  117 , Residual Echo &amp; Noise Canceller (RENC)  119 , NLP controller  109 , and Acoustic Echo Suppressor (AES)  111 . Aspects presented herein include example designs for a high performance simultaneous noise &amp; residual echo cancellation unit, an example design of a full-duplex NLP controller and an example design of an efficient frequency domain gain control unit. 
     The example system contains two delay compensation units: pure delay compensation and delay compensation with respect to a microphone signal, in order to synchronize the microphone signal with RENC output signal. The pure delay can be estimated using an ADF running in decimated domain. The estimation of pure delay is configurable. In an example, the algorithmic delay of Residual and Noise Cancellation (RENC) unit is 6 ms, so that a compensation delay is introduced to the microphone signal of about that amount to align with residual echo and Noise Canceller output signal. 
     Band Pass Filter (BPF) 
     It is to remove the DC and unwanted high frequency signal from the inputs. The cut-off frequencies of this filter are 0.0125 and 0.96. A 6th order IIR filter is used because of its simplicity and low processing requirement. 
     Band Splitter 
     It is to split the signal into two channels. Band splitter uses Quadrature Mirror Filter (QMF) filter for band splitting. For the two-bands of AEC processing, the input signal is split into 2 channels with a cut-off frequency of π/2. The sampling rate of each channel is reduced to half of the original sampling rate using decimation factor of 2. This sample rate reduction provides efficient processing of AEC. 
     De-Correlation Filter (DCF) 
     To avoid degradation of the performance of NLMS algorithms due to strong correlation of the speech signals, the farend signal is pre-whitened by applying a de-correlation filter before giving it to adaptive filter. De-correlation filter is a prediction error first order HPF, with its coefficient matched to the correlation properties of the speech signal. This filtering increases the rate of convergence of the adaptive filter for the speech signal. The typical value of filter co-efficient is 0.875. 
     Adaptive Filer (ADF) 
     Adaptive filter (ADF) uses delayed error NLMS algorithm. Since the filter is running in decimated and de-correlated domain with shorter filter length, the convergence of the filter is very faster. The maximum number of taps used per filter is 256. Each ADF has its own built-in near-end speech detector that activates/de-activates the weight adaptation. 
     Band Mixer 
     It is to combine echo estimates and error signals from the two bands after AEC processing to their single bands respectively. Echo estimates and error signals are up-sampled before combining by the synthesis filter bank into an original sampling rate signal. The combined structure for splitting the channels and combining again is called a Quadrature-Mirror Filter (QMF) bank. 
     Band Mixer  115 ,  117  outputs e(n) and y(n) are passed to RENC  119 , which as will be described below, further suppresses echo and background noise. RENC  119  also has an AGC  121 . The RENC  119  outputs signals including s(n) through AGC  120  (see  FIG. 2 ) to AES  111  and s′(n) to NLP controller  109 . s(n) is enhanced nearend signal after canceling residual echo and background noise. The signal s′(n) is an output of FAGC. 
     Since NLP controller  109  uses correlation between error and microphone signal, the output signal obtained before FAGC&#39;s action is given to it. The FAGC output is given to AES unit for further processing to eliminate unwanted very low level residual echo. The AES is controlled based on Non-linear Processor (NLP) decisions. 
     NLP controller  109  enables or disables Non-Linear Processing (NLP), and AES, as being part of NLP. NLP can completely remove the residual echo during single talk. The NLP decision also can ensure no signal clipping when passing from single talk to double-talk. The NLP controller  109  responds quickly, without hangover during start of near end signal present in the output of the microphone signal, this unit also can be called a Sensitive Double-Talk Detector (SNS DTD). 
     Acoustic Echo Suppresser (AES)  111  is a switched attenuator. AES comprises a noise parameter extractor and a Comfort Noise Injection units (CNI). During single talk, AES replaces residual echo by comfort noise generated by CNI unit. AES provides a smooth transition between the original signal and the comfort noise generated by CNI module at the beginning of single talk, as well as ensuring a smooth transition when moving from single talk to nearend speech or nearend background noise. For this seamless transition, AES performs Overlap and Add (OLA) using a triangular window on CNI generated noise and enhanced nearend signal s(n) from FAGC at the start of single talk and also at the end of single talk. During start of the single talk, CNI generated noise is multiplied by a rising ramp and is added to the s(n) multiplied by a falling ramp. Similarly, during end of the single talk, CNI generated noise is multiplied by a falling ramp and is added to s(n), which is multiplied by a rising ramp. In an example, the attenuation or rising factor of the ramp is 0.3662 over a 10 ms period. 
     The AGC output, s(n) is classified into Speech and noise frames. In an example, each frame is 10 ms in length. The classification uses an energy-based VAD algorithm. Average ambient noise level and Linear Predictive Coefficients (LPC) are extracted for each silence/noise frame. 
     CNI unit uses 10th order LPC parameters and Gaussian random number generator for generating comfort noise, which is used for matching the spectrum of nearend ambient noise. This simulated comfort noise replaces the residual echo without a noticeable transition (observable by user), when NLP is activated. 
     Residual Echo and Noise Canceller 
     A block diagram of an example RENC  119  is shown in  FIG. 3 . The example RENC  119  uses modified Frequency domain wiener filtering or an MMSE LSA estimator. In brief, the summation of an estimate of short-term spectral magnitude of ambient background noise and an estimate of short-term spectral magnitude of echo is used to estimate a spectral gain to be applied to an error signal, which includes residual echo, noise, and potentially, near end speech. 
     Assuming that the noise, v(n), is additive to near-end speech signal s(n) at respective discrete time indexes, denoted by the variable n, the noisy near-end speech signal d(n) is represented in equation (1).
 
 d ( n )= s ( n )+ v ( n )  (1)
 
     Error signal e(n) from Band Mixer will contain noisy near-end speech d(n) and residual echo ry(n), as denoted in equation (2). 
                                 e   ⁡     (   n   )       =       ⁢       d   ⁡     (   n   )       +     ry   ⁡     (   n   )                     =       ⁢       s   ⁡     (   n   )       +     v   ⁡     (   n   )       +     ry   ⁡     (   n   )                       (   2   )                 
Windowing
 
     An asymmetric trapezoidal window represented in equation (3). Where, D is the overlap length, L is the input frame length and M is the window length. Incoming samples are stored in a buffer of length L samples; last D samples from the previous frame are appended to this buffer and remaining samples are taken as zeros to make up for a buffer of length equal to window length M. In one example, the value of M is 176 samples, L is 80 samples and D is 48 samples. Buffered samples are windowed using trapezoidal window and then transformed into frequency domain for processing to reduce the jitter in packet transmission of packet based communication system such as VoIP. 
     
       
         
           
             
               
                 
                   
                     w 
                     ⁡ 
                     
                       ( 
                       n 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               
                                 sin 
                                 2 
                               
                               ⁡ 
                               
                                 ( 
                                 
                                   
                                     
                                       π 
                                       ⁡ 
                                       
                                         ( 
                                         
                                           n 
                                           + 
                                           0.5 
                                         
                                         ) 
                                       
                                     
                                     / 
                                     2 
                                   
                                   ⁢ 
                                   D 
                                 
                                 ) 
                               
                             
                             , 
                           
                         
                         
                           
                             
                               0 
                               ≤ 
                               n 
                               &lt; 
                               D 
                             
                             , 
                           
                         
                       
                       
                         
                           
                             1 
                             , 
                           
                         
                         
                           
                             
                               D 
                               ≤ 
                               n 
                               &lt; 
                               L 
                             
                             , 
                           
                         
                       
                       
                         
                           
                             
                               
                                 sin 
                                 2 
                               
                               ⁡ 
                               
                                 ( 
                                 
                                   
                                     
                                       π 
                                       ⁡ 
                                       
                                         ( 
                                         
                                           n 
                                           - 
                                           L 
                                           + 
                                           D 
                                           + 
                                           0.5 
                                         
                                         ) 
                                       
                                     
                                     / 
                                     2 
                                   
                                   ⁢ 
                                   D 
                                 
                                 ) 
                               
                             
                             , 
                           
                         
                         
                           
                             
                               L 
                               ≤ 
                               n 
                               &lt; 
                               
                                 D 
                                 + 
                                 L 
                               
                             
                             , 
                           
                         
                       
                       
                         
                           
                             0 
                             , 
                           
                         
                         
                           
                             
                               
                                 D 
                                 + 
                                 L 
                               
                               ≤ 
                               n 
                               &lt; 
                               M 
                             
                             , 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
     Frequency Domain Conversion: The error signal e(n) and scaled echo estimate r′y(n) are divided into overlapping frames by the application of a trapezoidal window function where r′ is a fixed correlation factor. The respective windowed signals are converted to frequency domain using Fourier Transform  160   161  (e.g., a Short-Time Fourier transform (STFT). 
     Let E k (l) and r′Y k (l) represent STFT of error signal e(n) and the scaled echo estimate r′y(n) respectively for the frame index l and frequency bin index k. Then error signal is given as
 
 E   k ( l )= S   k ( l )+ V   k ( l )+ Y   k ( l )  (4)
 
     Where, S k (l), V k (l) and Y k (l) represent STFT of nearend signal, s(n), the background noise, v(n) and residual echo y(n). 
     Reverberation Tracker 
     Since the AEC tail length used is short, it may not cancel the echoes completely when actual reverberation is longer than the tail length of the echo cancellation filters. So, to cancel them, a moving average filter with low attack rate and fast release rate is used on actual echo estimate obtained from echo cancellation filter. The estimation from moving average filters is controlled using appropriate logic when actual reverberation is within the tail length of echo cancellation filter. Equation 5 represents lengthen echo estimate R k (l). 
     
       
         
           
             
               
                 
                   
                     
                       R 
                       k 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               
                                 
                                   α 
                                   1 
                                 
                                 ⁢ 
                                 
                                   
                                     R 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     
                                       l 
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                               
                               + 
                               
                                 
                                   ( 
                                   
                                     1 
                                     - 
                                     
                                       α 
                                       1 
                                     
                                   
                                   ) 
                                 
                                 ⁢ 
                                 
                                   r 
                                   ′ 
                                 
                                 ⁢ 
                                 
                                   
                                     Y 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                               
                             
                             , 
                           
                         
                       
                       
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   
                                     r 
                                     ′ 
                                   
                                   ⁢ 
                                   
                                     
                                       Y 
                                       k 
                                     
                                     ⁡ 
                                     
                                       ( 
                                       l 
                                       ) 
                                     
                                   
                                 
                                 &gt; 
                                 
                                   
                                     R 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     
                                       l 
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 
                                   α 
                                   2 
                                 
                                 ⁢ 
                                 
                                   
                                     R 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     
                                       l 
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                               
                               + 
                               
                                 
                                   ( 
                                   
                                     1 
                                     - 
                                     
                                       α 
                                       2 
                                     
                                   
                                   ) 
                                 
                                 ⁢ 
                                 
                                   r 
                                   ′ 
                                 
                                 ⁢ 
                                 
                                   
                                     Y 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                               
                             
                             , 
                             else 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
         
         
           
             Where, α 2 &lt;α 1 &lt;1.
 
Noise Estimation
 
           
         
       
    
     Noise estimation uses external VAD. The VAD identifies presence of voice activity in the input error signal coming from ADF. When the VAD decision shows noise frame (i.e., VAD=0), noise estimation V k (l) is updated as per equation (6). 
                       V   k     ⁡     (   l   )       =     {               α   3     ⁢       V   k     ⁡     (     l   -   1     )         +       (     1   -     α   3       )     ⁢       V   k     ⁡     (   l   )                   if   ⁢           ⁢   VAD     =   0                 V   k     ⁡     (     l   -   1     )           otherwise                   (   6   )               
Cancellation Part
 
     The total signal that is suppressed from error signal in frequency domain at all frequency bins for a given frame l is given as
 
 NR   k ( l )= V   k ( l )+ R   k ( l )  (7)
 
Estimation Controller
 
     Even though equation (7) represents the unwanted components that are to be subtracted from error signal, there is a chance of over estimation possible in different platforms. This over estimation can be due to r′ value being greater than the ratio between actual residual echo and the echo estimate. To control the over estimate, moving average of error signal can be estimated using low pass filtering with dual α coefficient, such as in equation (8). 
     
       
         
           
             
               
                 
                   
                     
                       W 
                       k 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               
                                 α 
                                 4 
                               
                               ⁢ 
                               
                                 
                                   W 
                                   k 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                             
                             + 
                             
                               
                                 ( 
                                 
                                   1 
                                   - 
                                   
                                     α 
                                     4 
                                   
                                 
                                 ) 
                               
                               ⁢ 
                               
                                 
                                   E 
                                   k 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   
                                     W 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     
                                       l 
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                                 &gt; 
                                 
                                   
                                     E 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                               
                           
                         
                       
                       
                         
                           
                             
                               
                                 α 
                                 5 
                               
                               ⁢ 
                               
                                 
                                   W 
                                   k 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                             
                             + 
                             
                               
                                 ( 
                                 
                                   1 
                                   - 
                                   
                                     α 
                                     5 
                                   
                                 
                                 ) 
                               
                               ⁢ 
                               
                                 
                                   E 
                                   k 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   
                                     W 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     
                                       l 
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                                 ≤ 
                                 
                                   
                                     E 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
     To control the over estimation of cancellation part NR k (l), a ceiling operation is performed and modified cancellation part is estimated as given in equation (9).
 
 P   k ( l )=min( NR   k ( l ), W   k ( l ))  (9)
 
     The example RENC  119  filters out the cancellation part by modifying the spectral amplitudes of each frequency bins |E k (l)| in equation (4) by applying the gain estimates G k (l) as below
 
 S   k ( l )= G   k ( l ) E   k ( l ),for 0 ≦G   k ( l )≦1  (10)
 
     Gain estimates G k (l) is formed as a function of a posteriori SNR γ k (l), and a priori SNR ξ k (l). The γ k (l) and ξ k (l) are estimated as below using statistical variances of error signal or the expected clean near-end speech and the cancellation part signal. 
     
       
         
           
             
               
                 
                   
                     
                       γ 
                       k 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   ≡ 
                   
                     
                       
                          
                         
                           
                             E 
                             k 
                           
                           ⁡ 
                           
                             ( 
                             l 
                             ) 
                           
                         
                          
                       
                       2 
                     
                     
                       E 
                       ⁡ 
                       
                         ( 
                         
                           
                              
                             
                               
                                 P 
                                 k 
                               
                               ⁡ 
                               
                                 ( 
                                 l 
                                 ) 
                               
                             
                              
                           
                           2 
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       ξ 
                       k 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   ≡ 
                   
                     
                       E 
                       ⁡ 
                       
                         ( 
                         
                           
                              
                             
                               
                                 S 
                                 k 
                               
                               ⁡ 
                               
                                 ( 
                                 l 
                                 ) 
                               
                             
                              
                           
                           2 
                         
                         ) 
                       
                     
                     
                       E 
                       ⁡ 
                       
                         ( 
                         
                           
                              
                             
                               
                                 P 
                                 k 
                               
                               ⁡ 
                               
                                 ( 
                                 l 
                                 ) 
                               
                             
                              
                           
                           2 
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
           
         
       
     
     The statistical variance of clean near-end speech E(|S k (l)| 2 ) for the estimation of ξ k (l) is estimated using Decision-Directed (DD) method [1] proposed by Ephraim and Malah using 0&lt;α&lt;1 and is as. 
     
       
         
           
             
               
                 
                   
                     
                       ξ 
                       k 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     
                       α 
                       ⁢ 
                       
                         
                           
                              
                             
                               
                                 S 
                                 k 
                               
                               ⁡ 
                               
                                 ( 
                                 
                                   l 
                                   - 
                                   1 
                                 
                                 ) 
                               
                             
                              
                           
                           2 
                         
                         
                           E 
                           ⁡ 
                           
                             ( 
                             
                               
                                  
                                 
                                   
                                     P 
                                     k 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                                  
                               
                               2 
                             
                             ) 
                           
                         
                       
                     
                     + 
                     
                       
                         ( 
                         
                           1 
                           - 
                           α 
                         
                         ) 
                       
                       ⁢ 
                       
                         MAX 
                         ⁡ 
                         
                           ( 
                           
                             
                               
                                 
                                   γ 
                                   k 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                               - 
                               1 
                             
                             , 
                             0 
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
       FIG. 4  shows a block diagram of a gain estimator  175 . The formation of G k (l) function is done using: (1) Frequency domain wiener filtering or (2) MMSE LSA estimator. 
     Frequency Domain Weiner Filtering: The Wiener filter is a popular adaptive technique that has been used in many enhancement methods. Approach based on optimal filtering and the aim is to find the optimal filter that would minimize the mean square error between the desired signal (clean signal) and the estimated output. The Wiener filter gain G k (l) is estimated by solving an equation in which the derivative of the mean square error with respect to the filter coefficients is set to zero: 
     
       
         
           
             
               
                 
                   
                     
                       G 
                       k 
                       W 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         ξ 
                         k 
                       
                       ⁡ 
                       
                         ( 
                         l 
                         ) 
                       
                     
                     
                       
                         
                           ξ 
                           k 
                         
                         ⁡ 
                         
                           ( 
                           l 
                           ) 
                         
                       
                       + 
                       1 
                     
                   
                 
               
               
                 
                   ( 
                   14 
                   ) 
                 
               
             
           
         
       
     
     The Wiener filter emphasizes portions of the spectrum where the SNR is high, and attenuates portions of the spectrum where the SNR is low. Iterative Wiener filtering constructs an optimal linear filter using estimates of both the underlying speech and underlying noise spectra. 
     Minimum Mean-Square Error Log Spectral Amplitude (MMSE-LSA): This technique makes an assumption that Fourier expansion coefficients of noise components (V k (l) and RY k (l)) and near-end speech are statistically independent, and that they follow a Gaussian distribution. Log-spectra is used in distortion measures, and is motivation to examine the effect of an amplitude estimator constrained to minimizing mean-squared error of the log-spectra. Let A k  be the actual amplitude of the near-end speech signal and Ā k  be the estimated amplitude of the near-end speech signal. The cost function used to estimate the gain is given by
 
 E {(log  A   k −log  Ā   k ) 2 }  (15)
 
     The gain function is given by the equation (16), 
     
       
         
           
             
               
                 
                   
                     
                       G 
                       k 
                       LSA 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         
                           ξ 
                           k 
                         
                         ⁡ 
                         
                           ( 
                           l 
                           ) 
                         
                       
                       
                         1 
                         + 
                         
                           
                             ξ 
                             k 
                           
                           ⁡ 
                           
                             ( 
                             l 
                             ) 
                           
                         
                       
                     
                     ⁢ 
                     exp 
                     ⁢ 
                     
                       { 
                       
                         
                           1 
                           2 
                         
                         ⁢ 
                         
                           
                             ∫ 
                             
                               v 
                               k 
                             
                             ∞ 
                           
                           ⁢ 
                           
                             
                               
                                 ⅇ 
                                 
                                   - 
                                   t 
                                 
                               
                               t 
                             
                             ⁢ 
                             
                               ⅆ 
                               t 
                             
                           
                         
                       
                       } 
                     
                   
                 
               
               
                 
                   ( 
                   16 
                   ) 
                 
               
             
           
         
       
     
     Since the estimation of integral function over the exponential of equation (16) is very complex, the exponential integral in (16) can be evaluated using a functional approximation shown in equation 17. 
                       G   k   LSA     ⁡     (   l   )       =       (         ξ   k     ⁡     (   l   )         1   +       ξ   k     ⁡     (   l   )           )     ⁢     exp   ⁡     (       ⅇ       v   k     ⁡     (   l   )         2     )                 (   17   )               
Where, v k (l) and e v     k     (l)  are defined in the following equations (18) and (19) respectively.
 
                       v   k     ⁡     (   l   )       =           ξ   k     ⁡     (   l   )         1   +       ξ   k     ⁡     (   l   )           ⁢       γ   k     ⁡     (   l   )                 (   18   )                 ⅇ       v   k     ⁡     (   l   )         =     {                 -   2.31     ⁢           ⁢     log   10     ⁢       v   k     ⁡     (   l   )         -   0.6     ,               v   k     ⁡     (   l   )       &lt;   0.1                 10     -     (       0.52   ⁢           ⁢       v   k     ⁡     (   l   )         +   0.26     )         ,               v   k     ⁡     (   l   )       &gt;   1                     -   1.54     ⁢           ⁢     log   10     ⁢       v   k     ⁡     (   l   )         +   0.166     ,         otherwise                   (   19   )               
Gain Smoothing
 
     To avoid abrupt change across the frequency bins, gain smoothing is done as below. 
     
       
         
           
             
               
                 
                   
                     
                       G 
                       k 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               
                                 α 
                                 5 
                               
                               ⁢ 
                               
                                 
                                   G 
                                   k 
                                   W 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                             
                             + 
                             
                               
                                 ( 
                                 
                                   1 
                                   - 
                                   
                                     α 
                                     5 
                                   
                                 
                                 ) 
                               
                               ⁢ 
                               
                                 
                                   G 
                                   k 
                                   W 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                             
                           
                         
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   
                                     G 
                                     k 
                                     W 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     
                                       l 
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                                 &gt; 
                                 
                                   
                                     G 
                                     k 
                                     W 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 α 
                                 6 
                               
                               ⁢ 
                               
                                 
                                   G 
                                   k 
                                   W 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                             
                             + 
                             
                               
                                 ( 
                                 
                                   1 
                                   - 
                                   
                                     α 
                                     6 
                                   
                                 
                                 ) 
                               
                               ⁢ 
                               
                                 
                                   G 
                                   k 
                                   W 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                             
                           
                         
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   
                                     G 
                                     k 
                                     W 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     
                                       l 
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                                 ≤ 
                                 
                                   
                                     G 
                                     k 
                                     W 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             
                               G 
                               k 
                               W 
                             
                             ⁡ 
                             
                               ( 
                               l 
                               ) 
                             
                           
                         
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 l 
                                 &lt; 
                                 T 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   20 
                   ) 
                 
               
             
           
         
       
     
     2D Filtering: To smooth abrupt change in gain estimation across the frequency bins, smoothing is done as below.
 
 G   k   F ( l )=(α 7   G   k ( l− 1)+α 8   G   k ( l ))*(1/(α 7 +α 8 ))  (21)
         for (k&gt;1)       

                                                         TABLE 1                   Constants used by RENC 119                Constant   Value   Remarks                        α 1     0.61   Reverberation Tracker           α 2     0.21   smoothing factor           α 3     0.13   Noise estimation                   smoothing factor           α 4     0.61   Estimation Controller           α 5     0.21   smoothing factor           α   0.98   Decision Directed                   smoothing factor           α 5     0.98   Gain estimation           α 6     0.28   smoothing factor           α 7     7   2D filtering           α 8     1   smoothing factor           r′   2.8   Expected ratio between                   residual echo and echo                   estimate           T   40   Initial 40 frames           L   80   Frame size of 10 msec           D   48   Overlap size of 6 msec           M   176   Window length                    
Overlap and Add (OLA)
 
     The estimated Gain is applied on error signal as per equation (10) and the enhanced STSA S k (l) is obtained. Enhanced near-end speech s(n) is then reconstructed by applying the inverse FFT to the enhanced STSA, |S k (l)|, with the noisy phase ∠E k (l), followed by an appropriate overlap-and-add (OLA) procedure to compensate for the window effect and to alleviate abrupt signal changes between two consecutive frames. 
     Frequency Domain Automatic Gain Controller (FAGC) 
     The smoothed gain G k   F (l), and enhanced speech frequency bins S k (l) are used for estimating gain for each frequency bin to achieve target power level in the output. The high level architecture of the proposed AGC is shown in  FIG. 4 . VAD block estimates presence of voice activity for each frequency bin. If voice activity presence is detected at least on one frequency bin, the new gain is estimated by the computation module. Then the new gain is applied on the enhanced speech S k (l). 
     Voice Activity Detection (VAD) 
     Since calculating AGC gain for the silence frames is not needed, classification of a frame as speech/silence is required for gain calculations. Since, AGC is supposed to apply gain only on the nearend signal, it should not amplify echo or noise regions. So, the suppressor gain G k   F (l) is expected to be lower than unity for echo and noise regions. Also, the suppressor gain can be used for deciding the presence of nearend speech activity, as below.
 
 bvad   k ( l )=1 if ( G   k   F ( l )&gt;λ 1 )
 
 vad ( l )=1 if ( bvad   k ( l )==1),for any  k   (22)
 
     Where bvad k (l) represents VAD decision for k th  frequency bin in l th  frame. vad(l) represents global VAD decision for l th  frame. 
     The decision of VAD-activity for individual bins in a given frame are considered and if more than one bin is classified as a speech bin the frame is classified as a speech frame otherwise as silence frame. 
     Gain Computation Unit 
     The Gain Computation Unit estimates global frame gain from the RMS power level of nearend speech. The gain for each frequency bin is estimated using global frame gain G M (l) and low pass filtering. Total speech power level is given by
 
 P   sp ( l )=Σ( S   k   2 ( l )* bvad   k ( l ))  (23)
 
     Similarly, noise power is estimated as
 
 P   n ( l )=Σ( S   k   2 ( l )− P   sp ( l ))  (24)
 
     Global frame gain is estimated as given below, 
     
       
         
           
             
               
                 
                   
                     
                       G 
                       r 
                       M 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     
                       1 
                       
                         
                           msqr 
                           ⁡ 
                           
                             ( 
                             l 
                             ) 
                           
                         
                       
                     
                     * 
                     
                       ( 
                       TL 
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   25 
                   ) 
                 
               
             
           
         
       
     
     Where, TL is calibrated target power level considering the frame size and spectral leakage during windowing for the given actual target level in dB. Initial mean square value msqr(0) is given by equation (26).
 
msqr(0)=( TL*TL )  (26)
 
     Mean square values (msqr(l)) are estimated using a LPF as given below
 
msqr( l )=msqr( l− 1)+ P′   m ( l )  (27)
 
     Where, P′ m (l) is given by equation (27), and P m (l) is given by equation (28). 
     
       
         
           
             
               
                 
                   
                     tmp 
                     = 
                     
                       
                         
                           P 
                           m 
                         
                         ⁡ 
                         
                           ( 
                           l 
                           ) 
                         
                       
                       - 
                       
                         msqr 
                         ⁡ 
                         
                           ( 
                           
                             l 
                             - 
                             1 
                           
                           ) 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         P 
                         m 
                         ′ 
                       
                       ⁡ 
                       
                         ( 
                         l 
                         ) 
                       
                     
                     = 
                     
                       { 
                       
                         
                           
                             
                               
                                 
                                   P 
                                   m 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                               * 
                               
                                 λ 
                                 2 
                               
                             
                           
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 ( 
                                 
                                   tmp 
                                   &gt; 
                                   0 
                                 
                                 ) 
                               
                             
                           
                         
                         
                           
                             
                               
                                 
                                   P 
                                   m 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                               * 
                               
                                 λ 
                                 3 
                               
                             
                           
                           
                             otherwise 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   28 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       P 
                       m 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         P 
                         sp 
                       
                       ⁡ 
                       
                         ( 
                         l 
                         ) 
                       
                     
                     + 
                     
                       
                         
                           P 
                           n 
                         
                         ⁡ 
                         
                           ( 
                           l 
                           ) 
                         
                       
                       * 
                       
                         λ 
                         4 
                       
                     
                   
                 
               
               
                 
                   ( 
                   29 
                   ) 
                 
               
             
           
         
       
     
     The calculated gain is limited to the range of the allowable maximum and minimum values before applying it to the frames. In a case where low amplitude to high amplitude level transition is encountered in the input, the computed gain may exceed the limit and may cause a momentary transition spike. This phenomenon can be minimized through a condition to check gain blow over, by limiting the gain to a maximum gain value G MAX  to avoid any spiking and ensure smooth transition. 
     
       
         
           
             
               
                 
                   
                     
                       G 
                       r 
                       M 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             G 
                             MAX 
                           
                         
                         
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 
                                   G 
                                   r 
                                   M 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                             
                             &gt; 
                             
                               G 
                               MAX 
                             
                           
                         
                       
                       
                         
                           
                             G 
                             MIN 
                           
                         
                         
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 
                                   G 
                                   r 
                                   M 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   l 
                                   ) 
                                 
                               
                             
                             &lt; 
                             
                               G 
                               MIN 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   30 
                   ) 
                 
               
             
           
         
       
     
     To avoid high fluctuations between two frames that will result in signal distortion the gain is smoothed over time and is given below. 
     
       
         
           
             
               
                 
                   
                     tmp 
                     = 
                     
                       
                         
                           G 
                           r 
                           M 
                         
                         ⁡ 
                         
                           ( 
                           l 
                           ) 
                         
                       
                       - 
                       
                         
                           G 
                           M 
                         
                         ⁡ 
                         
                           ( 
                           
                             l 
                             - 
                             1 
                           
                           ) 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         G 
                         M 
                       
                       ⁡ 
                       
                         ( 
                         l 
                         ) 
                       
                     
                     = 
                     
                       { 
                       
                         
                           
                             
                               
                                 
                                   G 
                                   M 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                               + 
                               
                                 tmp 
                                 * 
                                 
                                   λ 
                                   5 
                                 
                               
                             
                           
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 ( 
                                 
                                   tmp 
                                   &gt; 
                                   0 
                                 
                                 ) 
                               
                             
                           
                         
                         
                           
                             
                               
                                 
                                   G 
                                   M 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                               + 
                               
                                 tmp 
                                 * 
                                 
                                   λ 
                                   6 
                                 
                               
                             
                           
                           
                             otherwise 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   31 
                   ) 
                 
               
             
           
         
       
     
     Different smoothing factors are applied for transitions from noise to speech and speech to noise respectively. These values are chosen in such a way that the attack time is faster than the release time. Attack Time should be fast for preventing harsh distortion when the amplitude rapidly increases and the decay Time should be relatively longer to avoid chopper effect to assure low distortion. 
     The computed gain is applied to speech and noise bins separately based on the VAD activity decision for each bin. To avoid distortion across frequency bins due to high gain differences across neighboring frequency bins, 2-D filtering on individual VAD decisions of each frequency bin is applied. 
     
       
         
           
             
               
                 
                   
                     
                       bvad 
                       k 
                       
                         2 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         d 
                       
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           1 
                         
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   
                                     bvad 
                                     i 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                                 == 
                                 1 
                               
                             
                           
                         
                       
                       
                         
                           
                             i 
                             = 
                             
                               k 
                               - 
                               
                                 1 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 or 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 k 
                               
                             
                           
                         
                         
                           
                             
                               or 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               k 
                             
                             + 
                             1 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   32 
                   ) 
                 
               
             
           
         
       
     
     With the knowledge of voice activity for each frame, individual frames are treated separately for the gain calculation. Gain to unvoiced portions that contain only background noise is set to unity. The AGC gain calculated for a given frame is given below for speech frequency bins bvad k   2d (l). 
     
       
         
           
             
               
                 
                   
                     tmp 
                     = 
                     
                       
                         
                           G 
                           M 
                         
                         ⁡ 
                         
                           ( 
                           l 
                           ) 
                         
                       
                       - 
                       
                         
                           G 
                           k 
                           AGC 
                         
                         ⁡ 
                         
                           ( 
                           
                             l 
                             - 
                             1 
                           
                           ) 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         G 
                         k 
                         AGC 
                       
                       ⁡ 
                       
                         ( 
                         l 
                         ) 
                       
                     
                     = 
                     
                       { 
                       
                         
                           
                             
                               
                                 
                                   G 
                                   k 
                                   AGC 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                               + 
                               
                                 tmp 
                                 * 
                                 
                                   λ 
                                   7 
                                 
                               
                             
                           
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 ( 
                                 
                                   tmp 
                                   &gt; 
                                   0 
                                 
                                 ) 
                               
                             
                           
                         
                         
                           
                             
                               
                                 
                                   G 
                                   k 
                                   AGC 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                               + 
                               
                                 ( 
                                 
                                   tmp 
                                   * 
                                   
                                     λ 
                                     8 
                                   
                                 
                                 ) 
                               
                             
                           
                           
                             otherwise 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   33 
                   ) 
                 
               
             
           
         
       
     
     If bvad k   2d (l) is noise, below equation is estimated for AGC gain (G k   AGC (l)). 
     
       
         
           
             
               
                 
                   
                     
                       G 
                       k 
                       AGC 
                     
                     ⁡ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               
                                 G 
                                 k 
                                 AGC 
                               
                               ⁡ 
                               
                                 ( 
                                 
                                   l 
                                   - 
                                   1 
                                 
                                 ) 
                               
                             
                             * 
                             
                               λ 
                               9 
                             
                           
                         
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   ( 
                                   
                                     
                                       
                                         G 
                                         k 
                                         AGC 
                                       
                                       ⁡ 
                                       
                                         ( 
                                         
                                           l 
                                           - 
                                           1 
                                         
                                         ) 
                                       
                                     
                                     &gt; 
                                     1 
                                   
                                   ) 
                                 
                                 || 
                                 
                                   ( 
                                   
                                     
                                       
                                         G 
                                         k 
                                         AGC 
                                       
                                       ⁡ 
                                       
                                         ( 
                                         
                                           l 
                                           - 
                                           1 
                                         
                                         ) 
                                       
                                     
                                     &gt; 
                                     
                                       
                                         G 
                                         M 
                                       
                                       ⁡ 
                                       
                                         ( 
                                         l 
                                         ) 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             
                               G 
                               k 
                               AGC 
                             
                             ⁡ 
                             
                               ( 
                               
                                 l 
                                 - 
                                 1 
                               
                               ) 
                             
                           
                         
                         
                           otherwise 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   34 
                   ) 
                 
               
             
           
         
       
     
     Finally, the computed gain is applied to respective frequency bins of enhanced speech coming out of residual echo suppressor.
 
 S′   k ( l )= G   k   AGC ( l )* S   k ( l )  (35)
 
     After gain multiplication on frequency domain, the frame is inverse transformed and the segments are put in order by overlap and add method (OLA) discussed in earlier sections. 
     
       
         
               
             
               
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 2 
               
             
             
               
                   
               
               
                 Constants used by FAGC 
               
             
          
           
               
                   
                 Constant 
                 Value 
                 Remarks 
               
               
                   
               
             
          
           
               
                   
                 λ 1   
                 0.732 
                 VAD decision factor 
               
               
                   
                   
                   
                 for each bin 
               
               
                   
                 λ 2   
                 0.793 
                 Multiplication factor 
               
               
                   
                 λ 3   
                 0.183 
                 Multiplication factor 
               
               
                   
                 λ 4   
                 0.5 
                 Multiplication factor to Noise power 
               
               
                   
                 G MAX   
                 8 
                 Gain Limitation 
               
               
                   
                 G MIN   
                 0.00015 
                   
               
               
                   
                 λ 3   
                 32 
                 Global Gain 
               
               
                   
                 λ 4   
                 0.6 
                 Smoothing factor 
               
               
                   
                 λ 5   
                 0.457 
                 AGC gain 
               
               
                   
                 λ 6   
                 0.793 
                 Smoothing factors 
               
               
                   
                 λ 7   
                 0.996 
                 AGC Gain limiter 
               
               
                   
               
             
          
         
       
     
     Non-Linear Processor (NLP) Controller 
       FIGS. 6 and 7  depict example aspects of NLP control and NLP decision logic (which is used in NLP control), which are performed in NLP controller  109 . NLP controller  109  enables or disables NLP to completely remove the residual echo during single talk. Also, it is a goal to ensure no signal clipping occurs while passing from single talk to double-talk and vice versa. The NLP decisions are made from the combination of normalized correlation between modified microphone signal and enhanced error signal by power of microphone signal and the normalized correlation between modified microphone signal and enhanced error signal by power of error signal. 
     NLP controller  109  outputs NLP decisions for discrete time intervals, nlp(n). NLP controller  109  uses several inputs in producing NLP decisions. The production of these inputs is collectively referred to as decision parameters estimation  305 . These inputs include correlation between error signal and microphone signal, ed enr (n). This correlation also can be used for echo detection, such that ed enr (n) also can be used as an indication of echo. Other inputs include, normalization parameters, such as error energy e enr (n), and microphone signal energy d enr (n), noise energy v enr (n), convergence indicator conv(n), long term average of reference signal amplitude ly(n), absolute value of error signal, e abs (n), and absolute value of modified microphone signal. NLP also uses counters for stability checking. These counters include counts for hangover. Before starting NLP decision making, hangover counts and NLP decision parameters are set as given below.
 
 nlp ( n )=0
 
distorsion( n )=0
 
 st _hngovr( n )= st _hngovr( n− 1)
 
 dt _hngovr( n )= dt _hngovr( n− 1)
 
 nlp   enr ( n )= nlp   enr ( n− 1)  (36)
 
     The input signals (microphone signal and error signal) to the NLP controller  109  are scaled to avoid saturation in computation using 16-bit registers. The scaling factor can be experimentally determined. The scaled down signals are called modified microphone signal d′(n) and enhanced error signal e n (n), and respectively are estimated by below equation (37).
 
 d ′( n )= d ( n−D   1 )/16
 
 e   n ( n )= s ′( n )/16  (37)
 
     Cross correlation ed enr (n) between modified microphone signal d′(n) and enhanced error signal e n (n) is called echo indicator parameter and is a major parameter deciding NLP activation/de-activation (decision to activate, not activate or deactivate). This parameter is estimated as below 
     
       
         
           
             
               
                 
                   
                     
                       ed 
                       enr 
                     
                     ⁡ 
                     
                       ( 
                       n 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         ed 
                         enr 
                       
                       ⁡ 
                       
                         ( 
                         
                           n 
                           - 
                           1 
                         
                         ) 
                       
                     
                     - 
                     
                       ( 
                       
                         
                           
                             d 
                             ′ 
                           
                           ⁡ 
                           
                             ( 
                             
                               n 
                               - 
                               K 
                             
                             ) 
                           
                         
                         * 
                         
                           
                             e 
                             n 
                           
                           ⁡ 
                           
                             ( 
                             
                               n 
                               - 
                               K 
                             
                             ) 
                           
                         
                       
                       ) 
                     
                     + 
                     
                       ( 
                       
                         
                           
                             d 
                             ′ 
                           
                           ⁡ 
                           
                             ( 
                             n 
                             ) 
                           
                         
                         * 
                         
                           
                             e 
                             n 
                           
                           ⁡ 
                           
                             ( 
                             n 
                             ) 
                           
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   38 
                   ) 
                 
               
             
           
         
       
     
     Other important parameters include normalization factors, including microphone energy d enr (n) and enhanced error energy e enr (n), and can be estimated as in equation (39) 
     
       
         
           
             
               
                 
                   
                     
                       
                         d 
                         enr 
                       
                       ⁡ 
                       
                         ( 
                         n 
                         ) 
                       
                     
                     = 
                     
                       
                         
                           d 
                           enr 
                         
                         ⁡ 
                         
                           ( 
                           
                             n 
                             - 
                             1 
                           
                           ) 
                         
                       
                       - 
                       
                         [ 
                         
                           
                             
                               d 
                               ′ 
                             
                             ⁡ 
                             
                               ( 
                               
                                 n 
                                 - 
                                 K 
                               
                               ) 
                             
                           
                           * 
                           
                             
                               d 
                               ′ 
                             
                             ⁡ 
                             
                               ( 
                               
                                 n 
                                 - 
                                 K 
                               
                               ) 
                             
                           
                         
                         ] 
                       
                       + 
                       
                         ( 
                         
                           
                             
                               d 
                               ′ 
                             
                             ⁡ 
                             
                               ( 
                               n 
                               ) 
                             
                           
                           * 
                           
                             
                               d 
                               ′ 
                             
                             ⁡ 
                             
                               ( 
                               n 
                               ) 
                             
                           
                         
                         ) 
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   
                     
                       
                         e 
                         enr 
                       
                       ⁡ 
                       
                         ( 
                         n 
                         ) 
                       
                     
                     = 
                     
                       
                         
                           e 
                           enr 
                         
                         ⁡ 
                         
                           ( 
                           
                             n 
                             - 
                             1 
                           
                           ) 
                         
                       
                       - 
                       
                         [ 
                         
                           
                             
                               e 
                               n 
                             
                             ⁡ 
                             
                               ( 
                               
                                 n 
                                 - 
                                 K 
                               
                               ) 
                             
                           
                           * 
                           
                             
                               e 
                               n 
                             
                             ⁡ 
                             
                               ( 
                               
                                 n 
                                 - 
                                 K 
                               
                               ) 
                             
                           
                         
                         ] 
                       
                       + 
                       
                         ( 
                         
                           
                             
                               e 
                               n 
                             
                             ⁡ 
                             
                               ( 
                               n 
                               ) 
                             
                           
                           * 
                           
                             
                               e 
                               n 
                             
                             ⁡ 
                             
                               ( 
                               n 
                               ) 
                             
                           
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   39 
                   ) 
                 
               
             
           
         
       
     
     Noise energy is another decision parameter that is used mainly for breaking hangover. Noise energy is estimated using a moving average filter as per (40).
 
 v   enr ( n )= v   enr ( n− 1)+β 1 ( e   enr ( n )− v   enr ( n− 1)) if ( e   enr ( n )&gt; v   enr ( n− 1))
 
 v   enr ( n )= v   enr ( n− 1)+β 2 ( e   enr ( n )− v   enr ( n− 1)) otherwise  (40)
 
     There are five counters used for stability and other purposes. Startup indicator counter m_cnt(n) is used to indicate initial session timing. This counter also indicates a number of samples processed by the proposed system before ADF convergence is achieved. This counter&#39;s maximum value is limited by the register length being used to avoid overflow.
 
 m   —   cnt ( n )= m   —   cnt ( n )+1 if ( m   —   cnt ( n )&lt;β 3 )  (41)
 
     Another counter counts recent noise frames. This counter uses VAD decisions (VAD(l)) from RENC  119 . 
     
       
         
           
             
               
                 
                   
                     v_cnt 
                     ⁢ 
                     
                       ( 
                       l 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             0 
                             , 
                           
                         
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   VAD 
                                   ⁡ 
                                   
                                     ( 
                                     l 
                                     ) 
                                   
                                 
                                 == 
                                 1 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 v_cnt 
                                 ⁢ 
                                 
                                   ( 
                                   
                                     l 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                               + 
                               1 
                             
                             , 
                           
                         
                         
                           else 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   42 
                   ) 
                 
               
             
           
         
       
     
     Another counter is an adaptation counter adp_cnt(n) used to indicate a number of samples, during which the ADFs have maintained convergence. Adaptation counter allows taking hard NLP decisions during start of convergence. After ADF convergence, the adaptation counter does not factor into NLP decision logic. 
     
       
         
           
             
               
                 
                   
                     adp_cnt 
                     ⁢ 
                     
                       ( 
                       n 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               
                                 adp_cnt 
                                 ⁢ 
                                 
                                   ( 
                                   
                                     n 
                                     - 
                                     1 
                                   
                                   ) 
                                 
                               
                               + 
                               1 
                             
                             , 
                           
                         
                         
                           
                             if 
                             ⁢ 
                             
                                 
                             
                             ⁢ 
                             
                               ( 
                               
                                 
                                   ADAP 
                                   ⁡ 
                                   
                                     ( 
                                     n 
                                     ) 
                                   
                                 
                                 == 
                                 1 
                               
                               ) 
                             
                           
                         
                       
                       
                         
                           
                             
                               adp_cnt 
                               ⁢ 
                               
                                 ( 
                                 
                                   n 
                                   - 
                                   1 
                                 
                                 ) 
                               
                             
                             , 
                           
                         
                         
                           else 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   43 
                   ) 
                 
               
             
           
         
       
     
     Another counter is suppressor activated counter, sup_cnt(n), which is similar to the startup indicator counter m_cnt(n). Suppressor activated counter is to indicate a number of samples during which the NLP is activated before convergence of the ADF. This counter is incremented by one for every NLP ON decision before convergence is achieved for a speech frame. The suppressor activated counter also does not have factor into NLP decision logic after ADF convergence. Balance convergence counter, con_cnt(n), is to indicate the number of samples ADFs are converged within the expected convergence. 
     The last counter used is called hist counter, his_cnt(n) is to check the stability of the convergence. Another decision parameters, absolute short term average error signal e abs (n), absolute short term average microphone signal d abs (n) and long term average of reference signal amplitude ly(n) are estimated as per below equations. 
     
       
         
           
             
               
                 
                   
                     tmp 
                     = 
                     
                       
                          
                         
                           
                             s 
                             ′ 
                           
                           ⁡ 
                           
                             ( 
                             n 
                             ) 
                           
                         
                          
                       
                       - 
                       
                         
                           e 
                           abs 
                         
                         ⁡ 
                         
                           ( 
                           
                             n 
                             - 
                             1 
                           
                           ) 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         e 
                         abs 
                       
                       ⁡ 
                       
                         ( 
                         n 
                         ) 
                       
                     
                     = 
                     
                       { 
                       
                         
                           
                             
                               
                                 
                                   e 
                                   abs 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   n 
                                   ) 
                                 
                               
                               + 
                               
                                 tmp 
                                 * 
                                 
                                   β 
                                   4 
                                 
                               
                             
                           
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 ( 
                                 
                                   
                                     ( 
                                     
                                       
                                          
                                         
                                           d 
                                           ⁡ 
                                           
                                             ( 
                                             
                                               n 
                                               - 
                                               
                                                 D 
                                                 1 
                                               
                                             
                                             ) 
                                           
                                         
                                          
                                       
                                       &lt; 
                                       
                                         β 
                                         5 
                                       
                                     
                                     ) 
                                   
                                   &amp;&amp; 
                                   
                                     ( 
                                     
                                       
                                         
                                           d 
                                           abs 
                                         
                                         ⁡ 
                                         
                                           ( 
                                           n 
                                           ) 
                                         
                                       
                                       &lt; 
                                       
                                          
                                         
                                           d 
                                           ⁡ 
                                           
                                             ( 
                                             
                                               n 
                                               - 
                                               
                                                 D 
                                                 1 
                                               
                                             
                                             ) 
                                           
                                         
                                          
                                       
                                     
                                     ) 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                         
                           
                             
                               
                                 
                                   
                                     e 
                                     abs 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     n 
                                     ) 
                                   
                                 
                                 + 
                                 
                                   tmp 
                                   * 
                                   
                                     β 
                                     6 
                                   
                                 
                               
                               , 
                             
                           
                           
                             otherwise 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   44 
                   ) 
                 
               
             
             
               
                 
                   
                     tmp 
                     = 
                     
                       
                          
                         
                           d 
                           ⁡ 
                           
                             ( 
                             
                               n 
                               - 
                               
                                 D 
                                 1 
                               
                             
                             ) 
                           
                         
                          
                       
                       - 
                       
                         
                           d 
                           abs 
                         
                         ⁡ 
                         
                           ( 
                           
                             n 
                             - 
                             1 
                           
                           ) 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         d 
                         abs 
                       
                       ⁡ 
                       
                         ( 
                         n 
                         ) 
                       
                     
                     = 
                     
                       { 
                       
                         
                           
                             
                               
                                 
                                   d 
                                   abs 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   n 
                                   ) 
                                 
                               
                               + 
                               
                                 tmp 
                                 * 
                                 
                                   β 
                                   4 
                                 
                               
                             
                           
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 ( 
                                 
                                   
                                     ( 
                                     
                                       
                                          
                                         
                                           d 
                                           ⁡ 
                                           
                                             ( 
                                             
                                               n 
                                               - 
                                               
                                                 D 
                                                 1 
                                               
                                             
                                             ) 
                                           
                                         
                                          
                                       
                                       &lt; 
                                       
                                         β 
                                         5 
                                       
                                     
                                     ) 
                                   
                                   &amp;&amp; 
                                   
                                     ( 
                                     
                                       
                                         
                                           d 
                                           abs 
                                         
                                         ⁡ 
                                         
                                           ( 
                                           n 
                                           ) 
                                         
                                       
                                       &lt; 
                                       
                                          
                                         
                                           d 
                                           ⁡ 
                                           
                                             ( 
                                             
                                               n 
                                               - 
                                               
                                                 D 
                                                 1 
                                               
                                             
                                             ) 
                                           
                                         
                                          
                                       
                                     
                                     ) 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                         
                           
                             
                               
                                 
                                   
                                     d 
                                     abs 
                                   
                                   ⁡ 
                                   
                                     ( 
                                     n 
                                     ) 
                                   
                                 
                                 + 
                                 
                                   tmp 
                                   * 
                                   
                                     β 
                                     6 
                                   
                                 
                               
                               , 
                             
                           
                           
                             otherwise 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   45 
                   ) 
                 
               
             
             
               
                 
                   
                     ly 
                     ⁡ 
                     
                       ( 
                       n 
                       ) 
                     
                   
                   = 
                   
                     
                       ( 
                       
                         
                           ly 
                           ⁡ 
                           
                             ( 
                             
                               n 
                               - 
                               1 
                             
                             ) 
                           
                         
                         * 
                         
                           ( 
                           
                             1 
                             - 
                             
                               β 
                               7 
                             
                           
                           ) 
                         
                       
                       ) 
                     
                     + 
                     
                       ( 
                       
                         
                            
                           
                             
                               x 
                               2 
                             
                             ⁡ 
                             
                               ( 
                               n 
                               ) 
                             
                           
                            
                         
                         * 
                         
                           β 
                           7 
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   46 
                   ) 
                 
               
             
           
         
       
     
     D 1  is a delay compensator factor for synchronizing microphone signal d(n) and error signal received from residual echo remover ś(n). 
     Another decision parameter is a convergence indicator and can be estimated (detection  307 ) as per pseudocode (47). When the ADF reaches convergence during single talk, the correlation between enhanced error signal and modified microphone signal decreases. Decreased correlation thus can be used as a detector for ADF convergence. For the detection of convergence, cross correlation ed enr (n) is normalized by microphone energy d enr (n) and compared with the predefined threshold. Since RENC  119  cancels background noise also, this normalized cross correlation check may pass during no speech region. So, convergence validation is checked during presence of speech activity using the v_cnt(l). 
                                                                                                                                                                                                                                                         if ((conv(n − 1) == 0) &amp; &amp; (v _ cnt(l) == 0))           {             if (d enr  (n) * β 9  &gt; ed enr  (n))             {                if ((his _ cnt(n − 1) &gt; β 10 ) &amp; &amp;                (adp _ cnt(n) &gt; β 37 ))                {                conv(n) = 1           sup _ cnt(n) = β 11             m _ cnt(n) = β 3                  }           else           {                his _ cnt(n) = his_ cnt(n − 1) + 1                }                 }                 else   (47)                 {                 if (his _ cnt(n − 1) &gt; β 38 )           {                 con _ cnt(n) =                con_ cnt(n − 1) + his _ cnt(n − 1)                  if (con _ cnt(n) &gt; β 10 ) &amp; &amp;                (adp _ cnt(n) &gt; β 37 )                  {                conv(n) = 1           sup _ cnt(n) = β 11             m _ cnt(n) = β 3                  }                }           his _ cnt(n) = 0                  }           }                        
Decision Logic— 309  &amp;  311 
 
       FIG. 7  depicts an example of NLP decision logic performed to update NLP decisions, in elements  309 / 311  of  FIG. 6 . The example of  FIG. 7  is exemplary and not limiting. A person of ordinary skill can adapt these disclosures to other implementations. The decision logic has two main stages; (1) Decision before convergence and (2) Decision after convergence. A Startup Decision Maker  354  is NLP decision maker before expected convergence is achieved. There are five sub-stages in the decision making after expected convergence is achieved. They are detailed in the subsequent sub sections. 
     Startup Decision Maker  354   
     Startup Decision Maker  354  uses a relaxed threshold and there is possibility that NLP might be activated sometimes during double talk. The startup decision maker is active for a short time during startup, and thus does not have a major effect on a conversation. Also, occurrence of double talk during start of a call is uncommon. 
                                                                                                                                 if ((m _ cnt(n) &lt; β 3 ) &amp; &amp;(sup _ cnt(n) &lt; β 11 )                 &amp; &amp;(d enr  (n) * β 12  &gt; ed enr  (n)))                {                nlp(n) = 1                if (v _ cnt(l) == 0)   (48)                {                sup _ cnt(n) = sup _ cnt(n) + 1                }                }                        
Coarse Decision Maker  356 
 
     A Coarse Decision Maker  356  uses normalized cross correlation ed enr (n)/d enr (n) for decision making. If the validation check is passed, the DT hangover is broken and ST hangover is set to β 14 . 
                                                                                         if (d enr  (n) * β 13  &gt; ed enr  (n))           {                nlp(n) = 1                st _ hngovr(n) = β 14     (49)                dt _ hngovr(n) = −1           distortion(n) = 1                }                        
Distorted Error Masker
 
     A Distorted Error Masker  358  is an energy comparator for low level signal. When the error signal is at a low level and also is much lower than the microphone signal level, this decision directs NLP activation. Activating the NLP under such conditions reduces situations where distorted low level noise can be heard by the user. 
                                                                                               if ((d enr  (n) &gt; e enr  (n) * β 15 ) &amp; &amp;(e enr  (n) &lt; β 16 ))                ∥ (d enr  (n) &gt; e enr  (n) * β 17 ) &amp; &amp;(e enr  (n) &lt; β 18 ))                {                (50)                nlp(n) = 1           dt _ hngovr(n) = −1                }                        
Coarse Decision Maker  360 
 
     A Coarse Decision Maker  360  uses a normalized cross correlation ed enr (n)/e enr (n) as a basis for outputting decisions for NLP activation. If the validation check is passed, the DT hangover is broken and ST hangover is set to β 20  if it is lower than that. 
                                                                                               if (e enr  (n) &gt; (ed enr  (n) * β 19 ))           {                nlp(n) = 1           if (st _ hngovr(n) &lt; β 20 )                (51)                st _ hngovr(n) = β 20                  dt _ hngovr(n) = −1           distortion(n) = 1                }                        
Double Talk Hangover Check
 
     If the NLP decision is OFF with the above validations, a DT Hangover Check  362  is performed. DT hangover is checked for transmitting the nearend signal passed out of AES until a current point. The hangover counter is decremented by one for every sample processing. 
                                                                           if (dt _ hngovr(n) &gt; 0)           {                (52)                dt _ hngovr(n) = dt _ hngovr(n) − 1                }                        
Coarse Decision Maker  365 
 
     If all decision making logics failed, then the coarse decision maker  365  becomes active (this example shows a serial flow, where any positive decision causes a NLP=1 decision, and the remainder of the flow need not be performed. A Coarse decision maker  365  applies a different threshold on the normalized cross correlation ed enr (n)/d enr (n) based on the convergence state of the adaptive filter as given below. 
     
       
         
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 if (d enr  (n) * β 21  &gt; ed enr  (n) ∥ ((d enr  (n) * 
               
             
          
           
               
                   
                 β 22  &gt; ed enr  (n)) &amp; &amp;(conv(n) == 0))) 
               
             
          
           
               
                   
                 { 
               
             
          
           
               
                   
                  nlp(n) = 1 
               
             
          
           
               
                   
                 (53) 
               
             
          
           
               
                   
                  dt _ hngovr(n) = 0 
               
               
                   
                  if (d enr  (n) * β 23  &gt; ed enr  (n)) 
               
             
          
           
               
                   
                 st _ hngovr(n) = β 24   
               
             
          
           
               
                   
                 } 
               
               
                   
                   
               
             
          
         
       
     
     The flow of  FIG. 7  completes by returning a decision for nlp(n)=0 or nlp(n)=1 to complete the flow of  FIG. 6 . 
     NLP Energy Threshold Updating  315   
     If the NLP Decision Logic enables NLP, then NLP energy threshold is updated  315  as given below. This threshold will be used for breaking ST hangover later. 
                                                                               tmp = e enr  (n) − nlp enr  (n)           if (tmp &gt; 0)                nlp enr  (n) = nlp enr  (n) + tmp * β 25     (54)                else                nlp enr  (n) = nlp enr  (n) + tmp * β 26                          
Double Talk Hangover Breaker  317 
 
     Sometimes there is change of residual echo passed to user due to hangover. So, there should be decision or other mechanism to break DT hangover based on a sudden fall in nearend energy or sudden rise in echo energy. The DT hangover is broken in this scenario based on the below condition: 
                                                                                               if ((e enr  (n) * β 27  &gt; d enr  (n))                ∥ (d enr  (n) &gt; e enr  (n) * β 28 ))                {                (55)                dt _ hngovr(n) = −1           nlp(n) = 1                }                        
Double Talk Hangover Setting  322 
 
     If the DT hangover breaking conditions failed and energy of the error signal is more than a predefined threshold, ST hangover is to be broken and DT hangover is to be set to another pre-defined value, as in the example below. 
                                                                               if (e enr  (n) &gt; β 29 )           {                dt _ hngovr(n) = β 20      (56)                st _ hngovr(n) = −1                }                        
Single Talk Hangover Breaker  320 
 
     The NLP threshold estimated is used for breaking the ST hangover. The ST hangover breaking validation condition is given below. 
     
       
         
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 if ((e enr  (n) &gt; nlp enr  (n) * β 30 ) ∥ 
               
             
          
           
               
                   
                 (e enr  (n) &gt; (nlp enr  (n) + β 31 )) 
               
               
                   
                 &amp; &amp;(e enr  (n) &gt; β 32 ) &amp; &amp;(distortion(n) == 0)) 
               
             
          
           
               
                   
                 (57) 
               
             
          
           
               
                   
                 { 
               
             
          
           
               
                   
                 st _ hngovr(n) = −1 
               
             
          
           
               
                   
                 } 
               
               
                   
                   
               
             
          
         
       
     
     If the hangover breaking validation is failed and ST hangover count is greater than 0 ( 325 ), NLP is activated ( 329 ) and ST hang over count is decremented by 1 ( 329 ). 
     Refine NLP Decision and ST Hangover  331   
     Refining the NLP decision and ST hangover are done based on the long term average amplitude of the reference signal  ly(n) , absolute average of error and modified microphone output signal as given below. 
     
       
         
               
               
             
               
               
               
             
               
               
             
               
               
             
               
               
             
               
               
               
             
               
               
             
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 if (ly(n) &lt; β 33 ) 
               
               
                   
                 { 
               
             
          
           
               
                   
                 nlp(n) = 0 
                 (58) 
               
             
          
           
               
                   
                 st _ hngovr(n) = −1 
               
             
          
           
               
                   
                 } 
               
               
                   
                 if (e abs  (n) &gt; d abs  (n) + β 34 ) ∥ 
               
             
          
           
               
                   
                 (e abs  (n) &gt; d abs  (n) * β 35 ) 
               
               
                   
                 &amp; &amp;(d abs  (n) &gt; 0) 
               
             
          
           
               
                   
                 { 
                 (59) 
               
             
          
           
               
                   
                 nlp(n) = 1 
               
               
                   
                 st _ hngovr(n) = β 36   
               
             
          
           
               
                   
                 } 
               
               
                   
                   
               
             
          
         
       
     
     
       
         
               
             
               
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 3 
               
             
             
               
                   
               
               
                 Constants used by NLP controller 109 
               
             
          
           
               
                   
                 Constant 
                 Value 
                 Remarks 
               
               
                   
               
             
          
           
               
                   
                 β 1   
                 0.0189 
                 Noise Energy 
               
               
                   
                 β 2   
                 0.1831 
                 Smoothing factor 
               
               
                   
                 β 3   
                 64000 
                 Max. value of 
               
               
                   
                   
                   
                 startup indication 
               
               
                   
                   
                   
                 counter 
               
               
                   
                 β 4   
                 0.5 
                 Smoothing factor 
               
               
                   
                 β 5   
                 50 
                 Constant 
               
               
                   
                 β 6   
                 0.03 
                 Smoothing factor 
               
               
                   
                 β 7   
                 128 
                 Constant 
               
               
                   
                 β 8   
                 480 
                 Max. value of adap_cnt 
               
               
                   
                 β 9   
                 0.0061 
                 Multiplication Factor 
               
               
                   
                 β 10   
                 1400 
                 his_cnt limit 
               
               
                   
                 β 11   
                 32000 
                 Constant 
               
               
                   
                 β 12   
                 0.4577 
                 Multiplication Factor 
               
               
                   
                 β 13   
                 0.0061 
                 Multiplication Factor 
               
               
                   
                 β 14   
                 4000 
                 st_hngovr limit 
               
               
                   
                 β 15   
                 3 
                 Constant 
               
               
                   
                 β 16   
                 7500 
                 e enr  limit 
               
               
                   
                 β 17   
                 2 
                 Constant 
               
               
                   
                 β 18   
                 2500 
                 e enr  limit 
               
               
                   
                 β 19   
                 2 
                 Constant 
               
               
                   
                 β 20   
                 540 
                 st_hngovr limit 
               
               
                   
                 β 21   
                 0.061 
                 Multiplication Factor 
               
               
                   
                 β 22   
                 0.3662 
                 Multiplication Factor 
               
               
                   
                 β 23   
                 0.4577 
                 Multiplication Factor 
               
               
                   
                 β 24   
                 240 
                 st_hngovr limit 
               
               
                   
                 β 25   
                 0.097 
                 NLP Energy 
               
               
                   
                 β 26   
                 0.0061 
                 Smoothing factor 
               
               
                   
                 β 27   
                 21845 
                 dt_hngovr limit 
               
               
                   
                 β 28   
                 0.0313 
                 Multiplication Factor 
               
               
                   
                 β 29   
                 35000 
                 e enr  limit 
               
               
                   
                 β 30   
                 4 
                 Constant 
               
               
                   
                 β 31   
                 0.2136 
                 Multiplication Factor 
               
               
                   
                 β 32   
                 12000 
                 e enr  limit 
               
               
                   
                 β 33   
                 6400 
                 ly limit 
               
               
                   
                 β 34   
                 900 
                 Constant 
               
               
                   
                 β 35   
                 8 
                 Constant 
               
               
                   
                 β 36   
                 1200 
                 st_hngovr limit 
               
               
                   
                 β 37   
                 300 
                 Constant 
               
               
                   
                 β 38   
                 20 
                 st_hngovr constant 
               
               
                   
                 K 
                 300 
                 Index 
               
               
                   
               
             
          
         
       
     
     Embodiments can be implemented in Fixed Point C on a RISC application processor, such as an Advanced RISC Machines (ARM) processor, such as an ARM 9E. In some implementations, other applications can execute on the same application processor and in some examples, processes can have preemptive scheduling provided by an OS kernel, for time-critical tasks. Good performance is shown on real platforms that have general purpose application processors, such as laptops, tablets, and desktops, such as Microsoft Windows desktop, laptop and mobile, as well as Android-based handsets. To demonstrate the proposed system&#39;s performance here, the ensemble average results are provided in this section. 
     Real-time captured farend and microphone output signals on different platforms are fed to the AEC module and respective block&#39;s output signals are captured and analyzed.  FIG. 8  depicts the ensemble average of ERLE for single talk test case. During single talk test case, microphone output signal has echo and background noise only. 
     In  FIG. 8 , it can be seen that ADFs ( 402 ) were able to provide ERLE of 8 dB only. With the Residual Echo and Noise Canceller (REnNC)  119 , ERLE can be increased up to 60 dB using modified Wiener gain estimation ( 404 ) and 40 dB using modified MMSE LSA gain estimation ( 406 ). The proposed method based on MMSE-LSA provides much less residual noise when compared to Weiner, while there is no perceptible difference in the enhanced quality of speech between these two methods. Further, residual noise sounds more uniform (more white), which is subjectively preferable. 
       FIG. 9  depicts the ensemble average of ERLE for Double Talk (DT) test case. In the  FIG. 9 , two DT regions are present. In all test cases, there is no clipping of nearend speech and complete cancellation of background noise is observed. 
       FIGS. 10-13  depict aspects of the performance of an implementation of the proposed FAGC. From  FIG. 10 , it can be noted that the target level tracking of the proposed FAGC is fast and accurate. 
     NLP controller  109  performance for real-time captured signal is depicted in  FIG. 14 . The captured signal has the combination of single talk, double talk and nearend signal. NLP is active during single talk and echo alone regions during double talk and it is deactivated in all the nearend regions.  FIG. 15  depicts the AES output for NLP decisions. AES output does not contain any residual echo. 
     Generally, any of the functions, methods, techniques or components described above can be implemented in modules using software, firmware, hardware (e.g., fixed logic circuitry), or any combination of these implementations. The terms “module,” “functionality,” “component”, “block” and “logic” are used herein to generally represent software, firmware, hardware, or any combination thereof. 
     In the case of a software implementation, the module, functionality, component or logic represents program code that performs specified tasks when executed on a processor (e.g. one or more CPUs). In one example, the methods described may be performed by a computer configured with software of a computer program product in machine readable form stored on a computer-readable medium. One such configuration of a computer-readable medium is signal bearing medium and thus is configured to transmit the instructions (e.g. as a carrier wave) to the computing device, such as via a network. The computer-readable medium may also be configured as a non-transitory computer-readable storage medium, which is not a propagating signal bearing medium (e.g., an EM signal propagating in free space or over a wire). Examples of a computer-readable storage medium include a random-access memory (RAM), read-only memory (ROM), an optical disc, flash memory, hard disk memory, and other memory devices that may use magnetic, optical, and other techniques to store instructions or other data and that can be accessed by a machine, but. 
     The software may be in the form of a computer program comprising computer program code for configuring a computer to perform the constituent portions of described methods or in the form of a computer program comprising computer program code means adapted to perform all the steps of any of the methods described herein when the program is run on a computer and where the computer program may be embodied on a computer readable medium. The program code can be stored in one or more computer readable media. The features of the techniques described herein are platform-independent, meaning that the techniques may be implemented on a variety of computing platforms having a variety of processors. 
     Those skilled in the art will also realize that all, or a portion of the functionality, techniques or methods may be carried out by a dedicated circuit, an application-specific integrated circuit, a programmable logic array, a field-programmable gate array, or the like. For example, the module, functionality, component or logic may comprise hardware in the form of circuitry. Such circuitry may include transistors and/or other hardware elements available in a manufacturing process. Such transistors and/or other elements may be used to form circuitry or structures that implement and/or contain memory, such as registers, flip flops, or latches, logical operators, such as Boolean operations, mathematical operators, such as adders, multipliers, or shifters, and interconnects, by way of example. Such elements may be provided as custom circuits or standard cell libraries, macros, or at other levels of abstraction. Such elements may be interconnected in a specific arrangement. The module, functionality, component or logic may include circuitry that is fixed function and circuitry that can be programmed to perform a function or functions; such programming may be provided from a firmware or software update or control mechanism. In an example, hardware logic has circuitry that implements a fixed function operation, state machine or process. 
     Aspects of the present disclosure encompass software (as represented by data recorded on a non-transitory medium) which “describes” or defines the configuration of hardware that implements a module, functionality, component or logic described above, such as HDL (hardware description language) software, as is used for designing integrated circuits, or for configuring programmable chips, to carry out desired functions. That is, there may be provided a computer readable storage medium having encoded thereon computer readable program code for generating a processing block configured to perform any of the methods described herein, or for generating a processing block comprising any apparatus described herein. 
     The term ‘processor’ and ‘computer’ are used herein to refer to any device, or portion thereof, with processing capability such that it can execute instructions, or a dedicated circuit capable of carrying out all or a portion of the functionality or methods, or any combination thereof. 
     Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. It will be understood that the benefits and advantages described above may relate to one example or may relate to several examples. 
     The actions of the methods described herein may be carried out in any suitable order, or simultaneously where appropriate and unless indicated otherwise by context or explicitly. Aspects of any of the examples described above may be combined with aspects of any of the other examples described to form further examples without losing the effect sought.