# Residual Echo Suppression Considering Harmonic Distortion and Temporal Correlation
## 4 Residual Echo Suppression Considering Harmonic Distortion and Temporal Correlation  
In this paper, we propose an extension of HDRES [28] which considers not only the harmonic distortion but also the temporal correlation of the relevant signals. The residual echo in the current time-frequency bin is modeled as a linear function of not only the linear echo estimates in the harmonically-related frequency bins in the current frame but also the linear echo estimate, residual echo estimate, and microphone signals in adjacent frames. Speech signal has strong temporal correlation that is beneficial to exploit in the vast areas of speech signal processing. In addition to the temporal correlation of the far-end speech signal or the filtered version of it that can help the estimation of the residual echo, the nonlinear part of the echo signal may have its own temporal correlation. Moreover, the microphone signals bears raw information in the captured signal that might be complementary to that in the estimated signals, which may also help the estimation of the residual echo in the current frame in the far-end single-talk regions. To utilize all the relevant signals considering spectral and temporal correlation, two estimates for the spectral magnitude of the residual echo for far-end single-talk and double-talk periods are maintained. The residual echo for the far-end single-talk period in the frame \(m\) and frequency \(f\) is modeled as:  
\[|\hat{D}_{ST}(m,f)|= \sum_{i=1}^{M}\sum_{j=1}^{H}\sum_{k=-K}^{K}\delta(i,j,k,f)W_{Hs}(i,j,k)|X^{\prime}(m,i)|+\sum_{p=1}^{T}W_{Ts1}(p,f)|X^{\prime}(m-p,f)|\] \[+\sum_{p=1}^{T}W_{Ts2}(p,f)|\hat{D}_{r}(m-p,f)|+\sum_{p=0}^{T}W_{Ts 3}(p,f)|Y(m-p,f)|, \tag{9}\]  
where \(T\) is the number of considered previous frames, \(W_{Hs}(i,j,k)\) is the weight for the linear echo estimates in the harmonically related frequencies and \(W_{Ts1}(p,f)\), \(W_{Ts2}(p,f)\), and \(W_{Ts3}(p,f)\) are the weights for the linear echo estimates, residual echo estimates, and microphone signals in the previous frames, respectively. It is noted that the summation index for \(Y\) starts with \(0\) to allow the effect of the microphone signal in the current frame. On the other hand, the residual echo in the double-talk period is estimated without the microphone signal to avoid the adverse effect of the near-end signals:  
\[|\hat{D}_{DT}(m,f)|= \sum_{i=1}^{M}\sum_{j=1}^{H}\sum_{k=-K}^{K}\delta(i,j,k,f)W_{Hd} (i,j,k)|X^{\prime}(m,i)|\] \[+\sum_{p=1}^{T}W_{Td1}(p,f)|X^{\prime}(m-p,f)|+\sum_{p=1}^{T}W_{ Td2}(p,f)|\hat{D}_{r}(m-p,f)|, \tag{10}\]  
in which \(W_{Hd}(i,j,k)\) is the weight for the linear echo estimates in the harmonically-related frequencies and \(W_{Td1}(p,f)\) and \(W_{Td2}(p,f)\) are the weights for the linear echo estimates, and residual echo estimates in the previous frames, respectively. With these two estimates, the estimate for the magnitude of the residual echo is determined depending on the result of the double-talk detection:  
\[|\hat{D}_{r}(m,f)|= \begin{cases}|\hat{D}_{DT}(m,f)|,&\text{ if double-talk is detected}\\ |\hat{D}_{ST}(m,f)|,&\text{ }otherwise\end{cases} . \tag{11}\]The weights in Equations (9) and (10) are updated during far-end single-talk periods using the NLMS algorithm that minimizes the mean square error between the AEC output signal and the residual echo estimate.