# An Optimized Differential Step-Size LMS Algorithm
## 2 System Model  
We begin with the traditional steepest descent update equation [11]:  
\[\mathbf{\hat{w}}(n)=\mathbf{\hat{w}}(n-1)+\mu\mathrm{E}[\mathbf{x}(n)\alpha(n)]. \tag{1}\]  
In Equation (1), we denote \(\mathbf{\hat{w}}(n)=\left[\begin{array}{cccc}\mathbf{\hat{w}}_{0}(n)& \mathbf{\hat{w}}_{1}(n)&\cdots&\mathbf{\hat{w}}_{L-1}(n)\end{array}\right]^{T}\) as the adaptive filter (of length \(L\)), where the superscript \({}^{T}\) is the transpose operator, \(\mu\) is the fixed step-size, \(\mathbf{x}(n)=\left[\begin{array}{cccc}x(n)&x(n-1)&\cdots&x(n-L+1)\end{array} \right]^{T}\) is a vector that contains the \(L\) most recent real-valued samples of the (zero-mean) input signal at the discrete time index \(n\), \(\alpha(n)\) is the a priori error signal, and \(\mathrm{E}(\cdot)\) is the statistical expectation. The goal of the adaptive filter is to estimate the impulse response of the unknown system denoted as \(\mathbf{w}(n)=[\begin{array}{cccc}w_{0}(n)&w_{1}(n)&\cdots&w_{L-1}(n)\end{array}]^{T}\). Using the previous notation, the a priori error is expressed as \(\alpha(n)=d(n)-\mathbf{x}^{T}(n)\mathbf{\hat{w}}(n-1)\), where the desired signal, \(d(n)=\mathbf{x}^{T}(n)\mathbf{w}(n)+v(n)\), consists of the output of the unknown system corrupted by the measurement noise, \(v(n)\), which is assumed to be a zero-mean Gaussian noise signal.  
Then, by applying the stochastic gradient principle and substituting the fixed step-size with a diagonal matrix, \(\mathbf{Y}(n)\), the coefficients of the estimated impulse response can be updated as follows:  
\[\mathbf{\hat{w}}(n)=\mathbf{\hat{w}}(n-1)+\mathbf{Y}(n)\mathbf{x}(n)\alpha(n). \tag{2}\]  
Considering a stochastic system model described by \(\mathbf{w}(n)=\mathbf{w}(n-1)+\delta_{\mathbf{w}}(n)\), where \(\delta_{\mathbf{w}}(n)\) is a zero-mean Gaussian random vector, the goal consists of finding the diagonal matrix, \(\mathbf{Y}(n)\), that minimizes at each iteration the MSD, \(m(n)=\mathrm{E}[\mathbf{c}^{T}(n)\mathbf{c}(n)]\), where \(\mathbf{c}(n)=\mathbf{\hat{w}}(n)-\mathbf{w}(n)\) denotes the a posteriori misalignment. Thereby, the a posteriori error of the adaptive filter is  
\[e(n)=d(n)-\mathbf{x}^{T}(n)\mathbf{\hat{w}}(n). \tag{3}\]  
Substituting the desired signal and the a posteriori misalignment in Equation (3), the a posteriori error becomes \(e(n)=v(n)-\mathbf{x}^{T}(n)\mathbf{c}(n)\). Proceeding in the same manner with the a priori error, we obtain \(\alpha(n)=\mathbf{x}^{T}(n)\left[\delta_{\mathbf{w}}(n)-\mathbf{c}(n-1) \right]+v(n)\). Therefore, we can introduce (2) in the a posteriori misalignment to obtain the recursive relation:  
\[\mathbf{c}(n)=\mathbf{c}(n-1)+\mathbf{w}(n-1)-\mathbf{w}(n)+\mathbf{Y}(n) \mathbf{x}(n)\alpha(n). \tag{4}\]  
Furthermore, we can process Equation (4) as follows:  
\[\mathbf{c}(n) =\mathbf{c}(n-1)-\delta_{\mathbf{w}}(n)+\mathbf{Y}(n)\mathbf{x} (n)\alpha(n)\] \[=\mathbf{c}(n-1)-\delta_{\mathbf{w}}(n)+\mathbf{Y}(n)\mathbf{x}(n )\{\mathbf{x}^{T}(n)\left[\delta_{\mathbf{w}}(n)-\mathbf{c}(n-1)\right]+v(n)\}\] \[=[\mathbf{I}_{L}-\mathbf{Y}(n)\mathbf{x}(n)\mathbf{x}^{T}(n)][ \mathbf{c}(n-1)-\delta_{\mathbf{w}}(n)]+\mathbf{Y}(n)\mathbf{x}(n)v(n), \tag{5}\]  
where \(\mathbf{I}_{L}\) is the identity matrix of size \(L\times L\).