# On Fast Converging Data-Selective Adaptive Filtering
## 3 Data-Selective Adaptive Filtering Algorithms
### LMSN and LMSQN  
Considering the same estimate of the gradient, \(\mathbf{\hat{g}_{w}}(k)=-e(k)\mathbf{x}(k)\), used in the LMS algorithm and also a variable step-size, we end-up with the following recursive coefficient updating formula [2]  
\[\mathbf{w}(k+1)=\mathbf{w}(k)+\frac{\
u}{\mathbf{x}^{T}(k)\mathbf{\hat{R}}^{- 1}(k)\mathbf{x}(k)}\mathbf{\hat{R}}^{-1}(k)\mathbf{x}(k)e(k), \tag{33}\]  
where \(\mu(k)=\
u/(\mathbf{x}^{T}(k)\mathbf{\hat{R}}^{-1}(k)\mathbf{x}(k))\) is a step-size parameter in which \(\
u\) is a positive constant and \(e(k)=d(k)-\mathbf{w}^{T}(k)\mathbf{x}(k)\) is the a priori error. The only difference between LMSN and the LMSQN algorithms is the way of matrix \(\mathbf{\hat{R}}^{-1}(k)\) is estimated. In the LMSN method, matrix \(\mathbf{\hat{R}}(k)\) is estimated via a Robbins-Monro procedure resulting in the following update of its inverse, given by [2]  
\[\mathbf{\hat{R}}^{-1}(k)= \frac{1}{1-\theta}\Bigg{\{}\mathbf{\hat{R}}^{-1}(k-1)-\frac{ \mathbf{\hat{R}}^{-1}(k-1)\mathbf{x}(k)\mathbf{x}^{T}(k)\mathbf{\hat{R}}^{-1 }(k-1)}{\frac{1-\theta}{\theta}+\mathbf{x}^{T}(k)\mathbf{\hat{R}}^{-1}(k) \mathbf{x}(k)}\Bigg{\}}, \tag{34}\]  
where \(\theta\) is a weight factor. The LMSQN algorithm updates matrix \(\mathbf{\hat{R}}^{-1}(k)\) by using the approach in [3] which ensures that \(\mathbf{\hat{R}}^{-1}(k)\) remains positive definite and bounded for a bounded input signal. As a result, the estimate \(\mathbf{\hat{R}}^{-1}(k)\) is obtained as  
\[\mathbf{\hat{R}}^{-1}(k)= \mathbf{\hat{R}}^{-1}(k-1)+\left(\frac{\
u}{2\mathbf{x}^{T}(k) \mathbf{\hat{R}}^{-1}(k-1)\mathbf{x}(k)}-1\right)\
u\frac{\mathbf{\hat{R}}^{- 1}(k-1)\mathbf{x}(k)\mathbf{x}^{T}(k)\mathbf{\hat{R}}^{-1}(k-1)}{\mathbf{x}^{ T}(k)\mathbf{\hat{R}}^{-1}(k-1)\mathbf{x}(k)}. \tag{35}\]  
These algorithms have been analyzed more deeply through the use of theory and simulations in [2, 3]. Although the estimate obtained by LMSN is accurate enough, it is not free of possible instability behavior. On the other hand, the LMSQN guarantees stability but can lead to poor estimations of \(\mathbf{R}^{-1}\). Since the LMSN and LMSQN algorithms update the filter coefficients in the same manner, both utilizing estimates of \(\mathbf{R}^{-1}\), the excess MSE is also the same [3]. Hence, as proposed in [14], the expression of the excess MSE at steady-state for both DS-LMSN and DS-LMSQN algorithms can be written as the following approximation  
\[\zeta_{exc}(k)\approx\frac{\
u P_{\text{up}}}{2-\
u P_{\text{up}}}\sigma_{n}^ {2},k\ \rightarrow\infty. \tag{36}\]  
As a result, we use a similar procedure to the one described in [18], in which the coefficient update of the DS-LMSN and DS-LMSQN algorithms are equivalent concerning their expected values. Hence, using this new update and by following the theoretical analysis demonstrated in [2], we obtain an approximation of the excess MSE at the steady-state in both algorithms. Thus, the value \(\rho\) can be obtained from the expression (36). The steps of both DS-LMSN and DS-LMSQN algorithms are summarized in Algorithm 1, where the quantities \(\mathbf{t}(k)\) and \(\psi(k)\) are included to simplify some steps in the computations. ```
DS-LMSN and DS-LMSQN algorithms  Initialize \(0<\
u\leq 1\), \(0<\theta\leq 1\) (for LMSN), \(\gamma\) small positive constant, \(\mathbf{w}(0)=\text{random vectors or zero vectors and }\mathbf{\hat{R}}^{-1}(0)=\gamma\mathbf{I}_{L+1}\) Prescribe \(P_{\text{up}}\) and choose \(\tau_{\text{max}}\) \(\sqrt{\tau}=\sqrt{(1+\rho)}Q^{-1}(\frac{P_{\text{up}}}{2})\) For prediction and equalizer use \(\rho=0\) For system identification use \(\rho=\frac{\
u P_{\text{up}}}{2-\
u P_{\text{up}}}\). Do for \(k>0\) acquire \(\mathbf{x}(k)\) and \(d(k)\) \(e(k)=d(k)-\mathbf{w}^{T}(k)\mathbf{x}(k)\) \(\delta(k)=\left\{ \begin{array}{ll}0,&\text{if }-\sqrt{\tau}\leq\frac{e(k)}{\sigma_{ \varepsilon}}\leq\sqrt{\tau}\\ 0,&\text{if }\frac{|e(k)|}{\sigma_{\varepsilon}}\geq\sqrt{\tau_{\text{max}}}\\ 1,&\text{otherwise}\end{array}