# A Variable Step Size Normalized Least-Mean-Square Algorithm Based on Data Reuse
## 3 Variable Step Size NLMS Algorithm  
Based on the previous considerations, we can design a simple VSS-NLMS algorithm, by implementing the previous approach in a \"reverse\" manner, using scheduled normalized step sizes. Thus, given the lower and the upper bounds for the normalized step size parameter, i.e., \(\alpha\) and \(\alpha_{\max}\), respectively, the update equation that defines the proposed VSS-NLMS algorithm is:  
\[\widehat{\mathbf{h}}(n)=\widehat{\mathbf{h}}(n-1)+\frac{\mu(n)\mathbf{x}(n)e( n)}{\mathbf{x}^{T}(n)\mathbf{x}(n)}, \tag{13}\]  
where the variable step size parameter \(\mu(n)\) is evaluated as follows (using (10) and (12)):  
* Step 1: \(\mu(n)=\alpha_{k_{\max}-q+1}\), for \(q=1,2,\ldots,k_{\max}\) and \(n=(q-1)\lceil IL\rceil+1,\ldots,q\lceil IL\rceil\) (with \(0<l\leq 1\));
* Step 2: \(\mu(n)=\alpha\), for \(n>k_{\max}\lceil IL\rceil\);
* Step 3: if \(\sigma_{e}^{2}(n)>\tilde{\varphi}\sigma_{v}^{2}\) (with \(1<\xi<2\)) for \(n>k_{\max}\lceil IL\rceil\), then go to Step 1;  
where \(\sigma_{e}^{2}(n)\) denotes the variance of the a priori error signal, \(\sigma_{v}^{2}\) is the power of the system noise (i.e., the external noise that usually corrupts the reference signal), and \(k_{\max}\) results based on (12), according to the values of \(\alpha\) and \(\alpha_{\max}\). In other words, in Step 1, the algorithm uses the normalized step sizes in the following order: \(\alpha_{k_{\max}}\), \(\alpha_{k_{\max}-1}\), \(\alpha_{k_{\max}-2}\), \(\ldots\), \(\alpha_{1}\). Each value is used for \(\lceil IL\rceil\) iterations, with \(0<l\leq 1\). For tracking purposes, in order to not stall the algorithm in Step 2 (i.e., when the lower bound of the normalized step size parameter \(\alpha\) is used), the condition from Step 3 is introduced. The power estimates required in Step 3 can be recursively evaluated as:  
\[\tilde{\sigma}_{e}^{2}(n) =\lambda\tilde{\sigma}_{e}^{2}(n-1)+(1-\lambda)e^{2}(n), \tag{14}\] \[\tilde{\sigma}_{v}^{2}(n) =\gamma\tilde{\sigma}_{v}^{2}(n-1)+(1-\gamma)e^{2}(n), \tag{15}\]  
where \(\lambda=1-1/(KL)\) and \(\gamma=1-1/(QL)\), with \(K>1\) and \(Q>K\)[29]. The power estimate of the system noise can be roughly estimated from the error signal by using a larger smoothing parameter \(\gamma\). By checking the condition from Step 3, we can identify the change of the system (in a system identification scenario) and reset the step size of the algorithm. Of course, more complex/performant detectors can be used in Step 3[15], but this is beyond the scope of this paper. The main parameters to be set are \(k_{\max}\) and \(l\). The first parameter, \(k_{\max}\), is related to the maximum value of the normalized step size, \(\alpha_{\max}\), as shown in (12). For example, if we consider \(\alpha_{\max}=0.99\) (which is high enough and very close to the fastest convergence mode) and the lower bound \(\alpha\ll 1\) (so that \(\log(1-\alpha)\approx-\alpha\)), according to (12) we obtain:  
\[k_{\max}\approx\left\lceil\frac{\log(0.01)}{-\alpha}\right\rceil-1\approx \left\lceil\frac{5}{\alpha}\right\rceil. \tag{16}\]The second parameter, \(l\) (with \(0<l\leq 1\)), determines how many iterations (i.e., \(\lceil IL\rceil\)) the algorithm will use a certain value of the step size (see Step 1). As we can notice from Step 2, the algorithm will reach the lower bound of the normalized step size \(\alpha\) after \(k_{\max}\lceil IL\rceil\) iterations; from this point of view, it is not recommended to set a very low value of \(\alpha\) (e.g., \(\alpha<0.01\)), which leads to a high value of \(k_{\max}\), according to (16). Since the convergence rate of the NLMS-based algorithms depends on the character of the input signal [1; 2; 3] (in terms of the condition number of the correlation matrix), the value of \(l\) should be chosen accordingly. For example, for white Gaussian inputs, we can choose a smaller value of \(l\), e.g., \(l<0.3\). For more correlated inputs (such as autoregressive processes), the value of \(l\) should be increased, e.g., \(0.3\leq l<0.5\).