# An Optimized Differential Step-Size LMS Algorithm
###### Abstract  
Adaptive algorithms with differential step-sizes (related to the filter coefficients) are well known in the literature, most frequently as \"proportionate\" algorithms. Usually, they are derived on a heuristic basis. In this paper, we introduce an algorithm resulting from an optimization criterion. Thereby, we obtain a benchmark algorithm and also another version with lower computational complexity, which is rigorously valid for less correlated input signals. Simulation results confirm the theory and outline the performance of the algorithms. Unfortunately, the good performance is obtained by an important increase in computational complexity. Nevertheless, the proposed algorithms could represent useful benchmarks in the field. adaptive filter; least-mean-square (LMS) algorithm; optimized differential step-size LMS algorithm; system identification; echo cancellation +
Footnote †: journal:  
0000-0001-5000-0000-0000  
0000-0000-0000-0000-0000-0000-0000-00000-  
manner. For example, in the case of the celebrated PNLMS algorithm [2], the equations used to calculate the step-size control factors are not based on any optimization criteria but are designed in an ad hoc way. For this reason, after an initial fast convergence phase, the convergence rate of the PNLMS algorithm significantly slows down. Nevertheless, following the PNLMS idea, other more sophisticated algorithms working on the same principle have been proposed with the ultimate purpose to fully exploit sparsity of impulse responses. Among them, the IPNLMS algorithm [4] represents a benchmark solution, which is more robust to the sparseness degree of the impulse response. A connection could be established between proportionate algorithms and variable step-size adaptive filters. In terms of their purpose, both categories adjust the step-size in order to enhance the overall performance. While proportionate filters aim to improve the convergence rate by increasing the adaptation step-size of the larger coefficients, variable step-size algorithms target the well-known compromise between the convergence rate and misadjustment [1]. In both cases, the adaptation control rules should rely on some specific optimization criteria. Based on the initial work reported in [11; 12], in this paper, we propose a least-mean-square (LMS) algorithm controlled by a diagonal matrix, \(\mathbf{Y}\), instead of a usual scalar step-size. The elements of the matrix are adjusted at each iteration by considering the mean-squared deviation (MSD) minimization. Hence, following a stochastic model [13], we obtain an optimized differential step-size LMS (ODSS-LMS) algorithm, tailored by a state variable model. The theory has been driven on the model of an unknown system characterized by a time-varying behavior. The algorithm developed in [12], namely the optimized LMS algorithm with individual control factors (OLMS-ICF), represents only a particular case of the ODSS-LMS algorithm. The motivation behind our approach can be summarized as follows. In many related works concerning optimized step-size algorithms, e.g., [14; 15; 16; 17; 18], the reference signal is considered the output of a time-invariant system corrupted by additive noise. However, in realistic scenarios, the system to be identified could be variable in time. For example, in acoustic echo cancellation, it can be assumed that the impulse response of the echo path is modeled by a time-varying system following a first-order Markov model [13]. In addition, it makes more sense to minimize the system misalignment (in terms of the MSD of the coefficients), instead of the classical error-based cost function. The reason is that, in real-world system identification problems, the goal of the adaptive filter is not to make the error signal goes to zero. The objective, instead, is to recover the additive noise (which corrupts the system's output) from the error signal of the adaptive filter, after this one converges to the true solution.