extreme eigenvalues of real symmetric toeplitz matrices we exploit the even and odd spectrum of real symmetric toeplitz matrices for the computation of their extreme eigenvalues which are obtained as the solutions of spectral or secular equations we also present a concise convergence analysis for a method to solve these spectral equations along with an efficient stopping rule an error analysis and extensive numerical results

the optimum scalar data nonlinearity in lms adaptation for arbitrary iid inputs the authors show that the optimum nonlinear scale operation upon the elements of the observation vector in the lms algorithm is exactly x 1 mu x sup 2 for any independent stochastic data input and any noise density moreover use of such a nonlinearity can yield a significant performance improvement in fast adaptation situations

stochastic analysis of the lms algorithm with a saturation nonlinearity following the adaptive filter output this paper presents a statistical analysis of the least mean square lms algorithm with a zero memory scaled error function nonlinearity following the adaptive filter output this structure models saturation effects in active noise and active vibration control systems when the acoustic transducers are driven by large amplitude signals the problem is first defined as a nonlinear signal estimation problem and the mean square error mse performance surface is studied analytical expressions are obtained for the optimum weight vector and the minimum achievable mse as functions of the saturation these results are useful for adaptive algorithm design and evaluation the lms algorithm behavior with saturation is analyzed for gaussian inputs and slow adaptation deterministic nonlinear recursions are obtained for the time varying mean weight and mse behavior simplified results are derived for white inputs and small step sizes monte carlo simulations display excellent agreement with the theoretical predictions even for relatively large step sizes the new analytical results accurately predict the effect of saturation on the lms adaptive filter behavior

adaptive filter performance with nonlinearities in the correlation multiplier it is well known that the correlation multiplier used to form the update signal in adaptive transversal filters need not be a true multiplier by using an approximate multiplier such as a sign bit only multiplier implementation can be made less costly in this paper we derive expressions for the convergence rate and misalignment noise of adaptive transversal filters with arbitrary correlation multiplier non linearities the analysis shows any such nonlinearities to always harm performance and quantifies the penalty suffered sign bit only multiplication suffers a significant penalty and power of two multiplication is suggested as a better alternative its performance is almost equal to that of a true multiplier and in a digital implementation at least it is only slightly more difficult to realize than a sign bit only multiplier the theory is developed in a way emphasizing physical understanding and has been verified to be in good agreement with simulation

the split levinson algorithm the classical levinson algorithm for computing the predictor polynomial relative to a real positive definite toeplitz matrix is shown to be redundant in complexity it can be broken down into two simpler algorithms either of which needs only to be processed this result can be interpreted in the framework of the theory of orthogonal polynomials on the real line as follows the symmetric and antisymmetric parts of the predictors relative to the sequence of toeplitz matrices constitute two families of polynomials orthogonal on the interval 1 1 with respect to some even weight functions it turns out that the recurrence relations for these orthogonal polynomials can be used efficiently to compute the desired predictor the resulting split levinson algorithm requires roughly one half the number of multiplications and the same number of additions as the classical levinson algorithm a simple derivation of cybenko s method for computing the pisarenko frequencies is obtained from the recurrence relations underlying the split levinson algorithm

the complex lms adaptive algorithm transient weight mean and covariance with applications to the ale the transient and steady state mean and covariance of the complex valued lms adaptive element weights are investigated when the inputs are samples from circularly normal processes it is shown that the data covariance diagonalizing matrix also diagonalizes the weight covariance matrix this result permits describing the complete transient behavior of the adaptive line enhancer ale weight covariance matrix in closed form for the case of multiple equal power narrow band statistically independent orthogonal rayleigh fading sinusoids in broad band noise based on these transient covariance results it is shown that for any stage of adaptation there exists an optimum ale gain constant that minimizes the excess mean squared prediction error this result is of particular significance when processing time limited data

common fixed point and invariant approximation results for noncommuting generalized f g nonexpansive maps we present common fixed point results for noncommuting generalized f g nonexpansive maps as application invariant approximation results are obtained our results unify and generalize various known results existing in the literature

on error saturation nonlinearities in lms adaptation the effect of a saturation type error nonlinearity in the weight update equation in least mean squares lms adaptation is investigated for a white gaussian data model nonlinear difference equations are derived for the eight first and second moments which include the effect of an error function erf saturation type nonlinearity on the error sequence driving the algorithm a nonlinear difference equation for the mean norm is explicitly solved using a differential equation approximation and integration by quadratures the steady state second moment weight behavior is evaluated exactly for the erf nonlinearity using the above results the tradeoff between the extent of error saturation steady state excess mean square error and rate of algorithm convergence is studied the tradeoff shows that 1 starting with a sign detector the convergence rate is increased by nearly a factor of two for each additional bit and 2 as the number of bits is increased further the additional bit by very little in convergence speed asymptotically approaching the behavior of the linear algorithm

variable explicit regularization in affine projection algorithm robustness issues and optimal choice a variable regularized affine projection algorithm vr apa is introduced without requiring the classical step size its use is supported from different points of view first it has the property of being hinfin optimal and it satisfies certain error energy bounds second the time varying regularization parameter is obtained by maximizing the speed of convergence of the algorithm although we first derive the vr apa for a linear time invariant lti system we show that the same expression holds if we consider a time varying system following a first order markov model we also find expressions for the power of the steady state error vector for the vr apa and the standard apa with no regularization parameter particularly we obtain quite different results with and without using the independence assumption between the a priori error vector and the measurement noise vector simulation results are presented to test the performance of the proposed algorithm and to compare it with other schemes under different situations an important conclusion is that the former independence assumption can lead to very inaccurate steady state results especially when high values of the projection order are used

tracking analysis of the sign algorithm in nonstationary environments a tracking analysis of the adaptive filters equipped with the sign algorithm and operating in nonstationary environments is presented under the assumption that the nonstationary can be modeled using a random disturbance it is shown that the long term time average of the mean absolute error is bounded and that there exists an optimal choice of the convergence constant mu which minimizes this quality applying the commonly used independence assumption and under the assumption that the nonstationarity is solely due to the time varying behavior of the optimal coefficients it is shown that the distributions of the successive coefficient misalignment vectors converge to a limiting distribution when the adaptive filter is used in the system identification mode finally under the additional assumption that the signals involved are zero mean and gaussian a set of nonlinear difference equations is derived that characterizes the mean and mean squared behavior of the filter coefficients and the mean squared estimation error during adaptation and tracking results of several experiments that show very good correlation with the theoretical analyses are presented

invariant approximations and r subweakly commuting maps the existence of invariant approximations for r subweakly commuting mappings is proved our results extend most of the known results to a new class of noncommuting mappings

a schur based algorithm for computing bounds to the smallest eigenvalue of a symmetric positive definite toeplitz matrix abstract recent progress in signal processing and estimation has generated c onsiderable interest in the problem of computing the smallest eigenvalue of symmetric positive definite toeplitz matrices several algorithms have been proposed in the literature they compute the smallest eigenvalue in an iterative fashion many of them relying on the levinson durbin solution of sequences of yule walker systems exploiting the properties of two algorithms recently developed for estimating a lower and an upper bound of the smallest singular value of upper triangular matrices respectively an algorithm for computing bounds to the smallest eigenvalue of a symmetric positive definite toeplitz matrix is derived the algorithm relies on the computation of the r factor of the qr factorization of the toeplitz matrix and the inverse of r the simultaneous computation of r and r 1 is efficiently accomplished by the generalized schur algorithm

behavior of the e normalized lms algorithm with gaussian inputs the first and second moment behavior of one variation of the normalized lms e nlms algorithm is investigated for a white covariance matrix and gaussian statistics for the data for this model it is shown that the e nlms algorithm has neither behavior independent of the input data power nor a performance significantly better than the lms algorithm for which the input power level also must be known a priori hence based upon the results of the analysis it is recommended that the algorithm not be used in place of the lms for known input power levels or in place of the nlms for unknown input power levels

computing the smallest eigenpair of a symmetric positive definite toeplitz matrix an algorithm for computing the smallest eigenvalue of a symmetric positive definite toeplitz matrix is presented the eigenvalue is approximated from below by newton s method applied to the characteristic polynomial of the matrix the newton s step is calculated by a levinson durbin type recursion simultaneously this recursion produces a realistic error bound of the actual approximation without additional computing effort as well as a simple and efficient way to compute the associated eigenvector

computing the minimum eigenvalue of a symmetric positive definite toeplitz matrix by newton type methods a method for computing the smallest eigenvalue of a symmetric positive definite toeplitz matrix by application of newton s method to the characteristic polynomial has been recently introduced by mastronardi and boley siam j sci comput 20 1999 pp 1921 1927 though considerably slower than methods developed by the authors w mackens and h voss siam j matrix anal appl 18 1997 pp 521 534 w mackens and h voss linear algebra appl 275 276 1998 pp 401 415 h voss linear algebra appl 287 1999 pp 359 371 the new approach is conceptually much simpler in this paper we improve the performance of the new method substantially while keeping its simplicity

adaptive equalizer using finite bit power of two quantizer the stochastic gradient algorithm using a simplified arithmetic is analyzed in this paper a power of two quantizer is used for the input of the multiplier to reduce the multiplication to at most a simple shift in spite of its simple implementation the performance is shown to be comparable to the classical lms algorithm a linearized approximation to the quantizer is first derived followed by the analysis of an exact nonlinear model the derivation is based on the gaussian assumption and the effects of removing the gaussian assumption are later considered the roundoff error due to the finite bit computation is calculated computer simulation results are provided to support the analysis

stochastic gradient adaptation under general error criteria examines a family of adaptive filter algorithms of the form w sub k 1 w sub k spl mu f d sub k w sub k sup t x sub k x sub k in which f spl middot is a memoryless odd symmetric nonlinearity acting upon the error such algorithms are a generalization of the least mean square lms adaptive filtering algorithm for even symmetric error criteria for this algorithm family the authors derive general expressions for the mean and mean square convergence of the filter coefficients for both arbitrary stochastic input data and gaussian input data they then provide methods for optimizing the nonlinearity to minimize the algorithm misadjustment for a given convergence rate using the calculus of variations it is shown that the optimum nonlinearity to minimize misadjustment near convergence under slow adaptation conditions is independent of the statistics of the input data and can be expressed as p x p x where p x is the probability density function of the uncorrelated plant noise for faster adaptation under the white gaussian input and noise assumptions the nonlinearity is shown to be x 1 spl mu spl lambda x sup 2 spl sigma sub k sup 2 where spl lambda is the input signal power and spl sigma sub k sup 2 is the conditional error power thus the optimum stochastic gradient error criterion for gaussian noise is not mean square it is shown that the equations governing the convergence of the nonlinear algorithm are exactly those which describe the behavior of the optimum scalar data nonlinear adaptive algorithm for white gaussian input simulations verify the results for a host of noise interferences and indicate the improvement using non mean square error criteria

a two step even odd split levinson algorithm for toeplitz systems abstract we use a two step durbin method rather than the single step version in the even odd split levinson algorithm for strongly nonsingular real symmetric toeplitz systems with arbitrary right hand side thereby slightly reducing the complexity of this algorithm to 19 8 n 2 o n flops on a sequential machine we also present extensive numerical results comparing several levinson type methods

analysis of the normalized lms algorithm with gaussian inputs the lms adaptive filter algorithm requires a priori knowledge of the input power level to select the algorithm gain parameter for stability and convergence since the input power level is usually one of the statistical unknowns it is normally estimated from the data prior to beginning the adaptation process it is then assumed that the estimate is perfect in any subsequent analysis of the lms algorithm behavior in this paper the effects of the power level estimate are incorporated in a data dependent that appears explicitly within the algorithm the transient mean and second moment behavior of the modified lms nlms algorithm are evaluated taking into account the explicit statistical dependence of upon the input data the mean behavior of the algorithm is shown to converge to the wiener weight a constant coefficient matrix difference equation is derived for the weight fluctuations about the wiener weight the equation is solved for a white data covariance matrix and for the adaptive line enhancer with a single frequency input in steady state for small expressions for the misadjustment error are also presented it is shown for the white data covariance matrix case that the averaging of about ten data samples causes negligible degradation as compared to the lms algorithm in the ale application the steady state weight fluctuations are shown to be mode dependent being largest at the frequency of the input

the least mean fourth lmf adaptive algorithm and its family new steepest descent algorithms for adaptive filtering and have been devised which allow error minimization in the mean fourth and mean sixth etc sense during adaptation the weights undergo exponential relaxation toward their optimal solutions time constants have been derived and surprisingly they turn out to be proportional to the time constants that would have been obtained if the steepest descent least mean square lms algorithm of widrow and hoff had been used the new gradient algorithms are insignificantly more complicated to program and to compute than the lms algorithm their general form is w j 1 w j 2 mu k epsilon j 2k 1 x j where w j is the present weight vector w j 1 is the next weight vector epsilon j is the present error x j is the present input vector mu is a constant controlling stability and rate of convergence and 2k is the exponent of the error being minimized conditions have been derived for weight vector convergence of the mean and of the variance for the new gradient algorithms the behavior of the least mean fourth lmf algorithm is of special interest in comparing this algorithm to the lms algorithm when both are set to have exactly the same time constants for the weight relaxation process the lmf algorithm under some circumstances will have a substantially lower weight noise than the lms algorithm it is possible therefore that a minimum mean fourth error algorithm can do a better job of least squares estimation than a mean square error algorithm this intriguing concept has implications for all forms of adaptive algorithms whether they are based on steepest descent or otherwise

convergence analysis of lms filters with uncorrelated gaussian data statistical analysis of the least mean squares lms adaptive algorithm with uncorrelated gaussian data is presented exact analytical expressions for the steady state mean square error mse and the performance degradation due to weight vector misadjustment are derived necessary and sufficient conditions for the convergence of the algorithm to the optimal wiener solution within a finite variance are derived it is found that the adaptive coefficient which controls the rate of convergence of the algorithm must be restricted to an interval significantly smaller than the domain commonly stated in the literature the outcome of this paper therefore places fundamental limitations on the mse performance and rate of convergence of the lms adaptive scheme

a stochastic newton algorithm with data adaptive step size a stochastic newton algorithm for adaptive filtering which is a generalization of the so called normalized least mean square algorithm is proposed expressions for the expected value of the algorithm weight error vector and second moment of the weight error vector norm are derived and have been verified by computer simulation experimental results which demonstrate the improved convergence of the algorithm over the algorithm after a large number of iterations are given

transient analysis of data normalized adaptive filters this paper develops an approach to the transient analysis of adaptive filters with data normalization among other results the derivation characterizes the transient behavior of such filters in terms of a linear time invariant state space model the stability of the model then translates into the mean square stability of the adaptive filters likewise the steady state operation of the model provides information about the mean square deviation and mean square error performance of the filters in addition to deriving earlier results in a unified manner the approach leads to stability and performance results without restricting the regression data to being gaussian or white the framework is based on energy conservation arguments and does not require an explicit recursion for the covariance matrix of the weight error vector

compatible maps and invariant approximations the existence of invariant best approximations for compatible maps is proved our results unify and generalize various known results to a more general class of noncommuting mappings

on the optimum data nonlinearity in lms adaptation the effect of an arbitrary nonlinear operation on the data input to the weight update equation in the lms adaptive algorithm is investigated for a gaussian data model exact difference equations are derived for the weight first and second moments that include the effects of an arbitrary nonlinear operation on the data sequence the difference equations are used to obtain expressions for the transient behavior of the mean square error the mean square error is minimized over the choice of nonlinearity for a fixed transient behavior the best choice of nonlinearity is shown exactly to be of the form x 1 bx2 when the input data are white for an arbitrary data covariance matrix the optimum nonlinearity is shown to be linear when the product of the algorithm step size and input power is much less than unity

numerical experience with a superfast real toeplitz solver abstract we briefly describe the generalized schur algorithm for the superfast solution of positive definite toeplitz systems of equations and its relationship with schur s algorithm and the szego recursions we then present some experimental results obtained with our fortran implementation of this superfast toeplitz solver the algorithm displays favorable behavior in that the growth rates of the resulting residuals and errors are comparable with those of the szego recursions

computation of the newton step for the even and odd characteristic polynomials of a symmetric positive definite toeplitz matrix we compute the newton step for the characteristic polynomial and for the even and odd characteristic polynomials of a symmetric positive definite toeplitz matrix as the reciprocal of the trace of an appropriate matrix we show that after the yule walker equations are solved this trace can be computed in 0 n additional arithmetic operations which is in contrast to existing methods which rely on a recursion requiring o n 2 additional arithmetic operations

on the convergence behavior of the lms and the normalized lms algorithms it is shown that the normalized least mean square nlms algorithm is a potentially faster converging algorithm compared to the lms algorithm where the design of the adaptive filter is based on the usually quite limited knowledge of its input signal statistics a very simple model for the input signal vectors that greatly simplifies analysis of the convergence behavior of the lms and nlms algorithms is proposed using this model answers can be obtained to questions for which no answers are currently available using other perhaps more realistic models examples are given to illustrate that even quantitatively the answers obtained can be good approximations it is emphasized that the convergence of the nlms algorithm can be speeded up significantly by employing a time varying step size the optimal step size sequence can be specified a priori for the case of a white input signal with arbitrary distribution

normalized data nonlinearities for lms adaptation properly designed nonlinearly modified lms algorithms in which various quantities in the stochastic gradient estimate are operated upon by memoryless nonlinearities have been shown to perform better than the lms algorithm in system identification type problems the authors investigate one such algorithm given by w sub k l w sub k spl mu d sub k w sub k sup t x sub k x sub k f x sub k in which the function f x sub k is a scalar function of the sum of the squares of the n elements of the input data vector x sub k this form of algorithm generalizes the so called normalized lms nlms algorithm they evaluate the expected behavior of this nonlinear algorithm for both independent input vectors and correlated gaussian input vectors assuming the system identification model by comparing the nonlinear algorithm s behavior with that of the lms algorithm they then provide a method of optimizing the form of the nonlinearity for the given input statistics in the independent input case they show that the optimum nonlinearity is a single parameter version of the nlms algorithm with an additional constant in the denominator and show that this algorithm achieves a lower excess mean square error mse than the lms algorithm with an equivalent convergence rate additionally they examine the optimum step size sequence for the optimum nonlinear algorithm and show that the resulting algorithm performs better and is less complex to implement than the optimum step size algorithm derived for another form of the nlms algorithm simulations verify the theory and the predicted performance improvements of the optimum normalized data nonlinearity algorithm

exact performance analysis of the epsilon nlms algorithm for colored circular gaussian inputs this paper presents exact mean square analysis of the nlms algorithm for circular complex correlated gaussian input the analysis is based on the derivation of a closed form expression for the cumulative distribution function cdf of random variables of the form ui d12 ui d12 and using that to derive the first and second moments of such variables these moments in turn completely characterize the mean square ms behavior of the nlms in explicit closed form expressions both transient and steady state behavior are analyzed consequently new explicit closed form expressions for the mean square error mse behavior are derived our simulations of the transient and steady state behavior of the filter match the expressions obtained theoretically for various degrees of input correlation and for various values of