on error saturation nonlinearities in nlms adaptation the effect of a saturation type error nonlinearity in the weight update equation in normalized least mean square nlms adaptation is investigated for system identification for a white gaussian data model nonlinear recursions are derived for the weight mean error and mean square deviation msd that include the effect of an error function erf saturation type nonlinearity on the error sequence driving the algorithm the nonlinear recursion for the msd is solved numerically and shown in excellent agreement with monte carlo simulations supporting the theoretical model assumptions the theory is extended to tracking a markov channel and accurately predicts the tracking behavior as well the saturation behavior of the algorithm is easily studied by varying a single parameter in the error function varying from a linear device to a hard limiter for the white data case the excess mean square error emse is simply related to the msd the tradeoff between the extent of error saturation steady state emse and algorithm convergence rate is studied using these results

roundoff noise analysis of signals represented using signed power of two terms it is a well known fact that the multiplication of a number by an integer power of two is a very simple process in binary arithmetic hence digital filters whose coefficient values are integer power of two are essentially multiplierless the design of digital filters with power of two coefficient values require time consuming optimization process and may not always be possible in some applications such as in adaptive filtering since hardware circuitry for real time conversion of a binary integer into a sum of a limited number of signed power of two spt terms is available if the signal is expressed in spt terms i e in digit code the filter is also multiplierless even though the coefficient values are not spt when each signal data is rounded to a limited number of spt terms a roundoff noise representing the roundoff error is introduced in the spt space the quantization step size is nonuniform and so the roundoff noise characteristic is different from that produced when the quantization step size is uniform this paper presents an analysis for the roundoff noise of signal represented using a limited number of spt terms the result is useful for determining the number of spt terms required to represent a signal subject to a given roundoff noise

the optimum scalar data nonlinearity in lms adaptation for arbitrary iid inputs the authors show that the optimum nonlinear scale operation upon the elements of the observation vector in the lms algorithm is exactly x 1 mu x sup 2 for any independent stochastic data input and any noise density moreover use of such a nonlinearity can yield a significant performance improvement in fast adaptation situations

performance advantage of complex lms for controlling narrow band adaptive arrays in narrow band adaptive array applications the mean square convergence of the discrete time real least mean square lms algorithm is slowed by image frequency noises generated in the lms loops the complex lms algorithm proposed by widrow et al is shown to eliminate these noises yielding convergence of the mean squared error mse at slightly over twice the rate this paper includes a comprehensive analysis of the mse of adaptation for lms the analysis is based upon the method developed in the 1968 dissertation by k d senne and it represents the most complete treatment of the subject published to date

stochastic analysis of the lms algorithm with a saturation nonlinearity following the adaptive filter output this paper presents a statistical analysis of the least mean square lms algorithm with a zero memory scaled error function nonlinearity following the adaptive filter output this structure models saturation effects in active noise and active vibration control systems when the acoustic transducers are driven by large amplitude signals the problem is first defined as a nonlinear signal estimation problem and the mean square error mse performance surface is studied analytical expressions are obtained for the optimum weight vector and the minimum achievable mse as functions of the saturation these results are useful for adaptive algorithm design and evaluation the lms algorithm behavior with saturation is analyzed for gaussian inputs and slow adaptation deterministic nonlinear recursions are obtained for the time varying mean weight and mse behavior simplified results are derived for white inputs and small step sizes monte carlo simulations display excellent agreement with the theoretical predictions even for relatively large step sizes the new analytical results accurately predict the effect of saturation on the lms adaptive filter behavior

adaptive filter performance with nonlinearities in the correlation multiplier it is well known that the correlation multiplier used to form the update signal in adaptive transversal filters need not be a true multiplier by using an approximate multiplier such as a sign bit only multiplier implementation can be made less costly in this paper we derive expressions for the convergence rate and misalignment noise of adaptive transversal filters with arbitrary correlation multiplier non linearities the analysis shows any such nonlinearities to always harm performance and quantifies the penalty suffered sign bit only multiplication suffers a significant penalty and power of two multiplication is suggested as a better alternative its performance is almost equal to that of a true multiplier and in a digital implementation at least it is only slightly more difficult to realize than a sign bit only multiplier the theory is developed in a way emphasizing physical understanding and has been verified to be in good agreement with simulation

saturation effects in lms adaptive echo cancellation for binary data the effect of a saturation type error nonlinearity in the weight update equation in least mean square lms adaptive echo cancellation is investigated for an independent binary data model a nonlinear difference equation is derived for the mean norm of the difference between the estimate and the unknown filter to be estimated by the algorithm the difference equation is evaluated numerically it is shown that far end binary data interference is much more deleterious to algorithm transient behavior than far end gaussian data interference the number of additional bits for the same cancellation convergence rates for binary versus gaussian interference of the same power is studied as a function of various system parameters algorithm convergence rates are studied as a function of an arbitrary probability density function pdf for the far end data it is shown that a binary pdf causes the worst degradation and a gaussian shaped pdf causes the least degradation

velocity vorticity helicity formulation and a solver for the navier stokes equations for the three dimensional incompressible navier stokes equations we present a formulation featuring velocity vorticity and helical density as independent variables we find the helical density can be observed as a lagrange multiplier corresponding to the divergence free constraint on the vorticity variable similar to the pressure in the case of the incompressibility condition for velocity as one possible practical application of this new formulation we consider a time splitting numerical scheme based on an alternating procedure between vorticity helical density and velocity bernoulli pressure systems of equations results of numerical experiments include a comparison with some well known schemes based on pressure velocity formulation and illustrate the competitiveness on the new scheme as well as the soundness of the new formulation

a block floating point treatment to the lms algorithm efficient realization and a roundoff error analysis an efficient scheme is presented for implementing the lms based transversal adaptive filter in block floating point bfp format which permits processing of data over a wide dynamic range at temporal and hardware complexities significantly less than that of a floating point processor appropriate bfp formats for both the data and the filter coefficients are adopted taking care so that they remain invariant to interblock transition and weight updating operation respectively care is also taken to prevent overflow during filtering as well as weight updating processes jointly by using a dynamic scaling of the data and a slightly reduced range for the step size with the latter having only marginal effect on convergence speed extensions of the proposed scheme to the sign sign lms and the signed regressor lms algorithms are taken up next in order to reduce the processing time further finally a roundoff error analysis of the proposed scheme under finite precision is carried out it is shown that in the steady state the quantization noise component in the output mean square error depends on the step size both linearly and inversely an optimum step size that minimizes this error is also found out

on the helicity in 3d periodic navier stokes equations i the non statistical case we study the asymptotic behavior of the statistical solutions to the navier stokes equations using the normalization map 9 it is then applied to the study of mean energy mean dissipation rate of energy and mean helicity of the spatial periodic flows driven by potential body forces the statistical distribution of the asymptotic beltrami flows are also investigated we connect our mathematical analysis with the empirical theory of decaying turbulence with appropriate mathematically defined ensemble averages the kolmogorov universal features are shown to be transient in time we provide an estimate for the time interval in which those features may still be present our collaborator and friend basil nicolaenko passed away in september of 2007 after this work was completed honoring his contribution and friendship we dedicate this article to him

analysis of conjugate gradient algorithms for adaptive filtering the paper presents and analyzes two approaches to the implementation of the conjugate gradient cg algorithm for filtering where several modifications to the original cg method are proposed the convergence rates and misadjustments for the two approaches are compared an analysis in the z domain is used in order to find the asymptotic performance and stability bounds are established the behavior of the algorithms in finite word length computation are described and dynamic range considerations are discussed it is shown that in finite word length computation and close to steady state the algorithms behaviors are similar to the steepest descent algorithm where the stalling phenomenon is observed using 16 bit fixed point number representation our simulations show that the algorithms are numerically stable

floating point arithmetic and digital filters the roundoff noise properties of floating point digital filters are examined to make the analysis tractable a high level model to deal with the errors in the inner product operation is developed this model establishes a broad connection between coefficient sensitivity and roundoff noise along with the model an efficient procedure to keep track of the addition scheme used in the inner product and to compute the statistics of the errors is introduced a systematic procedure based on the model is then developed to derive general expressions for the roundoff noise of fir direct form iir and state space filters the expressions in the context of state space filters are explored in some detail optimality issues are considered and it is shown that when double precision accumulation is used the optimal filters are similar in nature to those derived in the context of fixed point arithmetic with the essential difference that they also do depend on the spectrum of the input signal optimality with respect to addition schemes and second order filters are also examined in some detail

asymptotic expansion for solutions of the navier stokes equations with non potential body forces abstract we study the long time behavior of spatially periodic solutions of the navier stokes equations in the three dimensional space the body force is assumed to possess an asymptotic expansion or resp finite asymptotic approximation in sobolev gevrey spaces as time tends to infinity in terms of polynomial and decaying exponential functions of time we establish an asymptotic expansion or resp finite asymptotic approximation of the same type for the leray hopf weak solutions this extends previous results that were obtained in the case of potential forces to the non potential force case where the body force may have different levels of regularity and asymptotic approximation this expansion or approximation in fact reveals precisely how the structure of the force influences the asymptotic behavior of the solutions

locally optimum adaptive signal processing algorithms we propose a new analytic method for comparing constant gain adaptive signal processing algorithms specifically estimates of the convergence speed of the algorithms allow for the definition of a local measure of performance called the efficacy that can be theoretically evaluated by definition the efficacy is consistent with the fair comparison techniques used in signal processing applications using the efficacy as a performance measure we prove that the lms newton algorithm is optimum and is thus the fastest algorithm within a very rich algorithmic class furthermore we prove that regular lms is better than any of its variants that apply the same nonlinear transformation on the elements of the regression vector such as signed regressor quantized regressor etc for an important class of input signals simulations support all our theoretical conclusions

the complex lms adaptive algorithm transient weight mean and covariance with applications to the ale the transient and steady state mean and covariance of the complex valued lms adaptive element weights are investigated when the inputs are samples from circularly normal processes it is shown that the data covariance diagonalizing matrix also diagonalizes the weight covariance matrix this result permits describing the complete transient behavior of the adaptive line enhancer ale weight covariance matrix in closed form for the case of multiple equal power narrow band statistically independent orthogonal rayleigh fading sinusoids in broad band noise based on these transient covariance results it is shown that for any stage of adaptation there exists an optimum ale gain constant that minimizes the excess mean squared prediction error this result is of particular significance when processing time limited data

on error saturation nonlinearities in lms adaptation the effect of a saturation type error nonlinearity in the weight update equation in least mean squares lms adaptation is investigated for a white gaussian data model nonlinear difference equations are derived for the eight first and second moments which include the effect of an error function erf saturation type nonlinearity on the error sequence driving the algorithm a nonlinear difference equation for the mean norm is explicitly solved using a differential equation approximation and integration by quadratures the steady state second moment weight behavior is evaluated exactly for the erf nonlinearity using the above results the tradeoff between the extent of error saturation steady state excess mean square error and rate of algorithm convergence is studied the tradeoff shows that 1 starting with a sign detector the convergence rate is increased by nearly a factor of two for each additional bit and 2 as the number of bits is increased further the additional bit by very little in convergence speed asymptotically approaching the behavior of the linear algorithm

on the helicity in 3d periodic navier stokes equations ii the statistical case we study the asymptotic behavior of the statistical solutions to the navier stokes equations using the normalization map 9 it is then applied to the study of mean energy mean dissipation rate of energy and mean helicity of the spatial periodic flows driven by potential body forces the statistical distribution of the asymptotic beltrami flows are also investigated we connect our mathematical analysis with the empirical theory of decaying turbulence with appropriate mathematically defined ensemble averages the kolmogorov universal features are shown to be transient in time we provide an estimate for the time interval in which those features may still be present our collaborator and friend basil nicolaenko passed away in september of 2007 after this work was completed honoring his contribution and friendship we dedicate this article to him

nonlinear quantization effects in the frequency domain complex scalar lms adaptive algorithm nonlinear quantization effects in the frequency domain complex scalar lms adaptive algorithm are analyzed by using conditional expectations the probability density function of the quantizer input conditioned on the weight is derived the density is applied to finding the conditional characteristic function and the mth conditional moment at the quantizer output the first and second conditional moments of the quantizer output are used to derive difference equations that approximate the dynamical behavior of the first and second weight moments these difference equations are solved numerically and compare favorably to simulation results a model of the quantizer as an additive noise source is of no analytical value since the quantizatian noise has negligible effect on the mean square error when the model is valid finally a design approach is proposed for selecting the number of bits in the weight accumulator the moment equations are also used to determine the algorithm mean square error for different quantizer step sizes and the optimum algorithm step size when a fixed amount of input data is available for adaptation

nonlinear quantization effects in the lms and block lms adaptive algorithms a comparison digital implementations of the least mean square lms and block lms blms algorithms are compared with respect to finite word effects the algorithm stalling phenomenon is studied using gaussian data models and conditional expectation arguments it is shown that the blms algorithm requires 1 2 log sub 2 l k fewer bits for the same stalling behavior l block length and k lies between 0 2 and 1 0 depending on the precise definition of algorithm stalling on the other hand the lms algorithm requires log sub 2 l fewer bits than blms for the same level of saturation behavior transient response at algorithm initialization hence the lms algorithm requires 1 2 log sub 2 l k fewer bits than the blms algorithm for the same saturation and stalling effects

postsmoothing for the lms algorithm and a fixed point roundoff error analysis the effect of post algorithm smoothing on digital implementations of the least mean square lms algorithm is studied an expression is derived for the mean square error mse with post algorithm pa smoothing but without finite wordlength effects it is shown that the mse can be reduced from that of lms without pa smoothing pa smoothing results in reduced convergence speed the effect of fixed point finite precision on pa smoothing is studied an expression for the mse is derived monte carlo simulation results that support the result that digital lms with pa smoothing performs better than digital lms without pa smoothing are presented

power versus performance tradeoffs for reduced resolution lms adaptive filters low power implementation of digital adaptive filters for channel equalization and interference cancelling is an impor tant aspect of wireless communications transceiver design in this paper we propose a low power design methodology based on ex ploitation of the tradeoff between high bit resolution and low power consumption we focus on the widely used finite precision lms adaptive channel equalizer for which closed form approximations can be derived for the increase in mean square error when reducing both the number of data bits and filter coefficient bits the re sulting expressions are accurate as long as the data and coefficient wordlengths are such that the weight update signal does not fall into the finite resolution deadzone of the filter we give a sufficient condition on the wordlengths that ensures that this phenomenon known as stopping or slowdown does not occur under this con dition we obtain expressions for the power optimal bit allocation factor that determines the optimal proportion of bits to allocate to the data the remainder being allocated to the coefficients numer ical studies are presented for an exponential memory isi channel with gaussian training sequence

new insights on the transient and steady state behavior of the quantized lms algorithm this correspondence investigates the transient and steady state behavior of the quantized lms algorithm for gaussian inputs it is shown here that the so called stopping phenomenon is really a slow down phenomenon which because of an extremely slow convergence rate looks as if the algorithm has stopped the true steady state mse is shown to be nearly independent of the number of bits in the digital word length and very nearly the steady state mse of the infinite precision lms realization these results assume that the algorithm misadjustment effects due to coefficient quantization are negligible in comparison with those due to the stopping phenomena since the true steady state is rarely achievable with a finite number of iterations determination of the step size spl mu that minimizes the residual mse must be based on a stochastic model for the transient mode of algorithm operation it is shown that the finite word length and infinite precision design cases differ only in degree and not in kind as far as the selection of spl mu is concerned

maximum a posteriori estimation of multichannel bernoulli gaussian sequences general problems and solutions are described for maximum a posteriori estimation of multichannel bernoulli gaussian sequences which are inputs to a linear discrete time multivariable system the authors first develop a separation principle which indicates that one can estimate multichannel gaussian amplitudes and bernoulli events separately they then discuss approaches for estimation of these quantities

minimum variance and maximum likelihood recursive waveshaping in this paper we develop optimal recursive waveshaping filters in the framework of estimation theory and state variable models we develop a linear minimum variance waveshaper and a nonlinear maximum likelihood waveshaper both waveshapers consist of two components 1 stochastic inversion and 2 waveshaping the former is performed by means of minimum variance deconvolution simulations are given which illustrate results that can be obtained by both waveshapers in retrospect we view the minimum variance results of this paper as the recursive counterparts to those presented by treitel and robinson 14 which are for finite impulse response waveshaping

speech enhancement in discontinuous transmission systems using the constrained stability least mean squares algorithm in this paper a novel constrained stability least mean squares lms algorithm for filtering speech sounds is proposed in the adaptive noise cancellation anc problem it is based on the minimization of the squared euclidean norm of the weight vector change under a stability constraint over the a posteriori estimation errors to this purpose the lagrangian methodology has been used in order to propose a nonlinear adaptation in terms of the product of differential input and error convergence analysis is also studied in terms of the evolution of the natural modes to the optimal wiener hopf solution so that the stability performance depends exclusively on the adaptation parameter and the eigenvalues of the difference matrix r1 the algorithm shows superior performance over the referenced algorithms in the anc problem of speech discontinuous transmission systems which are characterized by rapid transitions of the desired signal the experimental analysis carried out on the aurora 3 speech databases provides an extensive performance evaluation together with an exhaustive comparison to the standard lms algorithms i e the normalized lms nlms and other recently reported lms algorithms such as the modified nlms the error nonlinearity lms or the normalized data nonlinearity lms adaptation

control theoretic design of the lms and the sign algorithms in nonstationary environments the feedback structure of the lms least mean squared algorithm proposed by b widrow et al proc ieee vol 64 p 1151 62 aug 1976 is reexamined from a control system design viewpoint the minimization of the misadjustments due to gradient noise and lag can then be recast as the disturbance rejection and tracking problems in control a frequency domain approach to the latter problems that has the advantages of transparency ease of computation and generality compared with the time domain approach previously used is presented with the same set of assumptions of white input and a markovian plant it is shown that the optimum step size obtained by the present approach is identical to that obtained by widrow applying the same approach the optimum step size of a simplified version of the lms algorithm the sign algorithm is derived for the case when the plant is slowly varying and the input signals are gaussian

transient and tracking performance analysis of the quantized lms algorithm for time varying system identification this paper investigates the statistical behavior of the finite precision lms adaptive filter in the identification of an unknown time varying stochastic system nonlinear recursions are derived for the mean and mean square behavior of the adaptive weights transient and tracking algorithm performance curves are generated from the recursions and shown to be in excellent agreement with monte carlo simulations our results demonstrate that linear models are inappropriate for analyzing the transient and the steady state algorithm behavior the performance curves indicate that the transient and tracking capabilities cannot be determined from perturbations about the infinite precision case it is shown that the transient phase of the algorithm increases as the digital wordlength or the speed of variation of the unknown system decrease design examples illustrate how the theory can be used to select the algorithm step size and the number of bits in the quantizer

fixed point roundoff error analysis of the exponentially windowed rls algorithm for time varying systems a fixed point roundoff error analysis of the exponentially windowed rls algorithm is presented it is shown that a tradeoff exists in the choice of the forgetting factor in order to reduce the sensitivity of the algorithm to additive noise must be chosen close to one on the other hand the roundoff error increases as lambda rightarrow 1 it is shown that the algorithm is stabilized with lambda rightarrow 1 to derive the theoretical results it is assumed that the input signal is a white gaussian random process finally simulations are presented which confirm the theoretical findings of the paper

global stability of large solutions to the 3d navier stokes equations we prove the stability of mildly decaying global strong solutions to the navier stokes equations in three space dimensions combined with previous results on the global existence of large solutions with various symmetries this gives the first global existence theorem for large solutions with approximately symmetric initial data the stability of unforced 2d flow under 3d perturbations is also obtained

on error saturation nonlinearities for lms adaptation in impulsive noise this correspondence extends the analytic results in n j bershad ldquoon error nonlinearities in lms adaptation rdquo ieee transactions on acoustics speech and signal processing vol assp 36 no 4 pp 440 452 april 1988 to least mean square lms adaptation in impulsive observation noise a scalar recursion for the weight misadjustment is derived for the white input data case monte carlo simulations verify the accuracy of the theoretical model the theoretical recursion is then used to study the effects of the impulse noise on algorithm convergence speed and steady state weight misadjustment for a wide variety of parameter values

unbiased and stable leakage based adaptive filters the paper develops a leakage based adaptive algorithm referred to as circular leaky which in addition to solving the drift problem of the classical least mean squares lms adaptive algorithm it also avoids the bias problem that is created by the standard leaky lms solution these two desirable properties of unbiased and bounded estimates are guaranteed by circular leaky at essentially the same computational cost as lms the derivation in the paper relies on results from averaging theory and from lyapunov stability theory and the analysis shows that the above properties hold not only in infinite precision but also in finite precision arithmetic the paper further extends the results to a so called switching spl sigma algorithm which is a leakage based solution used in adaptive control

general problems of minimum variance recursive waveshaping in this correspondence we sum up the general problems of minimum variance recursive waveshaping mvrw we first develop the general structure of minimum variance waveshaping mvw and then present the recursive algorithms of mvrw both on line and off line algorithms are considered

adaptive equalizer using finite bit power of two quantizer the stochastic gradient algorithm using a simplified arithmetic is analyzed in this paper a power of two quantizer is used for the input of the multiplier to reduce the multiplication to at most a simple shift in spite of its simple implementation the performance is shown to be comparable to the classical lms algorithm a linearized approximation to the quantizer is first derived followed by the analysis of an exact nonlinear model the derivation is based on the gaussian assumption and the effects of removing the gaussian assumption are later considered the roundoff error due to the finite bit computation is calculated computer simulation results are provided to support the analysis

asymptotic expansion for solutions of the navier stokes equations with potential forces abstract we derive an asymptotic expansion for smooth solutions of the navier stokes equations in weighted spaces this result removes previous restrictions on the number of terms of the asymptotics as well as on the range of the polynomial weights we also write the expansion in terms of expressions involving non linear quantities

a structural view of asymptotic convergence speed of adaptive iir filtering algorithms i infinite precision implementation for pt i see ibid vol 41 no 4 p 1493 1517 1993 finite precision fp implementation is the ultimately inevitable reality of all adaptive filters including adaptive infinite impulse response iir filters this paper continues to examine the asymptotic convergence speed of adaptive iir filters of various structures and algorithms including the simple constant gain type and the newton type but under fp implementation a stochastic differential equation sde approach is used in the analysis such an approach not only greatly simplifies the fp analysis which is traditionally very involved algebraically but it also provides valuable information about the first order as well as the second order moments that the latter are not available using the ordinary differential equation ode approach the asymptotic convergence speed as well as the convergent values of the pertinent moments of fp errors are examined in terms of unknown system pole zero locations the adverse effects of lightly damped low frequency ldlf poles resulting from fast sampling on the local transient and convergent behavior of various structures and algorithms are analyzed and compared the new results agree with the existing ones when reduced to the finite impulse response fir case in particular the explosive behavior of pertinent error variances of newton type iir algorithms when the forgetting factor spl lambda 1 is also concluded computer simulation verifies the predicted theoretical results

excursions of adaptive algorithms via the poisson clumping heuristic the authors detail the application of the poisson clumping heuristic pch to the least mean square lms adaptive algorithm and its signed variants under certain conditions on the input and disturbance statistics the parameter estimate errors form a markov process the pch asserts that large excursions of the parameter estimates occur in clumps and that these clumps are distributed in a poisson manner with parameter lambda sub b where b is the maximum parameter error expressions are derived for each of the four algorithms in the scalar case which allows calculation of lambda sub b in a relatively straightforward manner these values are compared to simulations of the algorithms given that the results are asymptotic in b the close agreement between simulated and theoretical values is striking even for very modest b the four algorithms are then compared in terms of lambda sub b some observations are made regarding the relative performance of the four variants no single lms variant always outperforms the others suggestions are made as to how this technique might be applied in the vector case and a crucial monotonicity property is verified

a nonlinear analytical model for the quantized lms algorithm the power of two step size case presents a study of the quantization effects in the finite precision lms algorithm with power of two step sizes deterministic nonlinear recursions are presented for the mean and second moment matrix of the weight vector about the wiener weight for white gaussian data models and small algorithm step size spl mu the numerical solutions of these recursions are shown to agree very closely with the monte carlo simulations during all phases of the adaptation process design examples demonstrate the selection of the number of quantizer bits and the adaptation step size spl mu to yield a desired transient behavior and cancellation performance the results obtained indicate that previous models are too conservative in predicting the converged mse for a given number of bits

on properties of information matrices of delta operator based adaptive signal processing algorithms in this paper we study the properties of information matrices of the difference operator or the so called delta spl delta operator based algorithms for adaptive signal processing we show that the conditioning of a transformed information matrix in the spl delta domain is always better than that of the original information matrix in the conventional q domain for ill conditioned problems the results obtained in this paper give an explanation of the advantages of using delta operator algorithms for adaptive signal processing that have been developed the analysis in this paper also helps to clarify the problem about the effect of spl delta in the delta operator algorithms and justify the use of sampling interval as spl delta in most cases when the ill conditioning is caused by fast sampling of continuous time systems

a fast exact least mean square adaptive algorithm a general block formulation of the least mean square lms algorithm for adaptive filtering is presented this formulation has an exact equivalence with the original lms algorithm hence it retains its convergence properties while allowing a reduction in arithmetic complexity even for very small block lengths working with small block lengths is interesting from an implementation point of view large blocks result in large memory and large system delay and allows a significant reduction in the number of operations tradeoffs between a number of operations and a convergence rate are obtainable by applying certain approximations to a matrix involved in the algorithm hence the usual block lms appears as a special case which explains its convergence behavior according to the type of input signal correlated or uncorrelated

adaptive algorithms with nonlinear data and error functions the tools of nonlinear system theory are used to examine several common nonlinear variants of the lms algorithm and derive a persistence of excitation criterion for local exponential stability the condition is tight when the inputs are periodic and a generic counterexample is demonstrated which gives local instability for a large class of such nonlinear versions of lms specifically those which utilize a nonlinear data function the presence of a nonlinear error function is found to be relatively benign in that it does not affect the stability of the error system rather it defines the cost function the algorithm tends to minimize specific examples include the dead zone modification the cubed data nonlinearity the cubed error nonlinearity the signed regressor algorithm and a single layer version of the backpropagation algorithm

a roundoff error analysis of the lms adaptive algorithm the steady state output error of the least mean square lms adaptive algorithm due to the finite precision arithmetic of a digital processor is analyzed it is found to consist of three terms 1 the error due to the input data quantization 2 the error due to the rounding of the arithmetic operations in calculating the filter s output and 3 the error due to the deviation of the filter s coefficients from the values they take when infinite precision arithmetic is used the last term is of paricular interest because its mean squared value is inversely proportional to the adaptation step size both fixed and floating point arithmetics are examined and the expressions for the final mean square error are found to be similar the relation between the quantization error and the error that occurs when adaptation possibly ceases due to quantization is also investigated

correlation function for the weight sequence of the frequency domain adaptive filter the statistical behavior of the single complex adaptive weight in the frequency domain adaptive filter is further investigated in a previous note 8 an expression was evaluated for the expected value of the magnitude square of the weight sequence at any time in this note the covariance of the weight at different times is evaluated and in steady state shown to be geometric with the same time constant as the frequency domain adaptive filter algorithm itself using these results the reduction in the fluctuations of the weight values are computed for an exponentially decaying post algorithm filter

on weight update saturation nonlinearities in lms adaptation the effect of a saturation type nonlinearity in the weight update equation in least mean square lms adaptation is investigated for a white gaussian data model nonlinear difference equations that include the effect of a 1 e sup ax saturation type nonlinearity on the update term driving the algorithm are derived for the weight first and second moments a nonlinear difference equation for the mean norm is explicitly solved using a differential equation approximation and integration by quadratures the steady state second moment weight behavior is evaluated approximately for the nonlinearity using these results the tradeoff between the extent of weight update saturation steady state excess mean square error and rate of algorithm convergence is studied for the same steady state misadjustment error the tradeoff results are discussed

recursive least squares ladder estimation algorithms recursive least squares ladder estimation algorithms have attracted much attention recently because of their excellent convergence behavior and fast parameter tracking capability compared to gradient based algorithms we present some recently developed square root normalized exact least squares ladder form algorithms that have fewer storage requirements and lower computational requirements than the unnormalized ones a hilbert space approach to the derivations of magnitude normalized signal and gain recursions is presented the normalized forms are expected to have even better numerical properties than the unnormalized versions other normalized forms such as joint process estimators e g adaptive line enhancer and arma pole zero models will also be presented applications of these algorithms to fast or zero startup equalizers adaptive noise and echo cancellers non gaussian event detectors and inverse models for control problems are also mentioned

a straightforward and unified approach to the derivation of minimum variance deconvolution algorithms the purpose of this note is to present a straightforward and unified approach for deriving the fixed point and fixed interval recursive algorithms of minimum variance deconvolution it is different from the approaches derived before

remark on the rate of decay of higher order derivatives for solutions to the navier stokes equations in rn we present a new derivation of upper bounds for the decay of higher order derivatives of solutions to the unforced navier stokes equations in rn the method based on so called gevrey estimates also yields explicit bounds on the growth of the radius of analyticity of the solution in time moreover under the assumption that the navier stokes solution stays sufficiently close to a solution of the heat equation in the l2 norm a result known to be true for a large class of initial data lower bounds on the decay of higher order derivatives can be obtained

the beltrami spectrum for incompressible fluid flows recently v yakhot s orszag and their co workers have suggested that turbulent flows in various regions of space organize into a coherent hierarchy of weakly interacting superimposed approximate beltrami flows a mathematical framework is developed here to study organized beltrami hierarchies in a systematic fashion this framework is applied to several important classes of examples with universal beltrami hierarchies an analysis of the persistence of such beltrami hierarchies is also presented for general solutions of the navier stokes equations

comparison of the convergence of two algorithms for adaptive fir digital filters the convergence properties of two different algorithms for the updating of the coefficients of an adaptive fir digital filter are investigated and compared with one another these algorithms are the stochastic iteration algorithm and the sign algorithm in this latter algorithm a one bit gradient estimation is used which makes its implementation very simple the convergence is characterized by the residual echo variance after convergence and a parameter that indicates the speed of the convergence it is shown that the convergence of the sign algorithm can always be assured but is much slower than that of the stochastic iteration algorithm if the same variance of the residual echo is to be obtained

a nonlinear analytical model for the quantized lms algorithm the arbitrary step size case this paper extends conditional moment techniques previously developed for the study of nonlinear versions of the lms algorithm to the study of the effects of quantizers in the finite precision case deterministic nonlinear recursions are derived for the mean and second moment matrix of the weight vector about the wiener weight for white gaussian data models and small algorithm step sizes spl mu these recursions are solved numerically and shown to be in very close agreement with the monte carlo simulations during all phases of the adaptation process a design example is presented that demonstrates how the theory can be used to select the number of quantizer bits and the adaptation step size spl mu to yield a desired transient behavior and cancellation performance

statistical analysis of the single layer backpropagation algorithm i mean weight behavior the single layer backpropagation algorithm is a gradient descent method that adjusts the connection weights of a single layer perceptron to minimize the mean square error at the output it is similar to the standard least mean square al gorithm except the output of the linear combiner contains a differentiable nonlinearity in this paper we present a statis tical analysis of the mean weight behavior of the single layer backpropagation algorithm for gaussian input signals it is based on a nonlinear system identification model of the desired response which is capable of generating an arbitrary hyper plane decision boundary it is demonstrated that although the weights grow unbounded the algorithm on average quickly learns the correct hyperplane associated with the system iden tification model

improved convergence analysis of stochastic gradient adaptive filters using the sign algorithm convergence analysis of stochastic gradient adaptive filters using the sign algorithm is presented in this paper the methods of analysis currently available in literature assume that the input signals to the filter are white this restriction is removed for gaussian signals in our analysis expressions for the second moment of the coefficient vector and the steady state error power are also derived simulation results are presented and the theoretical and empirical curves show a very good match

convergence analysis of lms filters with uncorrelated gaussian data statistical analysis of the least mean squares lms adaptive algorithm with uncorrelated gaussian data is presented exact analytical expressions for the steady state mean square error mse and the performance degradation due to weight vector misadjustment are derived necessary and sufficient conditions for the convergence of the algorithm to the optimal wiener solution within a finite variance are derived it is found that the adaptive coefficient which controls the rate of convergence of the algorithm must be restricted to an interval significantly smaller than the domain commonly stated in the literature the outcome of this paper therefore places fundamental limitations on the mse performance and rate of convergence of the lms adaptive scheme

asymptotic integration of navier stokes equations with potential forces ii an explicit poincare dulac normal form abstract we study the incompressible navier stokes equations with potential body forces on the three dimensional torus we show that the normalization introduced in the paper c foias j c saut linearization and normal form of the navier stokes equations with potential forces ann inst h poincare anal non lineaire 4 1 1987 1 47 produces a poincare dulac normal form which is obtained by an explicit change of variable this change is the formal power series expansion of the inverse of the normalization map each homogeneous term of a finite degree in the series is proved to be well defined in appropriate sobolev spaces and is estimated recursively by using a family of homogeneous gauges which is suitable for estimating homogeneous polynomials in infinite dimensional spaces

transient analysis of data normalized adaptive filters this paper develops an approach to the transient analysis of adaptive filters with data normalization among other results the derivation characterizes the transient behavior of such filters in terms of a linear time invariant state space model the stability of the model then translates into the mean square stability of the adaptive filters likewise the steady state operation of the model provides information about the mean square deviation and mean square error performance of the filters in addition to deriving earlier results in a unified manner the approach leads to stability and performance results without restricting the regression data to being gaussian or white the framework is based on energy conservation arguments and does not require an explicit recursion for the covariance matrix of the weight error vector

assessment of a vorticity based solver for the navier stokes equations we investigate numerically a recently proposed vorticity based formulation of the incompressible navier stokes equations the formulation couples a velocity pressure system with a vorticity helicity system and is intended to provide a numerical scheme with enhanced accuracy and superior conservation properties for a few benchmark problems we study the performance of a finite element method for this formulation and compare it with the commonly used velocity pressure based finite element method it is shown that both steady and unsteady discrete problems in the new formulation admit simple decoupling strategies followed by the application of iterative solves to auxiliary subproblems further we compare several iterative strategies to solve the discrete problems and study the interplay between the choice of stabilization parameters in the finite element method and the efficiency of linear algebra solvers

analysis and implementation of variable step size adaptive algorithms stochastic gradient adaptive filtering algorithms using variable step sizes are investigated the variable step size algorithm improves the convergence rate while sacrificing little in steady state error expressions describing the convergence of the mean and mean squared values of the coefficients are developed and used to calculate the mean square error evolution the initial convergence rate and the steady state error are also investigated the performance of the algorithm is studied when a power of two quantizer algorithm is used and finite word length effects are considered the analytical results are verified with simulations encompassing variable applications two cmos implementations of the variable step size power of two quantizer algorithm are presented to demonstrate that the performance gains are attainable with only a modest increase in circuit complexity

on the optimum data nonlinearity in lms adaptation the effect of an arbitrary nonlinear operation on the data input to the weight update equation in the lms adaptive algorithm is investigated for a gaussian data model exact difference equations are derived for the weight first and second moments that include the effects of an arbitrary nonlinear operation on the data sequence the difference equations are used to obtain expressions for the transient behavior of the mean square error the mean square error is minimized over the choice of nonlinearity for a fixed transient behavior the best choice of nonlinearity is shown exactly to be of the form x 1 bx2 when the input data are white for an arbitrary data covariance matrix the optimum nonlinearity is shown to be linear when the product of the algorithm step size and input power is much less than unity

unconstrained frequency domain adaptive filter an unconstrained frequency domain least mean squares uflms algorithm is presented the algorithm is based on the overlap save technique used in frequency domain filtering the proposed algorithm converges to the wiener solution without the time domain constraint on the impulse response as proposed by ferrara for a large number of taps geq32 the uflms algorithm offers a significant reduction in computation another advantage is its fast convergence for highly correlated input signals the performance of the algorithm is illustrated by a simulation

adaptive frequency sampling filters we present two new structures for adaptive filters based on the idea of frequency sampling filters and gradient based estimation algorithms these filters have a finite impulse response fir and can be thought of as attempting to approximate a desired frequency response at given points on the unit circle the filters operate in real time with no batch processing of signals as is the case when using the discrete fourier transform they result in a marked reduction in dimension of the time domain problem of fitting an nth order fir transversal filter to a collection of length 2 transversal filters and further to a collection of n scalar filters the advantages of this are then discussed

performance analysis of adaptive filters equipped with the dual sign algorithm a convergence analysis for stochastic gradient adaptive filters equipped with the dual sign algorithm dsa is presented expressions for the mean and mean squared values of the coefficient misalignment vector are derived under the assumption that the input signal is gaussian the main differences between the current analysis and previous analyses are 1 the present analysis is valid for arbitrary autocorrelation matrices and 2 the analysis does not approximate the behavior for the dsa as switching between two sign algorithms a certain amount of time after the dsa is initialized a simulation example comparing the analytical results with empirical ones is presented and the two curves show excellent agreement with each other

statistical analysis of the single layer backpropagation algorithm ii mse and classification performance for pt i see ibid p 583 91 1993 the analysis of pt i is extended to the evaluation of the mean square error and the probability of correct classification it is shown that the mean square error and the corresponding performance surface are such that the single layer perceptron is prevented from correctly classifying with probability one until the weights converge at infinity

performance of minimum variance deconvolution filter recently we observed zero phase and undershoot patterns in data processed by a minimum variance deconvolution mvd filter these observations motivated a careful analysis of the mvd filter which as we demonstrate in this paper explains both the zero phase and undershoot patterns this analysis also connects the mvd filter with the well known prediction error filter 6 and berkhout s two sided least squares inverse filter 7 we show that the performance of the mvd filter depends heavily on the bandwidth of the source wavelet and signal to noise ratio and only slightly on data length

finite precision error analysis of qrd rls and star rls adaptive filters the qr decomposition based recursive least squares rls adaptive filtering qrd rls algorithm is suitable for vlsi implementation since it has good numerical properties and can be mapped onto a systolic array a new fine grain pipelinable star rls algorithm was developed the pipelined star rls algorithm pstar rls is useful for high speed applications the stability of qrd rls star rls and pstar rls has been proved but the performance of these algorithms in finite precision arithmetic has not yet been analyzed the authors determine expressions for the degradation in the performance of these algorithms due to finite precision by exploiting the steady state properties of these algorithms simple expressions are obtained that depend only on known parameters this analysis can be used to compare the algorithms and to decide the wordlength to be used in an implementation since floating or fixed point arithmetic representations may be used in practice both representations are considered the results show that the three algorithms have about the same finite precision performance with pstar rls performing better than star rls which does better than qrd rls these algorithms can be implemented with as few as 8 bits for the fractional part depending on the filter size and the forgetting factor used the theoretical expressions are found to be in good agreement with the simulation results

normalized data nonlinearities for lms adaptation properly designed nonlinearly modified lms algorithms in which various quantities in the stochastic gradient estimate are operated upon by memoryless nonlinearities have been shown to perform better than the lms algorithm in system identification type problems the authors investigate one such algorithm given by w sub k l w sub k spl mu d sub k w sub k sup t x sub k x sub k f x sub k in which the function f x sub k is a scalar function of the sum of the squares of the n elements of the input data vector x sub k this form of algorithm generalizes the so called normalized lms nlms algorithm they evaluate the expected behavior of this nonlinear algorithm for both independent input vectors and correlated gaussian input vectors assuming the system identification model by comparing the nonlinear algorithm s behavior with that of the lms algorithm they then provide a method of optimizing the form of the nonlinearity for the given input statistics in the independent input case they show that the optimum nonlinearity is a single parameter version of the nlms algorithm with an additional constant in the denominator and show that this algorithm achieves a lower excess mean square error mse than the lms algorithm with an equivalent convergence rate additionally they examine the optimum step size sequence for the optimum nonlinear algorithm and show that the resulting algorithm performs better and is less complex to implement than the optimum step size algorithm derived for another form of the nlms algorithm simulations verify the theory and the predicted performance improvements of the optimum normalized data nonlinearity algorithm

approximate realization algorithms for truncated impulse response data the objectives of this paper are 1 to present a novel derivation of an approximate realization algorithm using singular value decomposition proposed by kung 2 to explain the difference between two versions of this algorithm which have been used interchangeably in the literature and 3 to introduce a new version of this algorithm which exhibits the accuracy of the original algorithm and the theoretical guarantee of stability