efficient arx identification algorithms with full parallelism a highly parallel algorithm for arx system identification is developed it involves only scalar operations which are of the schur type and can be implemented by inner product processors of the multiply and add form a by product of the above development is a levinson type algorithm in transversal form modularity of the arrays local communication and pipelineability make them suitable for vlsi implementation

paper error propagation properties of recursive least squares adaptation algorithms the numerical properties of implementations of the recursive least squares identification algorithm are of great importance for their continuous use in various adaptive schemes here we investigate how an error that is introduced at an arbitrary point in the algorithm propagates it is shown that conventional ls algorithms including bierman s ud factorization algorithm are exponentially stable with respect to such errors i e the effect of the error decays exponentially the base of the decay is equal to the forgetting factor the same is true for fast lattice algorithms the fast least squares algorithm sometimes known as the fast kalman algorithm is however shown to be unstable with respect to such errors

numerical stability properties of a qr based fast least squares algorithm the numerical stability of a recent qr based fast least squares algorithm is established from a backward stability perspective a stability domain approach applicable to any least squares algorithm constructed from the set of reachable states in exact arithmetic is presented the error propagation question is shown to be subordinate to a backward consistency constraint which requires that the set of numerically reachable variables be contained within the stability domain associated to the algorithm this leads to a conceptually lucid approach to the numerical stability question which frees the analysis of stationary assumptions on the filtered sequences and obviates the tedious linearization methods of previous approaches moreover initialization phenomena and considerations related to poorly exciting inputs admit clear interpretations from this perspective the algorithm under study is proved in contrast to many fast algorithms to be minimal

fast adaptive algorithms for multichannel filtering and system identification fast transversal and lattice least squares algorithms for adaptive multichannel filtering and system identification are developed models with different orders for input and output channels are allowed four topics are considered multichannel fir filtering rational iir filtering arx multichannel system identification and general linear system identification possessing a certain shift invariance structure the resulting algorithms can be viewed as fast realizations of the recursive prediction error algorithm computational complexity is then reduced by an order of magnitude as compared to standard recursive least squares and stochastic gauss newton methods the proposed transversal and lattice algorithms rely on suitable order step up step down updating procedures for the computation of the kalman gain stabilizing feedback for the control of numerical errors together with long run simulations are included

least squares adaptive lattice and transversal filters a unified geometric theory a unified theory is presented to characterize least squares adaptive filters in either lattice or transversal filter form for nonstationary processes the derivations are based upon a geometric formulation of least squares estimation and on the concept of displacement rank a few basic geometric relations are shown to underlie the various algorithms insights into the fundamental concepts that unify lattice and transversal filter approaches to least squares adaptive filters are also given the general results are illustrated by applications to the so called pre windowed and growing memory covariance formulations of the deterministic least squares problem

adaptive inference for the two sample scale problem some adaptive tests are proposed a for the general two sample scale problem and b for the restricted two sample scale problem in which the random variables are known to be right skewed and also positive valued these are based on either the ratio of the adaptively trimmed sample variances or adaptive rank tests of scale whose quantiles are estimated by means of the bootstrap simulations show that for practical sample sizes these tests tend to perform better than their competitors the methodology is illustrated by an application to a problem considered by nair regarding the breakdown times of an insulating fluid under elevated voltage stress

plots and tests for goodness of fit with randomly censored data summary in this paper we consider nonparametric procedures for assessing goodness of fit when the data may be subject to random censoring we introduce two classes of large sample tests based respectively on the maximum and average weighted difference between the specified survival function and the kaplan meier estimate of the true survival function the procedures in the first class are inverted to obtain simultaneous confidence bands for the survival and cumulative hazard functions these bands provide simultaneous bands associated with the percentage p p quantile q q and hazard plots for censored data some key word8 hazard function nonparametric procedure probability plot shift function survival function

the block processing ftf adaptive algorithm a new block type adaptive filtering algorithm is presented this new block adaptive filter differs from the frequency domain block adaptive filters of ferrara 1980 and of clark mitra and parker 1980 in that the new method applies a deterministic time domain least squares criteria within each of the data blocks information is carried from block to block via a weighted initial condition this new block fast transversal filters bftf algorithm is a numerically stable algorithm and can also be used to perform efficient least squares system identification on any one data block in which case it shows a moderate computational advantage over the previous most efficient single data block algorithms of morf et al 1977 of marple 1981 and of kalouptsidis manolakis and carayannis 1984 1985 mechanisms for tracking and varying block length from block to block are also presented and evaluated finally we indicate how the new algorithm could be pipelined for maximum throughput with delay proportional to the number of parameters after computation of the sample correlation lags

an application of the kantorovich theoremto nonlinear finite element analysis finite element solutions of strongly nonlinear elliptic boundary value problems are considered in this paper using the kantorovich theorem we show that if the frechet derivative of the nonlinear operator defined by the boundary value problem is an isomorphism at an exact solution then there exists a locally unique finite element solution near the exact solution moreover several a priori error estimates are obtained

floating point arithmetic and digital filters the roundoff noise properties of floating point digital filters are examined to make the analysis tractable a high level model to deal with the errors in the inner product operation is developed this model establishes a broad connection between coefficient sensitivity and roundoff noise along with the model an efficient procedure to keep track of the addition scheme used in the inner product and to compute the statistics of the errors is introduced a systematic procedure based on the model is then developed to derive general expressions for the roundoff noise of fir direct form iir and state space filters the expressions in the context of state space filters are explored in some detail optimality issues are considered and it is shown that when double precision accumulation is used the optimal filters are similar in nature to those derived in the context of fixed point arithmetic with the essential difference that they also do depend on the spectrum of the input signal optimality with respect to addition schemes and second order filters are also examined in some detail

fast rls type algorithm for unbiased equation error adaptive iir filtering based on approximate inverse power iteration in this paper a fast approximate inverse power aip iteration is applied to compute recursively the total least squares tls solution for unbiased equation error adaptive infinite impulse response iir filtering which is established by approximating the well known inverse power iteration with galerkin method the aip algorithm is based on an interesting modification of the inverse power iteration in which the first entry of the parameter vector is constrained to the negative one the distinctive feature of the proposed algorithm lies in its high computational efficiency which is characterized by efficient computation of the fast gain vector fgv adaptation of the interesting modification of the inverse power iteration and rank one update of the augmented correlation matrix the shift structure of the input data vector is exploited to develop a fast algorithm for computing the gain vector this fgv algorithm can be implemented at a numerical cost lower than the well known fast kalman algorithm since the first entry of the parameter vector has been fixed as the negative one and the weight vector is updated along the input data vector a very efficient aip algorithm is obtained by using the fgv algorithm the proposed aip algorithm is of computational complexity o l per iteration moreover with no need to use the well known matrix inversion lemma the aip algorithm has another attractive feature of numerical stability the proposed algorithm is shown to have global convergence simulation examples are included to demonstrate the effectiveness of the proposed aip algorithm

a priori error estimates of finite element solutions of parametrized strongly nonlinear boundary value problems abstract nonlinear boundary value problems with parameters are called parametrized nonlinear boundary problems this paper studies a priori error estimates of finite element solutions of second order parametrized strongly nonlinear boundary value problems in divergence form on one dimensional bounded intervals the banach space w 0 1 is chosen in formulation of the error analysis so that the nonlinear differential operators defined by the differential equations are nonlinear fredholm operators of index 1 finite element solutions are defined in a natural way and several a priori estimates are proved on regular branches and on branches around turning points in the proofs the extended implicit function theorem due to brezzi et al 1980 plays an essential role

efficient block implementation of exact sequential least squares problems an efficient blockwise algorithm namely the block sequential least squares bsls algorithm is presented for sequentially solving ls problems in realtime the information is carried from block to block by iterating some correlation vectors in the case of successive data blocks the exactness of the bsls algorithm is achieved at approximately the same computational requirement as characterizes the nonexact bftf block fast transversal filter algorithm which is significantly less than sample by sample rls recursive least squares algorithms however the bsls cannot accommodate the case of discontinuous blocks of data which can be accommodated at the expense of a nonexact solution by the bftf it is shown that the bsls algorithm allows efficient use of the fft fast fourier transform technique to make remarkable gains in computational complexity savings additionally the bsls algorithm can provide an improved numerical stability over the existing fast rls algorithms the numerical performance is illustrated by applications to adaptive equalization and online parameter identification

error sources and error propagation in the levinson durbin algorithm it is proved that there are two types of numerical error due to finite precision in the levinson durbin algorithms an erratic and a systematic one the erratic one depends on the value the input autocorrelation accidentally takes at an iteration and essentially it affects only the results obtained at this particular recursion on the contrary the systematic numerical error increases with the information the system carries and propagates essentially throughout the algorithm it is shown that for both types of error as well as the overall one there are specific intermediate quantities calculated in the evolution of the algorithm which may serve as precise indicators of the exact number of erroneous digits with which the various quantities are computed including the parcor coefficients and the filter coefficients therefore the generated numerical error can be accurately traced

a fast recursive total least squares algorithm for adaptive fir filtering this paper proposes a new fast recursive total least squares n rtls algorithm to recursively compute the tls solution for adaptive finite impulse response fir filtering the n rtls algorithm is based on the minimization of the constrained rayleigh quotient c rq in which the last entry of the parameter vector is constrained to the negative one as analysis results on the convergence of the proposed algorithm we study the properties of the stationary points of the c rq the high computational efficiency of the new algorithm depends on the efficient computation of the fast gain vector fgv and the adaptation of the c rq since the last entry of the parameter vector in the c rq has been fixed as the negative one a minimum point of the c rq is searched only along the input data vector and a more efficient n rtls algorithm is obtained by using the fgv as compared with davila s rtls algorithms the n rtls algorithm saves the 6m number of multiplies divides and square roots mads the global convergence of the new algorithm is studied by lasalle s invariance principle the performances of the relevant algorithms are compared via simulations and the long term numerical stability of the n rtls algorithm is verified

adaptive linearly constrained inverse qrd rls beamforming algorithm for moving jammers suppression a general linearly constrained lc recursive least squares rls array beamforming algorithm based on an inverse qr decomposition is developed for suppressing moving jammers efficiently in fact by using the inverse qr decomposition recursive least squares qrd rls algorithm approach the least squares ls weight vector can be computed without back substitution and is suitable for implementation using a systolic array to achieve fast convergence and good numerical properties the merits of this new constrained algorithm are verified by evaluating the performance in terms of the learning curve to investigate the convergence property and numerical efficiency and the output signal to interference and noise ratio we show that our proposed algorithm outperforms the conventional linearly constrained lms lclms algorithm and the one using the fast linear constrained rls algorithm and its modified version

finite dimensional approximation of non linear problems 3 simple bifurcation points note univ pavia cnr ist analisi numer i 27100 pavia italy ecole polytech f 91128 palaiseau france ecole polytech ctr math appl cnrs era 747 f 91128 palaiseau france brezzi f univ pavia ist mat appl i 27100 pavia italy isi document delivery no mn015times cited 75cited reference count 20 reference asn article 1981 001 record created on 2006 08 24 modified on 2017 05 12

improvement of the fast recursive least squares algorithms via normalization a comparative study this paper deals with the derivation and the properties of fast optimal least squares algorithms and particularly with their normalization it is shown how the well known fast kalman algorithm written in the most general form can be normalized through a purely algebraic point of view leading to the normalized least squares transversal filter derived by cioffi kailath and lev ari from the geometric approach an improved form of the algorithm is presented the different algorithms have been compared from a practical point of view as regards their convergence initialization procedures complexity and numerical properties normalized transversal algorithms are shown to be interesting because of their nice structured form simplicity of conception and improved good numerical behavior

fixed point roundoff error analysis of the exponentially windowed rls algorithm for time varying systems a fixed point roundoff error analysis of the exponentially windowed rls algorithm is presented it is shown that a tradeoff exists in the choice of the forgetting factor in order to reduce the sensitivity of the algorithm to additive noise must be chosen close to one on the other hand the roundoff error increases as lambda rightarrow 1 it is shown that the algorithm is stabilized with lambda rightarrow 1 to derive the theoretical results it is assumed that the input signal is a white gaussian random process finally simulations are presented which confirm the theoretical findings of the paper

analytical finite precision results for burg s algorithm and the autocorrelation method for linear prediction this paper derives new analytical results for quantifying the performance degradation induced by finite precision arithmetic upon the autocorrelation method and burg s algorithm for linear prediction the main results are analytical derivations of the resulting error in the reflection coefficient computation due to finite precision fp arithmetic implementation from analysis of the second order system it is shown that for the autocorrelation method fp effects are more dominant for signals having spectral poles which are near the unit circle and the real axis in the z plane in this circumstance analytical results show that the autocorrelation method indeed has severe degradation due to fp implementation however analytical results show that burg s algorithm has fp properties superior to the autocorrelation method finally experimental results are presented which show very close agreement between the analytical derivations and experimental results

a fast recursive least squares adaptive second order volterra filter and its performance analysis a fast recursive least squares rls adaptive nonlinear filter modeled using a second order volterra series expansion is presented the structure uses the ideas of fast rls multichannel filters and has a computational complexity of o n sup 3 multiplications where n 1 represents the memory span in number of samples of the nonlinear system model a theoretical performance analysis of its steady state behaviour in both stationary and nonstationary environments is presented the analysis shows that when the input is zero mean and gaussian distributed and the adaptive filter is operating in a stationary environment the steady state excess mean squared error due to the coefficient noise vector is independent of the statistics of the input signal the results of several simulation experiments show that the filter performs well in a variety of situations the steady state behaviour predicted by the analysis is in very good agreement with the experimental results

on the behavior of some estimators from probability plots abstract probability plots are popular graphical methods used to assess distributional assumptions under a location scale model the plot tends to lie on a straight line a common practice in this situation is to fit a line through the plot and use the intercept and slope of the fitted line as estimates of the location and scale parameters what are the properties of these estimators estimators from weighted least squares lines are considered and their asymptotic finite sample robustness and optimality properties are discussed included among these are the ordinary least squares estimators and estimators from least squares lines fitted after trimming or winsorizing some of the extreme order statistics

round off error propagation in four generally applicable recursive least squares estimation schemes abstract the numerical robustness of four generally applicable recursive least squares estimation schemes is analysed by means of a theoretical round off propagation study this study highlights a number of practical interesting insights into the widely used recursive least squares schemes these insights have been confirmed in an experimental verification study

on the sensitivity of transversal rls algorithms to random perturbations in the filter coefficients the author investigates the sensitivity of the rls algorithm to random perturbations about the optimum filter coefficients expressions are derived for the mean and variance of the deviation from the optimum error power for the prewindowed growing memory and the experimentally windowed lambda

on the numerical stability and accuracy of the conventional recursive least squares algorithm we study the nonlinear round off error accumulation system of the conventional recursive least squares algorithm and we derive bounds for the relative precision of the computations in terms of the conditioning of the problem and the exponential forgetting factor which guarantee the numerical stability of the finite precision implementation of the algorithm the positive definiteness of the finite precision inverse data covariance matrix is also guaranteed bounds for the accumulated round off errors in the inverse data covariance matrix are also derived in our simulations the measured accumulated roundoffs satisfied in steady state the analytically predicted bounds we consider the phenomenon of explosive divergence using a simplified approach we identify the situations that are likely to lead to this phenomenon simulations confirm our findings

on the discretization error of parametrized nonlinear equations many applications lead to nonlinear parameter dependent equations h y t y 0 where h y times t to y y 0 in operatorname rge h and the state space y is infinite dimensional while the parameter space t has finite dimension the case dim t 1 is of special interest in connection with continuation methods for this case a general theory is developed which provides for the existence of solution paths of a rather general class of such equations and of their finite dimensional approximations and which allows for an assessment of the error between these paths a principal tool in this analysis is the theory of nonlinear fredholm operators the results cover a more general class of operators than the mildly nonlinear mappings to which other approaches appear to be restricted

fast approximate inverse power iteration algorithm for adaptive total least squares fir filtering the presence of contaminating noises at both the input and the output of an finite impulse response fir system constitutes a major impediment to unbiased parameter estimation the total least squares tls method is known to be effective in achieving unbiased estimation in this correspondence we develop a fast recursive algorithm with a view to finding the tls solution for adaptive fir filtering given the fact that the tls solution is obtainable via inverse power iteration we introduce a novel but approximate inverse power iteration in combination with galerkin method so that the tls solution can be updated adaptively at a lower computational cost we also take advantage of the regular form of the tls solution to constrain the last element of the filter parameter vector to the negative one we further reduce the computational complexity of the developed algorithm by making efficient computation of the fast gain vector defined in and using rank one update of the augmented autocorrelation matrix the developed algorithm saves seven m mad s number of multiplies divides and square roots when compared with the recursive tls algorithm in moreover unlike the algorithms given in and the developed algorithm does not deal with the solution to a one variable quadratic equation and it avoids square root operation therefore it has the simpler structure and may be more easily implemented we then make a careful investigation into global convergence of the developed algorithm simulation results are provided that clearly illustrate appealing performance of the developed algorithm including its good long term numerical stability

brief parameter identification with derivative shift operator parametrization a way of parametrizing linear systems using an operator which combines the two commonly used operators the shift q and the delta d operators is proposed it is shown that the proposed parametrization improves the numerical property of parameter identification process by using the proposed operator the condition number of the information matrix in the identification algorithm is significantly lower than that using the q or d operator it is also shown that accurate and fast converging parameter identification can be achieved by using the new operator at fast sampling rates especially for systems with relatively wide bandwidth the numerical superiority of the new parametrization is demonstrated by simulation results

a structural view of asymptotic convergence speed of adaptive iir filtering algorithms i infinite precision implementation for pt i see ibid vol 41 no 4 p 1493 1517 1993 finite precision fp implementation is the ultimately inevitable reality of all adaptive filters including adaptive infinite impulse response iir filters this paper continues to examine the asymptotic convergence speed of adaptive iir filters of various structures and algorithms including the simple constant gain type and the newton type but under fp implementation a stochastic differential equation sde approach is used in the analysis such an approach not only greatly simplifies the fp analysis which is traditionally very involved algebraically but it also provides valuable information about the first order as well as the second order moments that the latter are not available using the ordinary differential equation ode approach the asymptotic convergence speed as well as the convergent values of the pertinent moments of fp errors are examined in terms of unknown system pole zero locations the adverse effects of lightly damped low frequency ldlf poles resulting from fast sampling on the local transient and convergent behavior of various structures and algorithms are analyzed and compared the new results agree with the existing ones when reduced to the finite impulse response fir case in particular the explosive behavior of pertinent error variances of newton type iir algorithms when the forgetting factor spl lambda 1 is also concluded computer simulation verifies the predicted theoretical results

precise finite element error analysis by yamamoto s explicit inversion formula for tridiagonal matrices an extension of babuska osborn s theorems in this paper we apply yamamoto s explicit inversion formula for tridiagonal matrices to error analysis of the piecewise linear finite element method for two point boundary value problems using the formula we extend the theorems by babuska osborn given in can a finite element method perform arbitrarily badly math comp 69 2000 443 462 to a general class of two point boundary value problems

a unified treatment of fast algorithms for identification we propose a unified description of several so called fast algorithms the projection operator technique is used to derive least squares ladder or lattice algorithms in the filter and the predictor forms this technique usually associated with orthogonal projection operations is extended to oblique projections the first application is an easy derivation of the exact instrumental variable ladder algorithm which to our knowledge has not previously been described in the literature a second application is a new derivation based on the operator technique of an algorithm for the fast calculation of gain matrices for recursive estimation schemes

finite dimensional approximation of nonlinear problems part ii limit points note univ paris 6 f 75230 paris 05 france ecole polytech ctr math appl cnrs equipe rech 747 f 91128 palaiseau france univ pavia cnr ist anal numer i 27100 pavia italy brezzi f univ pavia cnr ist math appl i 27100 pavia italy isi document delivery no lr761times cited 54cited reference count 13 reference asn article 1981 002doi 10 1007 bf01396184 record created on 2006 08 24 modified on 2017 05 12

local boundary value problems for the error in fe approximation of non linear diffusion systems in this work we investigate the a posteriori error estimation for a class of non linear multicomponent diffusion operators which includes the stefan maxwell equations the local error indicators for the global error are based on local boundary value problems which are chosen to approximate either the global residual of the finite element approximation or the global linearized error equation using representative numerical examples it is shown that the error indicators based on the latter approach are more accurate for estimating the global error for this problem class as the problem becomes more non linear and can even produce better adaptive mesh refinement amr in addition we propose a new local error indicator for the error in output functionals that is accurate inexpensive to compute and is suitable for amr as demonstrated by numerical examples

a floating point arithmetic error analysis of direct and indirect coefficient updating techniques for adaptive lattice filters the ways in which finite precision arithmetic effects can deleteriously manifest themselves in both the stochastic gradient and the recursive least squares adaptive lattice filters are discussed closed form expressions are derived for the steady state variance of the accumulated arithmetic error in a single adaptive lattice coefficient using a floating point stochastic arithmetic error analysis the analytical results show that the performance of adaptive lattice filters using a direct updating computational form is less sensitive to finite precision effects than that of adaptive lattice filters using an indirect updating computational form in addition a method for reducing the self generated noise is presented experimental results obtained on a 32 b floating point hardware implementation of the adaptive lattice filters and with computer simulations are included to verify the analytical results describing the effects of finite precision on adaptive lattice filters

an augmented ud identification algorithm an augmented ud identification audi algorithm for system identification is developed by rearranging the data vectors and augmenting the covariance matrix of bierman s ud factorization algorithm the structure of the augmented information covariance matrix is particularly easy to interpret and it is shown that the audi algorithm is a direct extension of the familiar recursive least squares rls algorithm the proposed algorithm permits simultaneous identification of the model parameters plus loss functions for all orders from 1 to n at each time step with approximately the same calculation effort as th order rls this provides a basis for simultaneous model order and parameter identification so that problems due to over and under estimation of model can be avoided based on its least squares properties numerical robustness theoretical basis and the fact that it simultaneously estimates multiple models the proposed audi algorithm is recommended for use in place of rls and bierman s ud factorizatio

pure time updates for least squares lattice algorithms this note presents some new time update formulas for certain types of lattice algorithms used in autoregressive modeling of stationary time series the new formulas enable the computation of the autoregressive coefficients in a number of operations per time step proportional to the model order

error accumulation effects for the a posteriori rlsl prediction filter this paper presents a numerical analysis of the a posteriori recursive least squares lattice filter with indirect updating the important problem of finite precision effects in adaptive rls lattice filters is addressed for this particular filter algorithm but the technique is applicable to many other versions new recursions for the filter residuals and the filter powers forward and backward are derived which describe the effects of arithmetic errors that originate at one stage m and are passed to the next stage m 1 these recursions permit an arithmetic error analysis at the zeroth filter stage to have meaning at all future filter stages effects from machine precision spl eta are linked to the forgetting factor spl lambda and time n to derive bounds for arithmetic error growth at the zeroth filter stage these results provide an explicit upper bound on the forgetting factor that ensures acceptable error propagation additionally we offer a computationally inexpensive way to monitor arithmetic error effects during the normal execution of the filter algorithm through a running error analysis

exponential convergence of adaptive identification and control algorithms output and equation error adaptive identification algorithms are shown to be exponentially convergent under a deterministic or stochastic persistently exciting or spanning condition on the system inputs together with several other standard conditions an adaptive control algorithm is shown to be exponentially convergent under a deterministic or stochastic persistently exciting condition on the reference trajectory together with some standard conditions

finite element analysis for parametrized nonlinear equations around turning points nonlinear equations with parameters are called parametrized nonlinear equations in this paper a priori error estimates of finite element solutions of parametrized nonlinear elliptic equations on branches around turning points are considered existence of a finite element solution branch is shown under suitable conditions on an exact solution branch around a turning point also some error estimates of distance between exact and finite element solution branches are given it is shown that error of a parameter is much smaller than that of functions approximation of nondegenerate turning points is also considered we show that if a turning point is nondegenerate there exists a locally unique finite element nondegenerate turning point at a nondegenerate turning point an elaborate error estimate of the parameter is proved

fixed point error analysis of the normalized ladder algorithm an attempt is made to analyze the fixed point error performance of the normalized ladder algorithm for autoregressive system identification assuming rounding arithmetic a preliminary simulation study of this algorithm has shown that the bias in the estimated reflection coefficients is much more predominant than the variance of the error in the estimate the study therefore is directed to find a model for predicting the bias in the estimated reflection coefficients the analysis shows that the roundoff errors associated with the square root operations in one of the algorithm equations are mainly responsible for the bias in the estimated reflection coefficients these errors arise because of the normalization procedure that makes the quantities under the square root operations very close to one two main results are presented in the paper 1 a simplified theoretical expression for predicting the average bias in the estimated reflection coefficients at any stage is derived 2 a recursive relation for the average error arising from the finite precision arithmetic in the squared residuals is derived this relation illustrates how the errors made in one stage affect the errors in the succeeding stages simulations are performed to check the theoretical models the experimental results agree very closely with the theoretical predictions

a note on the error propagation analysis of recursive least squares algorithms the article comments on the error propagation analysis of recursive least squares algorithms presented by ljung and ljung 1985 verhaegen 1989 and haykin 1991 in particular it corrects some incorrect statements about the error propagation given in the literature it considers the problem of spatial adaptive filtering using the rls minimization

finite dimensional approximation of nonlinear problems in the first two papers of this series 4 5 we have studied a general method of approximation of nonsingular solutions and simple limit points of nonlinear equations in a banach space we derive here general approximation results of the branches of solutions in the neighborhood of a simple bifurcation point the abstract theory is applied to the galerkin approximation of nonlinear variational problems and to a mixed finite element approximation of the von karman equations

sliding window order recursive least squares algorithms order recursive least squares orls algorithms employing a sliding window sw are presented the authors demonstrate that standard architectures that are well known for growing memory orls estimation e g triangular array lattice and multichannel lattice also apply to sliding window orls estimation a specific sw orls algorithm is the combination of two independent attributes its global architecture and its local cell implementation various forms of local cell implementation based on efficient time recursions of time varying coefficients are discussed in particular the authors show that time and order updates of any order recursive sliding window least squares algorithm can be realized solely in terms of 3 spl times 3 hyperbolic householder transformations hht finally the authors present two hht based algorithms the hht triangular array algorithm and the hht lattice algorithm

analysis of finite precision effects on a recursive set membership parameter estimation algorithm analysis of error propagation in an optimal bounding ellipsoid obe algorithm is performed which shows that the errors in the estimates due to an initial perturbation are bounded simulation results demonstrate that the obe algorithm can perform better than the conventional recursive least squares rls in small wordlength environments the analysis presented could also be applied for the finite precision analysis of recursive weighted least squares algorithms

a robust parallelizable o m a posteriori recursive least squares algorithm for efficient adaptive filtering this article presents a new recursive least squares rls adaptive algorithm the proposed computational scheme uses a finite window by means of a lemma for the system matrix inversion that is for the first time stated and proven here the new algorithm has excellent tracking capabilities moreover its particular structure allows for stabilization by means of a quite simple method its stabilized version performs very well not only for a white noise input but also for nonstationary inputs as well it is shown to follow music speech environmental noise etc with particularly good tracking properties the new algorithm can be parallelized via a simple technique its parallel form is very fast when implemented with four processors

a novel approach for stabilizing recursive least squares filters a novel approach for stabilizing recursive least squares rls filters is presented the approach relies on a detailed fixed point analysis which provides two important benefits the analysis reveals a bias in the error propagation mechanism providing an analytical basis for instability problems the analysis then indicates which specific roundoff errors are causing instability these roundoff errors are then biased in such a way that the overall filter is biased towards stable performance experimental results indicate that stability can be achieved with negligible loss in least squares performance

a robust recursive least squares algorithm a new algorithm is developed which guarantees the normalized bias in the weight vector due to persistent and bounded data perturbations to be bounded robustness analysis for this algorithm has been presented an approximate recursive implementation is also proposed it is termed as the robust recursive least squares rrls algorithm since it resembles the rls algorithm in its structure and is robust with respect to persistent bounded data perturbation simulation results are presented to illustrate the efficacy of the rrls algorithm

new levinson schur and lattice type algorithms for linear phase filtering the ls estimate of the impulse response of fir filters with linear phase is known to be given by a structured linear set of equations three new algorithms are presented for the efficient solution of such systems one is the unwindowed extension of a previously derived levinson type structurally symmetric algorithm the second is a novel exact least squares lattice algorithm for time recursive processing the third is a schur type structurally symmetric algorithm with high degree of parallelism the new structures have considerably lower computational complexity and more parsimonious parametrization compared to previously derived algorithms this is because in contrast to the previously derived algorithms the new algorithms are developed so as to respect the symmetry which is intrinsic in the linear phase problem

a hybrid least squares qr lattice algorithm using a priori errors this paper presents a new minimal and backward stable qr lsl algorithm obtained through the proper interpretation of the system matrix that describes the adaptation and filtering operations of qr rls algorithms the new algorithm is based on a priori prediction errors normalized by the a posteriori prediction error energy as suggested by the interpretation of the system matrix and uses the fact that the latter quantities can be computed via a lattice structure backward consistency and backward stability become guaranteed under simple numerical conventions in contrast with the known a posteriori qr lsl algorithm the new algorithm present fewer numerical complexity and backward consistency is guaranteed without the constraint of passive rotations in the recursive lattice section furthermore reordering of some operations results in a version with identical numerical behavior and inherent parallelism that can be exploited for fast implementations both a priori and a posteriori qr lsl algorithms are compared by means of simulations for small mantissa wordlengths and forgetting factors spl lambda not too close to 1 the proposed algorithm performs better due to dispensing with passive rotations for forgetting factors very close to one and small wordlengths both algorithms are sensitive to the accuracy of some well identified computations

transition to weak generalized synchrony in chaotically driven flows we study regimes of strong and weak generalized synchronization in chaotically forced nonlinear flows the transition between these dynamical states can occur via a number of different routes and here we examine the onset of weak generalized synchrony through intermittency and blowout bifurcations the quantitative characterization of this dynamical transition is facilitated by measures that have been developed for the study of strange nonchaotic motion weak and strong generalized synchronous motion show contrasting sensitivity to parametric variation and have distinct distributions of finite time lyapunov exponents

fast recursive least squares transversal filters for adaptive filtering fast transversal filter ftf implementations of recursive least squares rls adaptive filtering algorithms are presented in this paper substantial improvements in transient behavior in comparison to stochastic gradient or lms adaptive algorithms are efficiently achieved by the presented algorithms the true not approximate solution of the rls problem is always obtained by the ftf algorithms even during the critical initialization period first n iterations of the adaptive filter this true solution is recursively calculated at a relatively modest increase in computational requirements in comparison to stochastic gradient algorithms factor of 1 6 to 3 5 depending upon application additionally the fast transversal filter algorithms are shown to offer substantial reductions in computational requirements relative to existing fast rls algorithms such as the fast kalman algorithms of morf ljung and falconer 1976 and the fast ladder lattice algorithms of morf and lee 1977 1981 they are further shown to attain steady state unnormalized or improve upon first n initialization steps the very low computational requirements of the efficient rls solutions of carayannis manolakis and kalouptsidis 1983 finally several efficient procedures are presented by which to ensure the numerical stability of the transversal filter algorithms including the incorporation of soft constraints into the performance criteria internal bounding and rescuing procedures and dynamic range increasing square root normalized variations of the transversal filters

a class of mixed transversal and ladder adaptive filters with pure order updates a class of exact recursive least squares rls adaptive filter structures with pure order updates called mixed transversal and ladder mital filters is presented in which the propagation of numerical errors along in time is limited to at most m sampling periods so the long term numerical stability can be guaranteed the adaptive structures can be treated either as transversal forms or as ladder forms to meet the requirements of different applications the outstanding numerical performances of the adaptive filters are illustrated by computer simulations

statistical analysis of initialization methods for rls adaptive filters theoretical analysis is used to evaluate the mean and second moment properties of recursive least squares algorithms incorporating the fast exact initialization and soft constrained initialization methods during the initialization period it is shown that the weight vector mean and covariance produced by fast exact initialization are undefined for this period theoretical results are derived for soft constrained initialization that show that the weight vector mean and covariance are finite and expressions are given for these quantities simulations for various cases are presented to support the accuracy of these theoretical results

yamamoto s principle and its applications to precise finite element error analysis suppose that we discretize an elliptic boundary value problem and obtain a linear equation ax f in many case the inverse matrix a 1 is closely related to the green function of the original boundary value problem this fact is called yamamoto s principle in this paper using yamamoto s principle we develop a precise error analysis of the piecewise linear finite element method for two point boundary value problems with discontinuous and not necessarily positive coefficient functions we show that precise error estimations similar to known error bounds are obtained even in this case

solution manifolds and submanifolds of parametrized equations and their discretization errors the paper concerns solution manifolds of nonlinear parameterdependent equations 1 f u y0 involving a fredholm operatorf between infinite dimensional banach spacesx z andy and a finitedimensional parameter space differntial geometric ideas are used to discuss the connection between augmented equations and certain onedimensional submanifolds produced by numerical path tracing procedures then for arbitrary finite dimension of estimates of the error between the solution manifold of 1 and its discretizations are developed these estimates are shown to be applicable to rather general nonlinear boundaryvalue problems for partial differential equations

error bound method and its application to the lms algorithm a method to determine a bound on the error performance of an adaptive filter due to roundoff effects is described the method converts the analysis of a recursive algorithm into two much simpler sub problems convergence and momentary error to apply the method the input data has to be bounded by classifying convergence into different categories according to their rates it is observed that adaptive filtering algorithms that belong to a particular class share similar behavior due to roundoff error or other perturbation effects the merit of the method is its simplicity and general applicability based on this method a sufficient condition for the numerical stability of an adaptive filter is derived application of the method to the least mean square lms algorithm is described the analysis may also be generalized to include other perturbation effects

modular and numerically stable fast transversal filters for multichannel and multiexperiment rls the authors present scalar implementations of multichannel and multiexperiment fast recursive least squares algorithms in transversal filter form known as fast transversal filter ftf algorithms by processing the different channels and or experiments one at a time the multichannel and or multiexperiment algorithm decomposes into a set of intertwined single channel single experiment algorithms for multichannel algorithms the general case of possibly different filter orders in different channels is handled geometrically this modular decomposition approach corresponds to a gram schmidt orthogonalization of multiple error vectors algebraically this technique corresponds to matrix triangularization of error covariance matrices and converts matrix operations into a regular set of scalar operations modular algorithm structures that are amenable to vlsi implementation on arrays of parallel processors naturally follow from the present approach numerically the resulting algorithm benefits from the advantages of triangularization techniques in block processing

fast adaptive rls algorithms a generalized inverse approach and analysis a generalized inverse approach is used to derive two fast adaptive recursive least squares rls algorithms and an exact and stable initialization algorithm for the prewindowed signal case the partitioning techniques are directly applied to the signal matrix rather than the covariance matrix the simulations show that the method has better numerical properties than existing fast algorithms especially when the kalman gain is concerned interesting relations among the variables in the fast rls algorithms are provided these relations can be used to derive other variations of fast rls algorithms they can also be used to develop new rescue schemes finally a numerical analysis illustrates some pitfalls of the fast rls algorithms

on line parameter identification algorithms based on householder transformation a fast online least squares algorithm for both order determination and parameter identification of linear single variable dynamic systems is introduced an exponential weighting scheme is used to place heavier emphasis on the more recent data in the case of a time varying system this algorithm is derived on the basis of an orthogonal transformation the householder transformation rather than the matrix pseudoinverse in the solution of a normal equation so as to avoid worsening of the ill conditioning which occurs with most present online algorithms for parameter estimation from noisy measurements in complex stochastic environments a fast online generalized least squares algorithm a fast online extensive matrix algorithm and a fast online maximum likelihood algorithm are developed according to the proposed fast online least squares algorithm these algorithms can also estimate the order simultaneously with the parameters of a system to be identified

finite precision error analysis of qrd rls and star rls adaptive filters the qr decomposition based recursive least squares rls adaptive filtering qrd rls algorithm is suitable for vlsi implementation since it has good numerical properties and can be mapped onto a systolic array a new fine grain pipelinable star rls algorithm was developed the pipelined star rls algorithm pstar rls is useful for high speed applications the stability of qrd rls star rls and pstar rls has been proved but the performance of these algorithms in finite precision arithmetic has not yet been analyzed the authors determine expressions for the degradation in the performance of these algorithms due to finite precision by exploiting the steady state properties of these algorithms simple expressions are obtained that depend only on known parameters this analysis can be used to compare the algorithms and to decide the wordlength to be used in an implementation since floating or fixed point arithmetic representations may be used in practice both representations are considered the results show that the three algorithms have about the same finite precision performance with pstar rls performing better than star rls which does better than qrd rls these algorithms can be implemented with as few as 8 bits for the fractional part depending on the filter size and the forgetting factor used the theoretical expressions are found to be in good agreement with the simulation results

joint time delay estimation and adaptive recursive least squares filtering a general estimation model is defined in which two observations are available a noisy and a noise filtered and delayed version of the transmitted signal the delay and the filter must be estimated the joint estimator is composed of an adaptive delay element operating in conjunction with an adaptive transversal filter the delay is updated using a form of derivative with respect to the delay of the sum of squared errors the adaptive delay is limited to integer values and is defined as the lag the lag value is computed and updated so that the optimum least squares solution is attained the joint algorithm is obtained by combining the lag update relations with a version of the fast transversal filter rls algorithm simulations show that both stationary and time varying delays are effectively tracked and that the adaptive filter properly estimates the reference filter impulse response

powerful two sample tests based on the likelihood ratio a new approach to constructing nonparametric tests for the general two sample problem is proposed this approach not only generates traditional tests including the two sample kolmogorov smirnov cramer von mises and anderson darling tests but also produces new powerful tests based on the likelihood ratio although conventional two sample tests are sensitive to the difference in location most of them lack power to detect changes in scale and shape the new tests are location scale and shape sensitive so they are robust against variation in distribution

graphical estimators from probability plots with right censored data probability plots are popular graphical tools used by reliability engineers and other practitioners for assessing parametric distributional assumptions they are particularly well suited for location scale families or those that can be transformed to such families when the plot indicates a reasonable conformity to the assumed family it is common to estimate the underlying location and scale parameters by fitting a line through the plot this quick and easy method is especially useful with censored data indeed the current version of a popular statistical software package uses this as the default estimation method in this article we investigate the properties of graphical estimators with multiply right censored data and compare their performance with that of maximum likelihood estimators large sample results on consistency asymptotic normality and asymptotic variance expressions are obtained small sample properties are studied through simulation for selected distributions and censoring patterns th

vector channel lattice filters and identification of flexible structures a novel derivative of a least squares lattice filter is presented and applied to the identification of flexible structures the vector channel lattice given derived in an infinite dimensional history space without matrix manipulations or geometric arguments can constrain the autoregressive ar coefficients for several outputs to be the same numerical results for a simulated flexible structure compare the vector channel lattice to the standard lattice filter these results show that the frequencies and damping ratios for the most significant modes can be identified adaptively with lattice filters

efficient algorithms for adaptive capon and apes spectral estimation in this paper fast algorithms for adaptive capon and amplitude and phase estimation apes methods for spectral analysis of time varying signals are derived fast stable nonrecursive formulae are derived based on time shifting properties of the pertinent variables as a consequence efficient frequency domain recursive least squares rls based as well as fast rls based algorithms for the adaptive estimation of the power spectra are developed stability issues of the frequency domain estimators are considered and stabilization procedures are proposed the computational complexity of the proposed algorithms is lower than relevant existing methods the performance of the proposed algorithms is demonstrated through extensive simulations

fast adaptive algorithms for ar parameters estimation using higher order statistics time varying statistics in linear filtering and linear estimation problems necessitate the use of adaptive or time varying filters in the solution with the rapid availability of vast and inexpensive computation power models which are non gaussian even nonstationary are being investigated at increasing intensity statistical tools used in such investigations usually involve higher order statistics hos the classical instrumental variable iv principle has been widely used to develop adaptive algorithms for the estimation of arma processes despite the great number of iv methods developed in the literature the cumulant based procedures for pure autoregressive ar processes are almost nonexistent except lattice versions of iv algorithms this paper deals with the derivation and the properties of fast transversal algorithms hence by establishing a relationship between classical iv methods and cumulant based ar estimation problems new fast adaptive algorithms fast transversal recursive instrumental variable ftriv and generalized least mean squares glms are proposed for the estimation of ar processes the algorithms are seen to have better performance in terms of convergence speed and misadjustment even in low snr the extra computational complexity is negligible the performance of the algorithms as well as some illustrative tracking comparisons with the existing adaptive ones in the literature are verified via simulations the conditions of convergence are investigated for the glms

stabilizing the fast kalman algorithms the fast kalman algorithms are stabilized by introducing a quantity that measures the accumulation of the roundoff errors this quantity is used to correct the variables of the algorithm at every time step the correction is defined as the solution of a specific minimization problem the resulting algorithm still has the nice complexity properties of the original algorithm linear in the number of parameters to be estimated but it has a much more stable behavior

qr methods of o n complexity in adaptive parameter estimation recent attention in adaptive least squares parameter estimation has been focused on methods derived from the qr factorization owing to the fact that the qr based algorithms are much more numerically stable and accurate than the traditional pseudo inverse based algorithms also known as normal equation based algorithms even though the former is usually much slower than the latter this paper presents a fast adaptive least squares algorithm for the parameter estimation of linear and some nonlinear time varying systems the algorithm is based on householder transformations as verified by simulation results this algorithm exhibits good numerical stability and accuracy in addition the new algorithm requires computation and storage with order of o n rather than o n sup 2 where n is the number of unknown parameters to be estimated this algorithm can be easily extended to construct other kinds of algorithms such as the generalized adaptive least squares algorithm the augmented matrix algorithm and the maximum likelihood algorithm

applying numerical linear algebra techniques to analyzing algorithms in signal processing without theoretical evidence to support the use of an algorithm one cannot be certain that if it solves one problem successfully it will necessarily solve all problems successfully one can analyze an algorithm from many different settings each having the possibility for different focal points for results and insights in signal processing an analysis into the stability of an algorithm can have as its focal point stability in the sense of lyapunov or of numerical linear algebra backward forward etc unfortunately without a clear vocabulary to apply toward the focal point of interest the interpretation is ambiguous to avoid this problem a clear distinction must be made that delineates each significant contribution to the process of solving the problem from the definition of a problem to the derivation of the algorithm it requires that perturbations assumed at each juncture be described in a clear language that offers no chance for confusion we offer such a terminology and apply it to the fast transversal filter and its existing analyses an algorithm that has long been known to diverge yet has nonetheless received much interest regarding the reasons behind its divergence as well as inquiries into how it might be modified to one that enjoys more dependable and robust behavior