combined filtering and parameter estimation approximations and robustness abstract the filtering problem for a non linear discrete time model is considered that includes as a special case the problem of combined filtering and parameter estimation for linear models the filter derived in the paper is shown to be robust in a specific sense with respect to small variations in the a priori distributions in the model

sur la dimension de krull des anneaux noetheriens resume nous revisitons certains resultats importants de la theorie de la dimension sur les anneaux noetheriens en particulier nous arrivons au resultat suivant pour tout element x d un anneau noetherien a on pose l x kdim x n a et q x kdim a a x alors kdim a max l x q x l x q x ou est le symbole de kronecker cette formule implique par exemple le theoreme de l ideal principal de krull et sa formulation duale

model reference adaptive control of time varying and stochastic systems the adaptive control of general delay linear time varying systems is addressed a simple model reference adaptive control law is devised it does not require a priori knowledge of the sign of the high frequency gain this control law coupled with the parameter estimation equations allows a simple representation of the closed loop system this greatly eases the stability and performance analysis and has potential for generalization to adaptive pole placement and other control laws suitable for nonminimum phase systems for the kalman filter parameter estimator coupled with this control law stability is obtained without persistence of excitation or knowledge of the sign of the high frequency gain performance results for the extended least squares algorithm are described

parameter estimation problems with singular information matrices the case of a singular fisher information matrix fim represents a significant complication for the theory of the cramer rao lower bound crb that is usually handled by resorting to the pseudoinverse of the fisher matrix we take a different approach in which the crb is derived as the solution to an unconstrained quadratic maximization problem which enables us to handle the singular case in a simple yet rigorous manner when the fisher matrix is singular except under unusual circumstances any estimator having the specified bias derivatives that figure in the crb must have infinite variance

exploring estimator bias variance tradeoffs using the uniform cr bound we introduce a plane which we call the delta sigma plane that is indexed by the norm of the estimator bias gradient and the variance of the estimator the norm of the bias gradient is related to the maximum variation in the estimator bias function over a neighborhood of parameter space using a uniform cramer rao cr bound on estimator variance a delta sigma tradeoff curve is specified that defines an unachievable region of the delta sigma plane for a specified statistical model in order to place an estimator on this plane for comparison with the delta sigma tradeoff curve the estimator variance bias gradient and bias gradient norm must be evaluated we present a simple and accurate method for experimentally determining the bias gradient norm based on applying a bootstrap estimator to a sample mean constructed from the gradient of the log likelihood we demonstrate the methods developed in this paper for linear gaussian and nonlinear poisson inverse problems

a simple derivation of the constrained multiple parameter cramer rao bound j d gorman and a o hero 1988 obtained a remarkable extension to the classical multiple parameter cramer rao cr lower bound that accounts for deterministic nonlinear equality constraints on the parameters the virtue of gorman and hero s result is that the constrained cr bound on all of the parameters is obtained by subtracting an easily computed nonnegative definite correction matrix from the unconstrained cr bound matrix the author presents a new simple derivation of the constrained cr bound and a new necessary condition for an estimator to satisfy the constrained cr bound with equality

stability convergence and performance of an adaptive control algorithm applied to a randomly varying system the stability and performance of a stochastic adaptive control algorithm applied to a randomly varying linear system are investigated the authors demonstrate that loss functions on the input output process converge to their expectation with respect to an invariant probability at a geometric rate and hence a form of stochastic exponential asymptotic stability is established and when the parameter variation and measurement noise are small it is shown that the performance is nearly optimal and if an excitation signal is added in the control law near consistency of the parameter estimates is obtained further results include central limit theorems and the law of large numbers of the input output and parameter processes

discrete time stochastic adaptive control this paper establishes global convergence of a stochastic adaptive control algorithm for discrete time linear systems it is shown that with probability one the algorithm will ensure the system inputs and outputs are sample mean square bounded and the conditional mean square output tracking error achieves its global minimum possible value for linear feedback control thus asymptotically the adaptive control algorithm achieves the same performance as could be achieved if the system parameters were known

bounds on achievable performance in the identification and adaptive control of time varying systems treats the identification and adaptive control of time varying linear systems for linear systems with a gauss markov parameter process a global lower bound on the mean square error is obtained which is valid for any causal parameter estimator a similar lower bound is obtained for any causal one step ahead predictor these bounds are applied to the adaptive control of time varying systems to obtain a lower bound on closed loop mean square performance for any causal control law for a specific control law mean square stability is established and through simulations it is seen that the performance nearly meets the theoretical lower bound

model structure selection for multivariable systems by cross validation methods using cross validation ideas two procedures are proposed for making a choice between different model structures used for approximate modelling of multivariable systems the procedures are derived under fairly general conditions the true system does not need to be contained in the model set model structures do not need to be nested and different criteria may be used for model estimation and validation the proposed structure selection rules are shown to be invariant to parameter scaling under certain conditions essentially requiring that the system belongs to the model set and that the maximum likelihood method is used for parameter estimation they are shown to be asymptotically equivalent to the generalized akaike structure selection criteria

adaptive control of a simple time varying system adaptive control of a first order randomly varying stochastic process is considered several authors have treated this problem using either ergodic theory for markov processes or martingale limit theorems in both cases some initial variance calculations and bounds evaluations must be performed in order to apply these techniques and establish the sample path results generally considered in adaptive control these prior calculations are performed in the more general case where the single parameter follows a random walk and for an identification algorithm requiring no a priori knowledge of noise characteristics

cramer rao bound for doa estimators under the partial relaxation framework derivation and comparison a class of computationally efficient doa estimators under the partial relaxation pr framework has recently been proposed conceptually different from conventional doa estimation methods in the literature the estimators under the pr framework rely on the non complete relaxation of the array manifold while performing a spectral search in the field of view this particular type of relaxation essentially implies a modified signal model with partial information loss due to the relaxation the information loss and its impact on the doa estimation performance have not yet been analytically quantified in the literature in this paper the information loss induced by the relaxation of the array manifold is investigated through the cramer rao bound crb the closed form expression of the crb for doa estimation under the pr model on the one hand provides insight on the information loss in the asymptotic region where the number of snapshots tends to infinity on the other hand the proposed crb characterizes the lower bound for the doa estimation performance of all pr estimators we prove that under the assumptions of gaussian source signal and noise the crb of the pr signal model is lower bounded by the conventional stochastic crb we also prove that the previously proposed weighted subspace fitting estimator under the pr framework asymptotically achieves the crb of the pr signal model furthermore it is shown that the asymptotic mean squared errors of all weighted subspace fitting estimators under the pr framework for any positive definite weighting matrix are identical

an interpretation of the moore penrose generalized inverse of a singular fisher information matrix it is proved that in a non bayesian parametric estimation problem if the fisher information matrix fim is singular unbiased estimators for the unknown parameter will not exist cramer rao bound crb a popular tool to lower bound the variances of unbiased estimators seems inapplicable in such situations in this correspondence we show that the moore penrose generalized inverse of a singular fim can be interpreted as the crb corresponding to the minimum variance among all choices of minimum constraint functions this result ensures the logical validity of applying the moore penrose generalized inverse of an fim as the covariance lower bound when the fim is singular furthermore the result can be applied as a performance bound on the joint design of constraint functions and unbiased estimators

a constructive notion of codimension abstract we give a point free and constructively meaningful characterization of the notion of codimension for ideals of a commutative ring which with classical logic and the axiom of choice is equivalent to the customary one this characterization is based on notions from formal topology and upon the elementary characterization of krull dimension provided earlier by coquand lombardi and roy among other things we use this notion to prove krull s principal ideal theorem in the context of constructive algebra

brief paper stochastic adaptive control using a modified least squares algorithm recent papers on stochastic adaptive control have established global convergence for algorithms using a stochastic approximation iteration however to date global convergence has not been established for algorithms incorporating a least squares iteration this paper establishes global convergence for a slightly modified least squares stochastic adaptive control algorithm it is shown that with probability one the algorithm will ensure that the system inputs and outputs are sample mean square bounded and the mean square output tracking error achieves its global minimum possible value for linear feedback control

minimum variance in biased estimation with singular fisher information matrix this paper extends the work of y c eldar ldquominimum variance in biased estimation bounds and asymptotically optimal estimators rdquo in ieee trans signal process vol 52 pp 1915 1929 jul 2004 which deals with only nonsingular fisher information matrix in order to guarantee the uniform cramer rao bound to be a finite lower bound and also to have a feasible solution to the optimization problem in the work of y c eldar it is proved that the norms of bias gradient matrices of all biased estimators must have a nonzero exact lower bound which mainly depends on the rank of the singular fisher information matrix the smaller the rank of the singular fisher information matrix is the larger the lower bound of norms of bias gradient matrices of all biased estimators is for a specific frobenius norm the exact lower bound is simply the difference between the parameter dimension and the rank of the singular fisher information matrix

asymptotic properties of the maximum likelihood estimator in autoregressive models with markov regime an autoregressive process with markov regime is an autoregressive process for which the regression function at each time point is given by a nonobservable markov chain in this paper we consider the asymptotic properties of the maximum likelihood estimator in a possibly nonstationary process of this kind for which the hidden state space is compact but not necessarily finite consistency and asymptotic normality are shown to follow from uniform exponential forgetting of the initial distribution for the hidden markov chain conditional on the observations

brief paper on the adaptive control of a class of systems with random parameters and disturbances the adaptive control of discrete time parameter linear stochastic systems with random parameters is investigated it is shown that systems whose unknown autoregressive parameters undergo bounded martingale difference disturbances may be stabilized by the application of the so called modified least squares adaptive control algorithm asymptotically the sample mean square performance criterion is equal to the one step ahead minimum variance control loss which equals the prediction error variance when the system parameters are known plus a term which is bounded by a quantity proportional to the square of the bound on the parameter disturbance this latter term may be interpreted as the increase in the prediction error variance due to the random parameter variation

nonexistence of informative unbiased estimators in singular problems in many nonparametric problems such as density estimation nonparametric regression and so on all the existing informative estimators are biased asymptotic or finite sample there has long been a suspicion that either informative unbiased estimators do not exist for such problems or they must be quite complicated in this paper we clarify the nonexistence of informative unbiased estimators in all singular problems both for fixed sample size and asymptotically this includes most problems with optimal rate of convergence slower than n1 2 we also discuss situations in regular problems where such nonexistences can occur

lower bounds for parametric estimation with constraints a chapman robbins form of the barankin bound is used to derive a multiparameter cramer rao cr type lower bound on estimator error covariance when the parameter theta in r sup n is constrained to lie in a subset of the parameter space a simple form for the constrained cr bound is obtained when the constraint set theta sub c can be expressed as a smooth functional inequality constraint it is shown that the constrained cr bound is identical to the unconstrained cr bound at the regular points of theta sub c i e where no equality constraints are active on the other hand at those points theta in theta sub c where pure equality constraints are active the full rank fisher information matrix in the unconstrained cr bound must be replaced by a rank reduced fisher information matrix obtained as a projection of the full rank fisher matrix onto the tangent hyperplane of the full rank fisher matrix onto the tangent hyperplane of the constraint set at theta a necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable and examples for which the bound is achieved are presented in addition to providing a useful generalization of the cr bound the results permit analysis of the gain in achievable mse performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization

estimating time varying parameters by the kalman filter based algorithm stability and convergence convergence and stability properties of the kalman filter based parameter estimator are established for linear stochastic time varying regression models the main features are both the variances and sample path averages of the parameter tracking error are shown to be bounded the regression vector includes both stochastic and deterministic signals and no assumptions of stationarity or independence are requires and the unknown parameters are only assumed to have bounded variations in an average sense

a new approach to stochastic adaptive control the principal techniques used up to now for the analysis of stochastic adaptive control systems have been 1 super martingale often called stochastic lyapunov methods and 2 methods relying upon the strong consistency of some parameter estimation scheme optimal stochastic control and filtering methods have also been employed although there have been some successes the extension of these techniques to a broader class of adaptive control problems including the case of time varying parameters has been difficult in this paper a new approach is adopted if an underlying markovian state space system for the controlled process is available and if this process possesses stationary transition probabilities then the powerful ergodic theory of markov processes may be applied subject to technical conditions such as stability one may deduce 1 the existence of an invariant measure for the process and 2 the convergence almost surely of the sample averages of a function of the state process and of its expectation to its conditional expectation the technique is illustrated by an application to a previously unsolved problem involving a linear system with unbounded random time varying parameters

steady state and parameter tracking properties of self tuning minimum variance regulators abstract the self tuning regulator with a fixed gain and a fixed forgetting factor applied to control of an arx plant is considered the ordinary differential equation analysis of the closed loop system is performed in the neighbourhood of the stable equilibrium point and the approximate expressions for the steady state mean square parameter tracking and regulation errors are derived for the adaptive regulators based on the exponentially weighted least squares and stochastic approximation identification respectively it is shown that the concept of the effective memory length of the estimation algorithm has to be revised if the closed loop system identification with the gain fixing technique is used

comments on performance analysis of a deterministic channel estimator for block transmission systems with null guard intervals a deterministic algorithm was proposed for channel identification in block communication systems the method assumed that the channel has a finite impulse response fir and that null guard intervals of length greater than the channel order are inserted between successive blocks to prevent interblock interference and allow block synchronization in the absence of noise the algorithm provides error free channel estimates using a finite number of received data without requiring training sequences and without imposing a restriction neither on the channel except for finite order and time invariance nor on the symbol constellation using small perturbation analysis we derive approximate expressions of the estimated channel covariance matrix which are used to quantify the resilience of the estimation algorithm to additive noise and channel fluctuations specifically we consider channel fluctuations induced by transmitter receiver relative motion asynchronism and oscillators phase noise we also compare the channel estimation accuracy with the cramer rao bound crb and prove that our estimation method is statistically efficient at practical snr values for any data block length finally we validate our theoretical analysis with simulations and compare our transmission scheme with an alternative system using training sequences for channel estimation

joint covariance estimation with mutual linear structure we consider the problem of joint estimation of structured covariance matrices assuming the structure is unknown estimation is achieved using heterogeneous training sets namely given groups of measurements coming from centered populations with different covariances our aim is to determine the mutual structure of these covariance matrices and estimate them supposing that the covariances span a low dimensional affine subspace in the space of symmetric matrices we develop a new efficient algorithm discovering the structure and using it to improve the estimation our technique is based on the application of principal component analysis in the matrix space we also derive an upper performance bound of the proposed algorithm in the gaussian scenario and compare it with the cramer rao lower bound numerical simulations are presented to illustrate the performance benefits of the proposed method

stochastic adaptive control and martingale limit theory recently s p meyn and p e caines ibid vol ac 32 p 220 6 1987 have used ergodic theory for markov processes to give the first asymptotic stability analysis of a nontrivial stochastic adaptive control problem by nontrivial is meant a stochastic adaptive control problem whose parameter variation has finite nonzero power they correctly observed that the stochastic lyapunov function methods fail here because there is no almost sure parameter convergence it is shown here how martingale asymptotics can be used to produce many results close to those of meyn and caines as well as to supply some new observations strengths and weaknesses of both approaches are discussed

the zariski spectrum as a formal geometry we choose formal topology to deal in a basic manner with the zariski spectra of commutative rings and their structure sheaves by casting prime and maximal ideals in a secondary role we thus wish to prepare a constructive and predicative framework for abstract algebraic geometry in contrast to the classical approach neither points nor stalks need occur let alone any instance of the axiom of choice as compared with the topos theoretic treatments that may be rendered predicative as well the road we follow is built from more elementary material the formal counterpart of the structure sheaf which we present first is our guiding example for a notion of a sheaf on a formal topology we next define the category of formal geometries a natural abstraction from that of locally ringed spaces this allows us to eventually phrase and prove still within the language of opens and sections the universal property of the zariski spectrum our version appears to be the only one that is explicitly point free

bounds for a mixture of low rank compound gaussian and white gaussian noises we consider the problem of estimating the parameters of a low rank compound gaussian process in white gaussian noise this situation typically arises in radar applications where clutter is relevantly modeled as compound gaussian with a rank deficient covariance matrix of the speckle using a minimal and unconstrained parametrization of the problem we derive lower bounds for estimation of the parameters describing the covariance matrix first assuming the textures are deterministic the cramer rao bound is derived which enables one to assess the impact of the time varying textures on the estimation performance then considering the textures as random hybrid bounds are derived in addition we derive a lower bound for estimating the projector on the clutter subspace numerical simulations enable one to evaluate the impact of random time varying textures compared to the conventional case of constant texture i e of gaussian subspace signals in gaussian noise

a fundamental limitation on maximum parameter dimension for accurate estimation with quantized data it is revealed that there is a link between the quantization approach employed and the dimension of the vector parameter which can be accurately estimated by a quantized estimation system a critical quantity called inestimable dimension for quantized data idqd is introduced which does not depend on the quantization regions and the statistical models of the observations but instead depends only on the number of sensors and on the precision of the vector quantizers employed by the system it is shown that the idqd describes a quantization induced fundamental limitation on the estimation capabilities of the system to be specific if the dimension of the desired vector parameter is larger than the idqd of the quantized estimation system then the fisher information matrix for estimating the desired vector parameter is singular and moreover there exist infinitely many nonidentifiable vector parameter points in the vector parameter space furthermore it is shown that under some common assumptions on the statistical models of the observations and the quantization system a smaller idqd can be obtained which can specify an even more limiting quantization induced fundamental limitation on the estimation capabilities of the system

the cramer rao lower bound for bilinear systems estimation of the unknown parameters that characterize a bilinear system is of primary importance in many applications the cramer rao lower bound crlb provides a lower bound on the covariance matrix of any unbiased estimator of unknown parameters it is widely applied to investigate the limit of the accuracy with which parameters can be estimated from noisy data here it is shown that the crlb for a data set generated by a bilinear system with additive gaussian measurement noise can be expressed explicitly in terms of the outputs of its derivative system which is also bilinear a connection between the nonsingularity of the fisher information matrix and the local identifiability of the unknown parameters is exploited to derive local identifiability conditions of bilinear systems using the concept of the derivative system it is shown that for bilinear systems with piecewise constant inputs the crlb for uniformly sampled data can be efficiently computed through solving a lyapunov equation in addition a novel method is proposed to derive the asymptotic crlb when the number of acquired data samples approaches infinity these theoretical results are illustrated through the simulation of surface plasmon resonance experiments for the determination of the kinetic parameters of protein protein interactions

on non singular information matrices and local identifiability abstract the relation between non singularity of the information matrix and local identifiability of the parameters is discussed for some model structures these two properties arc shown to be equivalent however there are also cases where a singular information matrix does not contradict local identifiability

tyler s covariance matrix estimator in elliptical models with convex structure we address structured covariance estimation in elliptical distributions by assuming that the covariance is a priori known to belong to a given convex set e g the set of toeplitz or banded matrices we consider the general method of moments gmm optimization applied to robust tyler s scatter m estimator subject to these convex constraints unfortunately gmm turns out to be non convex due to the objective instead we propose a new coca estimator a convex relaxation which can be efficiently solved we prove that the relaxation is tight in the unconstrained case for a finite number of samples and in the constrained case asymptotically we then illustrate the advantages of coca in synthetic simulations with structured compound gaussian distributions in these examples coca outperforms competing methods such as tyler s estimator and its projection onto the structure set

nonlinear black box modeling in system identification a unified overview a nonlinear black box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics there has been considerable recent interest in this area with structures based on neural networks radial basis networks wavelet networks and hinging hyperplanes as well as wavelet transform based methods and models based on fuzzy sets and fuzzy rules this paper describes all these approaches in a common framework from a user s perspective it focuses on what are the common features in the different approaches the choices that have to be made and what considerations are relevant for a successful system identification application of these techniques it is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space these mappings are discussed separately the latter mapping is usually formed as a basis function expansion the basis functions are typically formed from one simple scalar function which is modified in terms of scale and location the expansion from the scalar argument to the regressor space is achieved by a radial or a ridge type approach basic techniques for estimating the parameters in the structures are criterion minimization as well as two step procedures where first the relevant basis functions are determined using data and then a linear least squares step to determine the coordinates of the function approximation a particular problem is to deal with the large number of potentially necessary parameters this is handled by making the number of used parameters considerably less than the number of offered parameters by regularization shrinking pruning or regressor selection