extended stochastic gradient identification algorithms for hammerstein wiener armax systems an extended stochastic gradient algorithm is developed to estimate the parameters of hammerstein wiener armax models the basic idea is to replace the unmeasurable noise terms in the information vector of the pseudo linear regression identification model with the corresponding noise estimates which are computed by the obtained parameter estimates the obtained parameter estimates of the identification model include the product terms of the parameters of the original systems two methods of separating the parameter estimates of the original parameters from the product terms are discussed the average method and the singular value decomposition method to improve the identification accuracy an extended stochastic gradient algorithm with a forgetting factor is presented the simulation results indicate that the parameter estimation errors become small by introducing the forgetting factor

a new kernel based approach for linear system identification this paper describes a new kernel based approach for linear system identification of stable systems we model the impulse response as the realization of a gaussian process whose statistics differently from previously adopted priors include information not only on smoothness but also on bibo stability the associated autocovariance defines what we call a stable spline kernel the corresponding minimum variance estimate belongs to a reproducing kernel hilbert space which is spectrally characterized compared to parametric identification techniques the impulse response of the system is searched for within an infinite dimensional space dense in the space of continuous functions overparametrization is avoided by tuning few hyperparameters via marginal likelihood maximization the proposed approach may prove particularly useful in the context of robust identification in order to obtain reduced order models by exploiting a two step procedure that projects the nonparametric estimate onto the space of nominal models the continuous time derivation immediately extends to the discrete time case on several continuous and discrete time benchmarks taken from the literature the proposed approach compares very favorably with the existing parametric and nonparametric techniques

on the estimation of transfer functions this paper treats the close conceptual relationships between basic approaches to the estimation of transfer functions of linear systems the classical methods of frequency and spectral analysis are shown to be related to the well known time domain methods of prediction error type via a common empirical transfer function estimate asymptotic properties of the estimates obtained by the respective methods are also described and discussed an important feature that is displayed by this treatment is a frequency domain weighting function that determines the distribution of bias in case the true system cannot be exactly described within the chosen model set the choice of this weighting function is made in terms of noise models for time domain methods the noise model thus has a dual character from the system approximation point of view

the steiglitz mcbride identification algorithm revisited convergence analysis and accuracy aspects the convergence and accuracy properties of the steiglitz and mcbride identification method are examined the analysis is valid for a sufficiently large number of data it is shown that the method can converge to the true parameter vector only when the additive output noise is white in that case the method is proved to be locally convergent to the true parameters the global convergence properties are also investigated it is pointed out that the method is not always globally convergent some sufficient conditions guaranteeing global convergence are given assuming convergence takes place the estimates are shown to be asymptotically gaussian distributed an explicit expression is given for their asymptotic covariance matrix

brief paper uniqueness of prediction error estimates of multivariable moving average models in prediction error pe identification of the parameter estimates is given by the global minimum of a scalar valued function of the innovation sample covariance matrix it may happen that the loss function has multiple local minimum points so that a numerical search routine can fail to find the global minimum such a situation usually referred to as lack of uniqueness of the estimates was experienced in practice and also theoretically examined for various model structures a unique minimum of the criterion is also crucial for convergence of recursive pe algorithms in this paper multivariable moving average ma models are considered it is proved that for such models any reasonable pe criterion has asymptotically a unique stationary point furthermore it is shown that this stationary point is a global minimum which corresponds to the true parameter vector this extends the result known for univariate ma models to the multivariate case

cost function shaping of the output error criterion identification of an output error model using the prediction error method leads to an optimization problem built on input output data collected from the system to be identified it is often hard to

partially coupled stochastic gradient identification methods for non uniformly sampled systems this technical note addresses identification problems of non uniformly sampled systems for the input output representation of non uniform discrete time systems a partially coupled stochastic gradient c sg algorithm is proposed to estimate the model parameters with high computational efficiency compared with the standard stochastic gradient sg algorithm the analysis indicates that the partially c sg algorithm can give more accurate parameter estimates than the sg algorithm the parameter estimates obtained using the partially c sg algorithm converge to their true values as the data length approaches infinity

non stationary stochastic embedding for transfer function estimation this paper presents a consistent framework for the quantification of noise and undermodelling errors in transfer function model estimation we use the so called stochastic embedding approach in which both noise and undermodelling errors are treated as stochastic processes in contrast to previous applications of stochastic embedding in this paper we represent the undermodelling as a multiplicative error characterised by random walk processes in the frequency domain the benefit of the present formulation is that it significantly simplifies the estimation of the parameters of the embedded process yielding a closed form expression for the model error quantification simulation and experimental examples illustrate how the random walks effectively capture typical cases of undermodelling found in practice including underdamped modes the examples also show how to use the method as a tool in the determination of model order and pole location in fixed denominator model structures

performance analysis of multi innovation gradient type identification methods it is well known that the stochastic gradient sg identification algorithm has poor convergence rate in order to improve the convergence rate we extend the sg algorithm from the viewpoint of innovation modification and present multi innovation gradient type identification algorithms including a multi innovation stochastic gradient misg algorithm and a multi innovation forgetting gradient mifg algorithm because the multi innovation gradient type algorithms use not only the current data but also the past data at each iteration parameter estimation accuracy can be improved finally the performance analysis and simulation results show that the proposed misg and mifg algorithms have faster convergence rates and better tracking performance than their corresponding sg algorithms

hardy littlewood sobolev inequalities via fast diffusion flows we give a simple proof of the d 2 cases of the sharp hardy littlewood sobolev inequality for d 3 and the sharp logarithmic hardy littlewood sobolev inequality for d 2 via a monotone flow governed by the fast diffusion equation

on resampling and uncertainty estimation in linear system identification linear system identification yields a nominal model parameter which minimizes a specific criterion based on the single input output data set here we investigate the utility of various methods for estimating the probability distribution of this nominal parameter using only the data from this single experiment the results are compared to the actual parameter distribution generated by many monte carlo runs of the data collection experiment the methods considered are collectively known as resampling schemes which include subsampling the jackknife and the bootstrap the broad aim is to generate an empirical parameter distribution function via the construction of a large number of new data records from the original single set of data based on an assumption that this data is representative of all possible data and then to run the parameter estimator on each of these new records to develop the distribution function the performance of these schemes is evaluated on a difficult almost unidentifiable system and compared to the standard results based on asymptotic normality in addition to the exploration of this example as a means to evaluate the strengths and weaknesses of these resampling schemes some new theoretical results are proven and demonstrated for subsampling schemes

sharp sobolev inequalities on the sphere and the moser trudinger inequality where de denotes normalized surface measure v is the conformal gradient and q 2n n 2 a modern folklore theorem is that by taking the infinitedimensional limit of this inequality one obtains the gross logarithmic sobolev inequality for gaussian measure which determines nelson s hypercontractive estimates for the hermite semigroup see 8 one observes using conformal invariance that the above inequality is equivalent to the sharp sobolev inequality on rn for which boundedness and extremal functions can be easily calculated using dilation invariance and geometric symmetrization the roots here go back to hardy and littlewood the advantage of casting the problem on the sphere is that the role of the constants is evident and one is led immediately to the conjecture that this inequality should hold whenever possible for example 2 q 0o if n 2 this is in fact true and will be demonstrated in section 2 a clear question at this point is what is the situation in dimension 2 two important arguments 25 26 27 dealt with this issue both motivated by geometric variational problems because q goes to infinity for dimension 2 the appropriate function space is the exponential class responding in part

locally robust identification of linear systems containing unknown gain elements with application to adapted iir lattice models abstract we consider the identification of stable linear systems whose unknown parameters may be interpreted as feedback gains by using the output error between the true system and a model of it containing adjustable parameters we develop a recursive algorithm for estimating the unknown parameters we describe a persistency of excitation condition on the system input which guarantees a robust local convergence property for the algorithm we then apply our results to the identification of the parameters of a tapped lattice model of a linear infinite impulse response iir plant considering the identification of a lattice rather than a direct form model of a linear system is attractive due to i the simplicity of its crucial stability check and maintenance procedure for the adapted iir parametrization and ii the numerical insensitivity properties of the lattice structure reproducible simulation evidence is presented that supports our results

maximum entropy vector kernels for mimo system identification abstract recent contributions have framed linear system identification as a nonparametric regularized inverse problem relying on l 2 type regularization which accounts for the stability and smoothness of the impulse response to be estimated these approaches have been shown to be competitive w r t classical parametric methods in this paper adopting maximum entropy arguments we derive a new l 2 penalty to do so we exploit the structure of the hankel matrix thus controlling at the same time complexity measured by the mcmillan degree stability and smoothness of the identified models as a special case we recover the nuclear norm penalty on the squared block hankel matrix in contrast with the previous literature on reweighted nuclear norm penalties our kernel is described by a small number of hyper parameters which are iteratively updated through marginal likelihood maximization constraining the structure of the kernel acts as a hyper regularizer which helps controlling the effective degrees of freedom of our estimator to optimize the marginal likelihood we adapt a scaled gradient projection sgp algorithm which is proved to be significantly computationally cheaper than other first and second order off the shelf optimization methods the paper also contains an extensive comparison with many state of the art methods on several monte carlo studies which confirms the effectiveness of our procedure

paper on the estimation of transfer functions this paper treats the close conceptual relationships between basic approaches to the estimation of transfer functions of linear systems the classical methods of frequency and spectral analysis are shown to be related to the well known time domain methods of prediction error type via a common empirical transfer function estimate asymptotic properties of the estimates obtained by the respective methods are also described and discussed an important feature that is displayed by this treatment is a frequency domain weighting function that determines the distribution of bias in case the true system cannot be exactly described within the chosen model set the choice of this weighting function is made in terms of noise models for time domain methods the noise model thus has a dual character from the system approximation point of view

an adaptive method for consistent estimation of real valued non minimum phase zeros in stable lti systems an adaptive algorithm consisting of a recursive estimator for a finite impulse response model having two non zero lags only and an adaptive input are presented the model is parametrized in terms of the first impulse response coefficient and the model zero for linear time invariant single input single output systems with real rational transfer functions possessing at least one real valued non minimum phase zero of multiplicity one it is shown that the model zero converges to such a zero of the true system in the case of multiple non minimum phase zeros the algorithm can be tailored to converge to a particular zero the result is shown to hold for systems and noise spectra of arbitrary degree the algorithm requires prior knowledge of the sign of the high frequency gain of the system as well as an interval to which the non minimum phase zero of interest belongs

functional inequalities thick tails and asymptotics for the critical mass patlak keller segel model we investigate the long time behavior of the critical mass patlak keller segel equation this equation has a one parameter family of steady state solutions rhohls lambda 0 with thick tails whose second moment is not bounded we show that these steady state solutions are stable and find basins of attraction for them using an entropy functional fcfd coming from the critical fast diffusion equation in r 2 we construct solutions of patlak keller segel equation satisfying an entropy entropy dissipation inequality for fcfd while the entropy dissipation for fcfd is strictly positive it turns out to be a difference of two terms neither of which need to be small when the dissipation is small we introduce a strategy of em controlled concentration to deal with this issue and then use the regularity obtained from the entropy entropy dissipation inequality to prove the existence of basins of attraction for each stationary state composed by certain initial data converging towards rhohls in the present paper we do not provide any estimate of the rate of convergence but we discuss how this would result from a stability result for a certain sharp gagliardo nirenberg sobolev inequality

brief paper some properties of the output error method the output error identification method is studied in various respects the stationary points of the associated loss function are investigated sufficient conditions for a unique local minimum are given the loss functions can be minimized using a quasilinearization algorithm such an algorithm will give good local convergence it is however shown that global convergence does not always occur the output error method is also compared with some other estimation methods from the accuracy point of view it is proved that a prediction error method will give better accuracy an instrumental variable technique may give better or worse accuracy depending on the actual noise correlation

prediction error identification of linear systems a nonparametric gaussian regression approach a novel bayesian paradigm for the identification of output error models has recently been proposed in which in place of postulating finite dimensional models of the system transfer function the system impulse response is searched for within an infinite dimensional space in this paper such a nonparametric approach is applied to the design of optimal predictors and discrete time models based on prediction error minimization by interpreting the predictor impulse responses as realizations of gaussian processes the proposed scheme describes the predictor impulse responses as the convolution of an infinite dimensional response with a low dimensional parametric response that captures possible high frequency dynamics overparameterization is avoided because the model involves only a few hyperparameters that are tuned via marginal likelihood maximization numerical experiments with data generated by armax and infinite dimensional models show the definite advantages of the new approach over standard parametric prediction error techniques and subspace methods both in terms of predictive capability on new data and accuracy in reconstruction of system impulse responses

useful redundancy in parameter and time delay estimation for continuous time models in this paper we propose an algorithm to estimate the parameters including time delay of continuous time systems based on instrumental variable identification methods to overcome the multiple local minima of the cost function associated with the estimation of a time delay system we utilize the useful redundancy technique specifically the cost function is filtered through a set of low pass filters to improve convexity with the useful redundancy technique exploited to achieve convergence to the global minimum of the optimization problem numerical examples are presented to demonstrate the effectiveness of the proposed algorithm

on error surfaces of sufficient order adaptive iir filters proofs and counterexamples to a unimodality conjecture it is shown that although a conjecture proposed by s stearns ibid vol assp 29 p 763 6 june 1981 concerning sufficient conditions to guarantee unimodality of error surfaces for adaptive pole zero infinite impulse response iir filters is valid for first and second order filters it is not true in general without an additional constraint introduced by t soderstrom autom vol 11 p 93 9 jan 1982 proofs of the conjecture for a first and second order adaptive filters are given counterexamples are presented to illustrate the failure of this conjecture for higher order filters when the condition of soderstrom is not satisfied for both matching order and overparameterizing cases

uncertain convex programs randomized solutions and confidence levels many engineering problems can be cast as optimization problems subject to convex constraints that are parameterized by an uncertainty or instance parameter two main approaches are generally available to tackle constrained optimization problems in presence of uncertainty robust optimization and chance constrained optimization robust optimization is a deterministic paradigm where one seeks a solution which simultaneously satisfies all possible constraint instances in chance constrained optimization a probability distribution is instead assumed on the uncertain parameters and the constraints are enforced up to a pre specified level of probability unfortunately however both approaches lead to computationally intractable problem formulations

closed loop experiment design for linear time invariant dynamical systems via lmis all stationary experimental conditions corresponding to a discrete time linear time invariant causal internally stable closed loop with real rational system and feedback controller are characterized using the youla kucera parametrization finite dimensional parametrizations of the input spectrum and the youla kucera parameter allow a wide range of closed loop experiment design problems based on the asymptotic in the sample size covariance matrix for the estimated parameters to be recast as computationally tractable convex optimization problems such as semi definite programs in particular for box jenkins models a finite dimensional parametrization is provided which is able to generate all possible asymptotic covariance matrices as a special case the very common situation of a fixed controller during the identification experiment can be handled and optimal reference signal spectra can be computed subject to closed loop signal constraints finally a brief numerical comparison with closed loop experiment design based on a high model order variance expression is presented

global convergence conditions in maximum likelihood estimation maximum likelihood estimation has been widely applied in system identification because of consistency its asymptotic efficiency and sufficiency however gradient based optimisation of the likelihood function might end up in local convergence in this article we derive various new non local minimum conditions in both open and closed loop system when the noise distribution is a gaussian process here we consider different model structures in particular ararmax bj and oe models

twenty one ml estimators for model selection classical approaches to determine a suitable model structure from observed input output data are based on hypothesis tests and information based criteria recently the model structure has been considered as a stochastic variable and standard estimation techniques have been proposed the resulting estimators are closely related to the aforementioned methods however it turns out that there are a number of prior choices in the problem formulation which are crucial for the estimators behavior the contribution of this paper is to clarify the role of the prior choices to examine a number of possibilities and to show which estimators are consistent this is done in a linear regression framework for autoregressive models we also investigate a novel prior assumption on stability and give the estimator for the model order and the parameters themselves

sobolev and hardy littlewood sobolev inequalities this paper is devoted to improvements of sobolev and onofri inequalities the additional terms involve the dual counterparts i e hardy littlewood sobolev type inequalities we focus our attention on optimal constants that can be achieved either by completion of the square methods or by using nonlinear flows and provide various new estimates

technical communique eigenvalue location of certain matrices arising in convergence analysis problems we give simple formulas for the eigenvalues of a certain type of matrices occurring in some convergence analysis problems application of the result to a pseudo linear regression algorithm gives explicit necessary and sufficient conditions for local convergence

on the mathematical foundations of stable rkhss abstract reproducing kernel hilbert spaces rkhss are key spaces for machine learning that are becoming popular also for linear system identification in particular the so called stable rkhss can be used to model absolutely summable impulse responses in combination e g with regularized least squares they can then be used to reconstruct dynamic systems from input output data in this paper we provide new structural properties of stable rkhss the relation between stable kernels and other fundamental classes like those containing absolutely summable or finite trace kernels is elucidated these insights are then brought into the feature space context first it is proved that any stable kernel admits feature maps induced by a basis of orthogonal eigenvectors in l 2 the exact connection with classical system identification approaches that exploit such kind of functions to model impulse responses is also provided then the necessary and sufficient stability condition for rkhss designed by formulating kernel eigenvectors and eigenvalues is obtained overall our new results provide novel mathematical foundations of stable rkhss with impact on stability tests impulse responses modeling and computational efficiency of regularized schemes for linear system identification

input design as a tool to improve the convergence of pem the prediction error method pem is related to an optimization problem built on input output data collected from the system to be identified it is often hard to find the global solution of this optimization problem because the corresponding objective function presents local minima and or the search space is constrained to a nonconvex set the shape of the cost function and hence the difficulty in solving the optimization problem depends directly on the experimental conditions more specifically on the spectrum of the input output data collected from the system therefore it seems plausible to improve the convergence to the global minimum by properly choosing the spectrum of the input in this paper we address this problem we present a condition for convergence to the global minimum of the cost function and propose its inclusion in the input design we present the application of the proposed approach to case studies where the algorithms tend to get trapped in nonglobal minima

on transient error surfaces of output error iir adaptive filtering addresses the convergence of iir output error adaptive algorithms we argue that convergence to global or local optimum is simultaneously determined by the shape of the transient error surface and the convergence speed of the algorithm this argument is confirmed by simulation examples comparing the convergence of lms based and ls based algorithms

l2 model reduction and variance reduction in this contribution we examine certain variance properties of model reduction the focus is on l 2 model reduction but some general results are also presented these general results can be used to analyze various other model reduction schemes the models we study are finite impulse response fir and output error oe models we compare the variance of two estimated models the first one is estimated directly from data and the other one is computed by reducing a high order model by l 2 model reduction in the fir case we show that it is never better to estimate the model directly from data compared to estimating it via l 2 model reduction of a high order fir model for oe models we show that the reduced model has the same variance as the directly estimated one if the reduced model class used contains the true system

a mildly weaker sufficient condition in iir adaptive filtering the cross covariance matrix of two stable autoregressive ar sequences is considered a mildly weaker condition is identified that ensures the nonsingularity of this matrix as one consequence of this result a weaker sufficient condition is obtained that would guarantee the unimodality of the mean square output error surface of an iir adaptive filter with white noise excitation

comparison of some instrumental variable methods consistency and accuracy aspects a general analysis of various instrumental variable methods is given the investigation includes methods for open loop operation as well as methods designed for closed loop operation the instrumental variables are formed as different combinations of inputs delayed inputs delayed outputs filtered inputs and external setpoint variations necessary conditions as well as sufficient conditions for consistency are explicitly derived and discussed the paper also includes some simple counter examples to general consistency for some common instrumental variable methods also asymptotic accuracy properties are reviewed it is e g studied how the use of filters will influence the covariance matrix of the parameter estimates

adaptive parameter estimation for a constrained low pass system a novel algorithm is presented for adaptive parameter estimation for a constrained low pass butterworth system model the algorithm will estimate the system cutoff frequency and gain online when it is known that the true system has a low pass butterworth structure or some transfer function similar to it and its true order is used the algorithm will lead to a substantial savings in computation and more accurate results than unconstrained algorithms potential applications include filter design and adaptive decision on nyquist rate for systems

brief paper identification and control joint input design and h state feedback with ellipsoidal parametric uncertainty via lmis one obstacle in connecting robust control with models generated from prediction error identification is that very few control design methods are able to directly cope with the ellipsoidal parametric uncertainty regions that are generated by such identification methods in this contribution we present a joint robust state feedback control input design procedure which guarantees stability and prescribed closed loop performance using models identified from experimental data this means that given h specifications on the closed loop transfer function are translated into sufficient requirements on the input signal spectrum used to identify the process the condition takes the form of a linear matrix inequality

auxiliary model based rels and mi els algorithm for hammerstein oema systems this paper considers the identification problem for hammerstein output error moving average oema systems an auxiliary model based recursive extended least squares rels algorithm and an auxiliary model based multi innovation extended least squares mi els algorithm are presented using the multi innovation identification theory the basic idea is to express the system output as a linear combination of the parameters by using the key term separation principle and auxiliary model method the proposed algorithms can give highly accurate parameter estimates the simulation results show the effectiveness of the proposed algorithms

on the estimation of transfer functions regularizations and gaussian processes revisited intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques we revisit the old problem of transfer function estimation from input output measurements we formulate a classical regularization approach focused on finite impulse response fir models and find that regularization is necessary to cope with the high variance problem this basic regularized least squares approach is then a focal point for interpreting other techniques like bayesian inference and gaussian process regression the main issue is how to determine a suitable regularization matrix bayesian prior or kernel several regularization matrices are provided and numerically evaluated on a data bank of test systems and data sets our findings based on the data bank are as follows the classical regularization approach with carefully chosen regularization matrices shows slightly better accuracy and clearly better robustness in estimating the impulse response than the standard approach the prediction error method maximum likelihood pem ml approach if the goal is to estimate a model of given order as well as possible a low order model is often better estimated by the pem ml approach and a higher order model is often better estimated by model reduction on a high order regularized fir model estimated with careful regularization moreover an optimal regularization matrix that minimizes the mean square error matrix is derived and studied the importance of this result lies in that it gives the theoretical upper bound on the accuracy that can be achieved for this classical regularization approach

a structural view of asymptotic convergence speed of adaptive iir filtering algorithms i infinite precision implementation for pt i see ibid vol 41 no 4 p 1493 1517 1993 finite precision fp implementation is the ultimately inevitable reality of all adaptive filters including adaptive infinite impulse response iir filters this paper continues to examine the asymptotic convergence speed of adaptive iir filters of various structures and algorithms including the simple constant gain type and the newton type but under fp implementation a stochastic differential equation sde approach is used in the analysis such an approach not only greatly simplifies the fp analysis which is traditionally very involved algebraically but it also provides valuable information about the first order as well as the second order moments that the latter are not available using the ordinary differential equation ode approach the asymptotic convergence speed as well as the convergent values of the pertinent moments of fp errors are examined in terms of unknown system pole zero locations the adverse effects of lightly damped low frequency ldlf poles resulting from fast sampling on the local transient and convergent behavior of various structures and algorithms are analyzed and compared the new results agree with the existing ones when reduced to the finite impulse response fir case in particular the explosive behavior of pertinent error variances of newton type iir algorithms when the forgetting factor spl lambda 1 is also concluded computer simulation verifies the predicted theoretical results

global positivity estimates and harnack inequalities for the fast diffusion equation abstract we investigate local and global properties of positive solutions to the fast diffusion equation u t u m in the range d 2 d m 1 corresponding to general nonnegative initial data for the cauchy problem posed in the whole euclidean space r d we prove sharp local positivity estimates weak harnack inequalities and elliptic harnack inequalities we use them to derive sharp global positivity estimates and a global harnack principle for the mixed initial and boundary value problem posed in a bounded domain of r d with homogeneous dirichlet condition we prove weak and elliptic harnack inequalities our work shows that these fast diffusion flows have regularity properties comparable and in some senses better than the linear heat flow

existence of extremal functions for the stein weiss inequalities on the heisenberg group abstract in this paper we establish the existence of extremals for two kinds of stein weiss inequalities on the heisenberg group more precisely we prove the existence of extremals for the stein weiss inequalities with full weights in theorem 1 1 and the stein weiss inequalities with horizontal weights in theorem 1 4 different from the proof of the analogous inequality in euclidean spaces given by lieb 26 using riesz rearrangement inequality which is not available on the heisenberg group we employ the concentration compactness principle to obtain the existence of the maximizers on the heisenberg group our result is also new even in the euclidean case because we don t assume that the exponents of the double weights in the stein weiss inequality 1 1 are both nonnegative see theorem 1 3 and more generally theorem 1 5 therefore we extend lieb s celebrated result of the existence of extremal functions of the stein weiss inequality in the euclidean space to the case where the exponents are not necessarily both nonnegative see theorem 1 3 furthermore since the absence of translation invariance of the stein weiss inequalities additional difficulty presents and one cannot simply follow the same line of lions idea to obtain our desired result our methods can also be used to obtain the existence of optimizers for several other weighted integral inequalities theorem 1 5

necessary and sufficient conditions for uniqueness of the minimum in prediction error identification the contribution of this paper is to establish computable necessary and sufficient conditions on the model structure and on the experiment under which the prediction error identification pei criterion has a unique global minimum we consider a broad class of rational model structures whose numerator and denominator are affine in the unknown parameter vector this class encompasses all classical model structures used in system identification the main results in this paper rely on the standard assumption that the system is in the model set while some intermediate results are valid even when this assumption does not hold in particular theorem 4 2 and lemma 6 1 this is achieved by first establishing necessary and sufficient conditions on the model structure and on the experiment under which a global minimum is isolated these conditions must hold locally at the global minimum a second contribution is to show that these conditions are equivalent to the nonsingularity of the information matrix at that minimum for open loop identification and with some additional constraints also for closed loop identification the nonsingularity of the information matrix is then also equivalent to the uniqueness of the global minimum

uniqueness of estimated k step prediction models of arma processes the direct estimation of the k step prediction models of arma processes is discussed the emphasis is on the uniqueness properties of the parameter estimates of such models obtained by using either a prediction error method pem or a pseudo linear regression plr algorithm the main result is that both pem and plr when applied to such models have a certain uniqueness property more specifically it is shown that all the limit models corresponding to either method behave precisely as the true optimal predictor furthermore if a minimal parameterization is used then the true predictor is the unique limit model both for pem and for plr

extremals of functionals with competing symmetries abstract we present a new method of producing optimizing sequences for highly symmetric functionals the sequences have good convergence properties built in we apply the method in different settings to give elementary proofs of some classical inequalities such as the hardy littlewood sobolev and the logarithmic sobolev inequality in their sharp form

set membership identification of nonlinear systems in the paper the problem of identifying nonlinear dynamic systems described in nonlinear regression form is considered using finite and noise corrupted measurements most methods in the literature are based on the estimation of a model within a finitely parametrized model class describing the functional form of involved nonlinearities a key problem in these methods is the proper choice of the model class typically realized by a search from the simplest to more complex ones linear bilinear polynomial neural networks etc in this paper an alternative approach based on a set membership framework is presented not requiring assumptions on the functional form of the regression function describing the relations between measured input and output but assuming only some information on its regularity given by bounds on its gradient in this way the problem of considering approximate functional forms is circumvented moreover noise is assumed to be bounded in contrast with statistical methods which rely on assumptions such as stationarity ergodicity uncorrelation type of distribution etc whose validity may be difficult to test reliably and is lost in presence of approximate modeling in this paper necessary and sufficient conditions are given for the validation of the considered assumptions an optimal interval estimate of the regression function is obtained providing its uncertainty range for any assigned regressor values the set estimate allows to derive an optimal identification algorithm giving estimates with minimal guaranteed l p error on the assigned domain of the regressors the properties of the optimal estimate are investigated and its worst case l p identification error is evaluated the presented approach is tested and compared with other nonlinear methods on the identification of a water heater a mechanical system with input saturation and a vehicle with controlled suspensions

decoupling of square singular systems via proportional state feedback necessary and sufficient conditions for the decoupling of a solvable square singular system ex t ax t bu t with output y t dx t through an admissible control law of the form u t kx t hr t where h is a square nonsingular matrix it has been shown that for a given singular system that satisfies these conditions a propagational state feedback exists for which the system s transfer function is a diagonal nonsingular and proper rational matrix the proofs of the main results are constructive and provide a procedure for computing an appropriate proportional state feedback

quantifying the error in estimated transfer functions with application to model order selection previous results on estimating errors or error bounds on identified transfer functions have relied on prior assumptions about the noise and the unmodeled dynamics this prior information took the form of parameterized bounding functions or parameterized probability density functions in the time or frequency domain with known parameters it is shown that the parameters that quantify this prior information can themselves be estimated from the data using a maximum likelihood technique this significantly reduces the prior information required to estimate transfer function error bounds the authors illustrate the usefulness of the method with a number of simulation examples how the obtained error bounds can be used for intelligent model order selection that takes into account both measurement noise and under modeling is shown another simulation study compares the method to akaike s well known fpe and aic criteria

identification of arx systems with non stationary inputs asymptotic analysis with application to adaptive input design a key problem in optimal input design is that the solution depends on system parameters to be identified in this contribution we provide formal results for convergence and asymptotic optimality of an adaptive input design method based on the certainty equivalence principle i e for each time step an optimal input design problem is solved exactly using the present parameter estimate and one sample of this input is applied to the system the results apply to stable arx systems with the input restricted to be generated by white noise filtered through a finite impulse response filter or a binary signal obtained from the latter by a static nonlinearity

analysis of a frequency domain adaptive iir filter convergence of frequency domain adaptive pole zero iir infinite impulse response filter is studied the algorithm is shown to converge in probability to an associated ordinary differential equation ode which in turn converges to a local minimum of its performance surface an analysis of the performance surface shows that the algorithm converges to one of n factorial members in an equivalence class of global minimum points where n is the number of adaptive poles saddle points exist on manifolds that separate members in the equivalence class this explains shoulders in the mse convergence curves and also suggests one way of avoiding these shoulders which cause slow convergence a second order simulation example confirms the above results

the decoupling of generalized state space systems via state feedback the input output decoupling problem of generalized state space systems gss via proportional state feedback is studied the following three major issues are resolved the necessary and sufficient conditions for the problem to have a solution the general analytical expressions of the controller matrices and the general analytical expressions of the diagonal elements of the closed loop system it is concluded that the approach presented seems to be a quite powerful tool for solving several feedback design problems for gss systems using this method problems such as decoupling and simultaneous pole assignment and decoupling via output feedback are successfully studied

time varying performance surfaces for adaptive iir filters geometric properties and implications for filter stability we present a new framework for understanding the performance of adaptive iir filters which enhances the understanding of filter stability during on line operation this new understanding arises from examining geometric properties of time varying performance surfaces which are defined by the data rather than the standard steady state error surfaces as defined by statistics of the data data dependent descent directions used in adaptive algorithms to update filter coefficients typically are viewed as functions of gradients defined on fixed performance surfaces and data is used to approximate these gradients at each iteration in contrast we view data dependent descent directions at each iteration as functions of exact gradients on time varying performance surfaces by examining the shape of these time varying performance surfaces near filter stability boundaries we are able to identify the origin of on line stability problems associated with existing adaptive iir filtering formulations and suggest corrective measures specifically by using exact z domain methods we define time varying performance surfaces which geometrically enforce filter stability and maintain the geometric and physical properties of the true error surface at each iteration development of adaptive algorithms based on this measure is expected to result in adaptive filters having improved stability performance during on line operation

self tuning control based on multi innovation stochastic gradient parameter estimation abstract this paper uses the multi innovation stochastic gradient misg algorithm to estimate the parameters of discrete time systems and presents an misg based self tuning control scheme furthermore we prove that the parameter estimation error converges to zero under persistent excitation and the parameter estimation based control algorithm can asymptotically achieve virtually optimal control and ensure that the closed loop systems are stable and globally convergent the simulation example is included

some characteristics of error surfaces for insufficient order adaptive iir filters the error surfaces associated with insufficient order adaptive filters with white input are considered it is shown that some sufficient but not necessary conditions for multimodality of these surfaces can be identified a convenient method of determining the stationary points of some insufficient order adaptive filters with colored inputs is presented

closed loop identification revisited identification of systems operating in closed loop has long been of prime interest in industrial applications the problem offers many possibilities and also some fallacies and a wide variety of approaches have been suggested many quite recently the purpose of the current contribution is to place most of these approaches in a coherent framework thereby showing their connections and display similarities and differences in the asymptotic properties of the resulting estimates the common framework is created by the basic prediction error method and it is shown that most of the common methods correspond to different parameterizations of the dynamics and noise models the so called indirect methods e g are indeed direct methods employing noise models that contain the regulator the asymptotic properties of the estimates then follow from the general theory and take different forms as they are translated to the particular parameterizations we also study a new projection approach to closed loop identification with the advantage of allowing approximation of the open loop dynamics in a given and user chosen frequency domain norm even in the case of an unknown nonlinear regulator

stability for a gns inequality and the log hls inequality with application to the critical mass keller segel equation starting from the quantitative stability result of bianchi and egnell for the 2 sobolev inequality we deduce several dierent stability results for a gagliardo nirenberg sobolev inequality in the plane then exploiting the connection between this inequality and a fast diusion equation we get stability for the log hls inequality finally using all these estimates we prove a quantitative convergence result for the critical mass keller segel system

internal stabilization and decoupling in linear multivariable systems by unity output feedback compensation we consider the problem of internally stabilizing and simultaneously diagonally decoupling a linear multivariable system by unity output feedback compensation a sufficient condition is derived for the existence of a cascade proper compensator c s such that when employed in a unity feedback loop involving the proper transfer function matrix p o of a free of unstable hidden modes system sigma p o will not only internally stabilize the feedback closed loop system sigma p o c but will also give rise to a closed loop transfer function matrix h yr diag which is nonsingular diagonal and has desired poles based on this analysis an algorithmic procedure for the computation of such a compensator is presented

an iterative algorithm for simulation error based identification of polynomial input output models using multi step prediction effective identification of polynomial input output models for applications requiring long range prediction or simulation performance relies on both careful model selection and accurate parameter estimation the simulation error minimisation sem approach has been shown to provide significant advantages in the model selection phase by ruling out candidate models with good short term prediction capabilities but unsuitable long term dynamics however sem based parameter estimation has been generally avoided due to excessive computational effort this article extends to the nonlinear case a computationally efficient approach for this task that was previously developed for linear models based on the iterative estimation of predictors with increasing prediction horizon conditions for the applicability of the approach to various model classes are also discussed finally some examples are provided to show the effectiveness and computational convenience of the proposed algorithm for polynomial input output

large signal to noise ratio quantification in mle for ararmax models it has been shown that closed loop linear system identification by indirect method can be generally transferred to open loop ararmax autoregressive autoregressive moving average with exogenous input estimation for such models the gradient related optimisation with large enough signal to noise ratio snr can avoid the potential local convergence in maximum likelihood estimation to ease the application of this condition the threshold snr needs to be quantified in this paper we build the amplitude coefficient which is an equivalence to the snr and prove the finiteness of the threshold amplitude coefficient within the stability region the quantification of threshold is achieved by the minimisation of an elaborately designed multi variable cost function which unifies all the restrictions on the amplitude coefficient the corresponding algorithm based on two sets of physically realisable system input output data details the minimisation and also points out how to use the gradient related method to est

input design via lmis admitting frequency wise model specifications in confidence regions a framework for reformulating input design problems in prediction error identification as convex optimization problems is presented for linear time invariant single input single output systems this framework unifies and extends existing results on open loop input design that are based on the finite dimensional asymptotic covariance matrix of the parameter estimates basic methods for parametrizing the input spectrum are provided and conditions on these parametrizations that guarantee that all possible covariance matrices for the asymptotic distribution of the parameter estimates can be generated are provided a wide range of model quality constraints can be handled in particular different frequency by frequency constraints can be used this opens up new applications of input design in areas such as robust control furthermore quality specifications can be imposed on all models in a confidence region thus allowing for statements such as with at least 99 probability the model quality specifications will be satisfied

asymptotics of the fast diffusion equation via entropy estimates we consider non negative solutions of the fast diffusion equation ut um with m 0 1 in the euclidean space mathbb r d d 3 and study the asymptotic behavior of a natural class of solutions in the limit corresponding to t for m mc d 2 d or as t approaches the extinction time when m mc for a class of initial data we prove that the solution converges with a polynomial rate to a self similar solution for t large enough if m mc or close enough to the extinction time if m mc such results are new in the range m mc where previous approaches fail in the range mc m 1 we improve on known results

input output decoupling of linear time varying singular systems the problem of input output decoupling by state feedback is considered for linear time varying singular systems assumptions are introduced and an algorithm is developed such that the system can be expressed in a simple normal form a time varying feedback law is then constructed which ensures under appropriate conditions that the closed loop system is regular impulse free and noninteractive

a counterexample to the uniqueness of the asymptotic estimate in armax model identification via the correlation approach abstract this paper deals with the identifiability of an armax system when the correlation approach is adopted in general identifiability depends on both the parametrization of the model class and on the informativeness of the data here we focus on the latter aspect and therefore a full order model class is considered the main goal is to provide a counterexample to the uniqueness of the asymptotic estimate when a persistently exciting input is adopted this shows the somehow counterintuitive fact that the identifiability of armax systems within the correlation approach is related to the color of the input

the cauchy problem for ut δum when 0 m 1 on considere le probleme de cauchy pour l equation de diffusion non lineaire u t u u m 1 0 sur 0 r n u 0 u 0 quand 0 m 1 on montre qu il existe une solution temporelle globale pour une fonction localement integrable u 0 on montre l unicite dans cette classe

nonlinear diffusions extremal properties of barenblatt profiles best matching and delays abstract in this paper we consider functionals based on moments and nonlinear entropies which have a linear growth in time in case of source type solutions to the fast diffusion or porous medium equations that are also known as barenblatt solutions as functions of time these functionals have convexity properties for generic solutions so that their asymptotic slopes are extremal for barenblatt profiles the method relies on scaling properties of the evolution equations and provides a simple and direct proof of sharp gagliardo nirenberg sobolev inequalities in scale invariant form the method also gives refined estimates of the growth of the second moment and as a consequence establishes the monotonicity of the delay corresponding to the best matching barenblatt solution compared to the barenblatt solution with same initial second moment here the notion of best matching is defined in terms of a relative entropy

uncertainty of transfer function modelling using prior estimated noise models assuming small model errors unmodelled dynamics and or nonlinear distortions and large signal to noise ratios we derive in this paper explicit expressions for the covariance matrix of a frequency domain estimator using prior estimated noise models these analytic expressions i give a clear insight in the behaviour of the covariance matrix as a function of the signal to noise ratio the unmodelled dynamics and the nonlinear distortions and ii allow to predict accurately the order of magnitude of the actual uncertainty of the estimates the link with the classical prediction error approach is also established

stable and efficient lattice algorithms for adaptive iir filtering previous attempts at applying lattice structures to adaptive infinite impulse response iir filtering have met with gradient computations of o n sup 2 complexity to overcome this computational burden two new lattice based algorithms are proposed for adaptive iir filtering and system identification with both algorithms of o n complexity the first algorithm is a reinterpretation of the steiglitz mcbride method 1965 while the second is a variation on the output error method state space models are employed to make the derivations transparent and the methods can be extended to other parameterizations if desired the set of possible stationary points of the algorithms is shown to be consistent with the convergent points obtained from the direct form versions of the steiglitz mcbride and output error methods whose properties are well studied the derived algorithms are as computationally efficient as existing direct form based algorithms while overcoming the stability problems associated with time varying direct form filters

uniformly improving the cramer rao bound and maximum likelihood estimation an important aspect of estimation theory is characterizing the best achievable performance in a given estimation problem as well as determining estimators that achieve the optimal performance the traditional cramer rao type bounds provide benchmarks on the variance of any estimator of a deterministic parameter vector under suitable regularity conditions while requiring a priori specification of a desired bias gradient in applications it is often not clear how to choose the required bias a direct measure of the estimation error that takes both the variance and the bias into account is the mean squared error mse which is the sum of the variance and the squared norm of the bias here we develop bounds on the mse in estimating a deterministic parameter vector x0 over all bias vectors that are linear in x0 which includes the traditional unbiased estimation as a special case in some settings it is possible to minimize the mse over all linear bias vectors more generally direct minimization is not possible since the optimal solution depends on the unknown x0 nonetheless we show that in many cases we can find bias vectors that result in an mse bound that is smaller than the cramer rao lower bound crlb for all values of x0 furthermore we explicitly construct estimators that achieve these bounds in cases where an efficient estimator exists by performing a simple linear transformation on the standard maximum likelihood ml estimator this leads to estimators that result in a smaller mse than the ml approach for all possible values of x0

output feedback decoupling of generalized state space systems abstract the input output decoupling problem of generalized state space systems via pure proportional output feedback is studied here for the first time wherein the following four major issues are resolved the necessary and sufficient conditions for the problem to have a solution the general analytical expressions of the decoupling controller matrices the general analytical expressions of the diagonal elements of the decoupled closed loop system and the structural properties of the closed loop system