nonlinear system identification via direct weight optimization a general framework for estimating nonlinear functions and systems is described and analyzed in this paper identification of a system is seen as estimation of a predictor function the considered predictor function estimate at a particular point is defined to be affine in the observed outputs and the estimate is defined by the weights in this expression for each given point the maximal mean square error or an upper bound of the function estimate over a class of possible true functions is minimized with respect to the weights which is a convex optimization problem this gives different types of algorithms depending on the chosen function class it is shown how the classical linear least squares is obtained as a special case and how unknown but bounded disturbances can be handled most of the paper deals with the method applied to locally smooth predictor functions it is shown how this leads to local estimators with a finite bandwidth meaning that only observations in a neighborhood of the target point will be used in the estimate the size of this neighborhood the bandwidth is automatically computed and reflects the noise level in the data and the smoothness priors the approach is applied to a number of dynamical systems to illustrate its potential

piecewise linear solution paths with application to direct weight optimization recently pathfollowing algorithms for parametric optimization problems with piecewise linear solution paths have been developed within the field of regularized regression this paper presents a generalization of these algorithms to a wider class of problems it is shown that the approach can be applied to the nonparametric system identification method direct weight optimization dwo and be used to enhance the computational efficiency of this method the most important design parameter in the dwo method is a parameter l controlling the bias variance trade off and the use of parametric optimization with piecewise linear solution paths means that the dwo estimates can be efficiently computed for all values of l simultaneously this allows for designing computationally attractive adaptive bandwidth selection algorithms one such algorithm for dwo is proposed and demonstrated in two examples

total least squares phased averaging and 3 d esprit for joint azimuth elevation carrier estimation the method of total least squares tls phased averaging for high performance subspace fitting in the three dimensional 3 d case of spectral estimation with 3 d esprit is introduced and applied to the joint azimuth elevation carrier estimation problem with two dimensional 2 d uniform rectangular arrays the method is highly efficient computationally and is suitable for large arrays detailed computer experiments and comparisons are provided for a 16 spl times 16 array of sensors and heavy noise tls phased averaging 3 d esprit exceeds the 3 d tls unitary esprit estimator by 300 in rmse performance

nonlinear black box models in system identification mathematical foundations we discuss several aspects of the mathematical foundations of the nonlinear black box identification problem we shall see that the quality of the identification procedure is always a result of a certain trade off between the expressive power of the model we try to identify the larger the number of parameters used to describe the model the more flexible is the approximation and the stochastic error which is proportional to the number of parameters a consequence of this trade off is the simple fact that a good approximation technique can be the basis of a good identification algorithm from this point of view we consider different approximation methods and pay special attention to spatially adaptive approximants we introduce wavelet and neuron approximations and show that they are spatially adaptive then we apply the acquired approximation experience to estimation problems finally we consider some implications of these theoretical developments for the practically implemented versions of the spatially adaptive algorithms

a recursive local linear estimator for identification of nonlinear arx systems asymptotical convergence and applications in this paper we propose a recursive local linear estimator rlle for nonparametric identification of nonlinear autoregressive systems with exogenous inputs narx first the rlle is introduced next the strong consistency as well as the asymptotical mean square error properties of the rlle are established and then an application of the rlle to an additive nonlinear system is discussed the rlle provides recursive estimates not only for the function values but also their gradients at fixed points a simulation example is provided to confirm the theoretical analysis

representation and identification of non parametric nonlinear systems of short term memory and low degree of interaction in this paper we propose a new representation which is particularly useful for a class of non parametric nonlinear systems that have short term memory and low degree of interaction advantages and disadvantages of this representation are discussed and compared to existing methods both theoretically and numerically furthermore results regarding structural estimation based on the analysis of variance and on full scale identification are also provided

a new structure exploiting derivation of recursive direct weight optimization the recursive direct weight optimization method is used to solve challenging nonlinear system identification problems this note provides a new derivation and a new interpretation of the method the key underlying the note is to acknowledge and exploit a certain structure inherent in the problem

bounding the zeros of an interval equation in this paper we consider the problem of finding reliably and with certainty all zeros of an interval equation within a given search interval for continuously differentiable functions over real numbers we propose a new formality of interval arithmetic which is treated in a theoretical manner to develop and prove a new method lying on the context of interval newton methods some important theoretical aspects of the new method are stated and proved finally an algorithmic realization of our method is proposed to be applied on a set of test functions where the promising theoretical results are verified

variable selection and identification of high dimensional nonparametric additive nonlinear systems this paper considers variable selection and identification of dynamic additive nonlinear systems via kernel based nonparametric approaches where the number of variables and additive functions may be large variable selection aims to find which additive functions contribute and which do not the proposed variable selection consists of two successive steps at the first step one estimates each additive function by kernel based nonparametric identification approaches without suffering from the curse of dimensionality at the second step a nonnegative garrote estimator is applied to identify which additive functions are nonzero by utilizing the obtained nonparametric estimates of each function under weak conditions the nonparametric estimates of each additive function can achieve the same asymptotic properties as for 1d nonparametric identification based on kernel functions it is also established that the nonnegative garrote estimator turns a consistent estimate for each additive function into a consistent variable selection with probability one as the number of samples tends to infinity two simulation examples are presented to verify the effectiveness of the variable selection and identification approaches proposed in the paper

projection based bayesian recursive estimation of arx model with uniform innovations abstract autoregressive model with exogenous inputs arx is a widely used black box type model underlying adaptive predictors and controllers its innovations stochastic unobserved stimulus of the model are white zero mean with time invariant variance mostly the innovations are assumed to be normal it induces least squares as the adequate estimation procedure the light tails of the normal distribution allow one to accept the unbounded support as a reasonable approximate description of bounded physical quantities in some cases however this approximation is too crude or does not fit subsequent processing for instance robust control design then techniques that deal with unknown but bounded equation errors are used more often than not these techniques give up a stochastic interpretation of innovations and develop estimation algorithms of a min max type the paper assumes bounded innovations but stays within the standard bayesian estimation framework by assuming uniformly distributed innovations the posterior probability density function pdf is first described and approximated by a pdf with a fixed dimensional statistic consequently the estimation can run in real time moreover its limited memory allows for tracking time varying parameters in this manner an alternative to popular forgetting techniques is also obtained the paper provides a complete algorithmic solution and illustrates its behavior

some comments on cp i study the typical configuration of a cp plot when the number of variables in the regression problem is large and there are many weak effects i show that a particular configuration that is very commonly seen can arise in a simple way i give a formula by means of which the risk incurred by the minimum cp rule can be estimated

input output parametric models for non linear systems part ii stochastic non linear systems recursive input output models for non linear multivariate discrete time systems are derived and sufficient conditions for their existence are defined the paper is divided into two parts the first part introduces and defines concepts such as nerode realization multistructural forms and results from differential geometry which are then used to derive a recursive input output model for multivariable deterministic non linear systems the second part introduces several examples compares the derived model with other representations and extends the results to create prediction error or innovation input output models for non linear stochastic systems these latter models are the generalization of the multivariable arm ax models for linear systems and are referred to as narmax or non linear autoregressive moving average models with exogenous inputs

a measurement fusion method for nonlinear system identification using a cooperative learning algorithm identification of a general nonlinear noisy system viewed as an estimation of a predictor function is studied in this article a measurement fusion method for the predictor function estimate is proposed in the proposed scheme observed data are first fused by using an optimal fusion technique and then the optimal fused data are incorporated in a nonlinear function estimator based on a robust least squares support vector machine ls svm a cooperative learning algorithm is proposed to implement the proposed measurement fusion method compared with related identification methods the proposed method can minimize both the approximation error and the noise error the performance analysis shows that the proposed optimal measurement fusion function estimate has a smaller mean square error than the ls svm function estimate moreover the proposed cooperative learning algorithm can converge globally to the optimal measurement fusion function estimate finally the proposed measurement fusion method is applied to arma signal and spatial temporal signal modeling experimental results show that the proposed measurement fusion method can provide a more accurate model

technical communique robust d stability via positivity the main objective of this paper is to convert the general problem of robust d stability of a complex polynomial to positivity in the real domain of the corresponding magnitude function in particular the obtained criterion for hurwitz stability is applied to polynomials with interval parameters and polynomic uncertainty structures the robust stability is verified by testing positivity of a real polynomial using the bernstein subdivision algorithm a new feature in this context is the stopping criterion which is applied whenever the algorithm is inconclusive after a large number of iterations but we can show that at least one zero of the polynomial is closer to the imaginary axis than a prescribed limit

locally weighted regression an approach to regression analysis by local fitting abstract locally weighted regression or loess is a way of estimating a regression surface through a multivariate smoothing procedure fitting a function of the independent variables locally and in a moving fashion analogous to how a moving average is computed for a time series with local fitting we can estimate a much wider class of regression surfaces than with the usual classes of parametric functions such as polynomials the goal of this article is to show through applications how loess can be used for three purposes data exploration diagnostic checking of parametric models and providing a nonparametric regression surface along the way the following methodology is introduced a a multivariate smoothing procedure that is an extension of univariate locally weighted regression b statistical procedures that are analogous to those used in the least squares fitting of parametric functions c several graphical methods that are useful tools for understanding loess estimates and checking the a

kernel based local order estimation of nonlinear nonparametric systems we consider the local order estimation of nonlinear autoregressive systems with exogenous inputs narx which may have different local dimensions at different points by minimizing the kernel based local information criterion introduced in this paper the strongly consistent estimates for the local orders of the narx system at points of interest are obtained the modification of the criterion and a simple procedure of searching the minimum of the criterion are also discussed the theoretical results derived here are tested by simulation examples

an algorithm to determine linear independence of a set of interval vectors this paper presents an algorithm to determine the linear dependence and independence of a set of interval vectors the algorithm is developed on the basis of a necessary and sufficient condition for linear independence of an arbitrary interval vector relative to other linearly independent interval vectors the necessary and sufficient condition can be checked by a linear programming method then the developed necessary and sufficient condition will be employed to check the linear independence of a set of interval vectors through numerical examples the validity of the developed algorithm is illustrated

optimal pointwise adaptive methods in nonparametric estimation the problem of optimal adaptive estimation of a function at a given point from noisy data is considered two procedures are proved to be asymptotically optimal for different settings first we study the problem of bandwidth selection for nonparametric pointwise kernel estimation with a given kernel we propose a bandwidth selection procedure and prove its optimality in the asymptotic sense moreover this optimality is stated not only among kernel estimators with a variable bandwidth the resulting estimator is asymptotically optimal among all feasible estimators the important feature of this procedure is that it is fully adaptive and it works for a very wide class of functions obeying a mild regularity restriction with it the attainable accuracy of estimation depends on the function itself and is expressed in terms of the ideal adaptive bandwidth corresponding to this function and a given kernel the second procedure can be considered as a specialization of the first one under the qualitative assumption that the function to be estimated belongs to some holder class l with unknown parameters l this assumption allows us to choose a family of kernels in an optimal way and the resulting procedure appears to be asymptotically optimal in the adaptive sense in any range of adaptation with 2

estimation theory and uncertainty intervals evaluation in presence of unknown but bounded errors linear families of models and estimators the problem of parameter estimation and of the evaluation of related uncertainty intervals is considered in the case that a probabilistic description of noise and errors is not available of suitable but only a bound on them is known in the present paper only linear parametrizations and estimators are considered very simple and computationally feasible algorithms are derived for evaluating two different types of uncertainty intervals estimates uncertainty intervals parameter uncertainty intervals the relationships between the eui s and the pui s are established and the solution to the problem of the minimum uncertainty intervals estimator is given the latter can be obtained by means of a simple linear programming algorithm

nonlinear black box modeling in system identification a unified overview a nonlinear black box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics there has been considerable recent interest in this area with structures based on neural networks radial basis networks wavelet networks and hinging hyperplanes as well as wavelet transform based methods and models based on fuzzy sets and fuzzy rules this paper describes all these approaches in a common framework from a user s perspective it focuses on what are the common features in the different approaches the choices that have to be made and what considerations are relevant for a successful system identification application of these techniques it is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space these mappings are discussed separately the latter mapping is usually formed as a basis function expansion the basis functions are typically formed from one simple scalar function which is modified in terms of scale and location the expansion from the scalar argument to the regressor space is achieved by a radial or a ridge type approach basic techniques for estimating the parameters in the structures are criterion minimization as well as two step procedures where first the relevant basis functions are determined using data and then a linear least squares step to determine the coordinates of the function approximation a particular problem is to deal with the large number of potentially necessary parameters this is handled by making the number of used parameters considerably less than the number of offered parameters by regularization shrinking pruning or regressor selection