asymptotic optimality of cl and generalized cross validation in ridge regression with application to spline smoothing the asymptotic optimality of mallows c l and generalized cross validation is demonstrated in the setting of ridge regression an application is made to spline smoothing in nonparametric regression a counterexample is given to help understand why sometimes gcv may not be asymptotically optimal the coefficient of variation for the eigenvalues of the information matrix must be large in order to guarantee the optimality of gcv the proff is based on the connection between gcv and stein s unbiased risk estimate

realisation and estimation of piecewise linear output error models piecewise linear systems in input output form can have different switching schedules in this article two categories instant and delayed switching are analysed even though a general piecewise linear state space model cannot be converted into input output form it is shown that it is possible to find state space models representing instant and delayed switching in addition a prediction error minimisation pem method for piecewise linear output error predictors is derived and it is concluded that the instant switching model candidate is not necessarily the most suitable for the parameter estimation procedure

dual mode predictive control algorithm for constrained hammerstein systems in the process industry there exist many systems which can be approximated by a hammerstein model moreover these systems are usually subjected to input magnitude constraints in this paper a multi channel identification algorithm mcia is proposed in which the coefficient parameters are identified by least squares estimation lse together with a singular value decomposition svd technique compared with traditional single channel identification algorithms the present method can enhance the approximation accuracy remarkably and provide consistent estimates even in the presence of coloured output noises under relatively weak assumptions on the persistent excitation pe condition of the inputs then to facilitate the following controller design this mcia is converted into a two stage single channel identification algorithm ts scia which preserves most of the advantages of mcia with this ts scia as the inner model a dual mode non linear model predictive control nmpc algorithm is developed in detail over a finite horizon an optimal input profile found by solving a open loop optimal control problem drives the non linear system state into the terminal invariant set afterwards a linear output feedback controller steers the state to the origin asymptotically in contrast to the traditional algorithms the present method has a maximal stable region a better steady state performance and a lower computational complexity finally simulation results on a heat exchanger are presented to show the efficiency of both the identification and the control algorithms

non parametric nonlinear system identification a data driven orthogonal basis function approach in this paper a data driven orthogonal basis function approach is proposed for non parametric fir nonlinear system identification the basis functions are not fixed a priori and match the structure of the unknown system automatically this eliminates the problem of blindly choosing the basis functions without a priori structural information further based on the proposed basis functions approaches are proposed for model order determination and regressor selection along with their theoretical justifications

identification of hammerstein systems with quantized observations this work is concerned with identification of hammerstein systems whose outputs are measured by quantized sensors the system consists of a memoryless nonlinearity that is polynomial and possibly noninvertible followed by a linear subsystem the parameters of linear and nonlinear parts are unknown but have known orders input design identification algorithms and their essential properties are presented under the assumptions that the distribution function of the noise is known and the quantization thresholds are known the concept of strongly scaled full rank signals is introduced to capture the essential conditions under which the hammerstein system can be identified with set valued observations under strongly scaled full rank conditions a strongly convergent algorithm is constructed asymptotic consistency and efficiency of the algorithm are investigated

variable bandwidth kernel estimators of regression curves on montre qu en termes de l erreur en moyenne quadratique integree asymptotique les estimateurs du noyau avec un certain choix de largeur de bande local sont superieurs aux estimateurs ordinaires du noyau avec un choix de largeur de bande global si on utilise des largeurs de bandes optimales

density estimation by wavelet thresholding density estimation is a commonly used test case for nonparametric estimation methods we explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients minimax rates of convergence are studied over a large range of besov function classes b pq and for a range of global l p error measures 1 p p some form of nonlinearity is essential since the minimax linear estimators are suboptimal by polynomial powers of n a second approach using an approximation of a gaussian white noise model in a mallows metric is used to attain exactly optimal rates of convergence for quadratic error p 2

nonparametric regression optimal local bandwidth choice summary kernel estimators of a regression function are investigated the bandwidths are locally chosen by a data driven method based on the minimization of a local cross validation criterion this method is shown to be asymptotically optimal with respect to local quadratic measures of errors monte carlo experiments are presented and finally the method is applied to some data of medical interest

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

nonlinear system identification via direct weight optimization a general framework for estimating nonlinear functions and systems is described and analyzed in this paper identification of a system is seen as estimation of a predictor function the considered predictor function estimate at a particular point is defined to be affine in the observed outputs and the estimate is defined by the weights in this expression for each given point the maximal mean square error or an upper bound of the function estimate over a class of possible true functions is minimized with respect to the weights which is a convex optimization problem this gives different types of algorithms depending on the chosen function class it is shown how the classical linear least squares is obtained as a special case and how unknown but bounded disturbances can be handled most of the paper deals with the method applied to locally smooth predictor functions it is shown how this leads to local estimators with a finite bandwidth meaning that only observations in a neighborhood of the target point will be used in the estimate the size of this neighborhood the bandwidth is automatically computed and reflects the noise level in the data and the smoothness priors the approach is applied to a number of dynamical systems to illustrate its potential

an adaptive multi scale basis method for modelling discrete time nonlinear dynamical systems based on locally applicable autoregressive moving average with exogenous armax models this paper proposes two new adaptive modelling schemes for discrete time non linear dynamic systems based on the concept of a multiscale which originates in approximation theory a general result on the convergence of modelling based on the multiscale basis functions with a least square estimated is derived with the advantage of the multiresolution nature of the multiscale basis the new modelling scheme can be used to capture both the global and local characteristics of a non linear system over distinct scales the algorithms also provide a trade off between the model structure model size and the modelling error one of which emphasizes the modelling error with variable model structure whilst the other stresses a desired model size with better modelling error data based modelling examples are used to demonstrate the effectiveness of the multiscale modelling schemes with different multiscale basis

identification of mimo hammerstein models using least squares support vector machines this paper studies a method for the identification of hammerstein models based on least squares support vector machines ls svms the technique allows for the determination of the memoryless static nonlinearity as well as the estimation of the model parameters of the dynamic arx part this is done by applying the equivalent of bai s overparameterization method for identification of hammerstein systems in an ls svm context the siso as well as the mimo identification cases are elaborated the technique can lead to significant improvements with respect to classical overparameterization methods as illustrated in a number of examples another important advantage is that no stringent assumptions on the nature of the nonlinearity need to be imposed except for a certain degree of smoothness

adaptive learning control for finite interval tracking based on constructive function approximation and wavelet using a constructive function approximation network an adaptive learning control alc approach is proposed for finite interval tracking problems the constructive function approximation network consists of a set of bases and the number of bases can evolve when learning repeats the nature of the basis allows the continuous adaptive learning of parameters when the network undergoes any structural changes and consequently offers the flexibility in tuning the network structure the expandability of the bases guarantees precision of the function approximation and avoids the trial and error procedure in structure selection for any fixed structure network two classes of unknown nonlinear functions namely either global l2 or local l2 with a known bounding function are taken into consideration using the lyapunov method the existence of solution and the convergence property of the proposed alc system are discussed in a rigorous manner by virtue of the celebrated orthonormal and multiresolution properties wavelet network is used as the universal function approximator with the weights tuned by the proposed adaptive learning mechanism

error estimates for approximate optimization by the extended ritz method an alternative to the classical ritz method for approximate optimization is investigated in the extended ritz method sets of admissible solutions are approximated by their intersections with sets of linear combinations of all n tuples of functions from a given basis this alternative scheme called variable basis approximation includes functions computable by trigonometric polynomials with free frequencies free node splines neural networks and other nonlinear approximating families estimates of rates of approximate optimization by the extended ritz method are derived upper bounds on rates of convergence of suboptimal solutions to the optimal one are expressed in terms of the degree n of variable basis functions the modulus of continuity of the functional to be minimized the modulus of tikhonov well posedness of the problem and certain norms tailored to the type of basis the results are applied to convex best approximation and to kernel methods in machine learning

brief fast approximate identification of nonlinear systems in this paper a method is presented to extend the classical identification methods for linear systems towards nonlinear modelling of linear systems that suffer from nonlinear distortions a well chosen general nonlinear model structure is proposed that is identified in a two step procedure first a best linear approximation is identified using the classical linear identification methods in the second step the nonlinear extensions are identified with a linear least squares method the proposed model not only includes wiener and hammerstein systems it is also suitable to model nonlinear feedback systems the stability of the nonlinear model can be easily verified the method is illustrated on experimental data

robust adaptive observers for nonlinear systems with bounded disturbances since existing adaptive observers for nonlinear systems may generate unbounded parameter estimates in the presence of bounded disturbances robust adaptive observers are presented which prevent parameter estimate drift in addition the input to state stability of the error dynamics with respect to disturbances and parameter time derivatives is guaranteed by generalizing a persistency of excitation result asymptotic convergence of state estimation errors is still achieved for systems in adaptive observer form when disturbances are not present by a suitable extension of barbalat s lemma

nonlinear black box models in system identification mathematical foundations we discuss several aspects of the mathematical foundations of the nonlinear black box identification problem we shall see that the quality of the identification procedure is always a result of a certain trade off between the expressive power of the model we try to identify the larger the number of parameters used to describe the model the more flexible is the approximation and the stochastic error which is proportional to the number of parameters a consequence of this trade off is the simple fact that a good approximation technique can be the basis of a good identification algorithm from this point of view we consider different approximation methods and pay special attention to spatially adaptive approximants we introduce wavelet and neuron approximations and show that they are spatially adaptive then we apply the acquired approximation experience to estimation problems finally we consider some implications of these theoretical developments for the practically implemented versions of the spatially adaptive algorithms

truncated chebyshev series approximation of fuzzy systems for control and nonlinear system identification the lack of mathematical models that pertains fuzzy control systems imposes a serious drawback regarding some important tasks such as stability analysis and system identification for that purpose the availability of analytical expressions for the fuzzy systems is very important this paper provides a systematic practical way of approximating fuzzy systems by chebyshev polynomials which depend on a finite number of parameters the proposed methodology is illustrated with two examples a a control problem concerning a tubular reactor which depending on the operating conditions may exhibit multiple steady states and b the problem of identifying a continuous stirred tank reactor which at certain values of structural parameters exhibits stable or unstable steady states or limit cycles

a bounded error approach to piecewise affine system identification this paper proposes a three stage procedure for parametric identification of piecewise affine autoregressive exogenous pwarx models the first stage simultaneously classifies the data points and estimates the number of submodels and the corresponding parameters by solving the partition into a minimum number of feasible subsystems min pfs problem for a suitable set of linear complementary inequalities derived from data second a refinement procedure reduces misclassifications and improves parameter estimates the third stage determines a polyhedral partition of the regressor set via two class or multiclass linear separation techniques as a main feature the algorithm imposes that the identification error is bounded by a quantity spl delta such a bound is a useful tuning parameter to trade off between quality of fit and model complexity the performance of the proposed pwa system identification procedure is demonstrated via numerical examples and on experimental data from an electronic component placement process in a pick and place machine

nonlinear adaptive control using neural networks and multiple models in this paper adaptive control of a class of nonlinear discrete time dynamical systems with boundedness of all signals is established by using a linear robust adaptive controller and a neural network based nonlinear adaptive controller and switching between them by a suitably defined switching law the linear controller when used alone assures boundedness of all the signals but not satisfactory performance the nonlinear controller may result in improved response but may also result in instability by using a switching scheme it is demonstrated that improved performance and stability can be simultaneously achieved

multiscale support vector learning with projection operator wavelet kernel for nonlinear dynamical system identification a giant leap has been made in the past couple of decades with the introduction of kernel based learning as a mainstay for designing effective nonlinear computational learning algorithms in view of the geometric interpretation of conditional expectation and the ubiquity of multiscale characteristics in highly complex nonlinear dynamic systems 1 3 this paper presents a new orthogonal projection operator wavelet kernel aiming at developing an efficient computational learning approach for nonlinear dynamical system identification in the framework of multiresolution analysis the proposed projection operator wavelet kernel can fulfill the multiscale multidimensional learning to estimate complex dependencies the special advantage of the projection operator wavelet kernel developed in this paper lies in the fact that it has a closed form expression which greatly facilitates its application in kernel learning to the best of our knowledge it is the first closed form orthogonal projection wavelet kernel reported in the literature it provides a link between grid based wavelets and mesh free kernel based methods simulation studies for identifying the parallel models of two benchmark nonlinear dynamical systems confirm its superiority in model accuracy and sparsity

parametric and nonparametric curve fitting we are concerned with convergence issues in the identification of a static nonlinear function our investigation focuses on properties of the input signal that ensure convergence of the estimate both parametric and nonparametric approaches are considered in the parametric case we offer sufficient conditions under which the estimated parameters converge to their true values almost surely for the nonparametric case we offer necessary and sufficient conditions under which the estimated function converges almost surely to the true nonlinearity

enhanced frequency analysis using wavelets an enhanced frequency analysis method is introduced using a phase shift wavelet representation to analyse individual frequency components and to reveal features of a given signal the frequency representation provides details at each time location without the loss caused by local orthogonality between the wavelet basis functions and the given signal which occurs when a normal wavelet decomposition is employed movement of the local orthogonality locations corresponding to the phase shift of the wavelet basis functions are studied and it is shown that these movements are important and can be used to reveal features of the signal in the frequency domain such as continuities smoothness and trends several examples are presented to illustrate the method and an application to a permanent magnet synchronous drive system is provided to demonstrate the approach

asymptotic properties of least squares estimates of hammerstein wiener models this paper investigates the asymptotic properties of least squares estimates of hammerstein wiener model structures and in doing so establishes consistency and asymptotic normality under fairly mild conditions on the additive noise process the inputs and the static non linearities in relation to the asymptotic distributional results a consistent procedure for the estimation of the asymptotic variance of the parameter estimates is provided a key theme of this paper is to demonstrate how recent results from the econometrics literature may be employed in an engineering setting in this respect the hammerstein wiener model structure serves as a demonstration example a simulation study complements the theoretical findings

a solution of the eigenproblem for undamped gyroscopic systems with the lanczos algorithm this paper presents an efficient numerical solution of the quadratic eigenproblem arising in the analysis of gyroscopic systems such problems are known to reduce to a generalized linear eigenproblem defined by two real non singular matrices one symmetric and one skew symmetric for this class of problems the general lanczos algorithm for unsymmetric matrices is shown to simplify considerably and yields an efficient solution of the problem full advantage can be taken of the sparsity of the matrices and of the specific nature of gyroscopic systems numerical examples are presented which demonstrate the efficiency and accuracy of the solution procedure

asymptotic optimality for c_p c_l cross validation and generalized cross validation discrete index set c p c l cross validation and generalized cross validation are useful data driven techniques for selecting a good estimate from a proposed class of linear estimates the asymptotic behaviors of these procedures are studied some easily interpretable conditions are derived to demonstrate the asymptotic optimality it is argued that cross validation and generalized cross validation can be viewed as some special ways of applying c l applications in nearest neighbor nonparametric regression and in model selection are discussed in detail

an interactive term approach to non parametric fir nonlinear system identification in this technical note a framework for designing specially structured input sequences for non parametric nonlinear system identification is presented so that interaction terms which describe interactions among variables can be identified separately in a sense the approach decomposes a general difficult nonlinear identification problem into a number of problems that are of lower orders corresponding identification algorithms are proposed

generalized neurofuzzy network modeling algorithms using bezier bernstein polynomial functions and additive decomposition this paper introduces a new neurofuzzy model construction algorithm for nonlinear dynamic systems based upon basis functions that are bezier bernstein polynomial functions this paper is generalized in that it copes with n dimensional inputs by utilising an additive decomposition construction to overcome the curse of dimensionality associated with high n this new construction algorithm also introduces univariate bezier bernstein polynomial functions for the completeness of the generalized procedure like the b spline expansion based neurofuzzy systems bezier bernstein polynomial function based neurofuzzy networks hold desirable properties such as nonnegativity of the basis functions unity of support and interpretability of basis function as fuzzy membership functions moreover with the additional advantages of structural parsimony and delaunay input space partition essentially overcoming the curse of dimensionality associated with conventional fuzzy and rbf networks this new modeling network is based on additive decomposition approach together with two separate basis function formation approaches for both univariate and bivariate bezier bernstein polynomial functions used in model construction the overall network weights are then learnt using conventional least squares methods numerical examples are included to demonstrate the effectiveness of this new data based modeling approach

a novel adaptive fuzzy variable structure control for a class of nonlinear uncertain systems via backstepping abstract this paper presents a new design of adaptive fuzzy variable structure control to solve the traditional problem of model reference adaptive control mrac for a class of single input single output minimum phase uncertain nonlinear systems via backstepping instead of taking the tedious coordinate transformation and yielding a hard high gain controller we introduce smooth b spline type membership functions into the controller so as to compensate for the uncertainties much softer i e in a much smoother and locally weighted manner to be rigorous it is shown that the stability of the closed loop system can be assured and the tracking error can globally approach to an arbitrary preset dead zone range in order to demonstrate the effectiveness of the developed novel controller an example is extensively simulated to provide quite satisfactory performance

identification of iir nonlinear systems without prior structural information in this paper we propose a kernel method for identification of infinite impulse response iir nonlinear systems without prior structural information the main result of the paper is to establish the conditions for asymptotic convergence in terms of the input to output exponential stability the performance of the method is tested on real world applications and computer simulations the results obtained show the efficacy of the method

projection based approximation and a duality with kernel methods projection pursuit regression and kernel regression are methods for estimating a smooth function of several variables from noisy data obtained at scattered sites methods based on local averaging can perform poorly in high dimensions curse of dimensionality intuition and examples have suggested that projection based approaches can provide better fits for what sorts of regression functions is this true when and by how much do projection methods reduce the curse of dimensionality we make a start by focusing on the two dimensional problem and study the l 2 approximation error bias of the two procedures with respect to gaussian measure let ra stand for a certain ppr type approximation and ka for a particular kernel type approximation building on a simple but striking duality for polynomials we show that ra behaves significantly better than the minimax rate of approximation for radial functions while ka performs significantly better than the minimax rate for harmonic functions in fact the rate improvements carry over to large classes ra behaving very well for functions with enough angular smoothness oscillating slowly with angle while ka behaves very well for functions with enough laplacian smoothness oscillations averaging out locally the rate improvements matter they are equivalent to lowering the dimensionality of the problem for example for functions with nice tail behavior ra behaves as if the dimensionality of the problem were 1 5 rather than its nominal value 2 also ra and ka are complementary for a given function if one method offers a dimensionality reduction the other does not

optimal bandwidth selection in nonparametric regression function estimation kernel estimators of an unknown multivariate regression function are investigated a bandwidth selection rule is considered which can be formulated in terms of cross validation under mild assumptions on the kernel and the unknown regression function it is seen that this rule is asymptotically optimal

bandwidth choice for nonparametric regression this paper is concerned with the problem of choosing a bandwidth parameter for nonparametric regression we analyze a tapered fourier series estimate and discuss the relationship of this estimate to a kernel estimate we first consider a method based on an unbiased estimate of mean square error and show that the bandwidth thus chosen is asymptotically optimal other methods are examined as well and are shown to be asymptotically equivalent a small simulation shows however that for small or moderate sample size the methods perform quite differently

identification of structured nonlinear systems this paper is concerned with the identification of static nonlinear components in a complex interconnected system these nonlinear components are treated nonparametrically in the sense that no natural parameterization is assumed to be available

variable bandwidth and local linear regression smoothers in this paper we introduce an appealing nonparametric method for estimating the mean regression function the proposed method combines the ideas of local linear smoothers and variable bandwidth hence it also inherits the advantages of both approaches we give expressions for the conditional mse and mise of the estimator minimization of the mise leads to an explicit formula for an optimal choice of the variable bandwidth moreover the merits of considering a variable bandwidth are discussed in addition we show that the estimator does not have boundary effects and hence does not require modifications at the boundary the performance of a corresponding plug in estimator is investigated simulations illustrate the proposed estimation method

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

minimax estimation via wavelet shrinkage we attempt to recover an unknown function from noisy sampled data using orthonormal bases of compactly supported wavelets we develop a nonlinear method which works in the wavelet domain by simple nonlinear shrinkage of the empirical wavelet coefficients the shrinkage can be tuned to be nearly minimax over any member of a wide range of triebel and besov type smoothness constraints and asymptotically minimax over besov bodies with p q linear estimates cannot achieve even the minimax rates over triebel and besov classes with p 2 so the method can significantly outperform every linear method e g kernel smoothing spline sieve in a minimax sense variants of our method based on simple threshold nonlinear estimators are nearly minimax our method possesses the interpretation of spatial adaptivity it reconstructs using a kernel which may vary in shape and bandwidth from point to point depending on the data least favorable distributions for certain of the triebel and besov scales generate objects with sparse wavelet transforms many real objects have similarly sparse transforms which suggests that these minimax results are relevant for practical problems sequels to this paper which was first drafted in november 1990 discuss practical implementation spatial adaptation properties universal near minimaxity and applications to inverse problems

hinging hyperplanes for regression classification and function approximation a hinge function y h x consists of two hyperplanes continuously joined together at a hinge in regression prediction classification pattern recognition and noiseless function approximation use of sums of hinge functions gives a powerful and efficient alternative to neural networks with computation times several orders of magnitude less than is obtained by fitting neural networks with a comparable number of parameters a simple and effective method for finding good hinges is presented

nonlinear regulation the piecewise linear approach this paper approaches nonlinear control problems through the use of discrete time piecewise linear systems these are systems whose next state and output maps are both described by pl maps i e by maps which are affine on each of the components of a finite polyhedral partition various results on state and output feedback observers and inverses standard for linear systems are proved for pl systems many of these results are then used in the study of more general both discrete and continuous time systems using suitable approximations

set membership prediction of nonlinear time series in this paper a prediction method for nonlinear time series based on a set membership sm approach is proposed the method does not require the choice of the functional form of the model used for prediction but assumes a bound on the rate of variation of the regression function defining the model at the contrary most of the existing prediction methods need the choice of a functional form of the regression function or of state equations piecewise linear quadratic etc and this choice is usually the result of heuristic searches these searches may be quite time consuming and lead only to approximate model structures whose errors may be responsible of bad propagation of prediction errors especially for the multistep ahead prediction moreover the method proposed in this paper assumes only that the noise is bounded in contrast with statistical approaches which rely on noise assumptions such as stationarity ergodicity uncorrelation type of distribution etc the validity of these assumptions may be difficult to be reliably tested in many applications and is certainly lost in presence of approximate modeling in the present sm approach using a result developed in a previous paper the values of the bounds on the gradient of the regression function and on the noise can be suitably assessed to verify the validity tests two almost optimal prediction algorithms are then derived the second one having improved optimal properties over the first one at the expense of an increased computational complexity the method is tested and compared with other literature methods on the well known wolf sunspot numbers series widely used in the time series literature as a benchmark test and on the prediction of vertical dynamics of vehicles with controlled suspensions a simulation example is also presented to investigate how much conservative the sm approach may be in the most adverse situation where data are generated by a linear autoregressive ar model driven by i i d gaussian white noise and the sm prediction is compared with the optimal statistical predictor which makes use of the exact assumptions

a quasi armax approach to modelling of non linear systems this paper proposes a class of quasi armax models for non linear systems similar to ordinary non linear armax models the quasi armax models are flexible black box models but they have various linearity properties similar to those of linear armax models a modelling scheme is introduced to construct models consisting of two parts a macro part and a kernel part by using taylor expansion and other mathematical transformation techniques it is first constructed as a class of quasi armax interfaces macro parts that have various linearity properties but contain some complicated coefficients mimo neurofuzzy models kernel parts are then introduced to represent the complicated coefficients it is shown that the proposed quasi armax models have both good approximation ability and some easy to use properties the proposed models have been successfully applied to prediction fault detection and adaptive control of non linear systems

wavelet shrinkage asymptopia much recent effort has sought asymptotically minimax methods for recovering infinite dimensional objects curves densities spectral densities images from noisy data a now rich and complex body of work develops nearly or exactly minimax estimators for an array of interesting problems unfortunately the results have rarely moved into practice for a variety of reasons among them being similarity to known methods computational intractability and lack of spatial adaptivity we discuss a method for curve estimation based on n noisy data translate the empirical wavelet coefficients towards the origin by an amount 2 log n n the proposal differs from those in current use is computationally practical and is spatially adaptive it thus avoids several of the previous objections further the method is nearly minimax both for a wide variety of loss functions pointwise error global error measured in l p norms pointwise and global error in estimation of derivatives and for a wide range of smoothness classes including standard holder and sobolev classes and bounded variation this is a much broader near optimality than anything previously proposed we draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves finally the theory underlying the method is interesting as it exploits a correspondence between statistical questions and questions of optimal recovery and information based complexity

fault detection and isolation in nonlinear dynamic systems a combined input output and local approach the problem of fault detection and isolation fdi is addressed for nonlinear dynamic systems modeled by polynomial differential algebraic equations ritt s algorithm is used to obtain an input output representation of the monitored system by eliminating unknown variables a local approach to change detection is then applied to the input output representation for fdi residual generation and evaluation simulation results on a microbial growth process are provided which illustrate the relevance of the proposed fdi method

inference of disjoint linear and nonlinear sub domains of a nonlinear mapping this paper investigates new ways of inferring nonlinear dependence from measured data the existence of unique linear and nonlinear sub spaces which are structural invariants of general nonlinear mappings is established and necessary and sufficient conditions determining these sub spaces are derived the importance of these invariants in an identification context is that they provide a tractable framework for minimising the dimensionality of the nonlinear modelling task specifically once the linear nonlinear sub spaces are known by definition the explanatory variables may be transformed to form two disjoint sub sets spanning respectively the linear and nonlinear sub spaces the nonlinear modelling task is confined to the latter sub set which will typically have a smaller number of elements than the original set of explanatory variables constructive algorithms are proposed for inferring the linear and nonlinear sub spaces from noisy data

unified set membership theory for identification prediction and filtering of nonlinear systems the problem of making inferences from data measured on nonlinear systems is investigated within a set membership sm framework and it is shown that identification prediction and filtering can be treated as specific instances of the general presented theory the sm framework presents an alternative view to the parametric statistical ps framework more widely used for studying the above specific problems in particular in the sm framework a bound only on the gradient of the model regression function is assumed at difference from ps methods which assume the choice of a parametric functional form of the regression function moreover the sm theory assumes only that the noise is bounded in contrast with ps approaches which rely on noise assumptions such as stationarity uncorrelation type of distribution etc the basic notions and results of the general inference making theory are presented moreover some of the main results that can be obtained for the specific inferences of identification prediction and filtering are reviewed concluding comments on the presented results are also reported focused on the discussion of two basic questions what may be gained in identification prediction and filtering of nonlinear systems by using the presented sm framework instead of the widely diffused ps framework why sm methods could provide stronger results than the ps methods requiring weaker assumptions on system and on noise

on board component fault detection and isolation using the statistical local approach we describe both the key principles and real application examples of a unified theory which allows us to perform the on board incipient fault detection and isolation tasks involved in monitoring for condition based maintenance this theory is known under the name of local approach and it is especially suited to component faults it aims at transforming complex detection problems concerning a parameterized stochastic process into the universal problem of monitoring the mean of a gaussian vector based on a small deviation assumption the key tools are the first order taylor expansion and the asymptotic gaussianity of a convenient parameter estimating function ml ls iv and subspace identification methods are addressed in this perspective in the case of dynamic processes given in state space form the approach may also call for observer based state estimation or state elimination experiments concerning both simulated and real data for linear and nonlinear dynamical processes are reported how the key principles and features of the local approach compare with those of other approaches is discussed

nonlinear identification with local model networks using gtls techniques and equality constraints local model networks approximate a nonlinear system through multiple local models fitted within a partition space the main advantage of this approach is that the identification of complex nonlinear processes is alleviated by the integration of structured knowledge about the process this paper extends these concepts by the integration of quantitative process knowledge into the identification procedure quantitative knowledge describes explicit dependences between inputs and outputs and is integrated in the parameter estimation process by means of equality constraints for this purpose a constrained generalized total least squares algorithm for local parameter estimation is presented furthermore the problem of proper integration of constraints in the partitioning process is treated where an expectation maximization procedure is combined with constrained parameter estimation the benefits and the applicability of the proposed concepts are demonstrated by means of two illustrative examples and a practical application using real measurement data

non linear system identification using hammerstein and non linear feedback models with piecewise linear static maps we consider the identification of hammerstein non linear feedback models by approximating internal non linearities using piecewise linear static maps the resulting method utilizes a point slope parameterization that leads to a computationally tractable optimization problem the computational appeal of this technique is derived from the fact that the method only requires a matrix inverse and singular value decomposition furthermore the identification method simultaneously identifies the linear dynamic and static non linear blocks without requiring prior assumptions on the form of the static non linearity

dependence of computational models on input dimension tractability of approximation and optimization tasks the role of input dimension d is studied in approximating in various norms target sets of d variable functions using linear combinations of adjustable computational units results from the literature which emphasize the number n of terms in the linear combination are reformulated and in some cases improved with particular attention to dependence on d for worst case error upper bounds are given in the factorized form d n where is nonincreasing typically n n 1 2 target sets of functions are described for which the function is a polynomial some important cases are highlighted where decreases to zero as d for target functions extent e g the size of domains in rd where they are defined scale e g maximum norms of target functions and smoothness e g the order of square integrable partial derivatives may depend on d and the influence of such dimension dependent parameters on model complexity is considered results are applied to approximation and solution of optimization problems by neural networks with perceptron and gaussian radial computational units

a neural state estimator with bounded errors for nonlinear systems a neural state estimator is described acting on discrete time nonlinear systems with noisy measurement channels a sliding window quadratic estimation cost function is considered and the measurement noise is assumed to be additive no probabilistic assumptions are made on the measurement noise nor on the initial state novel theoretical convergence results are developed for the error bounds of both the optimal and the neural approximate estimators to ensure the convergence properties of the neural estimator a minimax tuning technique is used the approximate estimator can be designed offline in such a way as to enable it to process on line any possible measure pattern almost instantly

brief paper initializing wiener hammerstein models based on partitioning of the best linear approximation this paper describes a new algorithm for initializing and estimating wiener hammerstein models which consist of two linear parts with a static nonlinearity in between the algorithm makes use of the best linear model of the system which is a consistent estimate of the systems dynamics for gaussian excitations the linear model is split in all possible ways into two sub models for all possible splits a wiener hammerstein model is initialized which means that a nonlinearity is introduced in between the two sub models the linear parameters of this nonlinearity can be estimated using least squares all initialized models can then be ranked depending on the fit typically one is only interested in the best one for which all parameters are fitted using prediction error minimization the paper explains the algorithm in detail and consistency of the initialization is proven computational aspects are investigated showing that in most realistic cases the number of splits of the initial linear model remains low enough to make the algorithm useful the algorithm is illustrated on an example where it is shown that the initialization is a tool to avoid many local minima

geometric upper bounds on rates of variable basis approximation in this paper approximation by linear combinations of an increasing number n of computational units with adjustable parameters such as perceptrons and radial basis functions is investigated geometric upper bounds on rates of convergence of approximation errors are derived the bounds depend on certain parameters specific for each function to be approximated the results are illustrated by examples of values of such parameters in the case of approximation by linear combinations of orthonormal functions

nonlinear black box modeling in system identification a unified overview a nonlinear black box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics there has been considerable recent interest in this area with structures based on neural networks radial basis networks wavelet networks and hinging hyperplanes as well as wavelet transform based methods and models based on fuzzy sets and fuzzy rules this paper describes all these approaches in a common framework from a user s perspective it focuses on what are the common features in the different approaches the choices that have to be made and what considerations are relevant for a successful system identification application of these techniques it is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space these mappings are discussed separately the latter mapping is usually formed as a basis function expansion the basis functions are typically formed from one simple scalar function which is modified in terms of scale and location the expansion from the scalar argument to the regressor space is achieved by a radial or a ridge type approach basic techniques for estimating the parameters in the structures are criterion minimization as well as two step procedures where first the relevant basis functions are determined using data and then a linear least squares step to determine the coordinates of the function approximation a particular problem is to deal with the large number of potentially necessary parameters this is handled by making the number of used parameters considerably less than the number of offered parameters by regularization shrinking pruning or regressor selection