a model based initial guess for estimating parameters in systems of ordinary differential equations summary the inverse problem of parameter estimation from noisy observations is a major challenge in statistical inference for dynamical systems parameter estimation is usually carried out by optimizing some criterion function over the parameter space unless the optimization process starts with a good initial guess the estimation may take an unreasonable amount of time and may converge to local solutions if at all in this article we introduce a novel technique for generating good initial guesses that can be used by any estimation method we focus on the fairly general and often applied class of systems linear in the parameters the new methodology bypasses numerical integration and can handle partially observed systems we illustrate the performance of the method using simulations and apply it to real data

realization of stable models with subspace methods abstract subspace methods for system identification estimate the dynamics of state space models either by using the shiftinvariance property of an estimated observability or controllability matrix or by estimating a state sequence and then solving a least squares problem to obtain the system matrices in either case it is possible for the estimated system to be unstable we present algorithms to find stable approximants to a least squares problem which can then be applied to subspace methods to ensure stability either asymptotic or marginal stability can be ensured in the latter case a pole or a pair of poles being forced to lie on the unit circle in addition some results on a sufficient condition for stability for least squares solutions obtained by the shift invariance approach are derived

multiple positive solutions for quasilinear problems with indefinite sublinear nonlinearity abstract we have established multiplicity of nontrivial solutions for the quasilinear elliptic problem p u h x u 1 g x u in u 0 in u 0 on where r n is a smooth bounded domain p 1 1 p h is a measurable function and g r r is a caratheodory function such that g x 0 0 which is asymptotically linear we also prove existence and nonexistence results when g x t k x t p 1

brief paper enforcing stability constraints in set membership identification of linear dynamic systems in this paper we consider the identification of linear systems a priori known to be stable from input output data corrupted by bounded noise by taking explicitly into account a priori information on system stability a formal definition of the feasible parameter set for a stable linear system is provided on the basis of a detailed analysis of the geometrical structure of the feasible set convex relaxation techniques are presented to solve nonconvex optimization problems arising in the computation of parameter uncertainty intervals properties of the computed relaxed bounds are discussed a simulated example is presented to show the effectiveness of the proposed technique

subspace identification with eigenvalue constraints standard subspace methods for the identification of discrete time linear time invariant systems are transformed into generalized convex optimization problems in which the poles of the system estimate are constrained to lie within user defined convex regions of the complex plane the transformation is done by restating subspace methods such as the minimization of a frobenius norm affine in the estimate parameters allowing the minimization to be augmented with convex constraints the constraints are created using linear matrix inequality regions which generalize standard lyapunov stability to arbitrary convex regions of the complex plane the algorithm is developed for subspace methods based on estimates of the extended observability matrix and methods based on estimates of state sequences but it is extendable to all subspace methods simulation examples demonstrate the utility of the proposed method

parameter estimation for differential equations a generalized smoothing approach summary we propose a new method for estimating parameters in models that are defined by a system of non linear differential equations such equations represent changes in system outputs by linking the behaviour of derivatives of a process to the behaviour of the process itself current methods for estimating parameters in differential equations from noisy data are computationally intensive and often poorly suited to the realization of statistical objectives such as inference and interval estimation the paper describes a new method that uses noisy measurements on a subset of variables to estimate the parameters defining a system of non linear differential equations the approach is based on a modification of data smoothing methods along with a generalization of profiled estimation we derive estimates and confidence intervals and show that these have low bias and good coverage properties respectively for data that are simulated from models in chemical engineering and neurobiology the performance of the method is demonstrated by using real world data from chemistry and from the progress of the autoimmune disease lupus

subspace identification for non linear systems with measured input non linearities this paper uses subspace methods to identify a class of multi input multi output discrete time non linear time varying systems specifically we identify systems that are non linear in measured data and linear in unmeasured states numerical examples are presented to demonstrate the efficacy of the method

kernels for linear time invariant system identification in this paper we study the problem of identifying the impulse response of a linear time invariant lti dynamical system from the knowledge of the input signal and a finite set of noisy output observations we adopt an approach based on regularization in a reproducing kernel hilbert space rkhs that takes into account both continuous and discrete time systems the focus of the paper is on designing spaces that are well suited for temporal impulse response modeling to this end we construct and characterize general families of kernels that incorporate system properties such as stability relative degree absence of oscillatory behavior smoothness or delay in addition we discuss the possibility of automatically searching over these classes by means of kernel learning techniques so as to capture different modes of the system to be identified

identification of positive real models in subspace identification by using regularization in time domain subspace methods for identifying linear time invariant dynamical systems the model matrices are typically estimated from least squares based on estimated kalman filter state sequences and the observed outputs and or inputs it is well known that for an infinite amount of data this least squares estimate of the system matrices is unbiased when the system order is correctly estimated however for a finite amount of data the obtained model may not be positive real in which case the algorithm is not able to identify a valid stochastic model in this note positive realness is imposed by adding a regularization term to a least squares cost function in the subspace identification algorithm the regularization term is the trace of a matrix which involves the dynamic system matrix and the output matrix

brief paper constrained state space system identification with application to structural dynamics constrained identification of state space models representing structural dynamic systems is addressed based on physical insight transfer function constraints are formulated in terms of the state space parametrization a simple example shows that a method tailored for this application which utilizes the non uniqueness of a state space model outperforms the classic sequential quadratic programming method in terms of robustness and convergence properties the method is also successfully applied to real experimental data of a plane frame structure

on consistency of subspace methods for system identification subspace methods for identification of linear time invariant dynamical systems typically consist of two main steps first a so called subspace estimate is constructed this first step usually consists of estimating the range space of the extended observability matrix secondly an estimate of system parametersis obtained based on the subspace estimate in this paper the consistency of a large class of methods for estimating the extended observability matrix is analyzed persistence of excitation conditions on the input signal are given which guarantee consistent estimates for systems with only measurement noise for systems with process noise it is shown that a persistence of excitation condition on the input is not sufficient more precisely an example for which the subspace methods fail to give a consistent estimate of the transfer function is given this failure occurs even if the input is persistently exciting of any order it is also shown that this problem can be eliminated if stronger conditions on the input signal are imposed

sieve estimation of constant and time varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error this article considers estimation of constant and time varying coefficients in nonlinear ordinary differential equation ode models where analytic closed form solutions are not available the numerical solution based nonlinear least squares nls estimator is investigated in this study a numerical algorithm such as the runge kutta method is used to approximate the ode solution the asymptotic properties are established for the proposed estimators considering both numerical error and measurement error the b spline is used to approximate the time varying coefficients and the corresponding asymptotic theories in this case are investigated under the framework of the sieve approach our results show that if the maximum step size of the p order numerical algorithm goes to zero at a rate faster than n 1 p 4 the numerical error is negligible compared to the measurement error this result provides a theoretical guidance in selection of the step size for numerical evaluations of odes moreover we have shown that the numerical solution based nls estimator and the sieve nls estimator are strongly consistent the sieve estimator of constant parameters is asymptotically normal with the same asymptotic co variance as that of the case where the true ode solution is exactly known while the estimator of the time varying parameter has the optimal convergence rate under some regularity conditions the theoretical results are also developed for the case when the step size of the ode numerical solver does not go to zero fast enough or the numerical error is comparable to the measurement error we illustrate our approach with both simulation studies and clinical data on hiv viral dynamics

brief a fast algorithm for the computation of an upper bound on the µ norm a fast algorithm for the computation of the optimally frequency dependent scaled h norm of a finite dimensional lti system is presented it is well known that this quantity is an upper bound to the m norm furthermore it was recently shown to play a special role in the context of slowly time varying uncertainty numerical experimentation suggests that the algorithm generally converges quadratically

subspace identification with guaranteed stability using constrained optimization in system identification the true system is often known to be stable however due to finite sample constraints modeling errors plant disturbances and measurement noise the identified model may be unstable we present a constrained optimization method to ensure asymptotic stability of the identified model in the context of subspace identification methods in subspace identification we first obtain an estimate of the state sequence or extended observability matrix and then solve a least squares optimization problem to estimate the system parameters to ensure asymptotic stability of the identified model we write the least squares optimization problem as a convex linear programming problem with mixed equality quadratic and positive semidefinite constraints suitable for existing convex optimization codes such as sedumi we present examples to illustrate the method and compare to existing approaches

convergence properties of the nelder mead simplex method in low dimensions the nelder mead simplex algorithm first published in 1965 is an enormously popular direct search method for multidimensional unconstrained minimization despite its widespread use essentially no theoretical results have been proved explicitly for the nelder mead algorithm this paper presents convergence properties of the nelder mead algorithm applied to strictly convex functions in dimensions 1 and 2 we prove convergence to a minimizer for dimension 1 and various limited convergence results for dimension 2 a counterexample of mckinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the nelder mead algorithm converges to a nonminimizer it is not yet known whether the nelder mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions

convex parameterizations and fidelity bounds for nonlinear identification and reduced order modelling model instability and poor prediction of long term behavior are common problems when modeling dynamical systems using nonlinear black box techniques direct optimization of the long term predictions often called simulation error minimization leads to optimization problems that are generally nonconvex in the model parameters and suffer from multiple local minima in this paper we present methods which address these problems through convex optimization based on lagrangian relaxation dissipation inequalities contraction theory and semidefinite programming we demonstrate the proposed methods with a model order reduction task for electronic circuit design and the identification of a pneumatic actuator from experiment

maximum likelihood identification of stable linear dynamical systems this paper concerns maximum likelihood identification of linear time invariant state space models subject to model stability constraints we combine expectation maximization em and lagrangian re

parameter estimation for differential equation models using a framework of measurement error in regression models differential equation de models are widely used in many scientific fields including engineering physics and biomedical sciences the so called forward problem the problem of simulations and predictions of state variables for given parameter values in the de models has been extensively studied by mathematicians physicists engineers and other scientists however the inverse problem the problem of parameter estimation based on the measurements of output variables has not been well explored using modern statistical methods although some least squares based approaches have been proposed and studied in this article we propose parameter estimation methods for ordinary differential equation ode models based on the local smoothing approach and a pseudo least squares psls principle under a framework of measurement error in regression models the asymptotic properties of the proposed psls estimator are established we also compare the psls method to the corresponding simulation extrapolation

differential equation modeling of hiv viral fitness experiments model identification model selection and multimodel inference many biological processes and systems can be described by a set of differential equation de models however literature in statistical inference for de models is very sparse we propose statistical estimation model selection and multimodel averaging methods for hiv viral fitness experiments in vitro that can be described by a set of nonlinear ordinary differential equations ode the parameter identifiability of the ode models is also addressed we apply the proposed methods and techniques to experimental data of viral fitness for hiv 1 mutant 103n we expect that the proposed modeling and inference approaches for the de models can be widely used for a variety of biomedical studies

identification of stable models in subspace identification by using regularization in subspace identification methods the system matrices are usually estimated by least squares based on estimated kalman filter state sequences and the observed inputs and outputs for a finite number of data points the estimated system matrix is not guaranteed to be stable even when the true linear system is known to be stable in this paper stability is imposed by using regularization the regularization term used here is the trace of a matrix which involves the dynamical system matrix and a positive semi definite weighting matrix the amount of regularization can be determined from a generalized eigenvalue problem the data augmentation method of chui and maciejowski 1996 is obtained by using specific choices for the weighting matrix in the regularization term

asymptotic efficiency and finite sample properties of the generalized profiling estimation of parameters in ordinary differential equations ordinary differential equations odes are commonly used to model dynamic behavior of a system because many parameters are unknown and have to be estimated from the observed data there is growing interest in statistics to develop efficient estimation procedures for these parameters among the proposed methods in the literature the generalized profiling estimation method developed by ramsay and colleagues is particularly promising for its computational efficiency and good performance in this approach the ode solution is approximated with a linear combination of basis functions the coefficients of the basis functions are estimated by a penalized smoothing procedure with an ode defined penalty however the statistical properties of this procedure are not known in this paper we first give an upper bound on the uniform norm of the difference between the true solutions and their approximations then we use this bound to prove the consistency and asymptotic normality of this estimation procedure we show that the asymptotic covariance matrix is the same as that of the maximum likelihood estimation therefore this procedure is asymptotically efficient for a fixed sample and fixed basis functions we study the limiting behavior of the approximation when the smoothing parameter tends to infinity we propose an algorithm to choose the smoothing parameters and a method to compute the deviation of the spline approximation from solution without solving the odes

local superlinearity and sublinearity for indefinite semilinear elliptic problems in this paper the usual notions of superlinearity and sublinearity for semilinear problems like du f x u are given a local form and extended to indefinite nonlinearities here f x s is allowed to change sign or to vanish for s near zero as well as for s near infinity some of the well known results of ambrosetti bre zis cerami are partially extended to this context

a new frequency domain subspace algorithm with restricted poles location through lmi regions and its application to a wind tunnel test abstractin this paper an innovative method is presented to identify models with a modified frequency domain subspace method this new approach allows to introduce in the subspace resolution constraints on the identified model poles location here a very general formulation is proposed to take into account regions in continuous discrete map this formulation is based on an lmi linear matrix inequalities description where the stability domain represents a particular case these lmi constraints are combined with the frequency domain subspace resolution to obtain identified models whose poles are situated in the specified lmi regions this approach is benchmarked with the loewner one which belongs to the class of frequency domain input output model identification and approximation methods besides the fact that they both belong to the data driven model approximation class they result to have slightly different objectives and show complementary performances this discussion is illustrated in practice wit

forcing function diagnostics for nonlinear dynamics this article investigates the problem of model diagnostics for systems described by nonlinear ordinary differential equations odes i propose modeling lack of fit as a time varying correction to the right hand side of a proposed differential equation this correction can be described as being a set of additive forcing functions estimated from data representing lack of fit in this manner allows us to graphically investigate model inadequacies and to suggest model improvements i derive lack of fit tests based on estimated forcing functions model building in partially observed systems of odes is particularly difficult and i consider the problem of identification of forcing functions in these systems the methods are illustrated with examples from computational neuroscience

nonlinear black box modeling in system identification a unified overview a nonlinear black box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics there has been considerable recent interest in this area with structures based on neural networks radial basis networks wavelet networks and hinging hyperplanes as well as wavelet transform based methods and models based on fuzzy sets and fuzzy rules this paper describes all these approaches in a common framework from a user s perspective it focuses on what are the common features in the different approaches the choices that have to be made and what considerations are relevant for a successful system identification application of these techniques it is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space these mappings are discussed separately the latter mapping is usually formed as a basis function expansion the basis functions are typically formed from one simple scalar function which is modified in terms of scale and location the expansion from the scalar argument to the regressor space is achieved by a radial or a ridge type approach basic techniques for estimating the parameters in the structures are criterion minimization as well as two step procedures where first the relevant basis functions are determined using data and then a linear least squares step to determine the coordinates of the function approximation a particular problem is to deal with the large number of potentially necessary parameters this is handled by making the number of used parameters considerably less than the number of offered parameters by regularization shrinking pruning or regressor selection