System Identification : Problems and perspectives

The paper presents a tutorial overview of the main problems arising from (quantitative) system identification . The fundamental issues of identifiability, overparametrization and model comparison are addressed within a probabilistic framework . After having classified models according to their linearity with respect to the unknown parameters, it is explained why system identification is definitely easier for linear-in-parameters models . The paper also illustrates some speck features of dynamic system identification, namely the distinction between output error and equation error models, the need for persistently exciting input signals and the use of prefiltering . Identification : From data to models Mathematical models of natural and man-ma e systems play an essential role in today's science and technology . The applications of models range from simulation and prediction to control and diagnosis in heterogeneous fields such as all branches of engineering, economics, medicine, physiology, geophysics, and many others . It is therefore natural to pose the question where mathematical models come from . If we depict a model as a box containing the mathematical laws that link the inputs (causes) with the outputs (effects), the three main modelling approaches can associated-with the "colour" ofthe box . inputs outputs -~ model White box modelling : The model is derived directly from some first principles by taking into account the connection between the components of the system . Typical examples are found in mechanical and electrical systems where the physical laws (F = ma, for instance) can be used to predict the effects given the causes. Rather than white, the box should be termed "transparent", in the sense that we know the internal structure of the system Grey box modelling : Sometimes the model obtained by System Identification: Problems and perspectives G. De Nicolao Dipartimento di Informatica e Sistemistica, Universiti di Pavia, Via Ferrata 1, 27100 Pavia, Italy, denicolao @conpro.unipv. it invoking the first principles is incomplete because the value of some parameter is missing . For instance, a planet is subject to the gravitation law but its mass is unknown . In this case, it is necessary to collect experimental data and proceed to a tuning of the unknown parameters until the outputs predicted by the model match the observed data . The internal structure of the box is only partially known (there are grey zones) . Black box modelling: When either the internal structure of the system is unknown or there are no first principles available, the only chance is to collect data and use them to guess the links between inputs and outputs . For instance, this is a common situation in economics and physiology. However, black box modelling is also useful to deal with very complex systems where the white box approach would be time consuming and expensive (an example : modelling the dynamics of an internal combustion engine in order to develop the idle-speed controller) . System identification is concerned with the development and analysis of methods for performing grey and black box modelling (Ljung, 1987), (Sbderstrom and Stoica, 1989), (Haber and Unbehauen, 1990) (Juditsky et al ., 1995), (Sjtiberg et al ., 1995) . Differently from white-box modelling that is intimately related to the specific knowledge domain (mechanics, thermodynamics, electromagnetism, . . . ), system identification covers a number of methodological issues that arise whenever data are processed to obtain a quantitative model . The present contribution is an attempt at giving a tutorial overview of the main problems arising from quantitative model identification . It is hoped that this could be a stimulus towards aprofitable interaction between the quantitative and qualitative viewpoint. Identification as hypersurface reconstruction A mathematical model can be thought of as a mapping f(-) that expresses the dependent variables (the outputs y) as a function of the independent ones (the inputs u) : To make a simple example, consider the fundamental law of dynamics F=ma, that predicts the acceleration (the effect) as function of the applied forceF (the cause) given the mass m. Then, y = a, u = F, and f(u) = u/m. In general, both y and u can be vectors : y _ [y! y2 . yp]', u = [u] u2 . u m ]' . For the sake of simplicity, hereafter it will be assumed that y is scalar (p = 1) . Then, in the simplest case (m = 1), the map (1) corresponds to a curve in the (x,y)-plane a curve. If m = 2, then (1) represents a surface . For m > 2, (1) is an hypersurface in a suitable space . When performing identification, only a finite number of noisy samples are available : y(k) = f(u(k)) + v(k), k = 1, 2, . , N (2) where the term v(k) accounts for the (unavoidable) measurement errors . If we postulate the existence of a model (1) that explains the data, then the identification process is equivalent to reconstructing ("learning") the hypersurface f(u) from the pairs (u(k),y(k)) ("examples", "training set") According to this viewpoint, there are clear connections with function approximation theory (Poggio and Girosi, 1990), learning theory, neural networks (Narendra and Parathasarathy, 1990) and, last but not least, statistics (Beck and Arnold, 1977) whenever the measurement errors are given a probabilistic description. Linear vs . nonlinearmodels At first sight there is no hope to reconstruct the hypersurface flu) from a finite set of pairs (u(k),y(k)) unless some further assumptions are introduced. In this respect it is common to assume that fu) belongs to a family of functions that share the same structure and differ for the values taken by suitable parameters . In other words, f(u) = f(u,0), e = [61 92 ep]' . To make an example, one may assume that a good approximation for f(u) is a third-order polynomial, i . e . f(u,6) = 61 + 02u + 63u2 + e4u 3 3) In this case, the identification problem boils down to estimating the values of the four parameters 6i . As will be discussed later on, the identification problem is harder for models that are nonlinear in parameters . Note that this has little or nothing to do with the possible linearity of the model with respect to the input u . For instance, f(u,e), in (3) is clearly nonlinear with respect to u, but linear with respect to the parameter vector e . Conversely, the model y =f(u,0) = exp(-0 u) (4) is nonlinear with respect to e . Letting y = ! n(y), model (4) can be reduced to a linear one, i . e.