stochastic theory of continuous time state space identification this paper presents theory algorithms and validation results for system identification of continuous time state space models from finite input output sequences the algorithms developed are methods of subspace model identification and stochastic realization adapted to the continuous time context the resulting model can be decomposed into an input output model and a stochastic innovations model using the riccati equation we have designed a procedure to provide a reduced order stochastic model that is minimal with respect to system order as well as the number of stochastic inputs thereby avoiding several problems appearing in standard application of stochastic realization to the model validation problem

mimo system identification state space and subspace approximations versus transfer function and instrumental variables the identification of multi input multi output mimo linear systems has previously received a new impetus with the introduction of the state space ss approach based on subspace approximations this approach has immediately gained popularity owing to the fact that it avoids the use of canonical forms requires the determination of only one structural parameter and has been empirically shown to yield mimo models with good accuracy in many cases however the ss approach suffers from several drawbacks there is no well established rule tied to this approach for determining the structural parameter and perhaps more important the ss parameter estimates depend on the data in a rather complicated way which renders almost futile any attempt to analyze and optimize the performance of the estimator in this paper we consider a transfer function tf approach based on instrumental variables iv as an alternative to the ss approach we use the simplest canonical tf parameterization in which the denominator is equal to a scalar polynomial times the identity matrix the analysis and optimization of the statistical accuracy of the tf approach is straightforward additionally a simple test tailored to this approach is devised for estimating the single structural parameter needed a simulation study in which we compare the performances of the ss and the tf approaches shows that the latter can provide more accurate models than the former at a lower computational cost

on the law of the iterated logarithm for trigonometric series with bounded gaps let n k k 1 be an increasing sequence of positive integers bobkov and gotze proved that if the distribution of label distr frac cos 2 pi n 1 x cdots cos 2 pi n n x sqrt n qquad quad quad quad 1 converges to a gaussian distribution then the value of the variance is bounded from above by 1 2 lim sup k 2n k in particular it is impossible that for a sequence n k k 1 with bounded gaps i e n k 1 n k c for some constant c the distribution of 1 converges to a gaussian distribution with variance 2 1 2 or larger in this paper we show that the situation is considerably different in the case of the law of the iterated logarithm we prove the existence of an increasing sequence of positive integers satisfying n k 1 n k leq 2 such that limsup n to infty frac sum k 1 n cos 2 pi n k x sqrt 2n log log n infty quad a e

identification of continuous time models the paper considers the problem of estimation of the transfer function of a continuous time dynamic system in the presence of colored noise the author introduces an operator transformation that allows for keeping a continuous time parametrization whereas the parameter estimation can be made by means of a discrete time maximum likelihood algorithm a comparison is made between the performance of the new method in comparison with a standard identification of an armax model the method is useful in cases where it is important to estimate the coefficients of a continuous time transfer function and where it is important to maintain a physical interpretation of the transfer function results

an improved lower bound on the sensitivity complexity of graph properties turan 1984 11 initiated the study of the sensitivity complexity of graph properties he conjectured that for any non trivial graph properties on n vertices the sensitivity complexity is at least n 1 he proved an email protected lower bound for sensitivity in his paper turan 1984 11 wegener 1985 12 proved this conjecture for all monotone graph properties in this paper we improve turan s lower bound to 617n 0 35n we hope that this will shed some light on the proof of turan s conjecture

brief paper stochastic system transformation using generalized orthonormal basis functions with applications to continuous time system identification this paper studies the system transformation using generalized orthonormal basis functions that include the laguerre basis as a special case the transformation of the deterministic systems is studied in the literature which is called the hambo transform the aim of the paper is to develop a transformation theory for stochastic systems the paper establishes the equivalence of continuous and transformed discrete time stochastic systems in terms of solutions the method is applied to the continuous time system identification problem it is shown that using the transformed signals the po moesp subspace identification algorithm yields consistent estimates for system matrices an example is included to illustrate the efficacy of the proposed identification method and to make a comparison with the method using the laguerre filter

sensitivity block sensitivity and l block sensitivity of boolean functions sensitivity is one of the simplest and block sensitivity one of the most useful invariants of a boolean function nisan siam j comput 20 6 1991 999 and nisan and szegedy comput complexity 4 4 1994 301 have shown that block sensitivity is polynomially related to a number of measures of boolean function complexity the main open question is whether or not a polynomial relationship exists between sensitivity and block sensitivity we define the intermediate notion of l block sensitivity and show that for any fixed l this new quantity is polynomially related to sensitivity we then achieve an improved though still exponential upper bound on block sensitivity in terms of sensitivity as a corollary we also prove that sensitivity and block sensitivity are polynomially related when the block sensitivity is n

maximal sensitivity of boolean nested canalizing functions abstract boolean nested canalizing functions ncf have important applications in molecular regulatory networks engineering and computer science in the literature there are two sensitivities to measure the complexity of a boolean function one is average sensitivity the other one is maximal sensitivity we follow the tradition and omit the word maximal in other words in this paper sensitivity is always maximal sensitivity using the past work of the authors and their coauthors on a characterization of ncf we obtain the formula of the sensitivity of any ncf we find that the sensitivity of any ncf is between n 2 2 and n both lower and upper bounds are tight we prove that the block sensitivity hence the l block sensitivity is the same as the sensitivity for ncf it is well known that monotone boolean functions mbf also have this property we characterize all functions which are both monotone and nested canalizing mncf the closed formula of the cardinality of the set of mncfs is also provided

the asymptotic variance of subspace estimates abstract we give new simple general expressions for the asymptotic covariance of the estimated system parameters a b c d in subspace identification the formulas can be applied to a whole class of subspace methods including n4sid moesp cva etc the asymptotic expressions highlight how the conditioning of the estimation problem influences the accuracy of the estimates

subspace identification for non linear systems with measured input non linearities this paper uses subspace methods to identify a class of multi input multi output discrete time non linear time varying systems specifically we identify systems that are non linear in measured data and linear in unmeasured states numerical examples are presented to demonstrate the efficacy of the method

crew prams and decision trees this paper gives a full characterization of the time needed to compute a boolean function on a crew pram with an unlimited number of processors the characterization is given in terms of a new compl

subspace based prediction of linear time varying stochastic systems in this paper a new subspace method for predicting time varying stochastic systems is proposed using the concept of angle between past and present subspaces spanned by the extended observability matrices the future signal subspace is predicted by rotating the present subspace in the geometrical sense and time varying system matrices are derived from the resultant signal subspace proposed algorithm is improved for fast varying systems furthermore recursive implementation of both algorithms is developed

subspace model identification part 2 analysis of the elementary output error state space model identification algorithm the elementary moesp algorithm presented in the first part of this series of papers is analysed in this paper this is done in three different ways first we study the asymptotic properties of the estimated state space model when only considering zero mean white noise perturbations on the output sequence it is shown that in this case the moespl implementation yields asymptotically unbiased estimates an important constraint to this result is that the underlying system must have a finite impulse response and subsequently the size of the hankel matrices constructed from the input and output data at the beginning of the computations depends on the number of non zero markov parameters this analysis however leads to a second implementation of the elementary moesp scheme namely moesp2 the latter implementation has the same asymptotic properties without the finite impulse response constraint secondly we compare the moesp2 algorithm with a classical state space model identification scheme the latter

a unifying theorem for three subspace system identification algorithms the aim of this paper is to indicate and explore the similarities between three different subspace algorithms for the identification of combined deterministic stochastic systems the similarities between these algorithms have been obscured due to different notations and backgrounds it is shown that all three algorithms are special cases of one unifying theorem the comparison also reveals that the three algorithms use exactly the same subspace to determine the order and the extended observability matrix but that the weighting matrix used to calculate a basis for the column space of the observability matrix is different in the three cases

subspace identification for continuous time stochastic systems via distribution based approach this paper presents a novel approach for system identification of continuous time stochastic state space models from random input output continuous data the approach is based on the introduction of random distribution theory in describing higher time derivatives of stochastic processes and the input output algebraic relationship is derived which is treated in the time domain the efficacy of the approach is examined by comparing with other approaches employing the filters

criteria for informative experiments with subspace identification informative experiments are identification experiments which contain sufficient information for an identification algorithm to discriminate between different models in an intended model set in this paper a particular set of identification algorithms namely subspace based identification is considered criteria for experiments to be informative with these methods in the deterministic setup and the combined deterministic stochastic setup are presented it is pointed out that if these criteria are not satisfied interesting phenomena in which perfect cancellations of the deterministic components and the stochastic components occur in a subspace projection may occur it is further shown that such cancellations can indeed be avoided under mild conditions

a central limit theorem for trigonometric series with bounded gaps in this paper it is proved that there exists a sequence nk of integers with 1 nk 1 nk 5 such that the distribution of cos 2 pi n 1 x dots cos 2 pi n n sqrt n on 0 1 b dx converges to a gaussian distribution it gives an affirmative answer to the long standing problem on lacunary trigonometric series which ask the existence of series with bounded gaps satisfying a central limit theorem

subspace based system identification weighting and pre filtering of instruments subspace based system identification is typically based on an estimate of the extended observability matrix it is thus of great interest to investigate and also optimize the estimate of the observability matrix of special interest in this paper is the fact that the influence of certain weighting matrices is an unresolved issue in the literature on subspace identification here an asymptotic analysis of the estimated observability matrix is presented the main result of the analysis is that novel weighting matrices and pre filters of instrumental variables are derived

on consistency of subspace methods for system identification subspace methods for identification of linear time invariant dynamical systems typically consist of two main steps first a so called subspace estimate is constructed this first step usually consists of estimating the range space of the extended observability matrix secondly an estimate of system parametersis obtained based on the subspace estimate in this paper the consistency of a large class of methods for estimating the extended observability matrix is analyzed persistence of excitation conditions on the input signal are given which guarantee consistent estimates for systems with only measurement noise for systems with process noise it is shown that a persistence of excitation condition on the input is not sufficient more precisely an example for which the subspace methods fail to give a consistent estimate of the transfer function is given this failure occurs even if the input is persistently exciting of any order it is also shown that this problem can be eliminated if stronger conditions on the input signal are imposed

concentration inequalities and limit theorems for randomized sums concentration properties and an asymptotic behaviour of distributions of normalized and self normalized sums are studied in the randomized model where the observation times are selected from prescribed consecutive integer intervals

identification of the deterministic part of mimo state space models given in innovations form from input output data the problem of linear multivariable state space model identification from input output data can under the presence of process and measurement noise be solved in a non iterative way when incorporating instrumental variables constructed from both input and output sequences in the recently developed class of multivariable output error state space model class of subspace model identification schemes abstract in this paper we describe two algorithms to identify a linear time invariant finite dimensional state space model from input output data the system to be identified is assumed to be excited by a measurable input and an unknown process tloise and the measurements are disturbed by unknown measurement noise both noise sequences are discrete zero mean white noise the first algorithm gives consistent estimates only for the case where the input also is zero mean white noise while the same result is obtained with the second algorithm without this constraint for the special case where the input signal is discrete zero mean white noise it is explicitly shown that this second algorithm is a special case of the recently developed multivariable output error state space moesp class of algorithms based on instrumental variables the usefulness of the presented schemes is highlighted in a realistic simulation study

analysis of state space system identification methods based on instrumental variables and subspace fitting subspace based state space system identification 4sid methods have recently been proposed as an alternative to more traditional techniques for multivariable system identification the advantages are that the user has simple and few design variables and that the methods have robust numerical properties and relatively low computational complexities though subspace techniques have been demonstrated to perform well in a number of cases the performance of these methods is neither fully understood nor analyzed our principal objective is to undertake a statistical investigation of subspace based system identification techniques the studied methods consist of two steps the subspace spanned by the extended observability matrix is first estimated the asymptotic properties of this subspace estimate are derived herein in the second step the structure of the extended observability matrix is used to find a system model estimate two possible methods are considered the simplest one only uses a certain shift invariance property while in the other method a parametric representation of the null space of the observability matrix is exploited explicit expressions for the asymptotic estimation error variances of the corresponding pole estimates are given

on line structured subspace identification with application to switched linear systems this article is concerned with the identification of switched linear multiple inputs multiple outputs state space systems in a recursive way first a structured subspace identification scheme for linear systems is presented which turns out to have many attractive features more precisely it does not require any singular value decomposition but is derived using orthogonal projection techniques it allows a computationally appealing implementation and it is closely related to input output models identification second it is shown that this method can be implemented on line to track both the range space of the extended observability matrix and its dimension and thereby the system matrices third by making use of an on line switching times detection strategy this method is applied to blindly identify switched systems and to label the obtained submodels simulation results on noisy data illustrate the abilities and the benefits of the proposed approach

trade offs between depth and width in parallel computation a new technique for proving lower bounds for parallel computation is introduced this technique enables us to obtain for the first time nontrivial tight lower bounds for shared memory models of parallel computation that allow several processors to have simultaneous access to the same memory location specifically we use a concurrent read concurrent write model of parallel computation it has p processors each has access to a common memory of size m also called communication width or width in short the input to the problem is located in an additional read only portion of the common memory for a wide variety of problems including parity majority and summation we show that the time complexity t depth and the communication width m are related by the trade off curve mt 2 omega n where n is the size of the input regardless of the number of processors moreover for every point on this curve with m o n log 2 n we give a matching upper bound with the optimal number of processors we e

comparison of some instrumental variable methods consistency and accuracy aspects a general analysis of various instrumental variable methods is given the investigation includes methods for open loop operation as well as methods designed for closed loop operation the instrumental variables are formed as different combinations of inputs delayed inputs delayed outputs filtered inputs and external setpoint variations necessary conditions as well as sufficient conditions for consistency are explicitly derived and discussed the paper also includes some simple counter examples to general consistency for some common instrumental variable methods also asymptotic accuracy properties are reviewed it is e g studied how the use of filters will influence the covariance matrix of the parameter estimates

subspace algorithms for the stochastic identification problem abstract in this paper we derive a new subspace algorithm to consistently identify stochastic state space models from given output data without forming the covariance matrix and using only semi infinite block hankel matrices the algorithm is based on the concept of principal angles and directions we describe how they can be calculated with qr and quotient singular value decomposition we also provide an interpretation of the principal directions as states of a non steady state kalman filter bank

on the law of the iterated logarithm for the discrepancy of lacunary sequences a classical result of philipp 1975 states that for any sequence n k k 1 of integers satisfying the hadamard gap condition n k 1 n k q 1 k 1 2 the discrepancy d n of the sequence n k x k 1 mod 1 satisfies the law of the iterated logarithm lil i e 1 4 lim sup nd n n k x n log log n 12 c q a e n the value of the lim sup is a long standing open problem recently fukuyama explicitly calculated the value of the lim sup for n k k 1 not necessarily integer we extend fukuyama s result to a large class of integer sequences n k characterized in terms of the number of solutions of a certain class of diophantine equations and show that the value of the lim sup is the same as in the chung smirnov lil for i i d random variables

identification methods in a unified framework the paper derives a framework suitable to discuss the classical koopmans levin kl and maximum likelihood ml algorithms to estimate parameters of errors in variables linear models in a unified way using the capability of the unified approach a new parameter estimation algorithm is presented offering flexibility to ensure acceptable variance in the estimated parameters the developed algorithm is based on the application of hankel matrices of variable size and can equally be considered as a generalized version of the kl method gkl or as a reduced version of the ml estimation the methodology applied to derive the gkl algorithm is used to present a straightforward derivation of the subspace identification algorithm

brief paper convergence analysis of instrumental variable recursive subspace identification algorithms the convergence properties of recently developed recursive subspace identification methods are investigated in this paper the algorithms operate on the basis of instrumental variable iv versions of the propagator method for signal subspace estimation it is proved that under suitable conditions on the input signal and the system the considered recursive subspace identification algorithms converge to a consistent estimate of the propagator and by extension to the state space system matrices

n4sid subspace algorithms for the identification of combined deterministic stochastic systems abstract recently a great deal of attention has been given to numerical algorithms for subspace state space system identification n4sid in this paper we derive two new n4sid algorithms to identify mixed deterministic stochastic systems both algorithms determine state sequences through the projection of input and output data these state sequences are shown to be outputs of non steady state kalman filter banks from these it is easy to determine the state space system matrices the n4sid algorithms are always convergent non iterative and numerically stable since they only make use of qr and singular value decompositions both n4sid algorithms are similar but the second one trades off accuracy for simplicity these new algorithms are compared with existing subspace algorithms in theory and in practice

sensitive functions and approximate problems properties of functions that are good measures of the crcw pram complexity of computing them are investigated while theblock sensitivityis known to be a good measure of the crew pram complexity no such measure is known for crcw prams it is shown that the complexity of computing a function is related to itseverywhere sensitivity introduced by vishkin and wigderson specifically the time required to compute a functionf dn rof everywhere sensitivityes f withpprocessors and unbounded memory is logloges f log d 4p es f this improves results of azar and of vishkin and wigderson this lower bound is used to derive new lower bounds for someapproximate problems these problems can often be solved faster than their exact counterparts and for many applications it is sufficient to solve the approximate problem it is shown thatapproximate selection approximate counting approximate compaction andpadded sortingall require time loglogn with a linear number of processors if the level of accuracy desired is moderately high for these levels of accuracy no lower bounds were known for these problems on the pram model the lower bounds for some of the problems are tight

asymptotic properties of subspace estimators since the proposal of subspace methods in the 1980s and early 1990s substantial efforts have been made in the analysis of the statistical properties of the algorithms this paper surveys the literature on the asymptotic properties of particular subspace methods used for linear dynamic time invariant discrete time systems the goals of this paper are threefold first this survey tries to present the most relevant results on the asymptotic properties of estimators obtained using subspace methods secondly the main methods and tools that have been used in the derivation of these results are presented to make the literature more accessible thirdly main unsolved questions and rewarding research topics are identified some of which can be attacked using the toolbox discussed in the paper

choice of state space basis in combined deterministic stochastic subspace identification this paper describes how the state space basis of models identified with subspace identification algorithms can be determined it is shown that this basis is determined by the input spectrum and by user defined input and output weightings through the connections between subspace identification and frequency weighted balancing the state space basis of the subspace identified models is shown to coincide with a frequency weighted balanced basis

a tighter relation between sensitivity complexity and certificate complexity abstract the sensitivity conjecture proposed by nisan and szegedy in 1994 which asserts that for any boolean function its sensitivity complexity is polynomially related to the block sensitivity complexity is one of the most important and challenging problems in the study of decision tree complexity despite a lot of efforts the best known upper bound of block sensitivity as well as the certificate complexity is still exponential in terms of sensitivity in this paper we give a better upper bound for certificate complexity and block sensitivity b s f c f 8 9 o 1 s f 2 s f 1 where b s f c f and s f are the block sensitivity certificate complexity and sensitivity respectively the proof is based on a deep investigation on the structure of the sensitivity graph we also provide a tighter relationship between the 0 certificate complexity c 0 f and 0 sensitivity s 0 f for functions with small 1 sensitivity s 1 f

identification of stable models in subspace identification by using regularization in subspace identification methods the system matrices are usually estimated by least squares based on estimated kalman filter state sequences and the observed inputs and outputs for a finite number of data points the estimated system matrix is not guaranteed to be stable even when the true linear system is known to be stable in this paper stability is imposed by using regularization the regularization term used here is the trace of a matrix which involves the dynamical system matrix and a positive semi definite weighting matrix the amount of regularization can be determined from a generalized eigenvalue problem the data augmentation method of chui and maciejowski 1996 is obtained by using specific choices for the weighting matrix in the regularization term

brief consistency analysis of subspace identification methods based on a linear regression approach in the literature results can be found which claim consistency for the subspace method under certain quite weak assumptions unfortunately a new result gives a counter example showing inconsistency under these assumptions and then gives new more strict sufficient assumptions which however does not include important model structures such as e g box jenkins based on a simple least squares approach this paper shows the possible inconsistency under the weak assumptions and develops only slightly stricter assumptions sufficient for consistency and which includes any model structure

subspace model identification part 1 the output error state space model identification class of algorithms in this paper we present two novel algorithms to realize a finite dimensional linear time invariant state space model from input output data the algorithms have a number of common features they are classified as one of the subspace model identification schemes in that a major part of the identification problem consists of calculating specially structured subspaces of spaces defined by the input output data this structure is then exploited in the calculation of a realization another common feature is their algorithmic organization an rq factorization followed by a singular value decomposition and the solution of an overdetermined set or sets of equations the schemes assume that the underlying system has an output error structure and that a measurable input sequence is available the latter characteristic indicates that both schemes are versions of the mimo output error state space model identification moesp approach the first algorithm is denoted in particular as the elementary moesp scheme

subspace based methods for the identification of linear time invariant systems subspace based methods for system identification have attracted much attention during the past few years this interest is due to the ability of providing accurate state space models for multivariable linear systems directly from input output data the methods have their origin in classical state space realization theory as developed in the 1960s the main computational tools are the qr and the singular value decompositions here an overview of existing subspace based techniques for system identification is given the methods are grouped into the classes of realization based and direct techniques similarities between different algorithms are pointed out and their applicability is commented upon we also discuss some recent ideas for improving and extending the methods a simulation example is included for comparing different algorithms the subspace based approach is found to perform competitive with respect to prediction error methods provided the system is properly excited

a novel subspace identification approach with enforced causal models subspace identification methods sims for estimating state space models have been proven to be very useful and numerically efficient they exist in several variants but all have one feature in common as a first step a collection of high order arx models are estimated from vectorized input output data in order not to obtain biased estimates this step must include future outputs however all but one of the submodels include non causal input terms the coefficients of them will be correctly estimated to zero as more data become available they still include extra model parameters which give unnecessarily high variance and also cause bias for closed loop data in this paper a new model formulation is suggested that circumvents the problem within the framework the system matrices a b c d and markov parameters can be estimated separately it is demonstrated through analysis that the new methods generally give smaller variance in the estimate of the observability matrix and it is supported by simulation studies that this gives lower variance also of the system invariants such as the poles

riemannian optimal identification method for linear systems with symmetric positive definite matrix this article develops identification methods for linear continuous time symmetric systems such as electrical network systems multiagent network systems and temperature dynamics in buildings to this end we formulate three system identification problems for the corresponding discrete time systems the first is a least squares problem in which we wish to minimize the sum of squared errors between the true and model outputs on the product manifold of the manifold of symmetric positive definite matrices and two euclidean spaces in the second problem to reduce the search dimensions the product manifold is replaced with the quotient set under a specified group action by the orthogonal group in the third problem the manifold of symmetric positive definite matrices in the first problem is replaced by the manifold of matrices with only positive diagonal elements in particular we examine the quotient geometry in the second problem we propose riemannian conjugate gradient methods for the three problems and select initial points using a popular subspace method the effectiveness of our proposed methods is demonstrated through numerical simulations and comparisons with the gauss newton method which is one of the most popular approach for solving least squares problems