a new kernel based approach for linear system identification this paper describes a new kernel based approach for linear system identification of stable systems we model the impulse response as the realization of a gaussian process whose statistics differently from previously adopted priors include information not only on smoothness but also on bibo stability the associated autocovariance defines what we call a stable spline kernel the corresponding minimum variance estimate belongs to a reproducing kernel hilbert space which is spectrally characterized compared to parametric identification techniques the impulse response of the system is searched for within an infinite dimensional space dense in the space of continuous functions overparametrization is avoided by tuning few hyperparameters via marginal likelihood maximization the proposed approach may prove particularly useful in the context of robust identification in order to obtain reduced order models by exploiting a two step procedure that projects the nonparametric estimate onto the space of nominal models the continuous time derivation immediately extends to the discrete time case on several continuous and discrete time benchmarks taken from the literature the proposed approach compares very favorably with the existing parametric and nonparametric techniques

higher order boundary value problems on time scales in this paper we study the existence of solutions for lidstone boundary value problems on time scale firstly by using schauder fixed point theorem in a cone we obtain the existence of solutions to a lidstone boundary value problem lbvp secondly existence result for this problem is also given by the monotone method finally by using krasnosel skii fixed point theorem it is proved that the lbvp has a positive solution

matchable observable linear models and direct filter tuning an approach to multivariable identification identification of linear time invariant multivariable systems can best be understood as comprising three separate problems selection of system model structure filter design and parameter estimation itself approaching the first using matchable observable models originally developed in the adaptive control literature and the second via direct or derivative free optimization effective least squares algorithms can be used for parameter estimation the accuracy robustness and moderate computational demands of the methods proposed are demonstrated via simulations with randomly generated models and applied to identification using real process data the results obtained are comparable or superior to the best results obtained using standard implementations of the algorithms described in the literature

robust em kernel based methods for linear system identification recent developments in system identification have brought attention to regularized kernel based methods this type of approach has been proven to compare favorably with classic parametric methods however current formulations are not robust with respect to outliers in this paper we introduce a novel method to robustify kernel based system identification methods to this end we model the output measurement noise using random variables with heavy tailed probability density functions pdfs focusing on the laplacian and the student s t distributions exploiting the representation of these pdfs as scale mixtures of gaussians we cast our system identification problem into a gaussian process regression framework which requires estimating a number of hyperparameters of the data size order to overcome this difficulty we design a new maximum a posteriori map estimator of the hyperparameters and solve the related optimization problem with a novel iterative scheme based on the expectation maximization em method in the presence of outliers tests on simulated data and on a real system show a substantial performance improvement compared to currently used kernel based methods for linear system identification

prediction error identification of linear systems a nonparametric gaussian regression approach a novel bayesian paradigm for the identification of output error models has recently been proposed in which in place of postulating finite dimensional models of the system transfer function the system impulse response is searched for within an infinite dimensional space in this paper such a nonparametric approach is applied to the design of optimal predictors and discrete time models based on prediction error minimization by interpreting the predictor impulse responses as realizations of gaussian processes the proposed scheme describes the predictor impulse responses as the convolution of an infinite dimensional response with a low dimensional parametric response that captures possible high frequency dynamics overparameterization is avoided because the model involves only a few hyperparameters that are tuned via marginal likelihood maximization numerical experiments with data generated by armax and infinite dimensional models show the definite advantages of the new approach over standard parametric prediction error techniques and subspace methods both in terms of predictive capability on new data and accuracy in reconstruction of system impulse responses

the quest for the right kernel in bayesian impulse response identification the use of obfs kernel based regularization approaches for impulse response estimation of linear time invariant lti systems have received a lot of attention recently the reason is that regularized least squares estimators may achieve a favorable bias variance trade off compared with classical prediction error minimization pem methods to fully exploit this property the kernel function needs to capture relevant aspects of the data generating system at hand hence it is important to design automatic procedures for kernel design based on data or prior knowledge the kernel models so far introduced focus on encoding smoothness and bibo stability of the expected impulse response while other properties like oscillatory behavior or the presence of fast and slow poles have not been successfully implemented in kernel design inspired by the representation theory of dynamical systems we show how to build stable kernels that are able to capture particular aspects of system dynamics via the use of orthonormal basis functions obfs in particular desired dynamic properties can be easily encoded via the generating poles of obfs such poles are seen as hyperparameters which are tuned via marginal likelihood optimization special cases of our kernel construction include laguerre kautz and generalized obfs gobfs based kernel structures monte carlo simulations show that the obfs based kernels perform well compared with stable spline tc kernels especially for slow systems with dominant poles close to the unit circle moreover the capability of kautz basis to model resonating systems is also shown

the harmonic analysis of kernel functions abstract kernel based methods have been recently introduced for linear system identification as an alternative to parametric prediction error methods adopting the bayesian perspective the impulse response is modeled as a non stationary gaussian process with zero mean and with a certain kernel i e covariance function choosing the kernel is one of the most challenging and important issues in the present paper we introduce the harmonic analysis of this non stationary process and argue that this is an important tool which helps in designing such kernel furthermore this analysis suggests also an effective way to approximate the kernel which allows to reduce the computational burden of the identification procedure

existence of nodal solutions for lidstone eigenvalue problems abstract we are concerned with determining values of r for which there exist nodal solutions of the lidstone eigenvalue problem 1 m u 2 m x r a x f u x 0 x 1 u 2 i 0 u 2 i 1 0 i 0 1 2 m 1 the proof of our main result is based upon eigenvalue theory and bifurcation techniques

existence multiplicity and infinite solvability of positive solutions to a nonlinear fourth order periodic boundary value problem abstract the existence of n and infinitely many positive solutions is proved for the nonlinear fourth order periodic boundary value problem u 4 t u t u t f t u t 0 t 1 u i 0 u i 1 i 0 1 2 3 where n is an arbitrary natural number and 2 2 0 1 2 2 2 2 4 2 1 0 this kind of fourth order boundary value problems usually describes the equilibrium state of an elastic beam with periodic boundary condition the main results show that the problem may have n or infinitely many positive solutions provided the growth rates of nonlinear term f t l are appropriate on some bounded subsets of its domain

bayes and empirical bayes semi blind deconvolution using eigenfunctions of a prior covariance we consider the semi blind deconvolution problem i e estimating an unknown input function to a linear dynamical system using a finite set of linearly related measurements where the dynamical system is known up to some system parameters without further assumptions this problem is often ill posed and ill conditioned we overcome this difficulty by modeling the unknown input as a realization of a stochastic process with a covariance that is known up to some finite set of covariance parameters we first present an empirical bayes method where the unknown parameters are estimated by maximizing the marginal likelihood posterior and subsequently the input is reconstructed via a tikhonov estimator with the parameters set to their point estimates next we introduce a bayesian method that recovers the posterior probability distribution and hence the minimum variance estimates for both the unknown parameters and the unknown input function both of these methods use the eigenfunctions of the random process covariance to obtain an efficient representation of the unknown input function and its probability distributions simulated case studies are used to test the two methods and compare their relative performance

monotone iterative technique and positive solutions of lidstone boundary value problems the purpose of this paper is to investigate the existence and iteration of n symmetric positive solutions for lidstone boundary value problem 1 nw 2 n t f t w t 0

multiple positive solutions of some boundary value problems abstract we study the existence of multiple positive solutions of the equations u t u subject to linear boundary conditions we show that there are at least two positive solutions if t u is superlinear at one end point zero or infinity and sublinear at the other applications of these results are provided to yield multiple positive solutions of some elliptic boundary value problems on an annulus

bifurcation techniques and positive solutions of discrete lidstone boundary value problems abstract let n 1 t n 5 be two positive integers and t n n n 1 t we consider the existence of solutions of the discrete lidstone boundary value problem 1 n 2 n u t n f t u t 2 u t 1 2 n 2 u t n 1 t t n 2 i u n i 1 2 i u t i 1 0 i 0 1 n 1 where f t n r n r is continuous

deligne periods of mixed motives theory and the entropy of certain ℤⁿ actions for a coherent sheaf m on the split n torus gn a spec a 7n over a commutative ring a there is a natural en action on its group of global sections f m f g aa m if we give f m the discrete topology we obtain a en action on the compact abelian pontrjagin dual f m the dynamical properties of this action for a z have been investigated for some time especially by k schmidt his book sch gives a comprehensive account of the theory developed so far in particular one may ask for the entropy 0 h m oc of the en action on f m i which measures to what extent repeated application of the action scatters around points note that for any action of zn on a compact abelian group topological entropy and metric entropy with respect to haar measure coincide the results of lind schmidt and ward lsw see also sch chap v can be formulated as follows a z the entropy of m is finite if and only if m belongs to the category t of coherent torsion sheaves on gn by yuzvinskii s addition formula sch th 14 1 entropy defines a homomorphism from the grothendieck group of t to the reals h ko t ri so that it suffices to calculate h oz for irreducible closed subschemes z c n with z g n if z is not defined by a laurent polynomial then h oz 0 on the other hand for any

system identification via sparse multiple kernel based regularization using sequential convex optimization techniques model estimation and structure detection with short data records are two issues that receive increasing interests in system identification in this paper a multiple kernel based regularization method is proposed to handle those issues multiple kernels are conic combinations of fixed kernels suitable for impulse response estimation and equip the kernel based regularization method with three features first multiple kernels can better capture complicated dynamics than single kernels second the estimation of their weights by maximizing the marginal likelihood favors sparse optimal weights which enables this method to tackle various structure detection problems e g the sparse dynamic network identification and the segmentation of linear systems third the marginal likelihood maximization problem is a difference of convex programming problem it is thus possible to find a locally optimal solution efficiently by using a majorization minimization algorithm and an interior point method where the cost of a single interior point iteration grows linearly in the number of fixed kernels monte carlo simulations show that the locally optimal solutions lead to good performance for randomly generated starting points

on kernel design for regularized lti system identification abstract there are two key issues for the kernel based regularization method one is how to design a suitable kernel to embed in the kernel the prior knowledge of the lti system to be identified and the other one is how to tune the kernel such that the resulting regularized impulse response estimator can achieve a good bias variance tradeoff in this paper we focus on the issue of kernel design depending on the type of the prior knowledge we propose two methods to design kernels one is from a machine learning perspective and the other one is from a system theory perspective we also provide analysis results for both methods which not only enhances our understanding for the existing kernels but also directs the design of new kernels

on the estimation of transfer functions regularizations and gaussian processes revisited intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques we revisit the old problem of transfer function estimation from input output measurements we formulate a classical regularization approach focused on finite impulse response fir models and find that regularization is necessary to cope with the high variance problem this basic regularized least squares approach is then a focal point for interpreting other techniques like bayesian inference and gaussian process regression the main issue is how to determine a suitable regularization matrix bayesian prior or kernel several regularization matrices are provided and numerically evaluated on a data bank of test systems and data sets our findings based on the data bank are as follows the classical regularization approach with carefully chosen regularization matrices shows slightly better accuracy and clearly better robustness in estimating the impulse response than the standard approach the prediction error method maximum likelihood pem ml approach if the goal is to estimate a model of given order as well as possible a low order model is often better estimated by the pem ml approach and a higher order model is often better estimated by model reduction on a high order regularized fir model estimated with careful regularization moreover an optimal regularization matrix that minimizes the mean square error matrix is derived and studied the importance of this result lies in that it gives the theoretical upper bound on the accuracy that can be achieved for this classical regularization approach

regularized lti system identification in the presence of outliers a variational em approach abstract regularized system identification of linear time invariant systems in the presence of outliers is investigated the finite impulse response fir model and the gaussian scale mixture are chosen to be the system model and the noise model respectively two special cases of the noise model are considered the well known student s t distribution and a proposed g confluent distribution both the fir model parameter and the latent variables in the noise model are treated as parameters of our statistical model and moreover the scale of the noise variance is treated as a hyper parameter besides the hyper parameters used to parameterize the priors of the impulse response and the latent variables then a variational expectation maximization algorithm is proposed for inference of the parameters and hyper parameters of the statistical model and the algorithm is guaranteed to converge to a stationary point monte carlo numerical simulations show that when the relative size of outliers is small the proposed approach performs comparably to a state of the art method and when the relative size of outliers and or the occurrence probability of outliers is large the proposed approach outperforms the state of the art method

a scaled gradient projection method for bayesian learning in dynamical systems a crucial task in system identification problems is the selection of the most appropriate model class and is classically addressed resorting to cross validation or using order selection criteria based on asymptotic arguments as recently suggested in the literature this can be addressed in a bayesian framework where model complexity is regulated by a few hyperparameters which can be estimated via marginal likelihood maximization it is thus of primary importance to design effective optimization methods to solve the corresponding optimization problem if the unknown impulse response is modeled as a gaussian process with a suitable kernel the maximization of the marginal likelihood leads to a challenging nonconvex optimization problem which requires a stable and effective solution strategy in this paper we address this problem by means of a scaled gradient projection algorithm in which the scaling matrix and the steplength parameter play a crucial role to provide a meaningful solution in a computatio

quantifying the error in estimated transfer functions with application to model order selection previous results on estimating errors or error bounds on identified transfer functions have relied on prior assumptions about the noise and the unmodeled dynamics this prior information took the form of parameterized bounding functions or parameterized probability density functions in the time or frequency domain with known parameters it is shown that the parameters that quantify this prior information can themselves be estimated from the data using a maximum likelihood technique this significantly reduces the prior information required to estimate transfer function error bounds the authors illustrate the usefulness of the method with a number of simulation examples how the obtained error bounds can be used for intelligent model order selection that takes into account both measurement noise and under modeling is shown another simulation study compares the method to akaike s well known fpe and aic criteria

maximum entropy properties of discrete time first order stable spline kernel the first order stable spline ss 1 kernel also known as the tuned correlated tc kernel is used extensively in regularized system identification where the impulse response is modeled as a zero mean gaussian process whose covariance function is given by well designed and tuned kernels in this paper we discuss the maximum entropy properties of this kernel in particular we formulate the exact maximum entropy problem solved by the ss 1 kernel without gaussian and uniform sampling assumptions under general sampling assumption we also derive the special structure of the ss 1 kernel e g its tridiagonal inverse and factorization have closed form expression also giving to it a maximum entropy covariance completion interpretation

eigenvalues of lidstone boundary value problems we consider the following boundary value problem 1 ny 2 n lf t y n 1 t 0 1 y 2 i 0 y 2 i 1 0 0 0 the values of l are characterized so that the boundary value problem has a positive solution in addition we derive explicit intervals of l such that for any l in the interval existence of a positive solution of the boundary value problem is guaranteed several examples are also included to dwell upon the importance of the results obtained

existence of positive solutions of lidstone boundary value problems abstract we show that there exists at least one positive solution for lidstone boundary value problem 1 n u 2 n t f t u t u t u 2 n 1 t 0 t 1 u 2 i 0 u 2 i 1 0 i 0 1 n 1 under some suitable conditions our proof is based upon global bifurcation techniques

maximum entropy kernels for system identification bayesian nonparametric approaches have been recently introduced in system identification scenario where the impulse response is modeled as the realization of a zero mean gaussian process whose covariance kernel has to be estimated from data in this scheme quality of the estimates crucially depends on the parametrization of the covariance of the gaussian process a family of kernels that have been shown to be particularly effective in the system identification framework is the family of diagonal correlated dc kernels maximum entropy properties of a related family of kernels the tuned correlated tc kernels have been recently pointed out in the literature in this technical note we show that maximum entropy properties indeed extend to the whole family of dc kernels the maximum entropy interpretation can be exploited in conjunction with results on matrix completion problems in the graphical models literature to shed light on the structure of the dc kernel in particular we prove that the dc kernel admits a closed form factorization inverse and determinant these results can be exploited both to improve the numerical stability and to reduce the computational complexity associated with the computation of the dc estimator

a regularized snpom for stable parameter estimation of rbf ar x model recently the radial basis function rbf network style coefficients autoregressive with exogenous inputs rbf ar x model identified by the structured nonlinear parameter optimization method snpom has attracted considerable interest because of its significant performance in nonlinear system modeling however this promising technique may occasionally confront the problem that the parameters are divergent in the optimization process which may be a potential issue ignored by most researchers in this paper a regularized snpom together with the regularization parameter detection technique is presented to estimate the parameters of rbf ar x models this approach first separates the parameters of an rbf ar x model into a linear parameters set and a nonlinear parameters set and then combines a gradient based nonlinear optimization algorithm for estimating the nonlinear parameters and the regularized least squares method for estimating the linear parameters several examples demonstrate that the proposed approach is effective to cope with the potential unstable problem in the parameters search process and may also yield better or similar multistep forecasting accuracy and better robustness than the previous method

implementation of algorithms for tuning parameters in regularized least squares problems in system identification there has been recently a trend to study linear system identification with high order finite impulse response fir models using the regularized least squares approach one key of this approach is to solve the hyper parameter estimation problem that is usually nonconvex our goal here is to investigate implementation of algorithms for solving the hyper parameter estimation problem that can deal with both large data sets and possibly ill conditioned computations in particular a qr factorization based matrix inversion free algorithm is proposed to evaluate the cost function in an efficient and accurate way it is also shown that the gradient and hessian of the cost function can be computed based on the same qr factorization finally the proposed algorithm and ideas are verified by monte carlo simulations on a large data bank of test systems and data sets

on the positive solutions of lidstone boundary value problems the existence nonexistence and multiplicity of positive solutions are considered for lidstone boundary value problem 1 email protected 2 n email protected t w t 0 0 by making use of krasnosel skii fixed point theorem some new results are obtained particularly it is proved that the lidstone boundary value problem has n positive solutions under suitable conditions where n is an arbitrary natural number

existence of n solutions and or positive solutions to a semipositone elastic beam equation abstract the existence of n solutions and or positive solutions is established for a nonlinear fourth order two point boundary value problem where n is an arbitrary natural number and the nonlinear term has a nonpositive lower bound in mechanics the problem describes deformation of the elastic beam whose two ends are simply supported the main results show that the problem has at least n solutions and or positive solutions provided that the heights of the nonlinear term are appropriate on some bounded sets

bifurcation techniques for lidstone boundary value problems abstract by employing bifurcation techniques this paper investigates the existence of nontrivial solutions single and multiple for lidstone boundary value problems depending on higher order derivatives our results improve on those in the literature