on the partial stochastic realization problem we describe a complete parameterization of the solutions to the partial stochastic realization problem in terms of a nonstandard matrix riccati equation our analysis of this covariance extension equation cee is based on a complete parameterization of all strictly positive real solutions to the rational covariance extension problem answering a conjecture due to georgiou 1987 in the affirmative we also compute the dimension of partial stochastic realizations in terms of the rank of the unique positive semidefinite solution to the cee yielding some insights into the structure of solutions to the minimal partial stochastic realization problem by combining this parameterization with some of the classical approaches in partial realization theory we are able to derive new existence and robustness results concerning the degrees of minimal stochastic partial realizations as a corollary to these results we note that in sharp contrast with the deterministic case there is no generic value of the degree of a minimal stochastic realization of partial covariance sequences of fixed length

uniform stability of switched linear systems extensions of lasalle s invariance principle this paper addresses the uniform stability of switched linear systems where uniformity refers to the convergence rate of the multiple solutions that one obtains as the switching signal ranges over a given set we provide a collection of results that can be viewed as extensions of lasalle s invariance principle to certain classes of switched linear systems using these results one can deduce asymptotic stability using multiple lyapunov functions whose lie derivatives are only negative semidefinite depending on the regularity assumptions placed on the switching signals one may be able to conclude just asymptotic stability or uniform exponential stability we show by counter example that the results obtained are tight

relative entropy and the multivariable multidimensional moment problem entropy like functionals on operator algebras have been studied since the pioneering work of von neumann umegaki lindblad and lieb the best known are the von neumann entropy rho trace rhologrho and a generalization of the kullback leibler distance s rhoparsigma trace rhologrho rhologsigma referred to as quantum relative entropy and used to quantify distance between states of a quantum system the purpose of this paper is to explore and s as regularizing functionals in seeking solutions to multivariable and multidimensional moment problems it will be shown that extrema can be effectively constructed via a suitable homotopy the homotopy approach leads naturally to a further generalization and a description of all the solutions to such moment problems this is accomplished by a renormalization of a riemannian metric induced by entropy functionals as an application we discuss the inverse problem of describing power spectra which are consistent with second order statistics which has been the main motivation behind the present work

analytic interpolation with a degree constraint for matrix valued functions we consider a nehari problem for matrix valued positive real functions and characterize the class of generically minimal degree solutions analytic interpolation problems such as the one studied herein for positive real functions arise in time series modeling and system identification the degree of positive real interpolants relates to the dimension of models and to the degree of matricial power spectra of vector valued time series the main result of the paper generalizes earlier results in scalar analytic interpolation with a degree constraint where the class of generically minimal degree solutions is characterized by an arbitrary choice of spectral zeros naturally in the current matricial setting there is freedom in assigning the jordan structure of the spectral zeros of the power spectrum i e the spectral zeros as well as their respective invariant subspaces the characterization utilizes rosenbrock s theorem on assignability of dynamics via linear state feedback

an extension of lasalle s invariance principle for switched systems in this paper we address invariance principles for a certain class of switched nonlinear systems we provide an extension of lasalle s invariance principle for these systems and state asymptotic stability criteria we also present some related results on the compactness of the trajectories of these switched systems

a complete parameterization of all positive rational extensions of a covariance sequence in this paper we formalize the observation that filtering and interpolation induce complementary or dual decompositions of the space of positive real rational functions of degree less than or equal to n from this basic result about the geometry of the space of positive real functions we are able to deduce two complementary sets of conclusions about positive rational extensions of a given partial covariance sequence on the one hand by viewing a certain fast filtering algorithm as a nonlinear dynamical system defined on this space we are able to develop estimates on the asymptotic behavior of the schur parameters 1918 of positive rational extensions on the other hand we are also able to provide a characterization of all positive rational extensions of a given partial covariance sequence indeed motivated by its application to signal processing speech processing and stochastic realization theory this characterization is in terms of a complete parameterization using familiar objects from systems theory and proves a conjecture made by georgiou 1983 1987 our basic result however also enables us to analyze the robustness of this parameterization with respect to variations in the problem data the methodology employed is a combination of complex analysis geometry linear systems and nonlinear dynamics

on the estimation of structured covariance matrices this paper discusses a method for estimating the covariance matrix of a multivariate stationary process w generated as the output of a given linear filter fed by a stationary process y the estimated covariance matrix must satisfy two constraints it must be positive semi definite and it must be consistent with the fact that w is the output of the given linear filter it turns out that these constraints force the estimated covariance to lie in the intersection of a cone with a linear space while imposing only the first of the two constraints is rather straightforward guaranteeing that both are satisfied is a non trivial issue to which quite a bit of attention has already been devoted in the literature our approach extends the method for estimating the toeplitz covariance matrix of order m of a process y based on the biased spectral estimator stoica moses 1997 this extension is based on the characterization of the output covariance matrix in terms of the filter parameters and the sequence of covariance lags of the input process after introducing our estimation method we propose a comparison performance between this one and other methods proposed in the literature simulation results show that our approach constitutes a valid estimation procedure

on the georgiou lindquist approach to constrained kullback leibler approximation of spectral densities we consider the georgiou lindquist constrained approximation of spectra in the kullback leibler sense we propose an alternative iterative algorithm to solve the corresponding convex optimization problem the lagrange multiplier is computed as a fixed point of a nonlinear matricial map simulation indicates that the algorithm is extremely effective

a uniform invariance principle for periodic systems with applications to synchronization abstract in this paper we propose an extension of the invariance principle which is uniform with respect to parameter uncertainties for the class of periodic ordinary differential equations this extension allows the derivative of the auxiliary function v commonly called a lyapunov function to be positive in some bounded sets this important feature has the potential to simplify the problem of exhibiting a function of lyapunov type and allows the application of the principle in systems that cannot be treated with the conventional principle either due to the nonexistence of a lyapunov type function or due to the difficulty in exhibiting it the extension of the invariance principle is useful to obtain estimates of attractors and regions of attraction that are uniform with respect to parameters the study of synchronization of periodic coupled systems illustrates an application of the principle

an invariance principle for nonlinear discrete autonomous dynamical systems this note proposes an extension of lasalle s invariance principle for nonlinear discrete autonomous dynamical systems the invariance principle is extended to allow the first difference of the auxiliar scalar function usually a lyapunov function to be positive in some bounded regions moreover a uniform version is proposed to deal with nonlinear discrete dynamical systems that vary with parameters both extensions have the original invariance principle as a particular case as a consequence a larger class of systems can be treated with this new theory the extensions are very useful to obtain attractor estimates as well as their corresponding stability regions the uniform version in particular is useful to obtain estimates that are uniform regarding parameters

from finite covariance windows to modeling filters a convex optimization approach the trigonometric moment problem is a classical moment problem with numerous applications in mathematics physics and engineering the rational covariance extension problem is a constrained version of this problem with the constraints arising from the physical realizability of the corresponding solutions although the maximum entropy method gives one well known solution in several applications a wider class of solutions is desired in a seminal paper georgiou derived an existence result for a broad class of models in this paper we review the history of this problem going back to carath eodory as well as applications to stochastic systems and signal processing in particular we present a convex optimization problem for solving the rational covariance extension problem with degree constraint given a partial covariance sequence and the desired zeros of the shaping filter the poles are uniquely determined from the unique minimum of the corresponding optimization problem in this way we obtain an algorithm for solving the covariance extension problem as well as a constructive proof of georgiou s existence result and his conjecture a generalized version of which we have recently resolved using geometric methods we also survey recent related results on constrained nevanlinna pick interpolation in the context of a variational formulation of the general moment problem

optimal prefix codes for sources with two sided geometric distributions a complete characterization of optimal prefix codes for off centered two sided geometric distributions of the integers is presented these distributions are often encountered in lossless image compression applications as probabilistic models for image prediction residuals the family of optimal codes described is an extension of the golomb codes which are optimal for one sided geometric distributions the new family of codes allows for encoding of prediction residuals at a complexity similar to that of golomb codes without recourse to the heuristic approximations frequently used when modifying a code designed for nonnegative integers so as to apply to the encoding of any integer optimal decision rules for choosing among a lower complexity subset of the optimal codes given the distribution parameters are also investigated and the relative redundancy of the subset with respect to the full family of optimal codes is bounded

existence of optimal prefix codes for infinite source alphabets it is proven that for every random variable with a countably infinite set of outcomes and finite entropy there exists an optimal prefix code which can be constructed from huffman codes for truncated versions of the random variable and that the average lengths of any sequence of huffman codes for the truncated versions converge to that of the optimal code also it is shown that every optimal infinite code achieves kraft s inequality with equality

generalized interpolation in h infty with a complexity constraint in a seminal paper sarason generalized some classical interpolation problems for h infinity functions on the unit disc to problems concerning lifting onto h 2 of an operator t that is defined on k

a convex optimization approach to the rational covariance extension problem in this paper we present a convex optimization problem for solving the rational covariance extension problem given a partial covariance sequence and the desired zeros of the modeling filter the poles are uniquely determined from the unique minimum of the corresponding optimization problem in this way we obtain an algorithm for solving the covariance extension problem as well as a constructive proof of georgiou s seminal existence result and his conjecture a stronger version of which we have resolved in byrnes et al ieee trans automat control ac 40 1995 pp 1841 1857

invariance principles for switching systems via hybrid systems techniques invariance principles and sufficient conditions for asymptotic stability for switching systems are given multiple lyapunov like functions are used and dwell time persistent dwell time and weak dwell time switching signals are considered the invariance principles are derived from general invariance principles for hybrid systems asymptotic stability is concluded under observability assumptions or common bounds on the lyapunov like functions

on the well posedness of a parametric spectral estimation problem and its numerical solution this paper concerns a spectral estimation problem in which we want to find a spectral density function that is consistent with estimated second order statistics it is an inverse problem admitting multiple solutions and selection of a solution can be based on prior functions we show that the problem is well posed when formulated in a parametric fashion and that the solution parameter depends continuously on the prior function in this way we are able to obtain a smooth parametrization of admissible spectral densities based on this result the problem is reparametrized via a bijective change of variables out of a numerical consideration and then a continuation method is used to compute the unique solution parameter numerical aspects such as convergence of the proposed algorithm and certain computational procedures are addressed a simple example is provided to show the effectiveness of the algorithm

optimal prefix codes for infinite alphabets with nonlinear costs let p p i be a measure of strictly positive probabilities on the set of nonnegative integers although the countable number of inputs prevents usage of the huffman algorithm there are nontrivial p for which known methods find a source code that is optimal in the sense of minimizing expected codeword length for some applications however a source code should instead minimize one of a family of nonlinear objective functions beta exponential means those of the form loga sigmaip i an i where n i is the length of the ith codeword and a is a positive constant applications of such minimizations include a novel problem of maximizing the chance of message receipt in single shot communications a 1 this paper introduces methods for finding codes optimal for such exponential means one method applies to geometric distributions while another applies to distributions with lighter tails the latter algorithm is applied to poisson distributions and both are extended to alphabetic codes as well as to minimizing maximum pointwise redundancy the aforementioned application of minimizing the chance of buffer overflow is also considered

a globally convergent matricial algorithm for multivariate spectral estimation in this paper we first describe a matricial newton type algorithm designed to solve the multivariable spectrum approximation problem we then prove its global convergence finally we apply this approximation procedure to multivariate spectral estimation and test its effectiveness through simulation simulation shows that in the case of short observation records this method may provide a valid alternative to standard multivariable identification techniques such as matlab s pem and matlab s n4sid

multivariate spectral estimation based on the concept of optimal prediction in this technical note we deal with a spectrum approximation problem arising in three like multivariate spectral estimation approaches the solution to the problem minimizes a suitable divergence index with respect to an a priori spectral density we derive a new divergence family between multivariate spectral densities which takes root in the prediction theory under mild assumptions on the a priori spectral density the approximation problem based on this new divergence family admits a family of solutions moreover an upper bound on the complexity degree of these solutions is provided

a new family of high resolution multivariate spectral estimators in this paper we extend the beta divergence family to multivariate power spectral densities similarly to the scalar case we show that it smoothly connects the multivariate kullback leibler divergence with the multivariate itakura saito distance we successively study a spectrum approximation problem based on the beta divergence family which is related to a multivariate extension of the three spectral estimation technique it is then possible to characterize a family of solutions to the problem an upper bound on the complexity of these solutions will also be provided finally we will show that the most suitable solution of this family depends on the specific features required from the estimation problem

a generalized entropy criterion for nevanlinna pick interpolation with degree constraint we present a generalized entropy criterion for solving the rational nevanlinna pick problem for n 1 interpolating conditions and the degree of interpolants bounded by n the primal problem of maximizing this entropy gain has a very well behaved dual problem this dual is a convex optimization problem in a finite dimensional space and gives rise to an algorithm for finding all interpolants which are positive real and rational of degree at most n the criterion requires a selection of a monic schur polynomial of degree n it follows that this class of monic polynomials completely parameterizes all such rational interpolants and it therefore provides a set of design parameters for specifying such interpolants the algorithm is implemented in a state space form and applied to several illustrative problems in systems and control namely sensitivity minimization maximal power transfer and spectral estimation

optimal prefix codes for pairs of geometrically distributed random variables optimal prefix codes are studied for pairs of independent integer valued symbols emitted by a source with a geometric probability distribution of parameter q 0 1 covering the range q 1 2 the described codes produce the expected reduction in redundancy with respect to the 1 d case while maintaining low complexity coding operations

a combinatorial approach to golomb forests optimal binary prefix free codes for infinite sources with geometrically distributed frequencies e g p pi 1 p i 0 0p1 were first implicitly suggested by golomb over 30 years ago in the context of run length encodings ten years later gallager and van voorhis exhibited such optimal codes for all values of p these codes were derived by using the hu2man encoding algorithm to build optimal codes for finite sources and then showing that the finite codes converge in a very specific sense to the infinite one in this note we present a new combinatorial approach to solve the same problem one that does not use the hu2man algorithm but instead treats a coding tree as an infinite sequence of integers and derives properties of the sequence one consequence of this new approach is a complete characterization of all of the optimal codes in particular it shows that for all p 0p1 except for an easily describable countable set there is a unique optimal code but for each p in this countable set there are an uncountable number of optimal codes another consequence is a derivation of infinite codes for geometric sources when the encoding alphabet is no longer restricted to be the binary one a final consequence is the extension of the results to optimal forests instead of being restricted to optimal trees

on weak invariance principles for nonlinear switched systems in this technical note we develop two weak invariance principles for nonlinear switched systems we first present a union weak invariance principle for switched systems which includes as a special case the integral invariance principle it is shown that the switched solution approaches the largest weakly invariant set of the combined zero loci of the output functions then we extend the union weak invariance principle to an intersection weak invariance principle which greatly reduces the convergence region unlike the existing results in which the constructions of lyapunov functions are inevitable our principles do not require the existence of lyapunov functions numerical examples are presented to demonstrate the feasibility of our principles as well as applications to multi agent consensus problems

identifiability and well posedness of shaping filter parameterizations a global analysis approach in this paper we study the well posedness of the problems of determining shaping filters from combinations of finite windows of cepstral coefficients covariance lags or markov parameters for example we determine whether there exists a shaping filter with a prescribed window of markov parameters and a prescribed window of covariance lags we show that several such problems are well posed in the sense of hadamard that is one can prove existence uniqueness identifiability and continuous dependence of the model on the measurements our starting point is the global analysis of linear systems where one studies an entire class of systems or models as a whole and where one views measurements such as covariance lags and cepstral coefficients or markov parameters from data as functions on the entire class this enables one to pose such problems in a way that tools from calculus optimization geometry and modern nonlinear analysis can be used to give a rigorous answer to such problems in an algorithm independent fashion in this language we prove that a window of cepstral coefficients and a window of covariance coefficients yield a bona fide coordinate system on the space of shaping filters thereby establishing existence uniqueness and smooth dependence of the model parameters on the measurements from data

on the well posedness of multivariate spectrum approximation and convergence of high resolution spectral estimators abstract in this paper we establish the well posedness of the generalized moment problems recently studied by byrnes georgiou lindquist and coworkers and by ferrante pavon ramponi we then apply these continuity results to prove the almost sure convergence of a sequence of high resolution spectral estimators indexed by the sample size

an extension of the invariance principle for dwell time switched nonlinear systems abstract in this paper we propose an extension of the invariance principle for nonlinear switched systems under dwell time switched solutions this extension allows the derivative of an auxiliary function v also called a lyapunov like function along the solutions of the switched system to be positive on some sets the results of this paper are useful to estimate attractors of nonlinear switched systems and corresponding basins of attraction uniform estimates of attractors and basin of attractions with respect to time invariant uncertain parameters are also obtained results for a common lyapunov like function and multiple lyapunov like functions are given illustrative examples show the potential of the theoretical results in providing information on the asymptotic behavior of nonlinear dynamical switched systems

the interpolation problem with a degree constraint the author previously 1983 1987 showed that there is a correspondence between nonnegative hermitian trigonometric polynomials of degree spl les n and solutions to the standard nevanlinna pick caratheodory interpolation problem with n 1 constraints which are rational and also of degree spl les n it was conjectured that the correspondence under suitable normalization is bijective and thereby that it results in a complete parametrization of rational solutions of degree spl les n the conjecture was proven by byrnes et al 1995 along with a detailed study of this parametrization however byrnes et al used a slightly restrictive assumption that the trigonometric polynomials are positive and accordingly the corresponding solutions have positive real part the purpose of the present note is to extend the result to the case of nonnegative trigonometric polynomials as well we present the arguments in the context of the general nevanlinna pick caratheodory fejer interpolation

a maximum entropy enhancement for a family of high resolution spectral estimators structured covariances occurring in spectral analysis filtering and identification need to be estimated from a finite observation record the corresponding sample covariance usually fails to possess the required structure this is the case for instance in the byrnes georgiou lindquist three like tunable high resolution spectral estimators there the output covariance of a linear filter is needed to initialize the spectral estimation technique the sample covariance estimate however is usually not compatible with the filter in this paper we present a new systematic way to overcome this difficulty the new estimate is obtained by solving an ancillary problem with an entropic type criterion extensive scalar and multivariate simulation shows that this new approach consistently leads to a significant improvement of the spectral estimators performances

realization of power spectra from partial covariance sequences in this paper we address the problem of realization of a spectral density function from incomplete information about the underlying stochastic process the standing assumption is the availability of an incomplete partial sequence of covariance samples of the process we study the set of rational extensions of this finite sequence to an infinite covariance function that agrees with the available samples the classical theory of orthogonal polynomials with respect to the unit circle and the theory of moments have been utilized extensively in a variety of engineering problems including the one we are dealing with these have been known to provide a unifying framework for a variety of current spectral estimation techniques maximum entropy method pisarenko s harmonic decomposition etc in this work we consider and study the set of all covariance realizations of dimension lower than or equal to the length of the partial sequence and equal to the dimension of the maximum entropy realization the me solution is a point in this set other points correspond to pole zero models a general formula is obtained for recursively updated pole zero models of dimension increasing with the data record information about the zeros is obtained from the asymptotic behavior of the partial autocorrelation coefficients our approach combines techniques from the theory of orthogonal polynomials on the unit circle the theory of moments and also techniques from degree theory topology our objective is to develop a theory that will provide a frame for constructing recursively pole zero realizations of increasing dimension

an invariance principle for nonlinear switched systems abstract in this paper we address the problem of extending lasalle invariance principle to switched system we prove an extension of the invariance principle relative to dwell time switched solutions and a second one relative to constrained switched systems

time and spectral domain relative entropy a new approach to multivariate spectral estimation the concept of spectral relative entropy rate is introduced for jointly stationary gaussian processes using classical information theoretic results we establish a remarkable connection between time and spectral domain relative entropy rates this naturally leads to a new spectral estimation technique where a multivariate version of the itakura saito distance is employed it may be viewed as an extension of the approach called three introduced by byrnes georgiou and lindquist in 2000 which in turn followed in the footsteps of the burg jaynes maximum entropy method spectral estimation is here recast in the form of a constrained spectrum approximation problem where the distance is equal to the processes relative entropy rate the corresponding solution entails a complexity upper bound which improves on the one so far available in the multichannel framework indeed it is equal to the one featured by three in the scalar case the solution is computed via a globally convergent matricial newton type algorithm simulations suggest the effectiveness of the new technique in tackling multivariate spectral estimation tasks especially in the case of short data records

extensions of weak invariance principle for nonlinear switched systems with time invariant and time varying subsystems abstract this paper addresses the stability problem for nonlinear switched systems with both time invariant and time varying subsystems given that lyapunov like functions are difficult to construct for nonlinear switched systems generalized invariance principles are established based on observer functions for nonlinear switched systems where the subsystems share lyapunov like functions the generalized invariance principles can be specialized to lyapunov based invariance principles where accurate convergent region can be obtained in addition to the above efforts the definitions of p limit system and limit system set are presented under which the proposed invariance principles are extended to nonlinear switched systems with time varying subsystems illustrative examples show the effectiveness of the theoretical results

huffman coding with an infinite alphabet a new type of sufficient condition is provided for a probability distribution on the nonnegative integers to be given an optimal d ary prefix code by a huffman type algorithm in the justification of our algorithm we introduce two new essentially one concepts as the definition of the optimality of a prefix d ary code which are shown to be equivalent to that defined in the traditional way these new concepts of the optimality are meaningful even for the case where the shannon entropy h p diverges

lyapunov stability theory of nonsmooth systems this paper develops nonsmooth lyapunov stability theory and lasalle s invariance principle for a class of nonsmooth lipschitz continuous lyapunov functions and absolutely continuous state trajectories computable tests based on filipov s differential inclusion and clarke s generalized gradient are derived the primary use of these results is in analyzing the stability of equilibria of differential equations with discontinuous right hand side such as in nonsmooth dynamic systems or variable structure control