passivity preserving model reduction by analytic interpolation abstract antoulas and sorensen have recently proposed a passivity preserving model reduction method of linear systems based on krylov projections the idea is to approximate a positive real rational transfer function with one of lower degree the method is based on an observation by antoulas in the single input single output case that if the approximant is preserving a subset of the spectral zeros and takes the same values as the original transfer function in the mirror points of the preserved spectral zeros then the approximant is also positive real however this turns out to be a special solution in the theory of analytic interpolation with degree constraint developed by byrnes georgiou and lindquist namely the maximum entropy central solution by tuning the interpolation points and the spectral zeros as prescribed by this theory one is able to obtain considerably better reduced order models we also show that in the multi input multi output case sorensen s algorithm actually amounts to tangential nevanlinna pick interpolation

passivity preserving model reduction via interpolation of spectral zeros abstract an algorithm is developed for passivity preserving model reduction of linear time invariant systems implementation schemes are described for both medium scale dense and large scale sparse applications the algorithm is based upon interpolation at selected spectral zeros of the original transfer function to produce a reduced transfer function that has the specified roots as its spectral zeros these interpolation conditions are satisfied through the computation of a basis for a selected invariant subspace of a certain block matrix which has the spectral zeros as its spectrum explicit interpolation is avoided and passivity of the reduced model is established instead through satisfaction of the necessary conditions of the positive real lemma it is also shown that this procedure indirectly solves the associated controllability and observability riccati equations and how to select the interpolation points to give maximal or minimal solutions of these equations from these a balancing transformation may be constructed to give a reduced model that is balanced as well as passive and stable

balanced parametrization of classes of linear systems canonical forms and parametrizations are presented for several sets of minimal systems of given dimension asymptotically stable systems allpass systems bounded real systems positive real systems minimum phase systems and the class of all minimal systems the approach is based on balancing techniques for these classes of systems applications of these results to hankel operators and model reduction are discussed

brownian integrated covariance functions for gaussian process modeling sigmoidal versus localized basis functions abstractgaussian process modeling or kriging is a popular method for modeling data from deterministic computer simulations and the most common choices of covariance function are gaussian power exponential and matern a characteristic of these covariance functions is that the basis functions associated with their corresponding response predictors are localized in the sense that they decay to zero as the input location moves away from the simulated input sites as a result the predictors tend to revert to the prior mean which can result in a bumpy fitted response surface in contrast a fractional brownian field model results in a predictor with basis functions that are nonlocalized and more sigmoidal in shape although it suffers from drawbacks such as inability to represent smooth response surfaces we propose a class of brownian integrated covariance functions obtained by incorporating an integrator as in the white noise integral representation of a fractional brownian field into any stationary

a complete parameterization of all positive rational extensions of a covariance sequence in this paper we formalize the observation that filtering and interpolation induce complementary or dual decompositions of the space of positive real rational functions of degree less than or equal to n from this basic result about the geometry of the space of positive real functions we are able to deduce two complementary sets of conclusions about positive rational extensions of a given partial covariance sequence on the one hand by viewing a certain fast filtering algorithm as a nonlinear dynamical system defined on this space we are able to develop estimates on the asymptotic behavior of the schur parameters 1918 of positive rational extensions on the other hand we are also able to provide a characterization of all positive rational extensions of a given partial covariance sequence indeed motivated by its application to signal processing speech processing and stochastic realization theory this characterization is in terms of a complete parameterization using familiar objects from systems theory and proves a conjecture made by georgiou 1983 1987 our basic result however also enables us to analyze the robustness of this parameterization with respect to variations in the problem data the methodology employed is a combination of complex analysis geometry linear systems and nonlinear dynamics

a computationally stable approach to gaussian process interpolation of deterministic computer simulation data for many expensive deterministic computer simulators the outputs do not have replication error and the desired metamodel or statistical emulator is an interpolator of the observed data realizations of gaussian spatial processes gp are commonly used to model such simulator outputs fitting a gp model to n data points requires the computation of the inverse and determinant of n n correlation matrices r that are sometimes computationally unstable due to near singularity of r this happens if any pair of design points are very close together in the input space the popular approach to overcome near singularity is to introduce a small nugget or jitter parameter in the model that is estimated along with other model parameters the inclusion of a nugget in the model often causes unnecessary over smoothing of the data in this article we propose a lower bound on the nugget that minimizes the over smoothing and an iterative regularization approach to construct a predictor that further improves the inter

bayesian prediction of deterministic functions with applications to the design and analysis of computer experiments abstract this article is concerned with prediction of a function y t over a multidimensional domain t given the function values at a set of sites t 1 t 2 t n in t and with the design that is with the selection of those sites the motivating application is the design and analysis of computer experiments where t determines the input to a computer model of a physical or behavioral system and y t is a response that is part of the output or is calculated from it following a bayesian formulation prior uncertainty about the function y is expressed by means of a random function y which is taken here to be a gaussian stochastic process the mean of the posterior process can be used as the prediction function t and the variance can be used as a measure of uncertainty this kind of approach has been used previously in bayesian interpolation and is strongly related to the kriging methods used in geostatistics here emphasis is placed on product linear and product cubic correlation func

optimal codes with limited kraft sum the well known huffman algorithm is an elegant approach to solve the basic problem of finding the optimal code among those with kraft sums smaller than or equal to 1 in this paper an extended problem is investigated where the kraft sum of the usable codes is limited to a given real number gamma 0 unlike the case gamma 1 i e huffman coding for gamma the kraft sum of the optimal code is not necessarily equal to gamma however we show that the kraft sum of the optimal code lies in a finite set of rational numbers which depend on the value of gamma this motivates us to first devise a recursive algorithm to obtain the optimal codelengths with kraft sum equal to gamma where gamma has a finite binary representation and 0 to do so the set of n tuple codelength vectors with a specific kraft sum gamma is characterized the binary representation of gamma plays a fundamental role in all problems that arise for gamma

a non stationary covariance based kriging method for metamodelling in engineering design metamodels are widely used to facilitate the analysis and optimization of engineering systems that involve computationally expensive simulations kriging is a metamodeling technique that is well known for its ability to build surrogate models of responses with nonlinear behavior however the assumption of a stationary covariance structure underlying kriging does not hold in situations where the level of smoothness of a response varies significantly although nonstationary gaussian process models have been studied for years in statistics and geostatistics communities this has largely been for physical experimental data in relatively low dimensions in this paper the nonstationary covariance structure is incorporated into kriging modeling for computer simulations to represent the nonstationary covariance structure we adopt a nonlinear mapping approach based on a parameterized density functions to avoid over parameterizing for the high dimension problems typical of engineering design we propose a modified version of the nonlinear map approach with a sparser yet flexible parameterization the effectiveness of the proposed method is demonstrated through both mathematical and engineering examples the robustness of the method is verified by testing multiple functions under various sampling settings we also demonstrate that our method is effective in quantifying prediction uncertainty associated with the use of metamodels nomenclature

nonparametric estimation of nonstationary spatial covariance structure abstract estimation of the covariance structure of spatial processes is a fundamental prerequisite for problems of spatial interpolation and the design of monitoring networks we introduce a nonparametric approach to global estimation of the spatial covariance structure of a random function z x t observed repeatedly at times ti i 1 t at a finite number of sampling stations xi i 1 2 n in the plane our analyses assume temporal stationarity but do not assume spatial stationarity or isotropy we analyze the spatial dispersions var z xi t z xj t as a natural metric for the spatial covariance structure and model these as a general smooth function of the geographic coordinates of station pairs xi xj the model is constructed in two steps first using nonmetric multidimensional scaling mds we compute a two dimensional representation of the sampling stations for which a monotone function of interpoint distances ij approximates the spatial dispersions mds transforms the problem

from finite covariance windows to modeling filters a convex optimization approach the trigonometric moment problem is a classical moment problem with numerous applications in mathematics physics and engineering the rational covariance extension problem is a constrained version of this problem with the constraints arising from the physical realizability of the corresponding solutions although the maximum entropy method gives one well known solution in several applications a wider class of solutions is desired in a seminal paper georgiou derived an existence result for a broad class of models in this paper we review the history of this problem going back to carath eodory as well as applications to stochastic systems and signal processing in particular we present a convex optimization problem for solving the rational covariance extension problem with degree constraint given a partial covariance sequence and the desired zeros of the shaping filter the poles are uniquely determined from the unique minimum of the corresponding optimization problem in this way we obtain an algorithm for solving the covariance extension problem as well as a constructive proof of georgiou s existence result and his conjecture a generalized version of which we have recently resolved using geometric methods we also survey recent related results on constrained nevanlinna pick interpolation in the context of a variational formulation of the general moment problem

the construction of huffman codes is a submodular convex optimization problem over a lattice of binary trees we show that the space of all binary huffman codes for a finite alphabet defines a lattice ordered by the imbalance of the code trees representing code trees as path length sequences we show that the imbalance ordering is closely related to a majorization ordering on real valued sequences that correspond to discrete probability density functions furthermore this tree imbalance is a partial ordering that is consistent with the total orderings given by either the external path length sum of tree path lengths or the entropy determined by the tree structure on the imbalance lattice we show the weighted path length of a tree the usual objective function for huffman coding is a submodular function as is the corresponding function on the majorization lattice submodular functions are discrete analogues of convex functions these results give perspective on huffman coding and suggest new approaches to coding as optimization over a lattice

generalized interpolation in h infty with a complexity constraint in a seminal paper sarason generalized some classical interpolation problems for h infinity functions on the unit disc to problems concerning lifting onto h 2 of an operator t that is defined on k

bayesian inference for non stationary spatial covariance structure via spatial deformations summary in geostatistics it is common practice to assume that the underlying spatial process is stationary and isotropic i e the spatial distribution is unchanged when the origin of the index set is translated and under rotation about the origin however in environmental problems such assumptions are not realistic since local influences in the correlation structure of the spatial process may be found in the data the paper proposes a bayesian model to address the anisot ropy problem following sampson and guttorp we define the correlation function of the spatial process by reference to a latent space denoted by d where stationarity and isotropy hold the space where the gauged monitoring sites lie is denoted by g we adopt a bayesian approach in which the mapping between g and d is represented by an unknown function d a gaussian process prior distribution is defined for d unlike the sampson guttorp approach the mapping of both gauged and ungauged sites is handled in a single framework and predictive inferences take explicit account of uncertainty in the mapping markov chain monte carlo methods are used to obtain samples from the posterior distributions two examples are discussed a simulated data set and the solar radiation data set that also was analysed by sampson and guttorp

a convex optimization approach to the rational covariance extension problem in this paper we present a convex optimization problem for solving the rational covariance extension problem given a partial covariance sequence and the desired zeros of the modeling filter the poles are uniquely determined from the unique minimum of the corresponding optimization problem in this way we obtain an algorithm for solving the covariance extension problem as well as a constructive proof of georgiou s seminal existence result and his conjecture a stronger version of which we have resolved in byrnes et al ieee trans automat control ac 40 1995 pp 1841 1857

sensitivity shaping with degree constraint by nonlinear least squares optimization this paper presents a new approach to shaping of the frequency response of the sensitivity function in this approach a desired frequency response is assumed to be specified at a finite number of frequency points a sensitivity shaping problem is formulated as an approximation problem to the desired frequency response with a function in a class of sensitivity functions with a degree bound the sensitivity shaping problem is reduced to a finite dimensional constrained nonlinear least squares optimization problem to solve the optimization problem numerically standard algorithms for an unconstrained version of nonlinear least squares problems are modified to incorporate the constraint numerical examples illustrate how these design parameters are tuned in an intuitive manner as well as how the design proceeds in actual control problems

analysis of computer experiments using penalized likelihood in gaussian kriging models kriging is a popular analysis approach for computer experiments for the purpose of creating a cheap to compute meta model as a surrogate to a computationally expensive engineering simulation model the maximum likelihood approach is used to estimate the parameters in the kriging model however the likelihood function near the optimum may be flat in some situations which leads to maximum likelihood estimates for the parameters in the covariance matrix that have very large variance to overcome this difficulty a penalized likelihood approach is proposed for the kriging model both theoretical analysis and empirical experience using real world data suggest that the proposed method is particularly important in the context of a computationally intensive simulation model where the number of simulation runs must be kept small because collection of a large sample set is prohibitive the proposed approach is applied to the reduction of piston slap an unwanted engine noise due to piston secondary motion issu

balanced stochastic realizations abstract the balanced stochastic realization is introduced as a balanced solution to the continuous time positive real equations and the dual positive real equations the structure of the associated balanced spectral factors is derived the main result being a product decomposition of the spectral factors the properties of the stochastic model reduction technique of balanced stochastic truncation are considered in particular it is shown that balanced stochastic truncation preserves the right half plane zeros of the spectral factors leading to a transformation approach to the approximation of non minimum phase stochastic systems

on the problem of stable rational interpolation abstract aspects of the stability issue in connection with rational interpolation are investigated in particular it is shown that unconstrained interpolation of a given set of points together with an associated mirror image set of points yields a one parameter family of stable interpolating functions as an application it is also shown that if a certain number of markov parameters are given by appropriate choice of the moments stable realizations are obtained

lifted brownian kriging models abstractgaussian processes have become a standard framework for modeling deterministic computer simulations and producing predictions of the response surface this article investigates a new covariance function that is shown to offer superior prediction compared to the more common covariances for computer simulations of real physical systems this is demonstrated via a gamut of realistic examples a simple closed form expression for the covariance is derived as a limiting form of a brownian like covariance model as it is extended to some hypothetical higher dimensional input domain and so we term it a lifted brownian covariance this covariance has connections with the multiquadric kernel through analysis of the kriging model this article offers some theoretical comparisons between the proposed covariance model and existing covariance models the major emphasis of the theory is explaining why the proposed covariance is superior to its traditional counterparts for many computer simulations of real phys

on the duality between filtering and nevanlinna pick interpolation positive real rational functions play a central role in both deterministic and stochastic linear systems theory as well as in circuit synthesis spectral analysis and speech processing for this reason results about positive real transfer functions and their realizations typically have many applications and manifestations in this paper we study certain manifolds and submanifolds of positive real transfer functions describing a fundamental geometric duality between filtering and nevanlinna pick interpolation not surprisingly then this duality while interesting in its own right has several corollaries which provide solutions and insight into some very interesting and intensely researched problems one of these is the problem of parameterizing all rational solutions of bounded degree of the nevanlinna pick interpolation problem which plays a central role in robust control and for which the duality theorem yields a complete solution in this paper we shall describe the duality theorem which we motivate in terms of both the interpolation problem and a fast algorithm for kalman filtering viewed as a nonlinear dynamical system on the space of positive real transfer functions we also outline a new proof of the recent solution to the rational nevanlinna pick interpolation problem using an algebraic topological generalization of hadamard s global inverse function theorem

the minimum average code for finite memoryless monotone sources the problem of selecting a code for finite monotone sources with n symbols is considered the selection criterion is based on minimizing the average redundancy called minave criterion instead of its maximum i e minimax criterion the average probability distribution pnmacr whose associated huffman code has the minimum average redundancy is derived the entropy of the average distribution i e h pnmacr and the average entropy of the monotone distributions i e h pnmacr are studied it is shown that both logn h pnmacr and logn h pnmacr are asymptotically equal to a constant sime0 61 therefore there is only a negligible penalty at most 1 61 bits symbol in using a simple fixed length code with respect to the optimal code an efficient near optimal encoding technique is also proposed the consequences of the two approaches i e minave and minimax are compared in terms of their associated distributions and associated codes in order to evaluate the average performance of the minimax code we prove that the informational divergence of the average distribution and minimax distribution asymptotically grows as 2 275 loglogn

identifiability and well posedness of shaping filter parameterizations a global analysis approach in this paper we study the well posedness of the problems of determining shaping filters from combinations of finite windows of cepstral coefficients covariance lags or markov parameters for example we determine whether there exists a shaping filter with a prescribed window of markov parameters and a prescribed window of covariance lags we show that several such problems are well posed in the sense of hadamard that is one can prove existence uniqueness identifiability and continuous dependence of the model on the measurements our starting point is the global analysis of linear systems where one studies an entire class of systems or models as a whole and where one views measurements such as covariance lags and cepstral coefficients or markov parameters from data as functions on the entire class this enables one to pose such problems in a way that tools from calculus optimization geometry and modern nonlinear analysis can be used to give a rigorous answer to such problems in an algorithm independent fashion in this language we prove that a window of cepstral coefficients and a window of covariance coefficients yield a bona fide coordinate system on the space of shaping filters thereby establishing existence uniqueness and smooth dependence of the model parameters on the measurements from data

choosing the sample size of a computer experiment a practical guide we provide reasons and evidence supporting the informal rule that the number of runs for an effective initial computer experiment should be about 10 times the input dimension our arguments quantify two key characteristics of computer codes that affect the sample size required for a desired level of accuracy when approximating the code via a gaussian process gp the first characteristic is the total sensitivity of a code output variable to all input variables the second corresponds to the way this total sensitivity is distributed across the input variables specifically the possible presence of a few prominent input factors and many impotent ones i e effect sparsity both measures relate directly to the correlation structure in the gp approximation of the code in this way the article moves toward a more formal treatment of sample size for a computer experiment the evidence supporting these arguments stems primarily from a simulation study and via specific codes modeling climate and ligand activa

the number of huffman codes compact trees and sums of unit fractions the number of nonequivalent compact huffman codes of length r over an alphabet of size t has been studied frequently equivalently the number of nonequivalent complete t ary trees has been examined we first survey the literature unifying several independent approaches to the problem then improving on earlier work we prove a very precise asymptotic result on the counting function consisting of two main terms and an error term

bayesian design and analysis of computer experiments use of derivatives in surface prediction this article is concerned with the problem of predicting a deterministic response function yo over a multidimensional domain t given values of yo and all of its first derivatives at a set of design sites points in t the intended application is to computer experiments in which yo is an output from a computer model of a physical system and each point in t represents a particular configuration of the input parameters it is assumed that the first derivatives are already available e g from a sensitivity analysis or can be produced by the code that implements the model a bayesian approach in which the random function that represents prior uncertainty about yo is taken to be a stationary gaussian stochastic process is used the calculations needed to update the prior given observations of yo and its first derivatives at the design sites are given and are illustrated in a small example the issue of experimental design is also discussed in particular the criterion of maximizing the reduction in entropy

the interpolation problem with a degree constraint the author previously 1983 1987 showed that there is a correspondence between nonnegative hermitian trigonometric polynomials of degree spl les n and solutions to the standard nevanlinna pick caratheodory interpolation problem with n 1 constraints which are rational and also of degree spl les n it was conjectured that the correspondence under suitable normalization is bijective and thereby that it results in a complete parametrization of rational solutions of degree spl les n the conjecture was proven by byrnes et al 1995 along with a detailed study of this parametrization however byrnes et al used a slightly restrictive assumption that the trigonometric polynomials are positive and accordingly the corresponding solutions have positive real part the purpose of the present note is to extend the result to the case of nonnegative trigonometric polynomials as well we present the arguments in the context of the general nevanlinna pick caratheodory fejer interpolation

realization of power spectra from partial covariance sequences in this paper we address the problem of realization of a spectral density function from incomplete information about the underlying stochastic process the standing assumption is the availability of an incomplete partial sequence of covariance samples of the process we study the set of rational extensions of this finite sequence to an infinite covariance function that agrees with the available samples the classical theory of orthogonal polynomials with respect to the unit circle and the theory of moments have been utilized extensively in a variety of engineering problems including the one we are dealing with these have been known to provide a unifying framework for a variety of current spectral estimation techniques maximum entropy method pisarenko s harmonic decomposition etc in this work we consider and study the set of all covariance realizations of dimension lower than or equal to the length of the partial sequence and equal to the dimension of the maximum entropy realization the me solution is a point in this set other points correspond to pole zero models a general formula is obtained for recursively updated pole zero models of dimension increasing with the data record information about the zeros is obtained from the asymptotic behavior of the partial autocorrelation coefficients our approach combines techniques from the theory of orthogonal polynomials on the unit circle the theory of moments and also techniques from degree theory topology our objective is to develop a theory that will provide a frame for constructing recursively pole zero realizations of increasing dimension

93 of the 3 over 4 conjecture is already verified one of the most important challenges in the context of fix free codes is proving the 3 4 conjecture which guarantees the existence of fix free codewords of lengths l1 l2 ln if i 1n2 li 3 4 although this conjecture has not become a theorem yet some researchers have proved the problem with some extra constraints on the codelengths one of those we call it yekhanin s constraint is i li 1 2 li 1 2 where minklk in this paper it is shown that such a constraint is not so restrictive we prove that almost 93 8 of the n tuple codelength vectors with kraft sum 3 4 do satisfy yekhanin s constraint one can optimistically interpret this result as almost 93 8 of the road of proving the 3 4 conjecture is paved

on the vector space of the automatic reals abstract a sequence a n n 0 is said to be k automatic if a n is a finite state function of the base k digits of n we say a real number is k b automatic if its fractional part has a base b expansion that forms a k automatic sequence and we denote the set of all such numbers as l k b lehr theoret comput sci 108 1993 385 391 proved that l k b forms a vector space over q in this paper we give a shortened version of the proof of lehr s result and answering a question of bach show that the dimension of the vector space l k b is infinite we also give an example of a transcendental number such that all of its positive powers are automatic the proof requires examining the coefficient of x n in the formal power series x x 2 x 4 x 8 r along the way we are led to examine several sequences of independent combinatorial interest finally solving an open problem we show that the automatic reals are not closed under 1 product 2 squaring and 3 reciprocal

a new result on passivity preserving model reduction abstract the problem of model reduction with preservation of passivity is investigated the approach is based on positive real interpolation and is inspired by the similarity between lowner and pick matrices the former are important in problems of general rational interpolation while the latter in problems of interpolation by positive real functions it follows that interpolation of the original set of data together with an appropriately defined mirror image set of data yields automatically positive real interpolants subsequently we show how this result can be implemented using a krylov projection procedure the ensuing model reduction method preserves stability and passivity and can be implemented efficiently for the large scale systems

canonical correlation analysis approximate covariance extension and identification of stationary time series abstract in this paper we analyze a class of state space identification algorithms for time series based on canonical correlation analysis in the light of recent results on stochastic systems theory in principle these so called subspace methods can be described as covariance estimation followed by stochastic realization the methods offer the major advantage of converting the nonlinear parameter estimation phase in traditional arma models identification into the solution of a riccati equation but introduce at the same time some nontrivial mathematical problems related to positivity the reason for this is that an essential part of the problem is equivalent to the well known rational covariance extension problem therefore the usual deterministic arguments based on factorization of a hankel matrix are not valid for generic data something that is habitually overlooked in the literature we demonstrate that there is no guarantee that several popular identification procedures based on the same principle will not fail to produce a positive extension unless some rather stringent assumptions are made which in general are not explicitly reported in this paper the statistical problem of stochastic modeling from estimated covariances is phrased in the geometric language of stochastic realization theory we review the basic ideas of stochastic realization theory in the context of identification discuss the concept of stochastic balancing and of stochastic model reduction by principal subsystem truncation the model reduction method of desai and pal 1982 a realization approach to stochastic model reduction proc 1st decision and control conf pp 1105 1112 based on truncated balanced stochastic realizations is partially justified showing that the reduced system structure has a positive covariance sequence but is in general not balanced as a byproduct of this analysis we obtain a theorem prescribing conditions under which the subspace identification methods produce bona fide stochastic systems