on mutual information likelihood ratios and estimation error for the additive gaussian channel this paper considers the model of an arbitrarily distributed signal x observed through an added independent white gaussian noise w y x w new relations between the minimal mean square error of the noncausal estimator and the likelihood ratio between y and w are derived this is followed by an extended version of a recently derived relation between the mutual information i x y and the minimal mean square error these results are applied to derive infinite dimensional versions of the fisher information and the de bruijn identity a comparison between the causal and noncausal estimation errors yields a restricted form of the logarithmic sobolev inequality the derivation of the results is based on the malliavin calculus

proper complex random processes with applications to information theory the covariance of complex random variables and processes when defined consistently with the corresponding notion for real random variables is shown to be determined by the usual complex covariance together with a quantity called the pseudo covariance a characterization of uncorrelatedness and wide sense stationarity in terms of covariance and pseudo covariance is given complex random variables and processes with a vanishing pseudo covariance are called proper it is shown that properness is preserved under affine transformations and that the complex multivariate gaussian density assumes a natural form only for proper random variables the maximum entropy theorem is generalized to the complex multivariate case the differential entropy of a complex random vector with a fixed correlation matrix is shown to be maximum if and only if the random vector is proper gaussian and zero mean the notion of circular stationarity is introduced for the class of proper complex random processes a discrete fourier transform correspondence is derived relating circular stationarity in the time domain to uncorrelatedness in the frequency domain an application of the theory is presented

second order asymptotics of mutual information a formula for the second order expansion of the input output mutual information of multidimensional channels as the signal to noise ratio snr goes to zero is obtained while the additive noise is assumed to be gaussian we deal with very general classes of input and channel distributions as special cases these channel models include fading channels channels with random parameters and channels with almost gaussian noise when the channel is unknown at the receiver the second term in the asymptotic expansion depends not only on the covariance matrix of the input signal but also on the fourth mixed moments of its components the study of the second order asymptotics of mutual information finds application in the analysis of the bandwidth power tradeoff achieved by various signaling strategies in the wideband regime

some relationships between logics of programs and complexity theory abstract we show that some open problems concerning comparative schematology and logics of programs are equivalent to open problems in complexity theory in particular we show that p space p time holds if and only if flow diagrams with arrays are of the same computational power as recursive procedures both conditions are equivalent to the statement that programming logics based on the mentioned classes of program schemes have equal expressive power a similar characterization is given for some other equalities between complexity classes

complete families of commuting functions for coisotropic hamiltonian actions abstract let g be an algebraic group defined over a field f of characteristic zero with g lie g the dual space g equipped with the lie poisson bracket is a poisson variety and each irreducible g invariant subvariety x g carries the induced poisson structure we prove that there is a set f 1 f l f x of algebraically independent polynomial functions which pairwise commute with respect to the poisson bracket such that l dim x tr deg f x g 2 we also discuss several applications of this result to complete integrability of hamiltonian systems on symplectic hamiltonian g varieties of corank zero and 2

an infinite pebble game and applications we generalize the pebble game to infinite directed acyclic graphs and use this generalization to give new and shorter proofs of the following well known results 1 that unbounded memory increases the power of logics of programs and 2 that there exists a context free grammar with infinite index

a new entropy power inequality a strengthened version of shannon s entropy power inequality for the case where one of the random vectors involved is gaussian is proved in particular it is shown that if independent gaussian noise is added to an arbitrary multivariate random variable the entropy power of the resulting random variable is a concave function of the variance power of the added noise the strengthened inequality is shown to hold for the class of stable distributions

optimum power allocation for parallel gaussian channels with arbitrary input distributions the mutual information of independent parallel gaussian noise channels is maximized under an average power constraint by independent gaussian inputs whose power is allocated according to the waterfilling policy in practice discrete signaling constellations with limited peak to average ratios m psk m qam etc are used in lieu of the ideal gaussian signals this paper gives the power allocation policy that maximizes the mutual information over parallel channels with arbitrary input distributions such policy admits a graphical interpretation referred to as mercury waterfilling which generalizes the waterfilling solution and allows retaining some of its intuition the relationship between mutual information of gaussian channels and nonlinear minimum mean square error mmse proves key to solving the power allocation problem

hessian and concavity of mutual information differential entropy and entropy power in linear vector gaussian channels within the framework of linear vector gaussian channels with arbitrary signaling the jacobian of the minimum mean square error and fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper capitalizing on prior research where the minimum mean square error and fisher information matrices were linked to information theoretic quantities through differentiation the hessian of the mutual information and the entropy are derived these expressions are then used to assess the concavity properties of mutual information and entropy under different channel conditions and also to derive a multivariate version of an entropy power inequality due to costa

representation of mutual information via input estimates a relationship between information theory and estimation theory was recently shown for the gaussian channel relating the derivative of mutual information with the minimum mean square error this paper generalizes the link between information theory and estimation theory to arbitrary channels giving representations of the derivative of mutual information as a function of the conditional marginal input distributions given the outputs we illustrate the use of this representation in the efficient numerical computation of the mutual information achieved by inputs such as specific codes or natural language

poisson commutative subalgebras and complete integrability on non regular coadjoint orbits and flag varieties the purpose of this paper is to bring together various loose ends in the theory of integrable systems for a semisimple lie algebra mathfrak g we obtain several results on the completeness of homogeneous poisson commutative subalgebras of mathscr s mathfrak g on coadjoint orbits this concerns in particular gelfand tsetlin and mishchenko fomenko subalgebras our results reveal the crucial role of nilpotent orbits and sheets in mathfrak g simeq mathfrak g

unbounded program memory adds to the expressive power of first order programming logic it is proved that no logic of programs with unbounded memory is reducible to a bounded memory programming logic this is achieved by carefully analyzing firstorder definability in the algebra of finite binary trees

the unwind property in certain algebras in the present paper we investigate abstract algebras having the so called unwind property i e algebras where every total computable function can be defined by open formulas in an adequate first order language a method of proving the unwind property is given for a special kind of algebra applying this method we prove that there exists a structure satisfying the unwind property for all flow charts but not for recursive procedures we prove also that algebraically closed and real closed fields with infinite degrees of transcendence over their prime subfields satisfy the unwind property for all schemas

mutual information and minimum mean square error in gaussian channels this paper deals with arbitrarily distributed finite power input signals observed through an additive gaussian noise channel it shows a new formula that connects the input output mutual information and the minimum mean square error mmse achievable by optimal estimation of the input given the output that is the derivative of the mutual information nats with respect to the signal to noise ratio snr is equal to half the mmse regardless of the input statistics this relationship holds for both scalar and vector signals as well as for discrete time and continuous time noncausal mmse estimation this fundamental information theoretic result has an unexpected consequence in continuous time nonlinear estimation for any input signal with finite power the causal filtering mmse achieved at snr is equal to the average value of the noncausal smoothing mmse achieved with a channel whose snr is chosen uniformly distributed between 0 and snr

on static logics dynamic logics and complexity classes several versions of quantified dynamic logic are shown to be equivalent in expressive power to static extensions of classical logics consequently by recent results of various researchers many connections between dynamic logics and complexity classes are obtained among other things a sequence of dynamic logics of increasing expressive power which correspond over appropriate finite structures to logspace ptime and pspace as well as to the sets definable in the logarithmic space polynomial time and arithmetical hierarchies is exhibited

weylgruppe und momentabbildung let g be a connected reductive group defined over an algebraically closed field of characteristic zero we assign to any g variety x a finite cristallographic reflection group w x by means of the moment map on the cotangent bundle this generalizes the little weyl group of a symmetric space the weyl group w x is related to the equivariant compactification theory of x we determine the closure of the image of the moment map and the generic isotropy group of the action of g on the cotangent bundle as a byproduct we determine the ideal of elements of mathfrak u g which act trivially on x as a differential operator

definability with bounded number of bound variables abstract a theory satisfies the k variable property if every first order formula is equivalent to a formula with at most k bound variables possibly reused gabbay has shown that a model of temporal logic satisfies the k variable property for some k if and only if there exists a finite basis for the temporal connectives over that model we give a model theoretic method for establishing the k variable property involving a restricted ehrenfeucht fraisse game in which each player has only k pebbles we use the method to unify and simplify results in the literature for linear orders we also establish new k variable properties for various theories of bounded degree trees and in each case obtain tight upper and lower bounds on k this gives the first finite basis theorems for branching time models of temporal logic

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters