an interval algorithm for constrained global optimization abstract an interval algorithm for bounding the solutions of a constrained global optimization problem is described the problem functions are assumed only to be continuous it is shown how the computational cost of bounding a set which satisfies equality constraints can often be reduced if the equality constraint functions are assumed to be continuously differentiable numerical results are presented

the 4m mixed memory markov model algorithm for finding genes in prokaryotic genomes in this paper we present a new algorithm called 4m mixed memory markov model for finding genes from the genomes of prokaryotes this is achieved by modeling the known coding regions of the genome as a set of sample paths of a multistep markov chain call it and the known non coding regions as a set of sample paths of another multistep markov chain call it the new feature of the 4m algorithm is that different states are allowed to have different memory lengths in contrast to a fixed multistep markov model used in genemark in its various versions at the same time compared with an algorithm like glimmer3 that uses an interpolation of markov models of different memory lengths the statistical significance of the conclusions drawn from the 4m algorithm is quite easy to quantify thus when a whole genome annotation is carried out and several new genes are predicted it is extremely easy to rank these predictions in terms of the confidence one has in the predictions the basis of the 4m algorithm is a simple rank condition satisfied by the matrix of frequencies associated with a markov chain the 4m algorithm is validated by applying it to 75 organisms belonging to practically all known families of bacteria and archae the performance of the 4m algorithm is compared with those of glimmer3 genemark2 5d and genemarkhmm2 6g it is found that in a vast majority of cases the 4m algorithm finds many more genes than it misses compared with any of the other three algorithms next the 4m algorithm is used to carry out whole genome annotation of 13 organisms by using 50 of the known genes as the training input for the coding model and 20 of the known non genes as the training input for the non coding model after this all of the open reading frames are classified it is found that the 4m algorithm is highly specific in that it picks out virtually all of the known genes while predicting that only a small number of the open reading frames whose status is unknown are genes

beyond the entropy power inequality via rearrangements a lower bound on the renyi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements for the special case of boltzmann shannon entropy this lower bound is better than that given by the entropy power inequality several applications are discussed including a new proof of the classical entropy power inequality and an entropy inequality involving symmetrization of levy processes

a new entropy power inequality for integer valued random variables the entropy power inequality epi yields lower bounds on the differential entropy of the sum of two independent real valued random variables in terms of the individual entropies versions of the epi for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy but no universal inequality is known beyond trivial ones more recently the sumset theory for the entropy function yields a sharp inequality h x x prime h x geq 1 over 2 o 1 when x x prime are independent identically distributed i i d with high entropy this paper provides the inequality h x x prime h x geq g h x where x x prime are arbitrary i i d integer valued random variables and where g is a universal strictly positive function on bbr satisfying g 0 0 extensions to nonidentically distributed random variables and to conditional entropies are also obtained

on interval enclosures using slope arithmetic this paper describes how a combination of interval derivative arithmetic and interval slope arithmetic can be used with the programming language pascal sc to obtain an enclosure of the range of a factorable function r n r 1 with c 2 d which is often narrower than that which is obtained by using slope arithmetic alone and which is always narrower than that which is obtained by using interval derivative arithmetic alone

the entropy power inequality for quantum systems when two independent analog signals x and y are added together giving z x y the entropy of z h z is not a simple function of the entropies h x and h y but rather depends on the details of x and y s distributions nevertheless the entropy power inequality epi which states that e2h z e2h x e2h y gives a very tight restriction on the entropy of z this inequality has found many applications in information theory and statistics the quantum analogue of adding two random variables is the combination of two independent bosonic modes at a beam splitter the purpose of this paper is to give a detailed outline of the proof of two separate generalizations of the epi to the quantum regime our proofs are similar in spirit to the standard classical proofs of the epi but some new quantities and ideas are needed in the quantum setting in particular we find a new quantum de bruijin identity relating entropy production under diffusion to a divergence based quantum fisher information furthermore this fisher information exhibits certain convexity properties in the context of beam splitters

an interval algorithm for nondifferentiable global optimization an interval algorithm for constrained nondifferentiable global optimization in which an exact penalty function is used is described and the determination of the penalty function parameter is discussed numerical results are presented

a strengthened entropy power inequality for log concave densities we show that shannon s entropy power inequality admits a strengthened version in the case in which the densities are log concave in such a case in fact one can extend the blachman stam argument to obtain a sharp inequality for the second derivative of shannon s entropy functional with respect to the heat semigroup

the conditional entropy power inequality for quantum additive noise channels we prove the quantum conditional entropy power inequality for quantum additive noise channels this inequality lower bounds the quantum conditional entropy of the output of an additive noise channel in terms of the quantum conditional entropies of the input state and the noise when they are conditionally independent given the memory we also show that this conditional entropy power inequality is optimal in the sense that we can achieve equality asymptotically by choosing a suitable sequence of gaussian input states we apply the conditional entropy power inequality to find an array of information theoretic inequalities for conditional entropies which are the analogs of inequalities which have already been established in the unconditioned setting furthermore we give a simple proof of the convergence rate of the quantum ornstein uhlenbeck semigroup based on entropy power inequalities we prove the quantum conditional entropy power inequality for quantum additive noise channels this inequality lower bounds the quantum conditional entropy of the output of an additive noise channel in terms of the quantum conditional entropies of the input state and the noise when they are conditionally independent given the memory we also show that this conditional entropy power inequality is optimal in the sense that we can achieve equality asymptotically by choosing a suitable sequence of gaussian input states we apply the conditional entropy power inequality to find an array of information theoretic inequalities for conditional entropies which are the analogs of inequalities which have already been established in the unconditioned setting furthermore we give a simple proof of the convergence rate of the quantum ornstein uhlenbeck semigroup based on entropy power inequalities

geometric inequalities from phase space translations we establish a quantum version of the classical isoperimetric inequality relating the fisher information and the entropy power of a quantum state the key tool is a fisher information inequality for a state which results from a certain convolution operation the latter maps a classical probability distribution on phase space and a quantum state to a quantum state we show that this inequality also gives rise to several related inequalities whose counterparts are well known in the classical setting in particular it implies an entropy power inequality for the mentioned convolution operation as well as the isoperimetric inequality and establishes concavity of the entropy power along trajectories of the quantum heat diffusion semigroup as an application we derive a log sobolev inequality for the quantum ornstein uhlenbeck semigroup and argue that it implies fast convergence towards the fixed point for a large class of initial states

intrinsic entropies of log concave distributions the entropy of a random variable is well known to equal the exponential growth rate of the volumes of its typical sets in this paper we show that for any log concave random variable x the sequence of the lfloor n theta rfloor text th intrinsic volumes of the typical sets of x in dimensions n geq 1 grows exponentially with a well defined rate we denote this rate by h x theta and call it the theta text th intrinsic entropy of x we show that h x theta is a continuous function of theta over the range 0 1 thereby providing a smooth interpolation between the values 0 and h x at the endpoints 0 and 1 respectively

on divergences and informations in statistics and information theory the paper deals with the f divergences of csiszar generalizing the discrimination information of kullback the total variation distance the hellinger divergence and the pearson divergence all basic properties of f divergences including relations to the decision errors are proved in a new manner replacing the classical jensen inequality by a new generalized taylor expansion of convex functions some new properties are proved too e g relations to the statistical sufficiency and deficiency the generalized taylor expansion also shows very easily that all f divergences are average statistical informations differences between prior and posterior bayes errors mutually differing only in the weights imposed on various prior distributions the statistical information introduced by de groot and the classical information of shannon are shown to be extremal cases corresponding to alpha 0 and alpha 1 in the class of the so called arimoto alpha informations introduced in this paper for 0 alpha 1 by means of the arimoto alpha entropies some new examples of f divergences are introduced as well namely the shannon divergences and the arimoto alpha divergences leading for alphauarr1 to the shannon divergences square roots of all these divergences are shown to be metrics satisfying the triangle inequality the last section introduces statistical tests and estimators based on the minimal f divergence with the empirical distribution achieved in the families of hypothetic distributions for the kullback divergence this leads to the classical likelihood ratio test and estimator

quantum harmonic analysis on phase space relative to an irreducible representation of the canonical commutation relations convolutions between quantum mechanical operators and between functions and operators are defined for which the usual weyl transform acts as a fourier transform basic properties of these operations are developed in close analogy to harmonic analysis on r2n using the quantum version of wiener s approximation theorem a natural one to one correspondence between the closed phase space translation invariant subspaces of classical and quantum observables is established

entropy production of doubly stochastic quantum channels we study the entropy increase of quantum systems evolving under primitive doubly stochastic markovian noise and thus converging to the maximally mixed state this entropy increase can be quantified by a logarithmic sobolev constant of the liouvillian generating the noise we prove a universal lower bound on this constant that stays invariant under taking tensor powers our methods involve a new comparison method to relate logarithmic sobolev constants of different liouvillians and a technique to compute logarithmic sobolev inequalities of liouvillians with eigenvectors forming a projective representation of a finite abelian group our bounds improve upon similar results established before and as an application we prove an upper bound on continuous time quantum capacities in the last part of this work we study entropy production estimates of discrete time doubly stochastic quantum channels by extending the framework of discrete time logarithmic sobolev inequalities to the quantum case

quantum logarithmic sobolev inequalities and rapid mixing a family of logarithmic sobolev inequalities on finite dimensional quantum state spaces is introduced the framework of non commutative lp spaces is reviewed and the relationship between quantum logarithmic sobolev inequalities and the hypercontractivity of quantum semigroups is discussed this relationship is central for the derivation of lower bounds for the logarithmic sobolev ls constants essential results for the family of inequalities are proved and we show an upper bound to the generalized ls constant in terms of the spectral gap of the generator of the semigroup these inequalities provide a framework for the derivation of improved bounds on the convergence time of quantum dynamical semigroups when the ls constant and the spectral gap are of the same order convergence bounds on finite dimensional state spaces are particularly relevant for the field of quantum information theory we provide a number of examples where improved bounds on the mixing time of several semigroups are obtained incl

a vector generalization of costa s entropy power inequality with applications this paper considers an entropy power inequality epi of costa and presents a natural vector generalization with a real positive semidefinite matrix parameter the new inequality is proved using a perturbation approach via a fundamental relationship between the derivative of mutual information and the minimum mean square error mmse estimate in linear vector gaussian channels as an application a new extremal entropy inequality is derived from the generalized costa epi and then used to establish the secrecy capacity regions of the degraded vector gaussian broadcast channel with layered confidential messages

a new entropy power inequality a strengthened version of shannon s entropy power inequality for the case where one of the random vectors involved is gaussian is proved in particular it is shown that if independent gaussian noise is added to an arbitrary multivariate random variable the entropy power of the resulting random variable is a concave function of the variance power of the added noise the strengthened inequality is shown to hold for the class of stable distributions

reverse brunn minkowski and reverse entropy power inequalities for convex measures we develop a reverse entropy power inequality for convex measures which may be seen as an anegeometric inverse of the entropy power inequality of shannon and stam the specialization of this inequality to log concave measures may be seen as a version of milman s reverse brunn minkowski inequality the proof relies on a demonstration of new relationships between the entropy of high dimensional random vectors and the volume of convex bodies and on a study of eective supports of convex measures both of which are of independent interest as well as on milman s deep technology of m ellipsoids and on certain information theoretic inequalities as a by product we also give a

hypercontractivity for a quantum ornstein uhlenbeck semigroup we prove hypercontractivity for a quantum ornstein uhlenbeck semigroup on the entire algebra mathcal b h of bounded operators on a separable hilbert space h we exploit the particular structure of the spectrum together with hypercontractivity of the corresponding birth and death process and a proper decomposition of the domain then we deduce a logarithmic sobolev inequality for the semigroup and gain an elementary estimate of the best constant

gaussian optimizers for entropic inequalities in quantum information we survey the state of the art for the proof of the quantum gaussian optimizer conjectures of quantum information theory these fundamental conjectures state that quantum gaussian input states are the solution to several optimization problems involving quantum gaussian channels these problems are the quantum counterpart of three fundamental results of functional analysis and probability the entropy power inequality the sharp young s inequality for convolutions and the theorem gaussian kernels have only gaussian maximizers quantum gaussian channels play a key role in quantum communication theory they are the quantum counterpart of gaussian integral kernels and provide the mathematical model for the propagation of electromagnetic waves in the quantum regime the quantum gaussian optimizer conjectures are needed to determine the maximum communication rates over optical fibers and free space the restriction of the quantum limited gaussian attenuator to input states diagonal in the fock basis coincide

hypercontractivity and the logarithmic sobolev inequality for the completely bounded norm we develop the notions of hypercontractivity hc and the log sobolev ls inequality for completely bounded norms of one parameter semigroups of super operators acting on matrix algebras we prove the equivalence of the completely bounded versions of hc and ls under suitable hypotheses we also prove a version of the gross lemma which allows ls at general q to be deduced from ls at q 2

counterexample to the vector generalization of costa s entropy power inequality and partial resolution we give a counterexample to the vector generalization of costa s entropy power inequality due to liu et al in particular the claimed inequality can fail if the matrix valued parameter in the convex combination does not commute with the covariance of the additive gaussian noise conversely the inequality holds if these two matrices commute

a filled function method applied to nonsmooth constrained global optimization the filled function method is an effective approach to find a global minimizer in this paper based on a new definition of the filled function for nonsmooth constrained programming problems a one parameter filled function is constructed to improve the efficiency of numerical computation then a corresponding algorithm is presented it is a global optimization method which modify the objective function as a filled function and which find a better local minimizer gradually by optimizing the filled function constructed on the minimizer previously found illustrative examples are provided to demonstrate the efficiency and reliability of the proposed filled function method

the concavity of p entropy power and applications in functional inequalities abstract in this paper we prove the concavity of p entropy power of probability densities solving the p heat equation on closed riemannian manifold with nonnegative ricci curvature as applications we give new proofs of l p euclidean nash inequality and l p euclidean logarithmic sobolev inequality moreover an improvement of l p logarithmic sobolev inequality is derived

on the similarity of the entropy power inequality and the brunn minkowski inequality corresp the entropy power inequality states that the effective variance entropy power of the sum of two independent random variables is greater than the sum of their effective variances the brunn minkowski inequality states that the effective radius of the set sum of two sets is greater than the sum of their effective radii both these inequalities are recast in a form that enhances their similarity in spite of this similarity there is as yet no common proof of the inequalities nevertheless their intriguing similarity suggests that new results relating to entropies from known results in geometry and vice versa may be found two applications of this reasoning are presented first an isoperimetric inequality for entropy is proved that shows that the spherical normal distribution minimizes the trace of the fisher information matrix given an entropy constraint just as a sphere minimizes the surface area given a volume constraint second a theorem involving the effective radii of growing convex sets is proved

gradient of mutual information in linear vector gaussian channels this paper considers a general linear vector gaussian channel with arbitrary signaling and pursues two closely related goals i closed form expressions for the gradient of the mutual information with respect to arbitrary parameters of the system and ii fundamental connections between information theory and estimation theory generalizing the fundamental relationship recently unveiled by guo shamai and verdu spl acute we show that the gradient of the mutual information with respect to the channel matrix is equal to the product of the channel matrix and the error covariance matrix of the best estimate of the input given the output gradients and derivatives with respect to other parameters are then found via the differentiation chain rule

the concavity of renyi entropy power we associate to the pth renyi entropy a definition of entropy power which is the natural extension of shannon s entropy power and exhibits a nice behavior along solutions to the p nonlinear heat equation in rn we show that the renyi entropy power of general probability densities solving such equations is always a concave function of time whereas it has a linear behavior in correspondence to the barenblatt source type solutions this result extends costa s concavity inequality for shannon s entropy power to renyi entropies

a strong entropy power inequality when one of the random summands is gaussian we sharpen the entropy power inequality epi in terms of the strong data processing function for gaussian channels among other consequences this strong epi generalizes the vector extension of costa s epi to non gaussian channels in a precise sense this leads to a new reverse epi and as a corollary sharpens stam s uncertainty principle relating entropy power and fisher information or equivalently gross logarithmic sobolev inequality applications to network information theory are also given including a short self contained proof of the rate region for the two encoder quadratic gaussian source coding problem and a new outer bound for the one sided gaussian interference channel

information theoretic proofs of entropy power inequalities while most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information up to now shannon s entropy power inequality epi is an exception existing information theoretic proofs of the epi hinge on representations of differential entropy using either fisher information or minimum mean square error mmse which are derived from de bruijn s identity in this paper we first present an unified view of these proofs showing that they share two essential ingredients 1 a data processing argument applied to a covariance preserving linear transformation 2 an integration over a path of a continuous gaussian perturbation using these ingredients we develop a new and brief proof of the epi through a mutual information inequality which replaces stam and blachman s fisher information inequality fii and an inequality for mmse by guo shamai and verdu used in earlier proofs the result has the advantage of being very simple in that it relies only on the basic properties of mutual information these ideas are then generalized to various extended versions of the epi zamir and feder s generalized epi for linear transformations of the random variables takano and johnson s epi for dependent variables liu and viswanath s covariance constrained epi and costa s concavity inequality for the entropy power

interval methods for global optimization this paper contains a brief survey of the fundamental ideas that underlie interval methods for global optimization some recent work is also surveyed and possible future developments are suggested

convexity and regularity properties for entropic interpolations abstract in this paper we prove a convexity property of the relative entropy along entropic interpolations solutions of the schrodinger problem and a regularity property of the entropic cost along the heat flow then we derive a dimensional evi inequality and a contraction property for the entropic cost along the heat flow as a consequence we recover the equivalent results in the wasserstein space proved by erbar kuwada and sturm

the kullback leibler divergence rate between markov sources in this work we provide a computable expression for the kullback leibler divergence rate lim sub n spl rarr spl infin 1 nd p sup n spl par q sup n between two time invariant finite alphabet markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions p sup n and q sup n respectively we illustrate it numerically and examine its rate of convergence the main tools used to obtain the kullback leibler divergence rate and its rate of convergence are the theory of nonnegative matrices and perron frobenius theory similarly we provide a formula for the shannon entropy rate lim sub n spl rarr spl infin 1 nh p sup n of markov sources and examine its rate of convergence

mutual information and minimum mean square error in gaussian channels this paper deals with arbitrarily distributed finite power input signals observed through an additive gaussian noise channel it shows a new formula that connects the input output mutual information and the minimum mean square error mmse achievable by optimal estimation of the input given the output that is the derivative of the mutual information nats with respect to the signal to noise ratio snr is equal to half the mmse regardless of the input statistics this relationship holds for both scalar and vector signals as well as for discrete time and continuous time noncausal mmse estimation this fundamental information theoretic result has an unexpected consequence in continuous time nonlinear estimation for any input signal with finite power the causal filtering mmse achieved at snr is equal to the average value of the noncausal smoothing mmse achieved with a channel whose snr is chosen uniformly distributed between 0 and snr

monotonicity thinning and discrete versions of the entropy power inequality we consider the entropy of sums of independent discrete random variables in analogy with shannon s entropy power inequality where equality holds for normals in our case infinite divisibility suggests that equality should hold for poisson variables we show that some natural analogues of the epi do not in fact hold but propose an alternative formulation which does always hold the key to many proofs of shannon s epi is the behavior of entropy on scaling of continuous random variables we believe that renyi s operation of thinning discrete random variables plays a similar role to scaling and give a sharp bound on how the entropy of ultra log concave random variables behaves on thinning in the spirit of the monotonicity results established by artstein ball barthe and naor we prove a stronger version of concavity of entropy which implies a strengthened form of our discrete epi

capacity and rate regions of a class of broadcast interference channels in this paper a class of broadcast interference channels bic is investigated where one of the two broadcast receivers is subject to interference coming from a point to point transmission for a general discrete memoryless broadcast interference channel dm bic an achievable scheme based on message splitting superposition and binning is proposed and a concise representation of the corresponding achievable rate region mathcal r is obtained two partial order broadcast conditions interference oblivious less noisy and interference cognizant less noisy are defined thereby extending the usual less noisy condition for a regular broadcast channel by taking interference into account under these conditions a reduced form of mathcal r is shown to be equivalent to a rate region based on a simpler scheme where the broadcast transmitter uses only superposition furthermore if interference is strong for the interference oblivious less noisy dm bic the capacity region is given by the aforementioned two equivalent rate regions for the interference cognizant less noisy dm bic it is argued that the strong but not very strong interference condition does not exist and in this case the capacity region for the very strong interference is obtained for a gaussian broadcast interference channel channel parameters are categorized into three regimes for the first two regimes which are closely related to the two partial order broadcast conditions achievable rate regions are derived by specializing the corresponding achievable schemes of dm bics with gaussian input distributions the entropy power inequality based outer bounds are obtained by combining bounding techniques for a gaussian broadcast channel and a gaussian interference channel these inner and outer bounds lead to either exact or approximate characterizations of capacity regions and sum capacity under various conditions for the remaining complementing regime inner and outer bounds are also provided

some applications of hypercontractive inequalities in quantum information theory hypercontractive inequalities have become important tools in theoretical computer science and have recently found applications in quantum computation in this note we discuss how hypercontractive inequalities in various settings can be used to obtain fairly concise proofs of several results in quantum information theory a recent lower bound of lancien and winter on the bias achievable by local measurements which are 4 designs spectral concentration bounds for k local hamiltonians and a recent result of pellegrino and seoane sepulveda giving general lower bounds on the classical bias obtainable in multiplayer xor games

normal form decomposition for gaussian to gaussian superoperators in this paper we explore the set of linear maps sending the set of quantum gaussian states into itself these maps are in general not positive a feature which can be exploited as a test to check whether a given quantum state belongs to the convex hull of gaussian states if one of the considered maps sends it into a non positive operator the above state is certified not to belong to the set generalizing a result known to be valid under the assumption of complete positivity we provide a characterization of these gaussian to gaussian not necessarily positive superoperators in terms of their action on the characteristic function of the inputs for the special case of one mode mappings we also show that any gaussian to gaussian superoperator can be expressed as a concatenation of a phase space dilatation followed by the action of a completely positive gaussian channel possibly composed with a transposition while a similar decomposition is shown to fail in the multi mode scenario we prove that it still holds at least under the further hypothesis of homogeneous action on the covariance matrix

non commutative nash inequalities a set of functional inequalities called nash inequalities are introduced and analyzed in the context of quantum markov process mixing the basic theory of nash inequalities is extended to the setting of non commutative l p spaces where their relationship to poincare and log sobolev inequalities is fleshed out we prove nash inequalities for a number of unital reversible semigroups

the conditional entropy power inequality for gaussian quantum states we propose a generalization of the quantum entropy power inequality involving conditional entropies for the special case of gaussian states we give a proof based on perturbation theory for symplectic spectra we discuss some implications for entanglement assisted classical communication over additive bosonic noise channels

simple proof of the concavity of the entropy power with respect to added gaussian noise a very simple proof of m h costa s result see ibid vol it 31 p 751 60 1985 that the entropy power of x sub t x n o ti is concave in t is derived as an immediate consequence of an inequality concerning fisher information this relationship between fisher information and entropy is found to be useful for proving the central limit theorem thus one who seeks new entropy inequalities should try first to find new equalities about fisher information or at least to exploit the existing ones in new ways

contractivity properties of a quantum diffusion semigroup we consider a quantum generalization of the classical heat equation and study contractivity properties of its associated semigroup we prove a nash inequality and a logarithmic sobolev inequality the former leads to an ultracontractivity result this in turn implies that the largest eigenvalue and the purity of a state with positive wigner function evolving under the action of the semigroup decrease at least inverse polynomially in time while its entropy increases at least logarithmically in time

a simple proof of the entropy power inequality this correspondence gives a simple proof of shannon s entropy power inequality epi using the relationship between mutual information and minimum mean square error mmse in gaussian channels

on the entropy power inequality for the renyi entropy of order 0 1 using a sharp version of the reverse young inequality and a renyi entropy comparison result due to fradelizi madiman and wang 2016 the authors derive renyi entropy power inequalities for log concave random vectors when renyi parameters belong to 0 1 furthermore the estimates are shown to be sharp up to absolute constants

interval mathematics algebraic equations and optimization this is a brief survey of some of the applications of interval mathematics to the solution of systems of linear and nonlinear algebraic equations and to the solution of unconstrained and constrained nonlinear optimization problems

capacity bounds via duality with applications to multiple antenna systems on flat fading channels a technique is proposed for the derivation of upper bounds on channel capacity it is based on a dual expression for channel capacity where the maximization of mutual information over distributions on the channel input alphabet is replaced with a minimization of average relative entropy over distributions on the channel output alphabet we also propose a technique for the analysis of the asymptotic capacity of cost constrained channels the technique is based on the observation that under fairly mild conditions capacity achieving input distributions escape to infinity the above techniques are applied to multiple antenna flat fading channels with memory where the realization of the fading process is unknown at the transmitter and unknown or only partially known at the receiver it is demonstrated that for high signal to noise ratio snr the capacity of such channels typically grows only double logarithmically in the snr to better understand this phenomenon and the rates at which it occurs we introduce the fading number as the second order term in the high snr asymptotic expansion of capacity and derive estimates on its value for various systems it is suggested that at rates that are significantly higher than the fading number communication becomes extremely power inefficient thus posing a practical limit on practically achievable rates upper and lower bounds on the fading number are also presented for single input single output siso systems the bounds coincide thus yielding a complete characterization of the fading number for general stationary and ergodic fading processes we also demonstrate that for memoryless multiple input single output miso channels the fading number is achievable using beam forming and we derive an expression for the optimal beam direction this direction depends on the fading law and is in general not the direction that maximizes the snr on the induced siso channel using a new closed form expression for the expectation of the logarithm of a noncentral chi square distributed random variable we provide some closed form expressions for the fading number of some systems with gaussian fading including siso systems with circularly symmetric stationary and ergodic gaussian fading the fading number of the latter is determined by the fading mean fading variance and the mean squared error in predicting the present fading from its past it is not directly related to the doppler spread for the rayleigh ricean and multiple antenna rayleigh fading channels we also present firm nonasymptotic upper and lower bounds on channel capacity these bounds are asymptotically tight in the sense that their difference from capacity approaches zero at high snr and their ratio to capacity approaches one at low snr

a concavity property for the reciprocal of fisher information and its consequences on costa s epi abstract we prove that the reciprocal of fisher information of a log concave probability density x in r n is concave in t with respect to the addition of a gaussian noise z t n 0 t i n as a byproduct of this result we show that the third derivative of the entropy power of a log concave probability density x in r n is nonnegative in t with respect to the addition of a gaussian noise z t for log concave densities this improves the well known costa s concavity property of the entropy power costa 1985

on the equivalence between stein and de bruijn identities this paper focuses on illustrating 1 the equivalence between stein s identity and de bruijn s identity and 2 two extensions of de bruijn s identity first it is shown that stein s identity is equivalent to de bruijn s identity under additive noise channels with specific conditions second for arbitrary but fixed input and noise distributions under additive noise channels the first derivative of the differential entropy is expressed by a function of the posterior mean and the second derivative of the differential entropy is expressed in terms of a function of fisher information several applications over a number of fields such as signal processing and information theory are presented to support the usefulness of the developed results in this paper

optimal kullback leibler aggregation via spectral theory of markov chains this paper is concerned with model reduction for complex markov chain models the kullback leibler divergence rate is employed as a metric to measure the difference between the markov model and its approximation for a certain relaxation of the bi partition model reduction problem the solution is shown to be characterized by an associated eigenvalue problem the form of the eigenvalue problem is closely related to the markov spectral theory for model reduction this result is the basis of a heuristic proposed for the m ary partition problem resulting in a practical recursive algorithm the results are illustrated with examples

applications of interval arithmetic in non smooth global optimization in this paper an expansion scheme for general functions is presented which can be used to achieve better enclosure of function ranges in the context of interval branch and bound methods for non smooth global optimization a pruning technique is given via the expansion an interval pruning test is established this pruning test offers the possibility to cut away a large part of the currently investigated box by the optimization algorithm which can be utilized as an accelerating device similar to the monotonicity test frequently used in interval methods for smooth problems numerical computation shows that the proposed method is reliable and can find all solutions of global optimization problem

remote source coding under gaussian noise dueling roles of power and entropy power the distributed remote source coding the so called ceo problem is studied in the case where the underlying source not necessarily gaussian has finite differential entropy and the observation noise is gaussian the main result is a new lower bound for the sum rate distortion function under arbitrary distortion measures when specialized to the case of mean squared error it is shown that the bound exactly mirrors a corresponding upper bound except that the upper bound has the source power variance whereas the lower bound has the source entropy power bounds exhibiting this pleasing duality of power and entropy power have been well known for direct and centralized source coding since shannon s work while the bounds hold generally their value is most pronounced when interpreted as a function of the number of agents in the ceo problem

nonlinear diffusions extremal properties of barenblatt profiles best matching and delays abstract in this paper we consider functionals based on moments and nonlinear entropies which have a linear growth in time in case of source type solutions to the fast diffusion or porous medium equations that are also known as barenblatt solutions as functions of time these functionals have convexity properties for generic solutions so that their asymptotic slopes are extremal for barenblatt profiles the method relies on scaling properties of the evolution equations and provides a simple and direct proof of sharp gagliardo nirenberg sobolev inequalities in scale invariant form the method also gives refined estimates of the growth of the second moment and as a consequence establishes the monotonicity of the delay corresponding to the best matching barenblatt solution compared to the barenblatt solution with same initial second moment here the notion of best matching is defined in terms of a relative entropy

information measures inequalities and performance bounds for parameter estimation in impulsive noise environments recent studies found that many channels are affected by additive noise that is impulsive in nature and is best explained by heavy tailed symmetric alpha stable distributions dealing with impulsive noise environments comes with an added complexity with respect to the standard gaussian environment the alpha stable probability density functions do not possess closed form expressions except in few special cases furthermore they have an infinite second moment and the nice hilbert space structure of the space of random variables having a finite second moment is lost along with its tools and methodologies this is indeed the case in estimation theory where classical tools to quantify the performance of an estimator are tightly related to the assumption of having finite variance variables in alpha stable environments expressions such as the mean square error and the cramer rao bound are hence problematic in this paper we tackle the parameter estimation problem in the impulsive noise environments and develop novel tools that are tailored to the alpha stable and heavy tailed noise environments tools that coincide with the standard ones adopted in the gaussian setup namely a generalized power measure and a generalized fisher information we generalize known information inequalities commonly used in the gaussian context the de bruijn identity the fisher information inequality the isoperimetric inequality for entropies and the cramer rao bound additionally we derive upper bounds on the differential entropy of independent sums having a stable component intermediately the new power measure is used to shed some light on the additive alpha stable noise channel capacity in a setup that generalizes the linear average power constrained additive white gaussian noise channel our theoretical findings are paralleled with numerical evaluations of various quantities and bounds using developed matlab packages

wasserstein continuity of entropy and outer bounds for interference channels it is shown that under suitable regularity conditions differential entropy is o sqrt n lipschitz as a function of probability distributions on mathbb r n with respect to the quadratic wasserstein distance under similar conditions discrete shannon entropy is shown to be o n lipschitz in distributions over the product space with respect to ornstein s bar d distance wasserstein distance corresponding to the hamming distance these results together with talagrand s and marton s transportation information inequalities allow one to replace the unknown multi user interference with its independent identically distributed approximations as an application a new outer bound for the two user gaussian interference channel is proved which in particular settles the missing corner point problem of costa 1985

on renyi entropy power inequalities this paper gives improved renyi entropy power inequalities r epis consider a sum s n sum k 1 n x k of n independent continuous random vectors taking values on mathbb r d and let alpha in 1 infty an r epi provides a lower bound on the order alpha renyi entropy power of s n that up to a multiplicative constant which may depend in general on n alpha d is equal to the sum of the order alpha renyi entropy powers of the n random vectors x k k 1 n for alpha 1 the r epi coincides with the well known entropy power inequality by shannon the first improved r epi is obtained by tightening the recent r epi by bobkov and chistyakov which relies on the sharpened young s inequality a further improvement of the r epi also relies on convex optimization and results on rank one modification of a real valued diagonal matrix