lower bounds on the quantum capacity and highest error exponent of general memoryless channels tradeoffs between the information rate and fidelity of quantum error correcting codes are discussed quantum channels to be considered are those subject to independent errors and modeled as tensor products of copies of a general completely positive cp linear map where the dimension of the underlying hilbert space is a prime number on such a quantum channel the highest fidelity of a quantum error correcting code of length n and rate r is proven to be lower bounded by 1 exp ne r o n for some function e r the e r is positive below some threshold r sub 0 a direct consequence of which is that r sub 0 is a lower bound on the quantum capacity this is an extension of the author s earlier result while the earlier work states the result for the depolarizing channel and a slight generalization of it pauli channels the result of this work applies to general discrete memoryless channels including channel models derived from a physical law of time evolution

rate region of the quadratic gaussian two encoder source coding problem we determine the rate region of the quadratic gaussian two encoder source coding problem this rate region is achieved by a simple architecture that separates the analog and digital aspects of the compression furthermore this architecture requires higher rates to send a gaussian source than it does to send any other source with the same covariance our techniques can also be used to determine the sum rate of some generalizations of this classical problem our approach involves coupling the problem to a quadratic gaussian ldquoceo problem rdquo

capacity and decoding rules for classes of arbitrarily varying channels the capacity of an arbitrarily varying channel avc is considered for deterministic codes with the average probability of error criterion and typically subject to at state constraint first sufficient conditions are provided that enable relatively simple decoding rules such as typicality maximum mutual information and minimum distance to attain capacity then the possibly noisy or channels and group adder channels are studied in detail for the former the capacity is explicitly determined and shown to be attainable by minimum distance decoding next for a large class of addictive avcs in addition to providing an intuitively suggestive simplification of the general avc capacity formula it is proven that capacity can be attained by a universal decoding rule finally the effect of random state selections on capacity is studied the merits and limitations of a previous mutual information game approach are also discussed

coding for positive rate in the source model key agreement problem a two party key agreement problem with public discussion known as the source model problem is considered by relating key agreement to hypothesis testing a new coding scheme is developed that yields a sufficient condition to achieve a positive secret key sk rate in terms of renyi divergence the merits of this coding scheme are illustrated by applying it to an erasure model for eve s side information and by deriving an upper bound on eve s erasure probabilities for which the sk capacity is zero this bound strictly improves on the best known single letter lower bound on the sk capacity moreover the bound is tight when alice s or bob s source is binary which extends a previous result for a doubly symmetric binary source the results motivate a new measure for the correlation between two random variables which is of independent interest

graph decomposition a new key to coding theorems a new and simple method is proposed for finding good encoders both for channels and for sources with side information this method relies on the continuous version of a graph decomposition result of lovasz the presently known best exponential error bounds for both problems follow in a unified manner with an improvement on the source coding bound the previous bounds for universal codes of the authors and marton are also improved

statistical inference under multiterminal data compression this paper presents a survey of the literature on the information theoretic problems of statistical inference under multiterminal data compression with rate constraints significant emphasis is put on problems 1 multiterminal hypothesis testing 2 multiterminal parameter estimation and 3 multiterminal pattern classification in either case of positive rates or zero rates in addition the paper includes three new results i e the converse theorems for all problems of multiterminal hypothesis testing multiterminal parameter estimation and multiterminal pattern classification at the zero rate

the rate distortion function for the quadratic gaussian ceo problem a new multiterminal source coding problem called the ceo problem was presented and investigated by berger zhang and viswanathan recently viswanathan and berger have investigated an extension of the ceo problem to gaussian sources and call it the quadratic gaussian ceo problem they considered this problem from a statistical viewpoint deriving some interesting results in this paper we consider the quadratic gaussian ceo problem from a standpoint of multiterminal rate distortion theory we regard the ceo problem as a certain multiterminal remote source coding problem with infinitely many separate encoders whose observations are conditionally independent if the remote source is given this viewpoint leads us to a complete solution to the problem we determine the tradeoff between the total amount of rate and squared distortion deriving an explicit formula of the rate distortion function the derived function has the form of a sum of two nonnegative functions one is a classical rate distortion function for single gaussian source and the other is a new rate distortion function which dominates the performance of the system for a relatively small distortion it follows immediately from our result that the conjecture of viswanathan and berger on the asymptotic behavior of minimum squared distortion for large rates is true

modules possessing projective resolutions of finite type letgbe a group in the classlhf of locally hierarchically decomposable groups and letrbe a stronglyg graded algebra we provide a characterization of ther modules of type fp under the assumptions thatr1is coherent and of finite global dimension

reliability in source coding with side information we study error exponents for source coding with side information both achievable exponents and converse bounds are obtained for the following two cases lossless source coding with coded information and lossy source coding with full side information wyner ziv these results recover and extend several existing results on source coding error exponents and are tight in some circumstances our bounds have a natural interpretation as a two player game between nature and the code designer with nature seeking to minimize the exponent and the code designer seeking to maximize it in the wyner ziv problem our analysis exposes a tension in the choice of test channel with the optimal test channel balancing two competing error events the gaussian and binary erasure cases are examined in detail

mismatched decoding revisited general alphabets channels with memory and the wide band limit the mismatch capacity of a channel is the highest rate at which reliable communication is possible over the channel with a given possibly suboptimal decoding rule this quantity has been studied extensively for single letter decoding rules over discrete memoryless channels dmcs here we extend the study to memoryless channels with general alphabets and to channels with memory with possibly non single letter decoding rules we also study the wide band limit and in particular the mismatch capacity per unit cost and the achievable rates on an additive noise spread spectrum system with single letter decoding and binary signaling

the error exponent for the noiseless encoding of finite ergodic markov sources a new approach to the classical fixed length noiseless source coding problem is proposed for the case of finite ergodic markov sources this approach is based on simple counting arguments the central notion of markov type a set containing all the source sequences having the same transition counts from letter to letter is introduced and the cardinality of such a set is evaluated via graph theoretical tools the error exponent is shown to be a weighted average of informational divergences and the universal character of the result is stressed as a corollary the classical source coding theorem determining the achievable rates is rederived

linear codes for sources and source networks error exponents universal coding for slepian wolf source networks the error exponents obtained by korner marton and the author are shown to be universally attainable by linear codes also improved exponents are derived for linear codes with large rates specializing the results to simple discrete memoryless sources reveals their relationship to the random coding and expurgated bounds for channels with additive noise one corollary is that there are universal linear codes for this class of channels which attain the random coding error exponent for each channel in the class the combinatorial approach of csiszar korner marton is used in particular all results are derived from a lemma specifying good encoders in terms of purely combinatorial properties

information embedding and the triple role of control we consider the problem of information embedding where the encoder modifies a white gaussian host signal in a power constrained manner to encode a message and the decoder recovers both the embedded message and the modified host signal this partially extends the recent work of sumszyk and steinberg to the continuous alphabet gaussian setting through a control theoretic lens we observe that the problem is a minimalist example of what is called the triple role of control actions we show that a dirty paper coding strategy achieves the optimal rate for perfect recovery of the modified host and the message for any message rate for imperfect recovery of the modified host by deriving bounds on the minimum mean square error mmse in recovering the modified host signal we show that dirty paper coding based strategies are guaranteed to attain within a uniform constant factor of 16 of the optimal weighted sum of power required in host signal modification and the mmse in the modified host signal reconstruction for all weights and all message rates when specialized to the zero rate case our results provide the tightest known lower bounds on the asymptotic costs for the vector version of a famous open problem in decentralized control the witsenhausen counterexample numerically this tighter bound helps us characterize the asymptotically optimal costs for the vector witsenhausen problem to within a factor of 1 3 for all problem parameters improving on the earlier best known bound of 2

inference under information constraints i lower bounds from chi square contraction multiple players are each given one independent sample about which they can only provide limited information to a central referee each player is allowed to describe its observed sample to the referee using a channel from a family of channels mathcal w which can be instantiated to capture among others both the communication and privacy constrained settings the referee uses the players messages to solve an inference problem on the unknown distribution that generated the samples we derive lower bounds for the sample complexity of learning and testing discrete distributions in this information constrained setting underlying our bounds is a characterization of the contraction in chi square distance between the observed distributions of the samples when information constraints are placed this contraction is captured in a local neighborhood in terms of chi square and decoupled chi square fluctuations of a given channel two quantities we introduce the former captures the average distance between distributions of channel output for two product distributions on the input and the latter for a product distribution and a mixture of product distribution on the input our bounds are tight for both public and private coin protocols interestingly the sample complexity of testing is order wise higher when restricted to private coin protocols

the suboptimality of randomized tests in distributed and quantized detection systems the design of decentralized or quantized detection systems requires simultaneous optimization of quantizer mappings and ultimate fusion rule it is shown that if the likelihood ratios of the unquantized or raw observations are independent and contain no point masses of probability the optimal test does not randomize this is so despite the fact that the data to be fused can be considered discrete the result is a considerable simplification in the design search

coordination capacity we develop elements of a theory of cooperation and coordination in networks rather than considering a communication network as a means of distributing information or of reconstructing random processes at remote nodes we ask what dependence can be established among the nodes given the communication constraints specifically in a network with communication rates ri j between the nodes we ask what is the set of all achievable joint distributions p x1 xm of actions at the nodes of the network several networks are solved including arbitrarily large cascade networks distributed cooperation can be the solution to many problems such as distributed games distributed control and establishing mutual information bounds on the influence of one part of a physical system on another

distributed rate distortion with common components we describe a scheme for rate distortion with distributed encoding in which the sources to be compressed contain a common component we show that this scheme is optimal in some situations and that it strictly improves upon existing schemes which do not make full use of common components this establishes that independent quantization followed by independent binning is not optimal for the two encoder problem with a distortion constraint on one source we also show that independent quantization and binning is suboptimal for the three encoder problem in which the goal is to reproduce one of the sources losslessly this provides a counterexample that is fundamentally different from one provided earlier by korner and marton the proofs rely on the binary analogue of the entropy power inequality and the existence of a rate loss for the binary symmetric wyner ziv problem

parameter estimation with multiterminal data compression the multiterminal estimation theory deals with an entirely novel problem which takes place in the void between information theory and statistics that is what amount of fisher information can be attained under a restriction on the amount of shannon information the key idea here is the indivisible fusion of the information theoretic universal coding problem and the statistical maximum likelihood parameter estimation problem the main result is the explicit establishment of maximum likelihood estimators attainable under the rate constrained universal coding scheme which is shown to have a variance equal to the inverse of the fisher information this may be regarded as giving a multiterminal generalization of the usual cramer rao bound relevant properties and examples of these maximum likelihood estimators are also shown

error exponents for distributed detection of markov sources the paper considers a binary hypothesis testing system in which two sensors simultaneously observe a discrete time finite valued stationary ergodic markov source and transmit m ary messages to a neyman pearson central detector the size m of the message alphabet increases at most subexponentially with the number of observations the asymptotic behavior of the type ii error rate is investigated as the number of observations increases to infinity and the associated error exponent is obtained under mild assumptions on the source distributions this exponent is independent of the test level spl epsiv and the actual codebook sizes m is achieved by a universally optimal sequence of acceptance regions and is characterized by an infimum of informational divergence rate over a class of infinite dimensional distributions important differences due to the observations being markov between the asymptotically optimal distributed tests and their nondistributed counterparts are highlighted the converse results require a blowing up lemma for stationary ergodic markov sources which is also proven

two way source coding with a helper consider the two way rate distortion problem in which a helper sends a common limited rate message to both users based on side information at its disposal we characterize the region of achievable rates and distortions when the markov relation helper user 1 user 2 holds the main insight of the result is that in order to achieve the optimal rate the helper may use a binning scheme as in wyner ziv where the side information at the decoder is the further user namely user 2 we derive these regions explicitly for the gaussian sources with square error distortion analyze a tradeoff between the rate from the helper and the rate from the source and examine a special case where the helper has the freedom to send different messages at different rates to the encoder and the decoder the converse proofs use a technique for verifying markov relations via undirected graphs

on certain homological invariants of groups recently the notions of sfli the supremum of the flat lengths of injective modules and silf the supremum of the injective lengths of flat modules have been studied by some authors these homological invariants are based on spli and silp invariants of gedrich and gruenberg and it is shown that they have enough potential to play an important role in studying homological conjectures in cohomology of groups in this paper we will study these invariants it turns out that for any group the finiteness of silf implies the finiteness of sfli but the converse is not known we investigate the situation in which sfli implies silf the statement holds for example for groups with the property that flat modules have finite projective dimension moreover we show that the gorenstein flat dimension of the trivial z module z that will be called gorenstein homological dimension of denoted ghd is completely related to these invariants

an improved outer bound for multiterminal source coding we prove a new outer bound on the rate distortion region for the multiterminal source coding problem this bound subsumes the best outer bound in the literature and improves upon it strictly in some cases the improved bound enables us to obtain a new conclusive result for the binary erasure version of the ceo problem the bound recovers many of the converse results that have been established for special cases of the problem including the recent one for the gaussian two encoder problem

some properties of sequential predictors for binary markov sources universal predictions of the next outcome of a binary sequence drawn from a markov source with unknown parameters is considered for a given source the predictability is defined as the least attainable expected fraction of prediction errors a lower bound is derived on the maximum rate at which the predictability is asymptotically approached uniformly over all sources in the markov class this bound is achieved by a simple majority predictor for bernoulli sources bounds on the large deviations performance are investigated a lower bound is derived for the probability that the fraction of errors will exceed the predictability by a prescribed amount delta 0 this bound is achieved by the same predictor if delta is sufficiently small

the wyner ziv problem with multiple sources this paper provides bounds on the rate distortion region for the distributed compression scenario where two or more sources are compressed separately for a decoder that has access to side information conclusive rate distortion results are found for the case where the sources are conditionally independent given the side information

asymptotic optimality of likelihood ratio threshold tests in decentralized detection two distributed systems are considered for discriminating between two finite alphabet bivariate memoryless sources and for detecting a known signal in stationary bivariate additive gaussian noise each system comprises two sensors m ary local quantizers and a fusion center which makes decisions based on quantized source observations the problem of asymptotically optimal quantization is considered in detail for the binary m 2 case it is shown that optimality is achieved by quantizing a locally computed likelihood ratio wherein one distribution is in general different from the appropriate source marginal for the problem of detection in gaussian noise it is further demonstrated that the optimal distributed system attains the same asymptotic performance as the optimal centralized system for appropriate choice of m

fluctuation theorem for stochastic systems the fluctuation theorem describes the probability ratio of observing trajectories that satisfy or violate the second law of thermodynamics it has been proved in a number of different ways for thermostatted deterministic nonequilibrium systems in the present paper we show that the fluctuation theorem is also valid for a class of stochastic nonequilibrium systems the theorem is therefore not reliant on the reversibility or the determinism of the underlying dynamics numerical tests verify the theoretical result

a converse coding theorem for mismatched decoding at the output of binary input memoryless channels an upper bound on the maximal transmission rate over binary input memoryless channels provided that the decoding decision rule is given is derived if the decision rule is equivalent to the maximum likelihood decoding matched decoding then the bound coincides with the channel capacity otherwise mismatched decoding it coincides with a known lower bound

on the duality between slepian wolf coding and channel coding under mismatched decoding in this paper slepian wolf coding with a mismatched decoding metric is studied two different dualities between slepian wolf coding and channel coding under mismatched decoding are established these two dualities provide a systematic framework for comparing linear slepian wolf codes nonlinear slepian wolf codes and variable rate slepian wolf codes in contrast with the fact that linear codes suffice to achieve the slepian wolf limit under matched decoding the minimum rate achievable with nonlinear slepian wolf codes under mismatched decoding can be strictly lower than that achievable with linear slepian wolf codes

rate distortion for correlated sources with partially separated encoders we resolve some open problems concerning the encoding of a pair of correlated sources with respect to a fidelity criterion two encoders are used one encoder observes only the output of the first source while the other is supplied both with the second source output and with partial information about the first source at some predetermined rate a general coding theorem is proved which established that cal s ast a certain region defined in terms of single letter information theoretic quantities is an inner bound to the region of all attainable vectors of rates and distortions for certain cases of interest the converse is proved too thereby establishing the rate distortion region for these cases

error exponents of typical random codes for the colored gaussian channel the error exponent of the typical random code is defined as the asymptotic normalized expectation of the logarithm of the probability of error as opposed to the traditional definition of the random coding exponent as the normalized logarithm of the expectation of the probability of error with respect to a given ensemble of codes for a certain ensemble of independent codewords with a given power spectrum and a generalized stochastic mismatched decoder we characterize the error exponent of the typical random codes trc for the colored gaussian channel with emphasis on the range of low rates where the trc error exponent differs in value from the ordinary random coding error exponent the error exponent formula which is exponentially tight at some range of low rates is presented as the maximum of a certain function with respect to one parameter only in the spirit of gallager s formulas in the case of matched decoding and two parameters in the case of mismatched decoding several aspects of the main results are discussed these include general properties a parametric representation critical rates phase transitions optimal input spectrum water pouring and comparison to the random coding exponent

splitting of the rate matrix as a definition of time reversal in master equation systems motivated by recent progress in nonequilibrium fluctuation relations we present a generalized time reversal for stochastic master equation systems with discrete states which is defined as a splitting of the rate matrix into irreversible and reversible parts an immediate advantage of this definition is that a variety of fluctuation relations can be attributed to different matrix splittings additionally we find that the accustomed total entropy production formula and conditions of the detailed balance must be modified appropriately to account for the reversible rate part which was previously ignored

on certain cohomological invariants of groups abstract let r be a left and right 0 noetherian ring we show that if all projective left and all projective right r modules have finite injective dimension then all injective left and all injective right r modules have finite projective dimension using this result we prove that the invariants silp z g and spli z g which were introduced by gedrich and gruenberg 1987 15 are equal for any group g as an application of the latter equality we show that a group g is finite if and only if cd g 0 where cd is the generalized cohomological dimension of groups introduced by ikenaga 1984 21

a counter example to the mismatched decoding converse for binary input discrete memoryless channels this paper studies the mismatched decoding problem for binary input discrete memoryless channels an example is provided for which an achievable rate based on superposition coding exceeds the csiszar korner hui rate thus providing a counter example to a previously reported converse result both numerical evaluations and theoretical results are used in establishing this claim

the homotopy category of flat modules and grothendieck duality let r be a ring we prove that the homotopy category k r proj is always aleph 1 compactly generated and depending on the ring r it may or may not be compactly generated we use this to give a description of k r proj as a quotient of k r flat the remarkable fact is that this new description of k r proj generalizes to non affine schemes this will appear in murfet s thesis

a homological characterization of locally finite groups abstract we show that a group g is locally finite if and only if tor 1 z g m i 0 for all z g modules m and i with m z free and i injective to that end we relate the finiteness of a finitely generated group g to the finiteness of the injective dimension of induced modules we also examine the relation between the finiteness of a group g and the finiteness of the projective dimension of coinduced modules

error exponents of typical random codes we define the error exponent of the typical random code trc as the long block limit of the negative normalized expectation of the logarithm of the error probability of the random code as opposed to the traditional random coding error exponent which is the limit of the negative normalized logarithm of the expectation of the error probability for the ensemble of uniformly randomly drawn fixed composition codes we provide exact error exponents of trcs for a general discrete memoryless channel and a wide class of stochastic decoders collectively referred to as the generalized likelihood decoder gld this ensemble of fixed composition codes is shown to be no worse than any other ensemble of independent codewords that are drawn under a permutation invariant distribution e g independent identically distributed codewords we also present relationships between the error exponent of the trc and the ordinary random coding error exponent as well as the expurgated exponent for the gld finally we demonstrate that our analysis technique is applicable also to more general communication scenarios such as list decoding for fixed size lists as well as the decoding with an erasure list option in forney s sense

rate region of the vector gaussian one helper source coding problem we determine the rate region of the vector gaussian one helper source coding problem under a covariance matrix distortion constraint the rate region is achieved by a simple scheme that separates the lossy vector quantization from the lossless spatial compression the converse is established by extending and combining three analysis techniques that have been employed in the past to obtain partial results for the problem

second order and moderate deviations asymptotics for successive refinement we derive the optimal second order coding region and moderate deviations constant for successive refinement source coding with a joint excess distortion probability constraint we consider two scenarios 1 a discrete memoryless source dms and arbitrary distortion measures at the decoders and 2 a gaussian memoryless source gms and quadratic distortion measures at the decoders for a dms with arbitrary distortion measures we prove an achievable second order coding region using type covering lemmas by kanlis and narayan and by no ingber and weissman we prove the converse using the perturbation approach by gu and effros when the dms is successively refinable the expressions for the second order coding region and the moderate deviations constant are simplified and easily computable for this case we also obtain new insights on the second order behavior compared with the scenario where separate excess distortion proabilities are considered for example we describe a dms for which the optimal second order region transitions from being characterizable by a bivariate gaussian to a univariate gaussian as the distortion levels are varied we then consider a gms with quadratic distortion measures to prove the direct part we make use of the sphere covering theorem by verger gaugry together with appropriately defined gaussian type classes to prove the converse we generalize kostina and verdu s one shot converse bound for point to point lossy source coding we remark that this proof is applicable to general successively refinable sources in the proofs of the moderate deviations results for both scenarios we follow a strategy similar to that for the second order asymptotics and use the moderate deviations principle

subblock constrained codes for real time simultaneous energy and information transfer consider an energy harvesting receiver that uses the same received signal both for decoding information and for harvesting energy which is employed to power its circuitry in the scenario where the receiver has limited battery size a signal with bursty energy content may cause power outage at the receiver since the battery will drain during intervals with low signal energy in this paper we analyze subblock energy constrained codes seccs which ensure that sufficient energy is carried within every subblock duration we consider discrete memoryless channels and characterize the secc capacity and the secc error exponent and provide useful bounds for these values we also study constant subblock composition codes csccs which are a subclass of seccs where all the subblocks in every codeword have the same fixed composition and this subblock composition is chosen to maximize the rate of information transfer while meeting the energy requirement compared with constant composition codes cccs we show that csccs incur a rate loss and that the error exponent for csccs is also related to the error exponent for cccs by the same rate loss term we exploit the symmetry in csccs to obtain a necessary and sufficient condition on the subblock length for avoiding power outage at the receiver furthermore for cscc sequences we present a tight lower bound on the average energy per symbol within a sliding time window we provide numerical examples highlighting the tradeoff between the delivery of sufficient energy to the receiver and achieving high information transfer rates it is observed that the ability to use energy in real time imposes less of penalty compared with the ability to use information in real time

a survey of the theory of source coding with a fidelity criterion the purpose of this paper is threefold 1 to acquaint the reader with the types of problems that have been considered in the area of source coding with a fidelity criterion 2 to survey results that have been obtained on these problems and 3 to outline future research trends in the area

statistical physics of random binning we consider the model of random binning and finite temperature decoding for slepian wolf codes from a statistical mechanical perspective while ordinary random channel coding is intimately related to the random energy model a statistical mechanical model of disordered magnetic materials it turns out that random binning for slepian wolf coding is analogous to another related statistical mechanical model of strong disorder which we call the random dilution model we use the latter analogy to characterize phase transitions pertaining to finite temperature slepian wolf decoding which are somewhat similar but not identical to those of finite temperature channel decoding we then provide the exact random coding exponent of the bit error rate as a function of the coding rate and the decoding temperature and discuss its properties finally a few modifications and extensions of our results are outlined and discussed

the method of types information theory the method of types is one of the key technical tools in shannon theory and this tool is valuable also in other fields in this paper some key applications are presented in sufficient detail enabling an interested nonspecialist to gain a working knowledge of the method and a wide selection of further applications are surveyed these range from hypothesis testing and large deviations theory through error exponents for discrete memoryless channels and capacity of arbitrarily varying channels to multiuser problems while the method of types is suitable primarily for discrete memoryless models its extensions to certain models with memory are also discussed

integer metrics for binary input symmetric output memoryless channels consider a coded communication system where antipodal signaling is used over symmetric memoryless channels a mismatch decoder employing metrics which assume values in a finite set of integers is used two performance measures for achievable rates are considered the generalized cutoff rate and the mismatch capacity two different integer metric assignments which maximize these criteria are found the optimal metric assignment holds for any binary input symmetric output memoryless channels it is calculated explicitly for the faded additive white gaussian noise channel with and without diversity an interference channel model and the additive cauchy channel the difference in terms of achievable rates between decoding using the optimal continuous metric and integer metric is calculated for these channels the degradation for fairly low number of quantization levels is found to be small e g for nine quantization levels degradation is less than 0 3 db for all analyzed channels the sensitivity of the generalized cutoff rate and the mismatch capacity to the exact location of the quantization levels is rather mild the difference between the mismatch capacity corresponding to the optimal integer quantization and the mismatch capacity obtained by the quantization which optimizes the generalized cutoff rate is found to be quite small

remote vector gaussian source coding with decoder side information under mutual information and distortion constraints let x y z be zero mean jointly gaussian random vectors of dimensions nx ny and nz respectively let p be the set of random variables w such that w harr y harr x z is a markov string we consider the following optimization problem wisinp min i y z subject to one of the following two possible constraints 1 i x w z ges ri and 2 the mean squared error between x and xcirc e x w z is less than d the problem under the first kind of constraint is motivated by multiple input multiple output mimo relay channels with an oblivious transmitter and a relay connected to the receiver through a dedicated link while for the second case it is motivated by source coding with decoder side information where the sensor observation is noisy in both cases we show that jointly gaussian solutions are optimal moreover explicit water filling interpretations are given for both cases which suggest transform coding approaches performed in different transform domains and that the optimal solution for one problem is in general suboptimal for the other

the gaussian many help one distributed source coding problem jointly gaussian memoryless sources are observed at n distinct terminals the goal is to efficiently encode the observations in a distributed fashion so as to enable reconstruction of any one of the observations say the first one at the decoder subject to a quadratic fidelity criterion our main result is a precise characterization of the rate distortion region when the covariance matrix of the sources satisfies a tree structure condition in this situation a natural analog digital separation scheme optimally trades off the distributed quantization rate tuples and the distortion in the reconstruction each encoder consists of a point to point gaussian vector quantizer followed by a slepian wolf binning encoder we also provide a partial converse that suggests that the tree structure condition is fundamental

on the optimality of binning for distributed hypothesis testing we study a hypothesis testing problem in which data are compressed distributively and sent to a detector that seeks to decide between two possible distributions for the data the aim is to characterize all achievable encoding rates and exponents of the type 2 error probability when the type 1 error probability is at most a fixed value for related problems in distributed source coding schemes based on random binning perform well and are often optimal for distributed hypothesis testing however the use of binning is hindered by the fact that the overall error probability may be dominated by errors in the binning process we show that despite this complication binning is optimal for a class of problems in which the goal is to test against conditional independence we then use this optimality result to give an outer bound for a more general class of instances of the problem

bounds on the decoding error probability of binary linear codes via their spectra bounds on the error probability of maximum likelihood decoding of a binary linear code are considered the bounds derived use the weight spectrum of the code and they are tighter than the conventional union bound in the case of large noise in the channel the bounds derived are applied to a code with an average spectrum and the result is compared to the random coding exponent the author shows that the bound considered for the binary symmetrical channel case coincides asymptotically with the random coding bound for the case of awgn channel the author shows that berlekamp s 1980 tangential bound can be improved but even this improved bound does not coincide with the random coding bound although it can be very close to it

on the reliability function of the discrete memoryless relay channel bounds on the reliability function for the discrete memoryless relay channel are derived using the method of types two achievable error exponents are derived based on partial decode forward and compress forward which are well known superposition block markov coding schemes the derivations require combinations of the techniques involved in the proofs of csiszar korner marton s packing lemma for the error exponent of channel coding and marton s type covering lemma for the error exponent of source coding with a fidelity criterion the decode forward error exponent is evaluated on sato s relay channel from this example it is noted that to obtain the fastest possible decay in the error probability for a fixed effective coding rate one ought to optimize the number of blocks in the block markov coding scheme assuming the blocklength within each block is large an upper bound on the reliability function is also derived using ideas from haroutunian s lower bound on the error probability for point to point channel coding with feedback

multiterminal detection with zero rate data compression the asymptotic performance of a multiterminal detection system comprising a central detector and two remote sensors that have access to discrete spatially dependent and temporally memoryless observations is discussed it is assumed that prior to transmitting information to the central detector each sensor compresses its observations at a rate which approaches zero as the sample size tends to infinity and that on the basis of the compressed data from both sensors the central detector seeks to determine whether the true distribution of the observations belongs to a null class ii or an alternative class xi under the criterion that stipulates minimization of the type ii error rate subject to an upper bound epsilon on the type i error rate it is shown that in the case of simple distribution classes mod pi mod mod xi mod 1 the error exponent achievable by such a system has a simple characterization is independent of the value of epsilon and is insensitive to changes in compression rate as long as the asymptotic rate on one of the sensors is zero

a dichotomy of functions f x y of correlated sources x y consider separate encoding of correlated sources x n x l cdots x n y n y l cdots y n for the decoder to reliably reproduce a function f x i y i n i 1 we establish the necessary and sufficient condition for the set of all achievable rates to coincide with the slepian wolf region whenever the probability density p x y is positive for all x y

improved source coding exponents via witsenhausen s rate we provide a novel upper bound on witsenhausen s rate the rate required in the zero error analogue of the slepian wolf problem our bound is given in terms of a new information theoretic functional defined on a certain graph and is derived by upper bounding complementary graph entropy we use the functional along with graph entropy to give a single letter lower bound on the error exponent for the slepian wolf problem under the vanishing error probability criterion where the decoder has full i e unencoded side information we demonstrate that our error exponent can beat the expurgated source coding exponent of csiszar and korner for some sources that have zeroes in the channel matrix connecting the source with the side information an extension of our scheme to the lossy case i e wyner ziv is given for the case in which the side information is a deterministic function of the source the exponent of our improved scheme agrees with the sphere packing bound exactly thus determining the reliability function an application of our functional to zero error channel capacity is also given

multiterminal source coding achievable rates and reliability some open problems concerning classical multiterminal configuration in which two correlated sources x and y are encoded separately and decoded by common decoder with respect to a fidelity criterion for messages of both sources are resolved the results of berger and yeung 1989 are expanded in two directions first we determine the rate distortion region r spl delta of achievable rates r sub x r sub y for a given spl delta spl delta sub x spl delta sub y with spl delta sub x the permissible distortion level for reproduction of x and spl delta sub y for y thus we give a full solution of the problem noted by kaspi and berger 1982 they constructed the inner bound for r spl delta and solved by berger and yeung for the case of one distortion second we introduce the notion of rate reliability distortion region for the multiterminal source encoding problem more precisely the region of admissible rates of codes ensuring exponential decrease with a given exponent e 0 of the probability of exceeding the distortion levels spl delta sub x or spl delta sub y the inner and outer bounds for this region r e spl delta are constructed the analytical representation obtained for r spl delta proved to be somewhat simpler than that considered by kaspi and berger and by berger and yeung ours uses only variables pertaining to the problem statement with no auxiliary ones

vector gaussian multiple description with two levels of receivers the problem of l multiple descriptions of a stationary and ergodic gaussian source with two levels of receivers is investigated each of the first level receivers receive an arbitrary subset k of the l descriptions k l the second level receiver receives all l descriptions all the receivers both at the first level and the second level reconstruct the source using the subset of descriptions they receive the corresponding reconstructions are subject to quadratic distortion constraints our main result is the derivation of an outer bound on the sum rate of the descriptions so that the distortion constraints are met we show that an analog digital separation architecture involving joint gaussian vector quantizers and a binning scheme meets this outer bound with equality for several scenarios these scenarios include the case when the distortion constraints are symmetric and the case for general distortion constraints with k 2 and l 3

multiterminal source encoding with one distortion criterion the authors unify earlier investigations concerning the encoding of two correlated sources x sub k y sub k by means of separate encoders decoding is done by a single decoder which receives the outputs from both encoders the reconstruction of x sub k is required to be perfect in the usual shannon sense the authors determine the admissible rate region r d where d is the distortion of the reconstruction of y sub k the binary hamming case is investigated explicitly

rate distortion theory for gaussian multiterminal source coding systems with several side informations at the decoder in this paper we consider the separate coding problem for l 1 correlated gaussian memoryless sources we deal with the case where l sources work as partial side information at the decoder for the reconstruction of the remaining source the determination problem of the rate distortion region for this system is the so called many help one problem and it has been known as a highly challenging problem for almost 20 years in this paper we give a partial solution to this problem we determine the rate distortion region in the case where the l sources working as partial side information are conditionally independent if the remaining source we wish to reconstruct is given the additive white gaussian noise ceo problem is a special case of this we also discuss the relation of the result previous results of ours

optimum tradeoffs between the error exponent and the excess rate exponent of variable rate slepian wolf coding we analyze the optimal tradeoff between the error exponent and the excess rate exponent for variable rate slepian wolf codes in particular we first derive upper converse bounds on the optimal error and excess rate exponents and then lower achievable bounds via a simple class of variable rate codes which assign the same rate to all source blocks of the same type class then using the exponent bounds we derive bounds on the optimal rate functions namely the minimal rate assigned to each type class needed in order to achieve a given target error exponent the resulting excess rate exponent is then evaluated iterative algorithms are provided for the computation of both bounds on the optimal rate functions and their excess rate exponents the resulting slepian wolf codes bridge between the two extremes of fixed rate coding which has minimal error exponent and maximal excess rate exponent and average rate coding which has maximal error exponent and minimal excess rate exponent

an integral homological characterization of finite groups abstract we show that a group g is finite if and only if every injective z g module has projective length one

quantization for decentralized hypothesis testing under communication constraints in a decentralized hypothesis testing network several peripheral nodes observe an environment and communicate their observations to a central node for the final decision the presence of capacity constraints introduces theoretical and practical problems the following problem is addressed given that the peripheral encoders that satisfy these constraints are scalar quantizers how should they be designed in order that the central test to be performed on their output indices is most powerful the scheme is called cooperative design separate encoding since the quantizers process separate observations but have a common goal they seek to maximize a system wide performance measure the bhattacharyya distance of the joint index space as such a criterion is suggested and a design algorithm to optimize arbitrarily many quantizers cyclically is proposed a simplified version of the algorithm namely an independent design separate encoding scheme where the correlation is either absent or neglected for the sake of simplicity is outlined performances are compared through worked examples

capacity and random coding exponents for channel coding with side information capacity formulas and random coding exponents are derived for a generalized family of gel fand pinsker coding problems these exponents yield asymptotic upper bounds on the achievable log probability of error in our model information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence and the channel is selected by a hypothetical adversary partial information about the state sequence is available to the encoder adversary and decoder the design of the transmitter is subject to a cost constraint two families of channels are considered 1 compound discrete memoryless channels cdmc and 2 channels with arbitrary memory subject to an additive cost constraint or more generally to a hard constraint on the conditional type of the channel output given the input both problems are closely connected the random coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder which may be thought of as an empirical generalized maximum a posteriori decoder for channels with arbitrary memory the random coding exponents are larger than their cdmc counterparts applications of this study include watermarking data hiding communication in presence of partially known interferers and problems such as broadcast channels all of which involve the fundamental idea of binning

the ceo problem multiterminal source coding we consider a new problem in multiterminal source coding motivated by the following decentralized communication estimation task a firm s chief executive officer ceo is interested in the data sequence x t sub t 1 sup spl infin which cannot be observed directly perhaps because it represents tactical decisions by a competing firm the ceo deploys a team of l agents who observe independently corrupted versions of x t sub t 1 sup spl infin because x t is only one among many pressing matters to which the ceo must attend the combined data rate at which the agents may communicate information about their observations to the ceo is limited to say r bits per second if the agents were permitted to confer and pool their data then in the limit as l spl rarr spl infin they usually would be able to smooth out their independent observation noises entirely then they could use their r bits per second to provide the ceo with a representation of x t with fidelity d r where d spl middot is the distortion rate function of x t in particular with such data pooling d can be made arbitrarily small if r exceeds the entropy rate h of x t suppose however that the agents are not permitted to convene agent i having to send data based solely on his own noisy observations y sub i t we show that then there does not exist a finite value of r for which even infinitely many agents can make d arbitrarily small furthermore in this isolated agents case we determine the asymptotic behavior of the minimal error frequency in the limit as l and then r tend to infinity

hypothesis testing with multiterminal data compression the multiterminal hypothesis testing h xy against h xy is considered where x n x n and y n y n are separately encoded at rates r 1 and r 2 respectively the problem is to determine the minimum beta n of the second kind of error probability under the condition that the first kind of error probability alpha n leq epsilon for a prescribed 0 a good lower bound theta l r 1 r 2 on the power exponent theta r 1 r 2 epsilon lim inf n rightarrow infty 1 n log beta n is given and several interesting properties are revealed the lower bound is tighter than that of ahlswede and csiszar furthermore in the special case of testing against independence this bound turns out to coincide with that given by them the main arguments are devoted to the special case with r 2 infty corresponding to full side information for y n y n in particular the compact solution is established to the complete data compression cases which are useful in statistics from the practical point of view

neyman pearson test for zero rate multiterminal hypothesis testing the problem of zero rate multiterminal hypothesis testing is revisited from the perspective of information spectrum approach and finite blocklength analysis a neyman pearson like test is proposed and its non asymptotic performance is clarified for a short block length it is numerically determined that the proposed test is superior to the previously reported hoeffding like test proposed by han kobayashi for a large deviation regime it is shown that our proposed test achieves an optimal trade off between the type i and type ii exponents presented by han kobayashi among the class of symmetric type based testing schemes when the type i error probability is non vanishing the proposed test is optimal up to the second order term of the type ii error exponent the latter term is characterized in terms of the variance of the projected relative entropy density the information geometry method plays an important role in the analysis as well as the construction of the test

duality between source coding and channel coding and its extension to the side information case we explore the information theoretic duality between source coding with side information at the decoder and channel coding with side information at the encoder we begin with a mathematical characterization of the functional duality between classical source and channel coding formulating the precise conditions under which the optimal encoder for one problem is functionally identical to the optimal decoder for the other problem we then extend this functional duality to the case of coding with side information by invoking this duality we are able to generalize the result of wyner and ziv 1976 relating to no rate loss for source coding with side information from gaussian to more arbitrary distributions we consider several examples corresponding to both discrete and continuous valued cases to illustrate our formulation for the gaussian cases of coding with side information we invoke geometric arguments to provide further insights into their duality our geometric treatment inspires the construction and dual use of practical coset codes for a large class of emerging applications for coding with side information such as distributed sensor networks watermarking and information hiding communication systems

nonasymptotic and second order achievability bounds for coding with side information we present a novel nonasymptotic or finite blocklength achievability bounds for three side information problems in network information theory these include 1 the wyner ahlswede korner wak problem of almost lossless source coding with rate limited side information 2 the wyner ziv wz problem of lossy source coding with side information at the decoder and 3 the gel fand pinsker gp problem of channel coding with noncausal state information available at the encoder the bounds are proved using ideas from channel simulation and channel resolvability our bounds for all three problems improve on all previous nonasymptotic bounds on the error probability of the wak wz and gp problems in particular those derived by verdu using our novel nonasymptotic bounds we recover the general formulas for the optimal rates of these side information problems finally we also present achievable second order coding rates by applying the multidimensional berry esseen theorem to our new nonasymptotic bounds numerical results show that the second order coding rates obtained using our nonasymptotic achievability bounds are superior to those obtained using existing finite blocklength bounds

an achievable rate region for joint compression and dispersive information routing for networks this paper considers the problem of minimum cost communication of correlated sources over a network with multi ple sinks which consists of distributed source coding followed by routing we introduce a new routing paradigm called dispersive information routing dir wherein the intermediate nodes are allowed to split a packet and forward subsets of the received bits on each of the forward paths this paradigm opens up a rich class of research problems which focus on the interplay between encoding and routing in a network unlike conventional routing methods such as in 1 dir ensures that each sink receives just the information needed to reconstruct the sources it is required to reproduce we demonstrate using simple examples that the proposed approach offers better asymptotic performance than conventional routing techniques we show that under certain assumptions on the cost function the problem of finding the minimum cost under dir essentially reduces to characterizing an achievable rate region for a new multiterminal information theoretic setup while it is possible to derive an achievable region for this setup using prior results from general multiterminal source coding 3 these techniques do not exploit the underlying problem structure and thereby lead to suboptimal regions in this paper we propose a new coding scheme using principles from multiple descriptions encoding 2 and show that it strictly improves upon a corresponding variant of coding scheme in 3 we further show that the new coding scheme achieves the complete rate region for certain special cases of the general setup and thereby achieves the minimum communication cost under this routing paradigm

guessing subject to distortion we investigate the problem of guessing a random vector x within distortion level d our aim is to characterize the best attainable performance in the sense of minimizing in some probabilistic sense the number of required guesses g x until the error falls below d the underlying motivation is that g x is the number of candidate codewords to be examined by a rate distortion block encoder until a satisfactory codeword is found in particular for memoryless sources we provide a single letter characterization of the least achievable exponential growth rate of the spl rho th moment of g x as the dimension of the random vector x grows without bound in this context we propose an asymptotically optimal guessing scheme that is universal both with respect to the information source and the value of spl rho we then study some properties of the exponent function e d spl rho along with its relation to the source coding exponents finally we provide extensions of our main results to the gaussian case guessing with side information and sources with memory

universal coding for the slepian wolf data compression system and the strong converse theorem universal coding for the slepian wolf 1973 data compression system is considered we shall demonstrate based on a simple observation that the error exponent given by csiszar and korner 1980 for the universal coding system can strictly be sharpened in general for a region of relatively higher rates this kind of observation can be carried over also to the case of lower rates outside the slepian wolf region which establishes the strong converse along with the optimal exponent

error exponent for multiple access channels upper bounds the problem of bounding the reliability function of a multiple access channel mac is studied two new upper bounds on the error exponent of a two user discrete memoryless dm mac are derived the first bound sphere packing is an upper bound on the exponent of the average probability of error and is the first bound of this type that is zero outside the capacity region and thus results in a tighter sphere packing exponent when compared with the tightest known exponent derived by haroutunian the second bound minimum distance is an upper bound on the exponent of the maximal as opposed to average probability of error to obtain this bound first an upper bound on the minimum bhattacharyya distance between codeword pairs is derived for a certain class of two user dm macs an upper bound on the exponent of maximal probability of error is derived as a consequence of the upper bound on the minimum bhattacharyya distance we analytically evaluate the sphere packing bound for uniform composition codes for an additive and nonsymmetric channel and show that it is tight near the boundary of the capacity region i e equal to the random coding lower bound

on error exponents of modulo lattice additive noise channels modulo lattice additive noise mlan channels appear in the analysis of structured binning codes for costa s dirty paper channel and of nested lattice codes for the additive white gaussian noise awgn channel in this paper we derive a new lower bound on the error exponents of the mlan channel with a proper choice of the shaping lattice and the scaling parameter the new lower bound coincides with the random coding lower bound on the error exponents of the awgn channel at the same signal to noise ratio snr in the sphere packing and straight line regions this result implies that at least for rates close to channel capacity 1 writing on dirty paper is as reliable as writing on clean paper and 2 lattice encoding and decoding suffer no loss of error exponents relative to the optimal codes with maximum likelihood decoding for the awgn channel

integral fluctuation theorem for the housekeeping heat the housekeeping heat qhk is the dissipated heat necessary to maintain the violation of detailed balance in nonequilibrium steady states by analysing the evolution of its probability distribution we prove an integral fluctuation theorem exp qhk 1 valid for arbitrary driven transitions between steady states we discuss gaussian limiting cases and the difference between the new theorem and both the hatano sasa and the jarzynski relation

a note on the asymptotics of distributed detection with feedback the effect of feedback on the performance of a distributed neyman pearson detection system consisting of n sensors two stage local quantizers and a global detector is investigated two feedback schemes are discussed only one of which yields an asymptotic gain in performance as measured by an appropriate error exponent

large deviations behavior of the logarithmic error probability of random codes this work studies the deviations of the error exponent of the constant composition code ensemble around its expectation known as the error exponent of the typical random code trc in particular it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the trc exponent is exponentially small upper and lower bounds for this exponent are given which coincide in some cases in addition the probability of randomly drawing a codebook whose error exponent is larger than the trc exponent is shown to be double exponentially small upper and lower bounds to the double exponential exponent are given the results suggest that codebooks whose error exponent is larger than the error exponent of the trc are extremely rare the key ingredient in the proofs is a new large deviations result of type class enumerators with dependent variables

universal wyner ziv coding for distortion constrained general side information we investigate the wyner ziv coding in which the statistics of the principal source is known but the statistics of the channel generating the side information is unknown except that it is in a certain class the class consists of channels such that the distortion between the principal source and side information is smaller than a threshold but channels may be neither stationary nor ergodic in this situation we define a new rate distortion function as the minimum rate such that there exists a wyner ziv code that is universal for every channel in the class then we show an upper bound and a lower bound on the rate distortion function and derive a matching condition such that the upper and lower bounds coincide the relation between the new rate distortion function and rate distortion function of the heegard berger problem is also discussed

a block decodable 1 8 runlength limited rate 8 12 code describes a d k 1 8 runlength limited rll rate 8 12 code with fixed codeword length 12 the code is block decodable a codeword can be decoded without knowledge of preceding or succeeding codewords the code belongs to the class of bounded delay block decodable bdb codes with one symbol 8 bits look ahead due to its format this code is particularly attractive for use in combination with error correcting codes such as reed solomon codes over the finite field gf 2 sup 8

physical origins of entropy production free energy dissipation and their mathematical representations a unifying mathematical theory of nonequilibrium thermodynamics of stochastic systems in terms of master equations is presented as generalizations of isothermal entropy and free energy two functions of state play central roles the gibbs entropy s and the relative entropy f which are related via the stationary distribution of the stochastic dynamics s satisfies the fundamental entropy balance equation ds dt e p ensuremath h d t with entropy production rate e p ensuremath ge 0 and heat dissipation rate h d while df dt ensuremath f d ensuremath le 0 for closed systems that satisfy detailed balance t e p t f d t for open systems one has t e p t f d t q hk t where the housekeeping heat q hk ensuremath ge 0 was first introduced in the phenomenological nonequilibrium steady state thermodynamics put forward by oono and paniconi q hk represents the irreversible work done by the surrounding to the system that is kept away from reaching equilibrium hence entropy production e p consists of free energy dissipation associated with spontaneous relaxation i e self organization f d and active energy pumping that sustains the open system q hk the amount of excess heat involved in the relaxation q ex h d ensuremath q hk f d ensuremath t ds dt two kinds of irreversibility and the meaning of the arrow of time emerge quasistationary processes adiabaticity and maximum principle for entropy are also generalized to nonequilibrium settings

on achievable rates and error exponents for channels with mismatched decoding the problem of characterizing achievable rates and error exponents for discrete memoryless channels with mismatched decoding is addressed a mismatched cognitive multiple access channel is introduced and an inner bound on its capacity region is derived using two alternative encoding methods 1 superposition coding and 2 random binning the inner bounds are derived by analyzing the average error probability of the code ensemble for both methods and by a tight characterization of the resulting error exponents random coding converse theorems are also derived a comparison of the achievable regions shows that in the matched case random binning performs as well as superposition coding i e the region achievable by random binning is equal to the capacity region the achievability results are further specialized to obtain a lower bound on the mismatch capacity of the single user channel by investigating a cognitive multiple access channel whose achievable sum rate serves as a lower bound on the single user channel s capacity while the achievable rate presented here may not improve the rate achieved by lapidoth s scheme in optimizing over the parameters of the random coding scheme it can improve the achieved rate for given parameters and thereby may reduce the computational complexity required to find a good code

on optimal data compression in multiterminal statistical inference the multiterminal theory of statistical inference deals with the problem of estimating or testing the correlation of letters generated from two or many correlated information sources under the restriction of a certain transmission rate for each source a typical example is two binary sources with joint probability p x y where the correlation of x and y is to be tested or estimated given n iid observations xn x1 xn and yn y1 yn only k rn 0 r 1 bits each can be transmitted to a common destination what is the optimal data compression for statistical inference a simple idea is to send the first k letters of xn and yn a simpler problem is the helper case where the optimal data compression of xn is searched for under the condition that all of yn are transmitted it is a long standing problem to determine if there is a better data compression scheme than this simple scheme of sending first k letters the present paper searches for the optimal data compression under the framework of linear threshold encoding and shows that there is a better data compression scheme depending on the value of correlation to this end we evaluate the fisher information in the class of linear threshold compression schemes it is also proved that the simple scheme is optimal when x and y are independent or their correlation is not too large