subblock constrained codes for real time simultaneous energy and information transfer consider an energy harvesting receiver that uses the same received signal both for decoding information and for harvesting energy which is employed to power its circuitry in the scenario where the receiver has limited battery size a signal with bursty energy content may cause power outage at the receiver since the battery will drain during intervals with low signal energy in this paper we analyze subblock energy constrained codes seccs which ensure that sufficient energy is carried within every subblock duration we consider discrete memoryless channels and characterize the secc capacity and the secc error exponent and provide useful bounds for these values we also study constant subblock composition codes csccs which are a subclass of seccs where all the subblocks in every codeword have the same fixed composition and this subblock composition is chosen to maximize the rate of information transfer while meeting the energy requirement compared with constant composition codes cccs we show that csccs incur a rate loss and that the error exponent for csccs is also related to the error exponent for cccs by the same rate loss term we exploit the symmetry in csccs to obtain a necessary and sufficient condition on the subblock length for avoiding power outage at the receiver furthermore for cscc sequences we present a tight lower bound on the average energy per symbol within a sliding time window we provide numerical examples highlighting the tradeoff between the delivery of sufficient energy to the receiver and achieving high information transfer rates it is observed that the ability to use energy in real time imposes less of penalty compared with the ability to use information in real time

large deviations hypotheses testing and source coding for finite markov chains let x n n geq 1 be a finite markov chain with transition probability matrix of strictly positive entries a large deviation theorem is proved for the empirical transition count matrix and is used to get asymptotically optimal critical regions for testing simple hypotheses about the transition matrix as a corollary the error exponent in the source coding theorem for x n is obtained these results generalize the corresponding results for the independent and identically distributed case

the reliability functions of the general source with fixed length coding the reliability function problems with fixed length source coding for the general source are studied for all rates r our fundamental philosophy in doing so is to convert all of the reliability function problems to the pertinent computation problems in the large derivation probability theory it turns out that this kind of new methodology which was previously developed by han see ibid vol 43 p 1145 64 1997 enables us to establish quite compact general formulas of the reliability function for general sources including all nonstationary and or nonergodic sources with countably infinite alphabet such general formulas are presented from the information spectrum point of view

lower bounds on the quantum capacity and highest error exponent of general memoryless channels tradeoffs between the information rate and fidelity of quantum error correcting codes are discussed quantum channels to be considered are those subject to independent errors and modeled as tensor products of copies of a general completely positive cp linear map where the dimension of the underlying hilbert space is a prime number on such a quantum channel the highest fidelity of a quantum error correcting code of length n and rate r is proven to be lower bounded by 1 exp ne r o n for some function e r the e r is positive below some threshold r sub 0 a direct consequence of which is that r sub 0 is a lower bound on the quantum capacity this is an extension of the author s earlier result while the earlier work states the result for the depolarizing channel and a slight generalization of it pauli channels the result of this work applies to general discrete memoryless channels including channel models derived from a physical law of time evolution

polynomially bounded error estimates for trapezoidal rule convolution quadrature abstract we clarify the dependence with respect to the time variable of some estimates about the convergence of the trapezoidal rule based convolution quadrature method applied to hyperbolic problems this requires a careful investigation of the article of lehel banjai where the first convergence estimates were introduced and of some technical results from a classical paper of christian lubich

a distributional version of kirchhoff s formula kirchhoff s formula is the equivalent to green s third theorem for transient waves representing the solution of the three dimensional wave equation in terms of its boundary data in this classical result two retarded layer potentials appear we show in this paper a precise description of these potentials as time convolution with adequate tempered distributions with values on operator spaces with these potentials in hands we give a self contained proof of the formula with minimal smoothness requirements

a survey of the theory of source coding with a fidelity criterion the purpose of this paper is threefold 1 to acquaint the reader with the types of problems that have been considered in the area of source coding with a fidelity criterion 2 to survey results that have been obtained on these problems and 3 to outline future research trends in the area

error exponents of typical random trellis codes in continuation to an earlier work where error exponents of typical random codes were studied in the context of general block coding with no underlying structure here we carry out a parallel study on typical random time varying trellis codes focusing on a certain range of low rates by analyzing an upper bound to the error probability of the typical random trellis code using the method of types we first derive a csiszar style error exponent formula with respect to the constraint length which allows to characterize properties of good codes and dominant error events we also derive a gallager style form which turns out to be related to the expurgated error exponent the main result is further extended to channels with memory and mismatch

polar codes for broadcast channels polar codes are introduced for discrete memoryless broadcast channels for m user deterministic broadcast channels polarization is applied to map uniformly random message bits from m independent messages to one codeword while satisfying broadcast constraints the polarization based codes achieve rates on the boundary of the private message capacity region for two user noisy broadcast channels polar implementations are presented for two information theoretic schemes 1 cover s superposition codes and 2 marton s codes due to the structure of polarization constraints on the auxiliary and channel input distributions are identified to ensure proper alignment of polarization indices in the multiuser setting the codes achieve rates on the capacity boundary of a few classes of broadcast channels e g binary input stochastically degraded the complexity of encoding and decoding is o n log n where n is the block length in addition polar code sequences obtain a stretched exponential decay of o 2 n beta of the average block error probability where 0 reproducible experiments for finite block lengths n 512 1024 2048 corroborate the theory

nonbinary quantum codes we present several results on quantum codes over general alphabets that is in which the fundamental units may have more than two states in particular we consider codes derived from finite symplectic geometry assumed to have additional global symmetries from this standpoint the analogs of calderbank shor steane codes and of gf 4 linear codes turn out to be special cases of the same construction this allows us to construct families of quantum codes from certain codes over number fields in particular we get analogs of quadratic residue codes including a single error correcting code encoding one letter in five for any alphabet size we also consider the problem of fault tolerant computation through such codes generalizing ideas of gottesman see phys rev a vol 57 no 1 p127 37 1998

disjointness of random sequence sets with respect to distinct probability measures it is shown that the set of deterministic random sequences of symbols from a finite alphabet with respect to a computable probability measure spl mu in martin lof s 1966 sense and the set of deterministic random sequences with respect to another computable probability measure spl nu are disjoint if spl mu and spl nu are different and the measures are either i i d or homogeneous finite order irreducible markov measures

new upper bounds on error exponents we derive new upper bounds on the error exponents for the maximum likelihood decoding and error detecting in the binary symmetric channels this is an improvement on the best earlier known bounds by shannon gallager berlekamp 1967 and mceliece omura 1977 for the probability of undetected error the new bounds are better than the bounds by levenshtein 1978 1989 and the bound by abdel ghaffar see ibid vol 43 p 1489 502 1997 moreover we further extend the range of rates where the undetected error exponent is known to be exact the new bounds are based on an analysis of possible distance distributions of the codes along with some inequalities relating the distance distributions to the error probabilities

detecting random walks on graphs with heterogeneous sensors we consider the problem of detecting a random walk on a graph based on observations of the graph nodes when visited by the walk each node of the graph observes a signal of elevated mean which we assume can be different across different nodes outside of the path of the walk and also in its absence nodes measure only noise assuming the neyman pearson setting our goal then is to characterize detection performance by computing the error exponent for the probability of a miss under a constraint on the probability of false alarm since the exact computation of the error exponent is known to be difficult equivalent to the computation of the lyapunov exponent we approximate its value by finding a tractable lower bound the bound reveals an interesting detectability condition the walk is detectable whenever the entropy of the walk is smaller than one half of the expected signal to noise ratio we derive the bound by extending the notion of markov types to gauss markov types these are sequences of the state observation pairs with a given number of node to node transition counts and the same average signal values across nodes computed from the measurements made during the times the random walk visited each node s respective location the lower bound has an intuitive interpretation among all gauss markov types that are asymptotically feasible in the absence of the walk the bound finds the most typical one under the presence of the walk finally we show by a sequence of judicious problem reformulations that computing the bound reduces to solving a convex optimization problem which is a result of in its interest own right

theoretical aspects of the application of convolution quadrature to scattering of acoustic waves in this paper we address several theoretical questions related to the numerical approximation of the scattering of acoustic waves in two or three dimensions by penetrable non homogeneous obstacles using convolution quadrature cq techniques for the time variable and coupled boundary element method finite element method for the space variable the applicability of cq to waves requires polynomial type bounds for operators related to the operator s 2 in the right half complex plane we propose a new systematic way of dealing with this problem both at the continuous and semidiscrete in space cases we apply the technique to three different situations scattering by a group of sound soft and hard obstacles by homogeneous and non homogeneous obstacles

the method of types information theory the method of types is one of the key technical tools in shannon theory and this tool is valuable also in other fields in this paper some key applications are presented in sufficient detail enabling an interested nonspecialist to gain a working knowledge of the method and a wide selection of further applications are surveyed these range from hypothesis testing and large deviations theory through error exponents for discrete memoryless channels and capacity of arbitrarily varying channels to multiuser problems while the method of types is suitable primarily for discrete memoryless models its extensions to certain models with memory are also discussed

asymptotically optimal classification for multiple tests with empirically observed statistics the decision problem of testing m hypotheses when the source is kth order markov and there are m or fewer training sequences of length n and a single test sequence of length n is considered k m n n are all given it is shown what the requirements are on m n n to achieve vanishing exponential error probabilities and how to determine or bound the exponent a likelihood ratio test that is allowed to produce a no match decision is shown to provide asymptotically optimal error probabilities and minimum no match decisions as an important serial case the binary hypotheses problem without rejection is discussed it is shown that for this configuration only one training sequence is needed to achieve an asymptotically optimal test

outer bounds on the admissible source region for broadcast channels with correlated sources two outer bounds on the admissible source region for broadcast channels with correlated sources are presented the first one is strictly tighter than the existing outer bound by gohari and anantharam while the second one provides a complete characterization of the admissible source region in the case where the two sources are conditionally independent given the common part these outer bounds are deduced from the general necessary conditions established for the lossy source broadcast problem via suitable comparisons between the virtual broadcast channel induced by the source and the reconstructions and the physical broadcast channel

an achievable rate region for joint compression and dispersive information routing for networks this paper considers the problem of minimum cost communication of correlated sources over a network with multi ple sinks which consists of distributed source coding followed by routing we introduce a new routing paradigm called dispersive information routing dir wherein the intermediate nodes are allowed to split a packet and forward subsets of the received bits on each of the forward paths this paradigm opens up a rich class of research problems which focus on the interplay between encoding and routing in a network unlike conventional routing methods such as in 1 dir ensures that each sink receives just the information needed to reconstruct the sources it is required to reproduce we demonstrate using simple examples that the proposed approach offers better asymptotic performance than conventional routing techniques we show that under certain assumptions on the cost function the problem of finding the minimum cost under dir essentially reduces to characterizing an achievable rate region for a new multiterminal information theoretic setup while it is possible to derive an achievable region for this setup using prior results from general multiterminal source coding 3 these techniques do not exploit the underlying problem structure and thereby lead to suboptimal regions in this paper we propose a new coding scheme using principles from multiple descriptions encoding 2 and show that it strictly improves upon a corresponding variant of coding scheme in 3 we further show that the new coding scheme achieves the complete rate region for certain special cases of the general setup and thereby achieves the minimum communication cost under this routing paradigm

rapid solution of the wave equation in unbounded domains in this paper we propose and analyze a new fast method for the numerical solution of time domain boundary integral formulations of the wave equation we employ lubich s convolution quadrature method for the time discretization and a galerkin boundary element method for the spatial discretization the coefficient matrix of the arising system of linear equations is a triangular block toeplitz matrix possible choices for solving the linear system arising from the above discretization include the use of fast fourier transform fft techniques and the use of data sparse approximations by using fft techniques the computational complexity can be reduced substantially while the storage cost remains unchanged and is typically high using data sparse approximations the gain is reversed i e the computational cost is approximately unchanged while the storage cost is substantially reduced the method proposed in this paper combines the advantages of these two approaches first the discrete convolution related to the block toeplitz system is transformed into the discrete fourier image thereby arriving at a decoupled system of discretized helmholtz equations with complex wave numbers a fast data sparse e g fast multipole or panel clustering method can then be applied to the transformed system additionally significant savings can be achieved if the boundary data are smooth and time limited in this case the right hand sides of many of the helmholtz problems are almost zero and hence can be disregarded finally the proposed method is inherently parallel we analyze the stability and convergence of these methods thereby deriving the choice of parameters that preserves the convergence rates of the unperturbed convolution quadrature we also present numerical results which illustrate the predicted convergence behavior

variable to fixed length codes provide better large deviations performance than fixed to variable length codes it is proved that for finite alphabet finite state unifilar sources variable to fixed length codes provide better large deviations performance of the empirical compression ratio than fixed to variable length codes it is shown how to construct a universal variable to fixed length code that achieves the optimal performance

nonbinary quantum stabilizer codes we define and show how to construct nonbinary quantum stabilizer codes our approach is based on nonbinary error bases it generalizes the relationship between self orthogonal codes over f sub 4 and binary quantum codes to one between self orthogonal codes over f q sup 2 and q ary quantum codes for any prime power q

error exponents for distributed detection of markov sources the paper considers a binary hypothesis testing system in which two sensors simultaneously observe a discrete time finite valued stationary ergodic markov source and transmit m ary messages to a neyman pearson central detector the size m of the message alphabet increases at most subexponentially with the number of observations the asymptotic behavior of the type ii error rate is investigated as the number of observations increases to infinity and the associated error exponent is obtained under mild assumptions on the source distributions this exponent is independent of the test level spl epsiv and the actual codebook sizes m is achieved by a universally optimal sequence of acceptance regions and is characterized by an infimum of informational divergence rate over a class of infinite dimensional distributions important differences due to the observations being markov between the asymptotically optimal distributed tests and their nondistributed counterparts are highlighted the converse results require a blowing up lemma for stationary ergodic markov sources which is also proven

on the minimum description length principle for sources with piecewise constant parameters universal lossless coding in the presence of finitely many abrupt changes in the statistics of the source at unknown points is investigated the minimum description length mdl principle is derived for this setting in particular it is shown that for any uniquely decipherable code for almost every combination of statistical parameter vectors governing each segment and for almost every vector of transition instants the minimum achievable redundancy is composed from 0 5 log n n bits for each unknown segmental parameter and log n n bits for each transition where n is the length of the input string this redundancy is shown to be attainable by a strongly sequential universal encoder i e an encoder that does not utilize the knowledge of a prescribed value of n

estimating with partial statistics the parameters of ergodic finite markov sources parameter estimation based on data emitted from a finite ergodic markov source is discussed this can be considered an extension of the memoryless case first an asymptotically optimal estimator is suggested for the case where the parametric model is completely known for an unknown parametric model e g unknown noise distribution with training sequences available a necessary condition is given for the existence of a universally optimum estimate a universal estimate is then suggested that is asymptotically nearly optimal the results hold under fairly mild regularity conditions

subset source coding this paper studies the fundamental limits of storage for structured data where statistics and structure are both critical to the application accordingly a framework is proposed for optimal lossless and lossy compression of subsets of the possible realizations of a discrete memoryless source dms for the lossless subset compression problem it turns out that the optimal source code may not index the conventional source typical sequences but rather index certain subset typical sequences consistent with the subset of interest building upon an achievability and a strong converse an analytic expression is given based on the shannon entropy relative entropy and subset entropy which identifies such subset typical sequences for a broad class of subsets of a dms intuitively subset typical sequences belong to those typical sets which highly intersect the subset of interest but are still closest to the source distribution in the sense of relative entropy for the lossy subset compression problem an upper bound is derived on the subset rate distortion function in terms of the subset mutual information optimized over the set of conditional distributions that satisfy the expected distortion constraint with respect to the subset typical distribution and over a set of certain auxiliary subsets by proving a strong converse result this upper bound is shown to be tight for a class of symmetric subsets as shown in our numerical examples more often than not one achieves a gain in the fundamental limits in that the optimal compression rate for the subset in both the lossless and lossy settings can be strictly smaller than the source entropy and the source rate distortion function respectively although exceptions are also possible

a large deviations approach to error exponents in source coding and hypothesis testing it is pointed out that the basic results can be proved fairly easily if one uses a sanov theorem for the distribution of types such a theorem comes easily from large deviation theory a caveat is that this technique only identifies the error exponent up to terms o n in the exponent whereas the combinatorial arguments give an estimate up to terms o log n in the exponent

on the multistep time discretization of linear initial boundary value problems and their boundary integral equations convergence estimates in terms of the data are shown for multistep methods applied to non homogeneous linear initial boundary value problems similar error bounds are derived for a new class of time discrete and fully discrete approximation schemes for boundary integral equations of such problems e g for the single layer potential equation of the wave equation in both cases the results are obtained from convergence and stability estimates for operational quadrature approximations of convolutions these estimates which are also proved here depend on bounds of the laplace transform of the distributional convolution kernel outside the stability region scaled by the time stepsize and on the smoothness of the data

an efficient time domain fem bem coupling approach based on fem implicit green s functions and truncation of bem time convolution process abstract this paper describes an efficient scheme for coupling the fe and the be methods the domain of the original problem is subdivided into two sub domains which are independently modelled by the fem and the bem both sub domains are analyzed separately and the coupling is enforced at their interface the fem time marching process is carried out through an algorithm based on implicit green s functions of the related mechanical system and the convolution process of the bem is compressed by proper interpolation of the time generated matrices the efficiency of the proposed scheme is improved by a special procedure to deal with different time step choices in each sub domain at the end of the paper numerical examples are presented in order to illustrate the accuracy and efficiency of the present approach

a universal model based on minimax average divergence given a set of training samples the commonly used approach to determine a universal model is accomplished by averaging the statistics over all training samples it is suggested to use average divergence as a measurement for the effectiveness of a universal model and a minimax universal model that minimizes the maximum average divergence among all training samples is proposed efficient searching algorithms are developed and experimental results are presented

entropy at a weight per symbol and embeddings of markov chains in 11 krieger gave a remarkable embedding result for shifts of finite type he proved that two simple necessary conditions strict inequality of the topological entropies of the shifts and the existence of an embedding for their periodic points are in fact sufficient for markov chains when the appropriate definitions of embedding and subsystem were supplied in 10 analogous necessary conditions strict inequality of functions and the existence of a weight preserving embedding for periodic points became apparent however an example in 10 also revealed these conditions to not be sufficient in this paper we bring to bear on embeddings of markov chains the ideas we introduced in 16 in fact we work in the closely related setting of triples z a a a f where 2 a aa is a shift of finite type and f is a function assigning weights to the edges of the graph associated with sa aa the func t ionf takes values in ir or more generally in n q for some qen the weight of a periodic orbit which corresponds to a cycle in the graph is then the sum of the weights of the edges lying on the cycle division by the length of the cycle gives the weight per symbol in 16 we continued the rational linear span of the set wps i of weights per symbol is a finite dimensional vector space vs the convex hull of wps i in v i is a polytope wpsi called the weight per symbol polytope consideration of the faces of wps i gives subsystems of r a aa f which are ordered according to the ordering of the faces the resulting nesting of subsystems is the scaffold of a o a f an embedding of a triple i a a a f in another 2 b ab g is a continuous shift commuting injection q 2 a 12 which preserves weights of periodic orbits the two necessary conditions mentioned above imply w p s i c w when the boundary c wpsi meets wpsg that is when a face f of wps i meets a face g of wpsg any embedding must carry the subsystem corresponding to f into that corresponding to g and this gives rise to further necessary conditons a discussion of these complications and a description of the example of 10 in these terms will be given in sect 7 before that the main embedding

coupling discontinuous galerkin methods and retarded potentials for transient wave propagation on unbounded domains this work deals with the numerical simulation of wave propagation on unbounded domains with localized heterogeneities to do so we propose to combine a discretization based on a discontinuous galerkin method in space and explicit finite differences in time on the regions containing heterogeneities with the retarded potential method to account the unbounded nature of the computational domain the coupling formula enforces a discrete energy identity ensuring the stability under the usual cfl condition in the interior moreover the scheme allows to use a smaller time step in the interior domain yielding to quasi optimal discretization parameters for both methods the aliasing phenomena introduced by the local time stepping are reduced by a post processing by averaging in time obtaining a stable and second order consistent in time coupling algorithm we compute the numerical rate of convergence of the method for an academic problem the numerical results show the feasibility of the whole discretization procedure

evaluation of marton s inner bound for the general broadcast channel the best known inner bound on the two receiver general broadcast channel is due to marton however this region is not computable except in certain special cases as no bounds on the cardinality of its auxiliary random variables exist nor is it even clear that the inner bound is a closed set the main obstacle in proving cardinality bounds is the fact that the traditional use of the caratheodory theorem the main known tool for proving cardinality bounds does not yield a finite cardinality result one of the main contributions of this paper is the introduction of a new tool based on an identity that relates the second derivative of the shannon entropy of a discrete random variable under a certain perturbation to the corresponding fisher information in order to go beyond the traditional caratheodory type arguments we identify certain properties that the auxiliary random variables corresponding to the extreme points of the inner bound need to satisfy these properties are then used to establish cardinality bounds on the auxiliary random variables of the inner bound thereby proving the computability of the region and its closedness lastly we establish a conjecture of nair and zizhou that marton s inner bound and the recent outer bound of nair and el gamal do not match in general

error exponents for finite hypothesis channel identification we consider the problem of designing optimal probing signals for finite hypothesis testing equivalently we cast the problem as the design of optimal channel input sequences for identifying a discrete channel under observation from a finite set of known channels the optimality criterion that we employ is the exponent of the bayesian probability of error in our study we consider a feedforward scenario where there is no feedback from the channel output to the signal selector at the channel input and a feedback scenario where the past channel outputs are revealed to the signal selector in the feedforward scenario only the type of the input sequence matters and our main result is an expression for the error exponent in terms of the limiting distribution of the input sequence in the feedback case we show that when discriminating between two channels the optimal scheme in the first scenario is simultaneously the optimal time invariant markov feedback policy of any order

achievable rate region for three user discrete broadcast channel based on coset codes we consider the problem of developing coding techniques and deriving achievable rate regions for discrete memoryless broadcast channels with three receivers 3 dbc we begin by identifying a novel vector additive 3 dbc for which we characterize an upper bound on the the largest achievable rate region based on unstructured codes henceforth referred to as pmb mathscr u mathcal m region we propose a coding technique based on coset codes that yield an achievable rate triple not contained within pmb mathscr u mathcal m region we generalize the proposed coding technique using a new ensemble of codes partitioned coset codes pcc containing both empirical and algebraic properties and evaluate its performance to derive an achievable rate region for the general 3 dbc the new elements in this derivation are binning and joint typicality encoding and decoding of statistically correlated pccs we validate the utility of this technique by identifying non additive instances of 3 dbc for which the proposed coding techniques based on pcc yield strictly larger rates

runge kutta convolution quadrature for operators arising in wave propagation an error analysis of runge kutta convolution quadrature is presented for a class of non sectorial operators whose laplace transform satisfies besides the standard assumptions of analyticity in a half plane re s 0 and a polynomial bound operatorname o s mu 1 there the stronger polynomial bound operatorname o s mu 2 in convex sectors of the form operatorname arg s leq pi 2 theta for 0 the order of convergence of the runge kutta convolution quadrature is determined by 2 and the underlying runge kutta method but is independent of 1 time domain boundary integral operators for wave propagation problems have laplace transforms that satisfy bounds of the above type numerical examples from acoustic scattering show that the theory describes accurately the convergence behaviour of runge kutta convolution quadrature for this class of applications our results show in particular that the full classical order of the runge kutta method is attained away from the scattering boundary

error exponent for multiple access channels upper bounds the problem of bounding the reliability function of a multiple access channel mac is studied two new upper bounds on the error exponent of a two user discrete memoryless dm mac are derived the first bound sphere packing is an upper bound on the exponent of the average probability of error and is the first bound of this type that is zero outside the capacity region and thus results in a tighter sphere packing exponent when compared with the tightest known exponent derived by haroutunian the second bound minimum distance is an upper bound on the exponent of the maximal as opposed to average probability of error to obtain this bound first an upper bound on the minimum bhattacharyya distance between codeword pairs is derived for a certain class of two user dm macs an upper bound on the exponent of maximal probability of error is derived as a consequence of the upper bound on the minimum bhattacharyya distance we analytically evaluate the sphere packing bound for uniform composition codes for an additive and nonsymmetric channel and show that it is tight near the boundary of the capacity region i e equal to the random coding lower bound

minimax universal noiseless coding for unifilar and markov sources corresp constructive upper bounds are presented for minimax universal noiseless coding of unifilar sources without any ergodicity assumptions these bounds are obtained by quantizing the estimated probability distribution of source letters with respect to the relative entropy they apply both to fixed length to variable length fv and variable length to fixed length vf codes unifilar sources are a generalization of the usual definition of markov sources so these results apply to markov sources as well these upper bounds agree asymptotically with the lower bounds given by davisson for fv coding of stationary ergodic markov sources

a posteriori probability distances between finite alphabet hidden markov models in this correspondence we consider a probability distance problem for a class of hidden markov models hmms the notion of conditional relative entropy between conditional probability measures is introduced as an a posteriori probability distance which can be used to measure the discrepancy between hidden markov models when a realized observation sequence is observed using a measure change technique we derive a representation for conditional relative entropy in terms of the parameters of the hmms and conditional expectations given measurements with this representation we show that this distance can be calculated using an information state approach

joint source channel coding error exponent for discrete communication systems with markovian memory we study the error exponent ej for reliably transmitting a discrete stationary ergodic markov sem source q over a discrete channel w with additive sem noise via a joint source channel jsc code we first establish an upper bound for ej in terms of the renyi entropy rates of the source and noise processes we next investigate the analytical computation of ej by comparing our bound with gallager s lower bound 1968 when the latter one is specialized to the sem source channel system we also note that both bounds can be represented in csiszar s form 1980 as the minimum of the sum of the source and channel error exponents our results provide us with the tools to systematically compare ej with the tandem separate coding exponent ej we show that as in the case of memoryless source channel pairs ej les 2er and we provide explicit conditions for which ej et numerical results indicate that ej ap 2et for many sem source channel pairs hence illustrating a substantial advantage of jsc coding over tandem coding for systems with markovian memory

on universal types the universal type class of a sequence xn is defined in analogy to the notion underlying the classical method of types two sequences of the same length are said to be of the same universal lz type if and only if they yield the same dictionary or equivalently parsing tree in the incremental parsing of ziv and lempel 1978 it is shown that for any finite order k the variational distance between the kth order empirical probability distributions of two sequences of the same universal type vanishes as the sequence length tends to infinity consequently for any k and any kth order probability assignment the difference between the normalized logarithms of the probabilities assigned to two sequences of the same universal type also vanishes asymptotically the size of a universal type class is studied and it is shown that its asymptotic behavior parallels that of the conventional counterpart with the lz78 code length playing the role of the empirical entropy the number of universal types for sequences of length n is estimated and shown to be of the form exp 1 o 1 gamman logn for a well characterized constant gamma algorithms for enumerating the sequences in a universal type class and for drawing a sequence from the class with uniform probability are described as an application the problem of universal simulation of individual sequences is considered a sequence drawn with uniform probability from the universal type class of xn is an optimal simulation of xn in a well defined mathematical sense

on the evaluation of marton s inner bound for two receiver broadcast channels marton s inner bound is the best known achievable rate region for a general two receiver discrete memoryless broadcast channel in this paper we establish improved bounds on the cardinalities of the auxiliary random variables appearing in this inner bound to the true rate region we combine a perturbation technique along with a representation using concave envelopes of information theoretic functions that involve the use of auxiliary random variables to achieve this improvement the new cardinality bounds lead to a proof that a randomized time division strategy achieves every rate triple in marton s region for binary input broadcast channels this extends the result by hajek and pursley which showed that the cover van der muelen region was exhausted by the randomized time division strategy

probabilistic distances between finite state finite alphabet hidden markov models this note considers the problem of evaluating a probabilistic distance between discrete time homogeneous first order finite state finite alphabet hidden markov models hmms our approach is based on a correspondence between probability measures and hmms established in this note using a probability measure transformation technique we obtain recursive expressions for the relative entropy between the marginal probability distributions of two hmms under consideration also the relative entropy rate as the limit of the time averaged value of the above relative entropy is obtained these expressions are given in terms of the parameters of the given hmms furthermore we show that the probabilistic distance between hmms used in the existing literature admits a representation in terms of a conditional expectation given the observation sequence this representation allows us to evaluate this distance using an information state approach

graph decomposition a new key to coding theorems a new and simple method is proposed for finding good encoders both for channels and for sources with side information this method relies on the continuous version of a graph decomposition result of lovasz the presently known best exponential error bounds for both problems follow in a unified manner with an improvement on the source coding bound the previous bounds for universal codes of the authors and marton are also improved

type classes of context trees it is well known that a tree model does not always admit a finite state machine fsm representation with the same minimal number of parameters therefore known characterizations of type classes for fsms do not apply in general to tree models in this paper the type class of a sequence with respect to a given context tree is studied an exact formula is derived for the size of the class extending whittle s formula for type classes with respect to fsms the derivation is more intricate than in the fsm case since some basic properties of fsm types do not hold in general for tree types the derivation also yields an efficient enumeration of the tree type class a formula for the number of type classes with respect to is also derived the formula is asymptotically tight up to a multiplicative constant and also extends the corresponding result for fsms the asymptotic behavior of the number of type classes and of the size of a class is expressed in terms of the so called minimal canonical extension of t a tree that is generally larger than but smaller than its fsm closure

achieving marton s region for broadcast channels using polar codes this paper presents polar coding schemes for the two user discrete memoryless broadcast channel dm bc which achieve marton s region with both common and private messages this is the best achievable rate region known to date and it is tight for all classes of two user dm bcs whose capacity regions are known to accomplish this task we first construct polar codes for both the superposition as well as binning strategy by combining these two schemes we obtain marton s region with private messages only finally we show how to handle the case of common information the proposed coding schemes possess the usual advantages of polar codes i e they have low encoding and decoding complexity and a superpolynomial decay rate of the error probability we follow the lead of goela abbe and gastpar who recently introduced polar codes emulating the superposition and binning schemes to align the polar indices for both schemes their solution involves some degradedness constraints that are assumed to hold between the auxiliary random variables and channel outputs to remove these constraints we consider the transmission of k blocks and employ a chaining construction that guarantees the proper alignment of the polarized indices the techniques described in this paper are quite general and they can be adopted to many other multiterminal scenarios whenever there polar indices need to be aligned

transparent boundary conditions for wave propagation in fractal trees convolution quadrature approach in this work we propose high order transparent boundary conditions for the weighted wave equation on a fractal tree with an application to the modeling of sound propagation in a human lung this article follows the recent work 29 dedicated to the mathematical analysis of the corresponding problem and the construction of low order absorbing boundary conditions the method proposed in this article consists in constructing the exact trans parent boundary conditions for the semi discretized problem in the spirit of the convolution quadrature method developed by ch lubich we analyze the stability and convergence of the method and propose an efficient algorithm for its implementation the exposition is concluded with numerical experiments

on zero rate error exponents of finite state channels with input dependent states we derive a single letter formula for the zero rate reliability error exponent of a finite state channel whose state variable depends deterministically and recursively on past channel inputs where the code complies with a given channel input constraint special attention is then devoted to the important special case of the gaussian channel with intersymbol interference where more explicit results are obtained

convolution quadrature and discretized operational calculus ii operational quadrature rules are applied to problems in numerical integration and the numerical solution of integral equations singular integrals power and logarithmic singularities finite part integrals multiple timescale convolution volterra integral equations wiener hopf integral equations frequency domain conditions which determine the stability of such equations can be carried over to the discretization

runge kutta methods for parabolic equations and convolution quadrature we study the approximation properties of runge kutta time discretizations of linear and semilinear parabolic equations including incompressible navier stokes equations we derive asymptotically sharp error bounds and relate the temporal order of convergence which is generally noninteger to spatial regularity and the type of boundary conditions the analysis relies on an interpretation of runge kutta methods as convolution quadratures in a different context these can be used as efficient computational methods for the approximation of convolution integrals and integral equations they use the laplace transform of the confolution kernal via a discrete operational calculus 23 refs 2 tabs

conditional limit theorems under markov conditioning let x 1 x 2 cdots be independent identically distributed random variables taking values in a finite set x and consider the conditional joint distribution of the first m elements of the sample x 1 cdots x n on the condition that x 1 x 1 and the sliding block sample average of a function h cdot cdot defined on x 2 exceeds a threshold alpha eh x 1 x 2 for m fixed and n rightarrow infty this conditional joint distribution is shown to converge m the m step joint distribution of a markov chain started in x 1 which is closest to x l x 2 cdots in kullback leibler information divergence among all markov chains whose two dimensional stationary distribution p cdot cdot satisfies sum p x y h x y geq alpha provided some distribution p on x 2 having equal marginals does satisfy this constraint with strict inequality similar conditional limit theorems are obtained when x 1 x 2 cdots is an arbitrary finite order markov chain and more general conditioning is allowed

duality of a source coding problem and the semi deterministic broadcast channel with rate limited cooperation the wyner ahlswede korner wak empirical coordination problem where the encoders cooperate via a finite capacity one sided link is considered the coordination capacity region is derived by combining several source coding techniques such as wyner ziv coding binning and superposition coding furthermore a semi deterministic sd broadcast channel bc with one sided decoder cooperation is considered duality principles relating the two problems are presented and the capacity region for the sd bc setting is derived the direct part follows from an achievable region for a general bc that is tight for the sd scenario a converse is established by using telescoping identities the sd bc is shown to be operationally equivalent to a class of relay bcs and the correspondence between their capacity regions is established the capacity region of the sd bc is transformed into an equivalent region that is shown to be dual to the admissible region of the wak problem in the sense that the information measures defining the corner points of both regions coincide achievability and converse proofs for the equivalent region are provided for the converse we use a probabilistic construction of auxiliary random variables that depends on the distribution induced by the codebook several examples illustrate the results

a statistical property of multiagent learning based on markov decision process we exhibit an important property called the asymptotic equipartition property aep on empirical sequences in an ergodic multiagent markov decision process mdp using the aep which facilitates the analysis of multiagent learning we give a statistical property of multiagent learning such as reinforcement learning rl near the end of the learning process we examine the effect of the conditions among the agents on the achievement of a cooperative policy in three different cases blind visible and communicable also we derive a bound on the speed with which the empirical sequence converges to the best sequence in probability so that the multiagent learning yields the best cooperative result

on marton s inner bound and its optimality for classes of product broadcast channels marton s inner bound is the tightest known inner bound on the capacity region of the broadcast channel it is not known however if this bound is tight in general one approach to settle this key open problem in network information theory is to investigate the multiletter extension of marton s bound which is known to be tight in general this approach has become feasible only recently through the development of a new method for bounding cardinalities of auxiliary random variables by gohari and anantharam this paper undertakes this long overdue approach to establish several new results including 1 establishing the optimality of marton s bound for new classes of product broadcast channels 2 showing that the best known outer bound by nair and el gamal is not tight in general and 3 finding sufficient conditions for a global maximizer of marton s bound that imply that the 2 letter extension does not increase the achievable rate motivated by the new capacity results we establish a new outer bound on the capacity region of product broadcast channels

on quantum fidelities and channel capacities for the discrete memoryless quantum channel we show the equivalence of two different notions of quantum channel capacity that which uses the entanglement fidelity as its criterion for success in transmission and that which uses the minimum fidelity of pure states in a subspace of the input hilbert space as its criterion as a corollary any source with entropy rate less than the capacity may be transmitted with high entanglement fidelity we also show that a restricted class of encodings is sufficient to transmit any quantum source which may be transmitted on a given channel this enables us to simplify a known upper bound for the channel capacity it also enables us to show that the availability of an auxiliary classical channel from encoder to decoder does not increase the quantum capacity

energy estimates for galerkin semidiscretizations of time domain boundary integral equations in this paper we present a battery of results related to how galerkin semidiscretization in space affects some formulations of wave scattering and propagation problems when retarded boundary integral equations are used

the error exponent for the noiseless encoding of finite ergodic markov sources a new approach to the classical fixed length noiseless source coding problem is proposed for the case of finite ergodic markov sources this approach is based on simple counting arguments the central notion of markov type a set containing all the source sequences having the same transition counts from letter to letter is introduced and the cardinality of such a set is evaluated via graph theoretical tools the error exponent is shown to be a weighted average of informational divergences and the universal character of the result is stressed as a corollary the classical source coding theorem determining the achievable rates is rederived

mismatched decoding revisited general alphabets channels with memory and the wide band limit the mismatch capacity of a channel is the highest rate at which reliable communication is possible over the channel with a given possibly suboptimal decoding rule this quantity has been studied extensively for single letter decoding rules over discrete memoryless channels dmcs here we extend the study to memoryless channels with general alphabets and to channels with memory with possibly non single letter decoding rules we also study the wide band limit and in particular the mismatch capacity per unit cost and the achievable rates on an additive noise spread spectrum system with single letter decoding and binary signaling

universal delay estimation for discrete channels the use of information theory concepts for universal estimation of delay for classes of discrete channels is discussed the problem is presented as one of hypothesis testing although the channel statistics are not known for large enough signal duration the exponent of the average error probability is equal to that associated with the optimal maximum likelihood ml decision procedure which utilizes full knowledge of the channel parameters two categories of problems are discussed the single channel problem where the random transmitted signal is known to the receiver and the two sensor problem where the random signal is unknown

an upper bound for the largest lyapunov exponent of a markovian product of nonnegative matrices we derive an upper bound for the largest lyapunov exponent of a markovian product of nonnegative matrices using markovian type counting arguments the bound is expressed as the maximum of a nonlinear concave function over a finite dimensional convex polytope of probability distributions

optimum tradeoffs between the error exponent and the excess rate exponent of variable rate slepian wolf coding we analyze the optimal tradeoff between the error exponent and the excess rate exponent for variable rate slepian wolf codes in particular we first derive upper converse bounds on the optimal error and excess rate exponents and then lower achievable bounds via a simple class of variable rate codes which assign the same rate to all source blocks of the same type class then using the exponent bounds we derive bounds on the optimal rate functions namely the minimal rate assigned to each type class needed in order to achieve a given target error exponent the resulting excess rate exponent is then evaluated iterative algorithms are provided for the computation of both bounds on the optimal rate functions and their excess rate exponents the resulting slepian wolf codes bridge between the two extremes of fixed rate coding which has minimal error exponent and maximal excess rate exponent and average rate coding which has maximal error exponent and minimal excess rate exponent

an information inequality and evaluation of marton s inner bound for binary input broadcast channels we establish an information inequality concerning five random variables this inequality is motivated by the sum rate evaluation of marton s inner bound for two receiver broadcast channels with a binary input alphabet we establish that randomized time division strategy achieves the sum rate of marton s inner bound for all binary input broadcast channels we also obtain an improved cardinality bound for evaluating the maximum sum rate given by marton s inner bound for all broadcast channels using these tools we explicitly evaluate the inner and outer bounds for the binary skew symmetric broadcast channel and demonstrate a gap between the bounds

the validity of johnson nedelec s bem fem coupling on polygonal interfaces in this short article we prove that the classical one equation or johnson nedelec coupling of finite and boundary elements can be applied with a lipschitz coupling interface because of the way it was originally approached from the analytical standpoint this bem fem scheme has generally required smooth boundaries and hence produced a consistency error in the finite element part with a variational argument we prove that this requirement is not needed and that stability holds for all pairs of discrete space as it inherits the underlying ellipticity of the problem

linear codes for sources and source networks error exponents universal coding for slepian wolf source networks the error exponents obtained by korner marton and the author are shown to be universally attainable by linear codes also improved exponents are derived for linear codes with large rates specializing the results to simple discrete memoryless sources reveals their relationship to the random coding and expurgated bounds for channels with additive noise one corollary is that there are universal linear codes for this class of channels which attain the random coding error exponent for each channel in the class the combinatorial approach of csiszar korner marton is used in particular all results are derived from a lemma specifying good encoders in terms of purely combinatorial properties

runge kutta convolution quadrature for the boundary element method abstract time domain boundary element formulations can be established either directly in time domain or via laplace or fourier domain somewhere in between are the convolution quadrature based boundary element formulations which utilize the laplace domain fundamental solution but establish a time stepping procedure up to now in applications mostly backward differential formulas of second order are used as the underlying multistep method however in recent mathematical literature also runge kutta methods have been applied here the use of runge kutta methods is explained in detail and some numerical studies are given in these studies the backward difference based procedures are compared to runge kutta methods for a non smooth problem an l 2 norm of the error is used as the basis of comparison the convergence of which is investigated theoretically as well the results confirm that the usage of the new techniques is preferable with regard to less numerical oscillations in the solution and better representation of wave fronts

a counter example to the mismatched decoding converse for binary input discrete memoryless channels this paper studies the mismatched decoding problem for binary input discrete memoryless channels an example is provided for which an achievable rate based on superposition coding exceeds the csiszar korner hui rate thus providing a counter example to a previously reported converse result both numerical evaluations and theoretical results are used in establishing this claim

on two stage sequential coding of correlated sources we study the problem of two stage sequential coding tssc which is an extension of sequential coding of correlated sources let x and y be dependent random variables the network contains two encoders and two decoders 1 a y encoder with input y 2 an x encoder with inputs x and y 3 a y decoder that reconstructs y and 4 an x decoder that reconstructs x the first stage is traditional sequential coding where the y encoder describes y to both the x decoder and y decoder and the x encoder describes x and y to the x decoder at the second stage the y encoder refines the description of y and the x encoder refines the description of x the tssc model is a theoretical abstraction of scalable video coding here y and x represent successive frames of a video sequence and the two stages together give an embedded description that allows the video to be decoded at two distinct rates we give an inner bound on the rate distortion region for this tssc model the tight bound on the rate distortion region is derived when y must be reconstructed losslessly in the usual shannon sense in the second stage we also study the minimum total rate of the tssc model and show that the minimum total rate of one stage sequential coding cannot be achieved at both stages for jointly gaussian sources this theoretical result can shed light on the rate distortion performance behavior of scalable video coding widely noted by practitioners

a lower bound on the quantum capacity of channels with correlated errors the highest fidelity of quantum error correcting codes of length n and rate r is proven to be lower bounded by 1 exp ne r o n for some function e r on noisy quantum channels that are subject to not necessarily independent errors the e r is positive below some threshold r0 which implies r0 is a lower bound on the quantum capacity this work is an extension of the author s previous works m hamada phys rev a 65 052305 2002 quant ph 0109114 lanl 2001 m hamada quant ph 0112103 lanl 2001 which presented the bound for channels subject to independent errors or channels modeled as tensor products of copies of a completely positive linear map the relation of the channel class treated in this article to those in the previous works is similar to that of markov chains to sequences of independent identically distributed random variables

monotone measures for non local correlations non locality is the phenomenon of observing strong correlations among the outcomes of local measurements of a multipartite physical system no signaling boxes are the abstract objects for studying non locality and wirings are local operations on the space of no signaling boxes this means that no matter how non local the nature is the set of physical non local correlations must be closed under wirings then one approach to identify the non locality of nature is to characterize the closed sets of non local correlations although non trivial examples of wirings of no signaling boxes are known there is no systematic way to study wirings in particular given a set of no signaling boxes we do not know a general method to prove that it is closed under wirings in this paper we propose the first general method to construct such closed sets of non local correlations we show that a well known measure of correlation called maximal correlation when appropriately defined for non local correlations is monotonically decreasing under wirings this establishes a conjecture about the impossibility of simulating isotropic boxes from each other implying the existence of a continuum of closed sets of non local boxes under wirings to prove our main result we introduce some mathematical tools that may be of independent interest we define a notion of maximal correlation ribbon as a generalization of maximal correlation and provide a connection between it and a known object called hypercontractivity ribbon we show that these two ribbons are monotone under wirings too

minimax noiseless universal coding for markov sources upper and lower bounds are presented on the minimax redundancy for markov source noiseless block to variable universal coding the number of states block size and markov order are arbitrary but finite unlike earlier results the upper and lower bounds are absolute that is not merely asymptotic the upper bound is established by combinatorial bounds the lower bound is established by a bound on the average redundancy for an arbitrary distribution on the transition probabilities the bound can be optimized across the choice of distribution

a fully discrete bem fem scheme for transient acoustic waves abstract we study a symmetric bem fem coupling scheme for the scattering of transient acoustic waves by bounded inhomogeneous anisotropic obstacles in a homogeneous field an incident wave in free space interacts with the obstacles and produces a combination of transmission and scattering the transmitted part of the wave is discretized in space by finite elements while the scattered wave is reduced to two fields defined on the boundary of the obstacles and is discretized in space with boundary elements we choose a coupling formulation that leads to a symmetric system of integro differential equations the retarded boundary integral equations are discretized in time by convolution quadrature and the interior field is discretized in time with the trapezoidal rule we show that the scattering problem generates a c 0 group of isometries in a hilbert space and use associated estimates to derive stability and convergence results we provide numerical experiments and simulations to validate our results and demonstrate the flexibility of the method

on marton s inner bound for the general broadcast channel we establish several new results on marton s inner bound on the capacity region of the general broadcast channel inspired by the fact that marton s coding scheme without superposition coding is optimal in the gaussian case we consider the class of binary input degraded broadcast channels with no common message that have the same property we characterize this class we also establish new properties of marton s inner bound that help restrict the search space for computing the marton sum rate in particular we establish an extension of the xor case of the binary inequality of nair wang and geng

entropies and combinatorics of random branching processes and context free languages the entropies and combinatorics of trees that branch according to fixed but finite numbers of rules are studied context free grammars are used to categorize the ways in which nodes branch to yield daughter nodes thus providing an organized setting to examine the entropies for random branching processes whose realizations are trees and whose probabilities are determined by probabilities associated with the substitution rules of the grammar normalized entropy rates h are derived for the critical branching rate and supercritical branching rate processes an equipartition theorem is proved for the supercritical processes a strong departure from classical theorems for markov sources occurs for supercritical branching processes as the typical sets have supergeometric growth rates the combinatorics of the set of all trees that can be generated from the context free substitution rules is also studied

csiszar s cutoff rates for arbitrary discrete sources csiszar s 1995 forward spl beta cutoff rate given a fixed spl beta 0 for a discrete source is defined as the smallest number r sub 0 such that for every r r sub 0 there exists a sequence of fixed length codes of rate r with probability of error asymptotically vanishing as e sup n spl beta r r0 for a discrete memoryless source dms the forward spl beta cutoff rate is shown by csiszar to be equal to the source renyi 1961 entropy an analogous concept of reverse spl beta cutoff rate regarding the probability of correct decoding is also characterized by csiszar in terms of the renyi entropy in this work csiszar s results are generalized by investigating the spl beta cutoff rates for the class of arbitrary discrete sources with memory it is demonstrated that the limsup and liminf renyi entropy rates provide the formulas for the forward and reverse spl beta cutoff rates respectively consequently new fixed length source coding operational characterizations for the renyi entropy rates are established

large deviations for coding markov chains and gibbs random fields fixed block coding schemes for gibbs random fields are proposed in which the empirical expectations of the local interactions of the gibbs measure is compared to its expectation with respect to all gibbs measures having those interactions the exponential decay of the error probabilities is proven and it is shown that the code rates equal the entropy of the random field in addition it is shown that any coding scheme based on regarding the field as a 1 d sequence of symbols has rate greater than the entropy of the field the theory of fixed length coding is approached from the point of view of large deviations both for the calculation of the error exponents or error probabilities and for the calculation of the encoding rates or the asymptotic combinatorics of the coding schemes this approach is also applied to fixed length coding schemes of markov sources for which estimates on the error exponents and on the rates are derived