asymmetric evaluations of erasure and undetected error probabilities the problem of channel coding with the erasure option is revisited for discrete memoryless channels the interplay between the code rate the undetected and total error probabilities is characterized using the information spectrum method a sequence of codes of increasing blocklengths n is designed to illustrate this tradeoff furthermore for additive discrete memoryless channels with uniform input distribution we establish that our analysis is tight with respect to the ensemble average this is done by analyzing the ensemble performance in terms of a tradeoff between the code rate the undetected and total errors this tradeoff is parameterized by the threshold in a generalized likelihood ratio test two asymptotic regimes are studied first the code rate tends to the capacity of the channel at a rate slower than n 1 2 corresponding to the moderate deviations regime in this case both error probabilities decay subexponentially and asymmetrically the precise decay rates are characterized second the code rate tends to capacity at a rate of n 1 2 in this case the total error probability is asymptotically a positive constant while the undetected error probability decays as exp bn 1 2 for some b 0 the proof techniques involve the applications of a modified or shifted version of the gartner ellis theorem and the type class enumerator method to characterize the asymptotic behavior of a sequence of cumulant generating functions

subblock constrained codes for real time simultaneous energy and information transfer consider an energy harvesting receiver that uses the same received signal both for decoding information and for harvesting energy which is employed to power its circuitry in the scenario where the receiver has limited battery size a signal with bursty energy content may cause power outage at the receiver since the battery will drain during intervals with low signal energy in this paper we analyze subblock energy constrained codes seccs which ensure that sufficient energy is carried within every subblock duration we consider discrete memoryless channels and characterize the secc capacity and the secc error exponent and provide useful bounds for these values we also study constant subblock composition codes csccs which are a subclass of seccs where all the subblocks in every codeword have the same fixed composition and this subblock composition is chosen to maximize the rate of information transfer while meeting the energy requirement compared with constant composition codes cccs we show that csccs incur a rate loss and that the error exponent for csccs is also related to the error exponent for cccs by the same rate loss term we exploit the symmetry in csccs to obtain a necessary and sufficient condition on the subblock length for avoiding power outage at the receiver furthermore for cscc sequences we present a tight lower bound on the average energy per symbol within a sliding time window we provide numerical examples highlighting the tradeoff between the delivery of sufficient energy to the receiver and achieving high information transfer rates it is observed that the ability to use energy in real time imposes less of penalty compared with the ability to use information in real time

fundamental limits of communication with low probability of detection this paper considers the problem of communication over a discrete memoryless channel dmc or an additive white gaussian noise awgn channel subject to the constraint that the probability that an adversary who observes the channel outputs can detect the communication is low in particular the relative entropy between the output distributions when a codeword is transmitted and when no input is provided to the channel must be sufficiently small for a dmc whose output distribution induced by the off input symbol is not a mixture of the output distributions induced by other input symbols it is shown that the maximum amount of information that can be transmitted under this criterion scales like the square root of the blocklength the same is true for the awgn channel exact expressions for the scaling constant are also derived

second order asymptotics for the gaussian mac with degraded message sets this paper studies the second order asymptotics of the gaussian multiple access channel with degraded message sets for a fixed average error probability varepsilon in 0 1 and an arbitrary point on the boundary of the capacity region we characterize the speed of convergence of rate pairs that converge to that boundary point for codes that have asymptotic error probability no larger than varepsilon as a stepping stone to this local notion of the second order asymptotics we study a global notion and establish relationships between the two we provide a numerical example to illustrate how the angle of approach to a boundary point affects the second order coding rate this is the first conclusive characterization of the second order asymptotics of a network information theory problem in which the capacity region is not a polygon

covert communication over noisy channels a resolvability perspective we consider the situation in which a transmitter attempts to communicate reliably over a discrete memoryless channel while simultaneously ensuring covertness low probability of detection with respect to a warden who observes the signals through another discrete memoryless channel we develop a coding scheme based on the principle of channel resolvability which generalizes and extends prior work in several directions first it shows that irrespective of the quality of the channels it is possible to communicate on the order of sqrt n reliable and covert bits over n channel uses if the transmitter and the receiver share on the order of sqrt n key bits this improves upon earlier results requiring on the order of sqrt n log n key bits second it proves that if the receiver s channel is better than the warden s channel in a sense that we make precise it is possible to communicate on the order of sqrt n reliable and covert bits over n channel uses without a secret key this generalizes earlier results established for binary symmetric channels we also identify the fundamental limits of covert and secret communications in terms of the optimal asymptotic scaling of the message size and key size and we extend the analysis to gaussian channels the main technical problem that we address is how to develop concentration inequalities for low weight sequences the crux of our approach is to define suitably modified typical sets that are amenable to concentration inequalities

uniform random number generation from markov chains non asymptotic and asymptotic analyses in this paper we derive non asymptotic achievability and converse bounds on the random number generation with without side information our bounds are efficiently computable in the sense that the computational complexity does not depend on the block length we also characterize the asymptotic behaviors of the large deviation regime and the moderate deviation regime by using our bounds which implies that our bounds are asymptotically tight in those regimes we also show the second order rates of those problems and derive single letter forms of the variances characterizing the second order rates furthermore we address the relative entropy rate and the modified mutual information rate for these problems

awgn goodness is enough capacity achieving lattice codes based on dithered probabilistic shaping in this paper we show that any sequence of infinite lattice constellations which is good for the unconstrained gaussian channel can be shaped into a capacity achieving sequence of codes for the power constrained gaussian channel under lattice decoding and non uniform signaling unlike previous results in the literature our scheme holds with no extra condition on the lattices e g quantization goodness or vanishing flatness factor thus establishing a direct implication between awgn goodness in the sense of poltyrev and capacity achieving codes our analysis uses properties of the discrete gaussian distribution in order to obtain precise bounds on the probability of error and achievable rates in particular we obtain a simple characterization of the finite blocklength behavior of the scheme showing that it approaches the optimal dispersion coefficient for high signal to noise ratio we further show that for low signal to noise ratio the discrete gaussian over centered lattice constellations cannot achieve capacity and thus a shift or dither is essentially necessary

on intertwining dilations on considere la description de l ensemble l x de tous les relevements de nagy foias de x en combinant la methode des noyaux de toeplitz generalises avec certaines idees de arov et grossman sur les fonctions analytiques contractibles

positive definite completions of partial hermitian matrices abstract the question of which partial hermitian matrices some entries specified some free may be completed to positive definite matrices is addressed it is shown that if the diagonal entries are specified and principal minors composed of specified entries are positive then if the undirected graph of the specified entries is chordal a positive definite completion necessarily exists furthermore if this graph is not chordal then examples exist without positive definite completions in case a positive definite completion exists there is a unique matrix in the class of all positive definite completions whose determinant is maximal and this matrix is the unique one whose inverse has zeros in those positions corresponding to unspecified entries in the original partial hermitian matrix additional observations regarding positive definite completions are made

extensions of positive definite functions on free groups abstract an analogue of krein s extension theorem is proved for operator valued positive definite functions on free groups the proof gives also the parametrization of all extensions by means of a generalized type of szego parameters one singles out a distinguished completion called central which is related to quasi multiplicative positive definite functions an application is given to factorization of noncommutative polynomials

distributed channel synthesis two familiar notions of correlation are rediscovered as the extreme operating points for distributed synthesis of a discrete memoryless channel in which a stochastic channel output is generated based on a compressed description of the channel input wyner s common information is the minimum description rate needed however when common randomness independent of the input is available the necessary description rate reduces to shannon s mutual information this paper characterizes the optimal tradeoff between the amount of common randomness used and the required rate of description we also include a number of related derivations including the effect of limited local randomness rate requirements for secrecy applications to game theory and new insights into common information duality our proof makes use of a soft covering lemma known in the literature for its role in quantifying the resolvability of a channel the direct proof achievability constructs a feasible joint distribution over all parts of the system using a soft covering from which the behavior of the encoder and decoder is inferred with no explicit reference to joint typicality or binning of auxiliary interest this paper also generalizes and strengthens this soft covering tool

bounds on the decoding error probability of binary linear codes via their spectra bounds on the error probability of maximum likelihood decoding of a binary linear code are considered the bounds derived use the weight spectrum of the code and they are tighter than the conventional union bound in the case of large noise in the channel the bounds derived are applied to a code with an average spectrum and the result is compared to the random coding exponent the author shows that the bound considered for the binary symmetrical channel case coincides asymptotically with the random coding bound for the case of awgn channel the author shows that berlekamp s 1980 tangential bound can be improved but even this improved bound does not coincide with the random coding bound although it can be very close to it

on single antenna rayleigh block fading channels at finite blocklength this article concerns the maximum coding rate at which data can be transmitted over a noncoherent single antenna rayleigh block fading channel using an error correcting code of a given blocklength with a block error probability not exceeding a given value a high snr normal approximation of the maximum coding rate is presented that becomes accurate as the signal to noise ratio snr and the number of coherence intervals l over which we code tend to infinity numerical analyses suggest that the approximation is accurate at snr values above 15db and when the number of coherence intervals is 10 or more

finite dimensional infinite constellations in the setting of a gaussian channel without power constraints proposed by poltyrev in 1994 the codewords are points in an n dimensional euclidean space an infinite constellation and the tradeoff between their density and the error probability is considered the normalized log density nld plays the role of the communication rate and capacity as well as error exponent bounds for this setting are known this paper considers the infinite constellation setting in the finite block length dimension regime a simplified expression for poltyrev s achievability bound is found and it is shown to be closely related to the sphere converse bound and to a recently proposed achievability bound based on point processes the bounds are then analyzed asymptotically for growing n for fixed nld the bounds turn out to be extremely tight compared to previous error exponent analysis for fixed error probability e it is shown that the gap of the highest achievable nld to the optimal nld poltyrev s capacity is approximately 1 2n q 1 e where q is the standard complementary gaussian cumulative distribution function thus extending the channel dispersion analysis to infinite constellations connections to the error exponent of the power constrained gaussian channel and to the volume to noise ratio as a figure of merit are discussed finally the new tight bounds are compared to state of the art coding schemes

lexicography and degeneracy can a pure cutting plane algorithm work we discuss an implementation of the lexicographic version of gomory s fractional cutting plane method for ilp problems and of two heuristics mimicking the latter in computational testing on a battery of miplib problems we compare the performance of these variants with that of the standard gomory algorithm both in the single cut and in the multi cut rounds of cuts version and show that they provide a radical improvement over the standard procedure in particular we report the exact solution of ilp instances from miplib such as stein15 stein27 and bm23 for which the standard gomory cutting plane algorithm is not able to close more than a tiny fraction of the integrality gap we also offer an explanation for this surprising phenomenon

on gaussian channels with feedback under expected power constraints and with non vanishing error probabilities in this paper we consider single and multi user gaussian channels with feedback under expected power constraints and with non vanishing error probabilities in the first of two contributions we study asymptotic expansions for the additive white gaussian noise awgn channel with feedback under the average error probability formalism by drawing ideas from gallager and nakiboglu s work for the direct part and the meta converse for the converse part we establish the varepsilon capacity and show that it depends on varepsilon in general and so the strong converse fails to hold furthermore we provide bounds on the second order term in the asymptotic expansion we show that for any positive integer l the second order term is bounded between a term proportional to ln l n where ln l cdot is the l fold nested logarithm function and a term proportional to n ln n 1 2 where n is the blocklength the lower bound on the second order term shows that feedback does provide an improvement in the maximal achievable rate over the case where no feedback is available in our second contribution we establish the varepsilon capacity region for the awgn multiple access channel with feedback under the expected power constraint by combining ideas from hypothesis testing information spectrum analysis ozarow s coding scheme and power control

signal codes convolutional lattice codes the coded modulation scheme proposed in this paper has a simple construction an integer sequence representing the information is convolved with a fixed continuous valued finite impulse response fir filter to generate the codeword a lattice point due to power constraints the code construction includes a shaping mechanism inspired by precoding techniques such as the tomlinson harashima filter we naturally term these codes convolutional lattice codes or alternatively signal codes due to the signal processing interpretation of the code construction surprisingly properly chosen short fir filters can generate good codes with large minimal distance decoding can be done efficiently by sequential decoding or for better performance by bidirectional sequential decoding error analysis and simulation results indicate that for the additive white gaussian noise awgn channel convolutional lattice codes with computationally reasonable decoders can achieve low error rate close to the channel capacity

channel coding rate in the finite blocklength regime this paper investigates the maximal channel coding rate achievable at a given blocklength and error probability for general classes of channels new achievability and converse bounds are given which are tighter than existing bounds for wide ranges of parameters of interest and lead to tight approximations of the maximal achievable rate for blocklengths n as short as 100 it is also shown analytically that the maximal rate achievable with error probability isclosely approximated by c v n q 1 where c is the capacity v is a characteristic of the channel referred to as channel dispersion and q is the complementary gaussian cumulative distribution function

a study on universal codes with finite block lengths based on random codes and typical set decoding an alternative proof of root and varaiya s compound channel coding theorem for linear gaussian channels is presented the performance limit of codes with finite block length under a compound channel is studied through error bounds and simulation although the theorem promises uniform convergence of the probability of error as the block length approaches infinity with short block lengths the performance can differ considerably for individual channels simulation results show that universal performance can be a practical goal as the block lengths become large

short non binary low density parity check codes for phase noise channels this paper considers the design of short non binary low density parity check ldpc codes over finite fields of order m for channels with phase noise in particular m ary differential phase shift keying dpsk modulated code symbols are transmitted over an additive white gaussian noise awgn channel with the wiener phase noise at the receiver side non coherent detection takes place with the help of a multi symbol detection algorithm followed by a non binary decoding step both the detector and the decoder operate on a joint factor graph as a benchmark finite length bounds and information rate expressions are computed and compared with the codeword error rate cer performance as well as the iterative threshold of the obtained codes as a result performance within 1 2 db from finite length bounds is obtained down to a cer of 10 3

inheritance principles for chordal graphs abstract we prove two results concerning positive completions of partial positive matrices first we give a parametrization of all positive invertible completions of a partial positive matrix having as support a chordal graph along a fixed increasing chordal sequence then a formula for the determinant of each completion is obtained in terms of the associated parameters as an application we obtained a proof of a conjecture of w w barrett c r johnson and l rodman regarding an inheritance principle for chordal graphs

variable length compression allowing errors this paper studies the fundamental limits of the minimum average length of lossless and lossy variable length compression allowing a nonzero error probability epsilon for lossless compression we give nonasymptotic bounds on the minimum average length in terms of erokhin s rate distortion function and we use those bounds to obtain a gaussian approximation on the speed of approach to the limit which is quite accurate for all but small blocklengths 1 epsilon k h mathsf s k v mathsf s 2 pi 1 2 mathop rm exp q 1 left epsilon right 2 2 where q 1 left cdot right is the functional inverse of the standard gaussian complementary cumulative distribution function and v mathsf s is the source dispersion a nonzero error probability thus not only reduces the asymptotically achievable rate by a factor of 1 epsilon but this asymptotic limit is approached from below i e larger source dispersions and shorter blocklengths are beneficial variable length lossy compression under an excess distortion constraint is shown to exhibit similar properties

completions of matrices and the commutant lifting theorem abstract completion results are obtained for partially defined matrices with an infinite number of entries as a corollary the commutant lifting theorem of sz nagy and foia is proved by solving a completion problem

an information theoretic model for steganography an information theoretic model for steganography with a passive adversary is proposed the adversary s task of distinguishing between an innocent cover message c and a modified message s containing hidden information is interpreted as a hypothesis testing problem the security of a steganographic system is quantified in terms of the relative entropy or discrimination between the distributions of c and s which yields bounds on the detection capability of any adversary it is shown that secure steganographic schemes exist in this model provided the covertext distribution satisfies certain conditions a universal stegosystem is presented in this model that needs no knowledge of the covertext distribution except that it is generated from independently repeated experiments

on achievable rates of awgn energy harvesting channels with block energy arrival and non vanishing error probabilities this paper investigates the achievable rates of an additive white gaussian noise energy harvesting eh channel with an infinite battery the eh process is characterized by a sequence of blocks of harvested energy which is known causally at the source the harvested energy remains constant within a block while the harvested energy across different blocks is characterized by a sequence of independent and identically distributed random variables the blocks have length l which can be interpreted as the coherence time of the energy arrival process if l is a constant or grows sublinearly in the blocklength n we fully characterize the first order term in the asymptotic expansion of the maximum transmission rate subject to a fixed tolerable error probability varepsilon the first order term is known as the varepsilon capacity in addition we obtain lower and upper bounds on the second order term in the asymptotic expansion which reveal that the second order term is proportional to l n 1 2 for any varepsilon less than 1 2 the lower bound is obtained through analyzing the save and transmit strategy if l grows linearly in n we obtain lower and upper bounds on the varepsilon capacity which coincide whenever the cumulative distribution function of the eh random variable is continuous and strictly increasing in order to achieve the lower bound we have proposed a novel adaptive save and transmit strategy which chooses different save and transmit codes across different blocks according to the energy variation across the blocks

new nonasymptotic channel coding theorems for structured codes new nonasymptotic random coding theorems with error probability epsilon and finite block length n based on gallager parity check ensemble and general parity check ensembles are derived in this paper the resulting nonasymptotic achievability bounds when combined with nonasymptotic equipartition properties developed in this paper can be easily computed analytically these nonasymptotic achievability bounds are shown to be asymptotically tight up to the second order of the coding rate as n goes to infinity with either constant or subexponentially decreasing epsilon in the case of gallager parity check ensemble and to imply that low density parity check ldpc codes be capacity achieving in the case of ldpc ensembles numerically they are also compared favorably for finite n and epsilon of practical interest with existing nonasymptotic achievability bounds in the literature

finite length analysis of low density parity check codes on the binary erasure channel in this paper we are concerned with the finite length analysis of low density parity check ldpc codes when used over the binary erasure channel bec the main result is an expression for the exact average bit and block erasure probability for a given regular ensemble of ldpc codes when decoded iteratively we also give expressions for upper bounds on the average bit and block erasure probability for regular ldpc ensembles and the standard random ensemble under maximum likelihood ml decoding finally we present what we consider to be the most important open problems in this area

tight exponential analysis of universally composable privacy amplification and its applications motivated by the desirability of universal composability we analyze in terms of l1 distinguishability the task of secret key generation from a joint random variable under this secrecy criterion using the renyi entropy of order 1 s for s 0 1 we derive a new upper bound of eve s distinguishability under the application of the universal2 hash functions it is also shown that this bound gives the tight exponential rate of decrease in the case of independent and identical distributions the result is applied to the wiretap channel model and to secret key generation distillation by public discussion

a hard integer program made easy by lexicography a small but notoriously hard integer program formulated by donald knuth fifty years ago is solved by three versions of a lexicographic algorithm using gomory cuts the lexicographic cutting plane algorithms are faster than cplex on this problem by a factor of at least 10

sphere packing bounds revisited for moderate block lengths the main reference of this paper is the sphere packing bound of 1967 sp67 derived by shannon gallager and berlekamp it offers a lower bound on the decoding error probability over a very large variety of channels if it has failed so far to provide any usable material in practical implementation of telecommunication systems it is due to an original focus on asymptotic results making it inapplicable for moderate code lengths and to the difficulty of the involved methods which makes the derivation of sp67 quite hermetic and uninspiring for further research the purpose of this paper is two fold 1 to stir up some renewed interest in the topic on which shannon concluded his career in information theory thanks to a qualitative rather than technical review of the derivation of sp67 introduced by a review of the simpler sphere packing bound derived by shannon in 1959 2 to prove the practical interest of sp67 by extending its field of application to continuous output channels and particularly the additive white gaussian noise awgn channel used with any particular modulation scheme and by improving its lower bound for the moderate code length case so that it becomes the best lower bound for most iteratively decodable codes turbo codes low density parity check ldpc codes repeat accumulate ra codes etc of usual lengths

unequal message protection asymptotic and non asymptotic tradeoffs we study a form of unequal error protection that we term unequal message protection ump the message set of a ump code is a union of m disjoint message classes each class has its own error protection requirement with some classes needing better error protection than others we analyze the tradeoff between rates of message classes and the levels of error protection our analysis reveals new tradeoffs which were not captured by prior works on ump codes to obtain our results we generalize finite block length achievability and converse bounds due to polyanskiy poor verdu we evaluate our bounds for the binary symmetric and binary erasure channels and analyze the asymptotic characteristic of the bounds in the fixed error and moderate deviations regimes in addition we consider two questions related to the practical construction of ump codes first we study a header construction that prefixes the message class into a header followed by data protection using a standard homogeneous classical code we show that in general this construction is not optimal at finite block lengths we further demonstrate that our main ump achievability bound can be obtained using coset codes which suggests a path to implementation of tractable ump codes

exact exponent for soft covering this paper establishes the exact exponents for the soft covering phenomenon of a memoryless channel under the total variation metric when random i i d and constant composition channel codes are used the exponents established herein are strict improvements in both directions on bounds found in the literature this complements the recent literature establishing the exact exponents under the relative entropy metric however the proof techniques have significant differences and thus neither result trivially implies the other the found exponents imply new and improved bounds for various problems that use soft covering as their achievability argument including new lower bounds for the resolvability exponent and the secrecy exponent in the wiretap channel

on the energy delay tradeoff in streaming data finite blocklength analysis this paper investigates basic trade offs between energy and delay in wireless communication systems using finite blocklength theory we first assume that data arrive in constant stream of bits which are put into packets and transmitted over a communications link our results show that depending on exactly how energy is measured in general energy depends on sqrt d 1 or sqrt d 1 log d where d is the delay this means that the energy decreases quite slowly with increasing delay furthermore to approach the absolute minimum of 1 59 db on energy bandwidth has to increase very rapidly much more than what is predicted by infinite blocklength theory we then consider the scenario when data arrive stochastically in packets and can be queued we devise a scheduling algorithm based on finite blocklength theory and develop bounds for the energy delay performance our results again show that the energy decreases quite slowly with increasing delay

strong secrecy from channel resolvability we analyze physical layer security based on the premise that the coding mechanism for secrecy over noisy channels is tied to the notion of channel resolvability instead of considering capacity based constructions which associate to each message a subcode that operates just below the capacity of the eavesdropper s channel we consider channel resolvability based constructions which associate to each message a subcode that operates just above the resolvability of the eavesdropper s channel building upon the work of csiszar and hayashi we provide further evidence that channel resolvability is a powerful and versatile coding mechanism for secrecy by developing results that hold for strong secrecy metrics and arbitrary channels specifically we show that at least for symmetric wiretap channels random capacity based constructions fail to achieve the strong secrecy capacity while channel resolvability based constructions achieve it we then leverage channel resolvability to establish the secrecy capacity region of arbitrary broadcast channels with confidential messages and a cost constraint for strong secrecy metrics finally we specialize our results to study the secrecy capacity of wireless channels with perfect channel state information csi mixed channels and compound channels with receiver csi as well as the secret key capacity of source models for secret key agreement by tying secrecy to channel resolvability we obtain achievable rates for strong secrecy metrics with simple proofs

on the enumerative nature of gomory s dual cutting plane method for 30 years after their invention half a century ago cutting planes for integer programs have been an object of theoretical investigations that had no apparent practical use when they finally proved their practical usefulness in the late eighties that happened in the framework of branch and bound procedures as an auxiliary tool meant to reduce the number of enumerated nodes to this day pure cutting plane methods alone have poor convergence properties and are typically not used in practice our reason for studying them is our belief that these negative properties can be understood and thus remedied only based on a thorough investigation of such procedures in their pure form in this paper the second in a sequence we address some important issues arising when designing a computationally sound pure cutting plane method we analyze the dual cutting plane procedure proposed by gomory in 1958 which is the first and most famous convergent cutting plane method for integer linear programming we focus on the enumerative nature of this method as evidenced by the relative computational success of its lexicographic version as documented in our previous paper on the subject and we propose new versions of gomory s cutting plane procedure with an improved performance in particular the new versions are based on enumerative schemes that treat the objective function implicitly and redefine the lexicographic order on the fly to mimic a sound branching strategy preliminary computational results are reported

linear programming based converses for finite blocklength lossy joint source channel coding a linear programming lp based framework is presented for obtaining converses for finite blocklength lossy joint source channel coding problems the framework applies for any loss criterion generalizes certain previously known converses and also extends to multi terminal settings the finite blocklength problem is posed equivalently as a nonconvex optimization problem and using a lift and project like method a close but tractable lp relaxation of this problem is derived lower bounds on the original problem are obtained by the construction of feasible points for the dual of the lp relaxation a particular application of this approach leads to new converses which recover and improve on the converses of kostina and verdu for finite blocklength lossy joint source channel coding and lossy source coding for finite blocklength channel coding the lp relaxation recovers the converse of polyanskiy poor and verdu and leads to a new improvement on the converse of wolfowitz showing thereby that our lp relaxation is asymptotically tight with increasing blocklengths for channel coding lossless source coding and joint source channel coding with the excess distortion probability as the loss criterion using a duality based argument a new converse is derived for finite blocklength joint source channel coding for a class of source channel pairs employing this converse the lp relaxation is also shown to be tight for all blocklengths for the minimization of the expected average symbolwise hamming distortion of a q ary uniform source over a q ary symmetric memoryless channel for any q in mathbb n the optimization formulation and the lift and project method are extended to networked settings and demonstrated by obtaining an improvement on a converse of zhou et al for the successive refinement problem for successively refinable source distortion measure triplets

a general formula for channel capacity a formula for the capacity of arbitrary single user channels without feedback not necessarily information stable stationary etc is proved capacity is shown to equal the supremum over all input processes of the input output inf information rate defined as the liminf in probability of the normalized information density the key to this result is a new converse approach based on a simple new lower bound on the error probability of m ary hypothesis tests among equiprobable hypotheses a necessary and sufficient condition for the validity of the strong converse is given as well as general expressions for spl epsiv capacity

first and second order asymptotics in covert communication we study the first and second order asymptotics of covert communication over binary input discrete memoryless channels for three different covertness metrics and under maximum probability of error constraint when covertness is measured in terms of the relative entropy between the channel output distributions induced with and without communication we characterize the exact first and second order asymptotics of the number of bits that can be reliably transmitted with a maximum probability of error less than epsilon and a relative entropy less than delta when covertness is measured in terms of the variational distance between the channel output distributions or in terms of the probability of missed detection for fixed probability of false alarm we establish the exact first order asymptotics and bound the second order asymptotics pulse position modulation achieves the optimal first order asymptotics for all three metrics as well as the optimal second order asymptotics for relative entropy the main conceptual contribution of this paper is to clarify how the choice of a covertness metric impacts the information theoretic limits of covert communications the main technical contribution underlying our results is a detailed expurgation argument to show the existence of a code satisfying the reliability and covertness criteria

an improved sphere packing bound for finite length codes over symmetric memoryless channels this paper derives an improved sphere packing isp bound for finite length error correcting codes whose transmission takes place over symmetric memoryless channels and the codes are decoded with an arbitrary list decoder we first review classical results i e the 1959 sphere packing sp59 bound of shannon for the gaussian channel and the 1967 sphere packing sp67 bound of shannon et al for discrete memoryless channels an improvement on the sp67 bound as suggested by valembois and fossorier is also discussed these concepts are used for the derivation of a new lower bound on the error probability of list decoding referred to as the isp bound which is uniformly tighter than the sp67 bound and its improved version the isp bound is applicable to symmetric memoryless channels and some of its applications are presented its tightness under maximum likelihood ml decoding is studied by comparing the isp bound to previously reported upper and lower bounds on the ml decoding error probability and also to computer simulations of iteratively decoded turbo like codes this paper also presents a technique which performs the entire calculation of the sp59 bound in the logarithmic domain thus facilitating the exact calculation of this bound for moderate to large block lengths without the need for the asymptotic approximations provided by shannon

divergence measures based on the shannon entropy a novel class of information theoretic divergence measures based on the shannon entropy is introduced unlike the well known kullback divergences the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved more importantly their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds these bounds are crucial in many applications of divergence measures the measures are also well characterized by the properties of nonnegativity finiteness semiboundedness and boundedness

near optimal finite length scaling for polar codes over large alphabets for any prime power q mori and tanaka introduced a family of q ary polar codes based on the q times q reed solomon polarization kernels for transmission over a q ary erasure channel they also derived a closed form recursion for the erasure probability of each effective channel in this paper we use that expression to analyze the finite length scaling of these codes on the q ary erasure channel with erasure probability epsilon in 0 1 our primary result is that for any gamma 0 and delta 0 there is a q 0 such that for all q geq q 0 the fraction of effective channels with erasure rate at most n gamma is at least 1 epsilon o n 1 2 delta where n q n is the blocklength since this fraction cannot be larger than 1 epsilon o n 1 2 this establishes near optimal finite length scaling for this family of codes our approach can be seen as an extension of a similar analysis for binary polar codes by hassani alishahi and urbanke a similar analysis is also considered for q ary polar codes with m times m polarizing matrices this separates the effect of the alphabet size from the effect of the matrix size if the polarizing matrix at each stage is drawn independently and uniformly from the set of invertible m times m matrices then the linear operator associated with the lyapunov function analysis can be written in the closed form to prove near optimal scaling for polar codes with fixed q as m increases however two technical obstacles remain thus we conclude by stating two concrete mathematical conjectures that if proven would imply near optimal scaling for fixed q

a linear program for the finite block length converse of polyanskiy poor verdu via nonsignaling codes motivated by recent work on entanglement assisted codes for sending messages over classical channels the larger easily characterized class of nonsignaling codes is defined analyzing the optimal performance of these codes yields an alternative proof of the finite block length converse of polyanskiy poor and verdu and shows that they achieve this converse this provides an explicit formulation of the converse as a linear program which has some useful features for discrete memoryless channels it is shown that nonsignaling codes attain the channel capacity with zero error probability if and only if the dispersion of the channel is zero

information spectrum approach to second order coding rate in channel coding in this paper second order coding rate of channel coding is discussed for general sequence of channels the optimum second order transmission rate with a constant error constraint epsiv is obtained by using the information spectrum method we apply this result to the discrete memoryless case the discrete memoryless case with a cost constraint the additive markovian case and the gaussian channel case with an energy constraint we also clarify that the gallager bound does not give the optimum evaluation in the second order coding rate

on converse bounds for classical communication over quantum channels we explore several new converse bounds for classical communication over quantum channels in both the one shot and asymptotic regimes first we show that the matthews wehner meta converse bound for entanglement assisted classical communication can be achieved by activated no signaling assisted codes suitably generalizing a result for classical channels second we derive a new efficiently computable meta converse on the amount of classical information unassisted codes can transmit over a single use of a quantum channel as applications we provide a finite resource analysis of classical communication over quantum erasure channels including the second order and moderate deviation asymptotics third we explore the asymptotic analogue of our new meta converse the upsilon information of the channel we show that its regularization is an upper bound on the classical capacity which is generally tighter than the entanglement assisted capacity and other known efficiently computable strong converse bounds for covariant channels we show that the upsilon information is a strong converse bound

a lower bound on the probability of error in multihypothesis testing consider two random variables x and y where x is finitely or countably infinitely valued and where y is arbitrary let spl epsiv denote the minimum probability of error incurred in estimating x from y it is shown that spl epsiv spl ges sub 0 spl les spl alpha spl les 1 sup sup 1 spl alpha p spl pi x y spl les spl alpha where spl pi x y denotes the posterior probability of x given y this bound finds information theoretic applications in the proof of converse channel coding theorems it generalizes and strengthens previous lower bounds due to shannon and to verdu and han 1994

the dispersion of nearest neighbor decoding for additive non gaussian channels we study the second order asymptotics of information transmission using random gaussian codebooks and nearest neighbor decoding over a power limited stationary memoryless additive non gaussian noise channel we show that the dispersion term depends on the non gaussian noise only through its second and fourth moments thus complementing the capacity result lapidoth 1996 which depends only on the second moment furthermore we characterize the second order asymptotics of point to point codes over k sender interference networks with non gaussian additive noise specifically we assume that each user s codebook is gaussian and that nn decoding is employed i e that interference from the k 1 unintended users gaussian interfering signals is treated as noise at each decoder we show that while the first order term in the asymptotic expansion of the maximum number of messages depends on the power of the interfering codewords only through their sum this does not hold for the second order term

on the evaluation of the polyanskiy poor verdu converse bound for finite block length coding in awgn a tight converse bound to the channel coding rate in the finite block length regime and under additive white gaussian noise conditions was recently proposed by polyanskiy poor verdu ppv the bound is a generalization of a number of other classical results and it was also claimed to be equivalent to shannon s 1959 cone packing bound in this paper we investigate methods for a reliable numerical evaluation of the bound which is troublesome even for not too large values of the block length n by compactly expressing the polyanskiy poor and verdu ppv converse bound in terms of non central chi squared distributions and by evaluating those through an integral expression and a corresponding series expansion which exploit a method proposed by temme as a result a robust evaluation method and new insights on the bound s asymptotics as well as new approximate expressions are obtained