variable length lossy source code using a constrained random number generator a variable length lossy source code is introduced with a rate distortion pair close to the rate distortion function random numbers that satisfy a condition specified by a function and its value are used to construct a stochastic encoder the proof of the theorem is based on the balanced coloring property of an ensemble of functions since an ensemble of systematic sparse matrices has this property we can construct a tractable code for a memoryless source some algorithms for implementing the code are introduced and compared by simulation

the entropy theorem via coding bounds the entropy theorem also known as the shannon mcmillan breiman theory or the asymptotic equipartition theorem asserts that for a stationary ergodic finite alphabet process the sequence 1 n log p x sub 1 sup n converges almost surely to the entropy rate h of the process the entropy theorem has been used to establish asymptotic bounds on the performance of noiseless codes here the coding theorems are established without using the entropy theorem and the coding theorems are then used to prove the entropy theorem the principle feature is the direct use of coding ideas to obtain the entropy theorem

a survey of the theory of source coding with a fidelity criterion the purpose of this paper is threefold 1 to acquaint the reader with the types of problems that have been considered in the area of source coding with a fidelity criterion 2 to survey results that have been obtained on these problems and 3 to outline future research trends in the area

characterizing the statistical properties of mutual information in mimo channels we consider gaussian multiple input multiple output mimo frequency selective spatially correlated fading channels assuming that the channel is unknown at the transmitter and perfectly known at the receiver for gaussian codebooks using results from multivariate statistics we derive an analytical expression for a tight lower bound on the ergodic capacity of such channels at any signal to noise ratio snr we show that our bound is tighter than previously reported analytical lower bounds and we proceed to analytically quantify the impact of spatial fading correlation on ergodic capacity based on a closed form approximation of the variance of mutual information in correlated flat fading mimo channels we provide insights into the multiplexing diversity tradeoff for gaussian code books furthermore for a given total number of antennas we consider the problem of finding the optimal ergodic capacity maximizing number of transmit and receive antennas and we reveal the snr dependent nature of the maximization strategy finally we present numerical results and comparisons between our capacity bounds and previously reported bounds

results on the wess zumino consistency condition for arbitrary lie algebras the so called covariant poincare lemma on the induced cohomology of the space time exterior derivative in the cohomology of the gauge part of the brst differential is extended to cover the case of arbitrary nonreductive lie algebras as a consequence the general solution of the wess zumino consistency condition with a nontrivial descent can for arbitrary super lie algebras be computed in the small algebra of the one form potentials the ghosts and their exterior derivatives for particular lie algebras that are the semidirect sum of a semisimple lie subalgebra with an ideal a theorem by hochschild and serre is used to characterize more precisely the cohomology of the gauge part of the brst differential in the small algebra in the case of an abelian ideal this leads to a complete solution of the wess zumino consistency condition in this space as an application the consistent deformations of 2 1 dimensional chern simons theory based on iso 2 1 are rediscussed

the empirical distribution of rate constrained source codes let x x sub 1 be a stationary ergodic finite alphabet source x sup n denote its first n symbols and y sup n be the codeword assigned to x sup n by a lossy source code the empirical kth order joint distribution q spl circ sup k x sup n y sup n spl rceil x sup k y sup k is defined as the frequency of appearances of pairs of k strings x sup k y sup k along the pair x sup n y sup n our main interest is in the sample behavior of this random distribution letting i q sup k denote the mutual information i x sup k y sup k when x sup k y sup k spl sim q sup k we show that for any sequence of lossy source code s of rate spl les r lim sup sub n spl rarr spl infin 1 k i q spl circ sup k x sup n y sup n spl rfloor spl les r 1 k h x sub 1 sup k h x a s where h x denotes the entropy rate of x this is shown to imply for a large class of sources including all independent and identically distributed i i d sources and all sources satisfying the shannon lower bound with equality that for any sequence of codes which is good in the sense of asymptotically attaining a point on the rate distortion curve q spl circ sup k x sup n y sup n spl rfloor spl rarr sup d p x sup k y sup k a s whenever p sub x sup k sub y sup k is the unique distribution attaining the minimum in the definition of the kth order rate distortion function consequences of these results include a new proof of kieffer s sample converse to lossy source coding as well as performance bounds for compression based denoisers

an on line universal lossy data compression algorithm via continuous codebook refinement ii optimality for phi mixing source models for pt i see ibid vol 42 no 3 p 803 21 1996 two versions of the gold washing data compression algorithm one with codebook innovation interval and the other with finitely many codebook innovations are considered the version of the gold washing algorithm with codebook innovation interval k is a variant of the gold washing algorithm such that the codebook is innovated once every k 1 source words during the process of encoding the entire source it is demonstrated that when this version of the gold washing algorithm is applied to encode a stationary spl phi mixing source the expected distortion performance converges to the distortion rate function of the source as the codebook length goes to infinity furthermore if the source to be encoded is a markov source or a finite state source then the corresponding sample distortion performance converges almost surely to the distortion rate function the version of the gold washing algorithm with finitely many codebook innovations is a variant of the gold washing algorithm in which after finitely many codebook innovations the codebook is held fixed and reused to encode the forthcoming source sequence block by block similar results are shown for this version of the gold washing algorithm in addition the convergence speed of the algorithm is discussed

exactly soluble diffeomorphism invariant theories a class of diffeomorphism invariant theories is described for which the hilbert space of quantum states can be explicitly constructed these theories can be formulated in any dimension and include witten s solution to 2 1 dimensional gravity as a special case higher dimensional generalizations exist which start with an action similar to the einstein action inn dimensions many of these theories do not involve a spacetime metric and provide examples of topological quantum field theories one is a version of yang mills theory in which the only quantum states ons3 r are the vacua finally it is shown that the three dimensional chern simons theory which witten has shown is intimately connected with knot theory arises naturally from a four dimensional topological gauge theory

distortion program size complexity with respect to a fidelity criterion and rate distortion function a novel concept of distortion kolmogorov chaitin complexity is proposed bearing analogy to ordinary kolmogorov chaitin complexity that represents the information content of a single message distortion kolmogorov chaitin complexity can be regarded as the amount of information about a single message that must be conveyed in order to reproduce it with a bounded distortion this explanation is justified by the properties of distortion kolmogorov chaitin complexity and the equivalences between distortion kolmogorov chaitin complexity and the rate distortion function which are analogous to those between ordinary kolmogorov chaitin complexity and the shannon entropy some implications for universal almost sure data compression are also discussed

strong converses in source coding relative to a fidelity criterion two strong converses are obtained for an abstract alphabet stationary ergodic source coded relative to an appropriate fidelity criterion it is shown that given a distortion rate point d r that lies below the rate distortion curve 1 block codes that operate at rate level r must encode sample source blocks at a rate exceeding d with probability tending to one as the block length tends to infinity and 2 variable rate codes that operate at distortion level d must encode sample source blocks at a rate exceeding r with probability tending to one as the block length tends to infinity the previously known weak converses guarantee only that the indicated probabilities remain bounded away from zero as block length tends to infinity the proofs of the strong converses involve sample converses in source coding theory

asymptotic properties of data compression and suffix trees recently wyner and ziv see ibid vol 35 p 1250 8 1989 have proved that the typical length of a repeated subword found within the first n positions of a stationary ergodic sequence is 1 h log n in probability where h is the entropy of the alphabet this finding was used to obtain several insights into certain universal data compression schemes most notably the lempel ziv data compression algorithm wyner and ziv have also conjectured that their result can be extended to a stronger almost sure convergence in this paper we settle this conjecture in the negative in the so called right domain asymptotic that is during a dynamic phase of expanding the data base we prove under an additional assumption involving mixing conditions that the length of a typical repeated subword oscillates almost surely a s between 1 h sub 1 log n and 1 h sub 2 log n where d

pointwise redundancy in lossy data compression and universal lossy data compression we characterize the achievable pointwise redundancy rates for lossy data compression at a fixed distortion level pointwise redundancy refers to the difference between the description length achieved by an nth order block code and the optimal nr d bits for memoryless sources we show that the best achievable redundancy rate is of order o spl radic n in probability this follows from a second order refinement to the classical source coding theorem in the form of a one sided central limit theorem moreover we show that along almost any source realization the description lengths of any sequence of block codes operating at distortion level d exceed nr d by at least as much as c spl radic nloglogn infinitely often corresponding direct coding theorems are also given showing that these rates are essentially achievable the above rates are in sharp contrast with the expected redundancy rates of order o log n reported by various authors our approach is based on showing that the compression performance of an arbitrary sequence of codes is essentially bounded below by the performance of shannon s random code we obtain partial generalizations of the above results for arbitrary sources with memory and we prove lossy analogs of barron s lemma barron 1985

strongly consistent code based identification and order estimation for constrained finite state model classes observations are made of data generated by a stationary ergodic finite alphabet information source according to an unknown statistical model two modeling problems the identification problem and the order estimation problem are considered it is required that the given model class in the identification problem and each given model class in the order estimation problem be a constrained finite state model class which is a type that includes many model classes of information theoretic interest strongly consistent decision rules are exhibited in both the identification problem and the order estimation problem the decision rules are code based in that a model class is chosen based on how well a certain code for that class encodes the observed data the code used for a model class is based on the maximum likelihood code for that class and asymptotic code performance is gauged by means of a key property of divergence rate distance

on information divergence measures and a unified typicality strong typicality which is more powerful for theorem proving than weak typicality can be applied to finite alphabets only while weak typicality can be applied to countable alphabets in this paper the relation between typicality and information divergence measures is discussed the new definition of information divergence measure in this paper leads to the definition of a unified typicality for finite or countably infinite alphabets which is stronger than both weak typicality and strong typicality unified typicality retains the asymptotic equipartition property and the structural properties of strong typicality and it can potentially be used to generalize those theorems which are previously established by strong typicality to countable alphabets the applications in rate distortion theory and multisource network coding problems are discussed

a sandwich proof of the shannon mcmillan breiman theorem let x t be a stationary ergodic process with distribution p admitting densities p x 0 ldots x n 1 relative to a reference measure m that is finite order markov with stationary transition kernel let i m p denote the relative entropy rate then n 1 log p x 0 ldots x n 1 rightarrow i m p mathrm a s p we present an elementary proof of the shannon mcmillan breiman theorem and the preceding generalization obviating the need to verify integrability conditions and also covering the case i m p infty a sandwich argument reduces the proof to direct applications of the ergodic theorem

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

the context tree weighting method extensions first we modify the basic binary context tree weighting method such that the past symbols x sub 1 d x sub 2 d x sub 0 are not needed by the encoder and the decoder then we describe how to make the context tree depth d infinite which results in optimal redundancy behavior for all tree sources while the number of records in the context tree is not larger than 2t 1 here t is the length of the source sequence for this extended context tree weighting algorithm we show that with probability one the compression ratio is not larger than the source entropy for source sequence length t spl rarr spl infin for stationary and ergodic sources

chern simons field theories with non semisimple gauge group of symmetry the subject of this work is a class of chern simons field theories with non semisimple gauge group which may well be considered as the most straightforward generalization of an abelian chern simons field theory as a matter of fact these theories which are characterized by a non semisimple group of gauge symmetry have cubic interactions like those of non abelian chern simons field theories but are free from radiative corrections moreover at the tree level in the perturbative expansion there are only two connected tree diagrams corresponding to the propagator and to the three vertex originating from the cubic interaction terms for such theories it is derived here a set of brst invariant observables which lead to metric independent amplitudes the vacuum expectation values of these observables can be computed exactly from their expressions it is possible to isolate the gauss linking number and an invariant of the milnor type which describes the topological relations among three or more closed curves

on the performance of data compression algorithms based upon string matching lossless and lossy data compression algorithms based on string matching are considered in the lossless case a result of wyner and ziv 1989 is extended in the lossy case a data compression algorithm based on approximate string matching is analyzed in the following two frameworks 1 the database and the source together form a markov chain of finite order 2 the database and the source are independent with the database coming from a markov model and the source from a general stationary ergodic model in either framework it is shown that the resulting compression rate converges with probability one to a quantity computable as the infimum of an information theoretic functional over a set of auxiliary random variables the quantity is strictly greater than the rate distortion function of the source except in some symmetric cases in particular this result implies that the lossy algorithm proposed by steinberg and gutman 1993 is not optimal even for memoryless or markov sources

fixed length lossy compression in the finite blocklength regime this paper studies the minimum achievable source coding rate as a function of blocklength n and probability that the distortion exceeds a given level d tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength for stationary memoryless sources with separable distortion the minimum rate achievable is shown to be closely approximated by r d v d n q 1 where r d is the rate distortion function v d is the rate dispersion a characteristic of the source which measures its stochastic variability and q 1 is the inverse of the standard gaussian complementary cumulative distribution function

on universal types the universal type class of a sequence xn is defined in analogy to the notion underlying the classical method of types two sequences of the same length are said to be of the same universal lz type if and only if they yield the same dictionary or equivalently parsing tree in the incremental parsing of ziv and lempel 1978 it is shown that for any finite order k the variational distance between the kth order empirical probability distributions of two sequences of the same universal type vanishes as the sequence length tends to infinity consequently for any k and any kth order probability assignment the difference between the normalized logarithms of the probabilities assigned to two sequences of the same universal type also vanishes asymptotically the size of a universal type class is studied and it is shown that its asymptotic behavior parallels that of the conventional counterpart with the lz78 code length playing the role of the empirical entropy the number of universal types for sequences of length n is estimated and shown to be of the form exp 1 o 1 gamman logn for a well characterized constant gamma algorithms for enumerating the sequences in a universal type class and for drawing a sequence from the class with uniform probability are described as an application the problem of universal simulation of individual sequences is considered a sequence drawn with uniform probability from the universal type class of xn is an optimal simulation of xn in a well defined mathematical sense

twice universal simulation of markov sources and individual sequences the problem of universal simulation given a training sequence is studied both in a stochastic setting and for individual sequences in the stochastic setting the training sequence is assumed to be emitted by a markov source of unknown order extending previous work where the order is assumed known and leading to the notion of twice universal simulation a simulation scheme which partitions the set of sequences of a given length into classes is proposed for this setting and shown to be asymptotically optimal this partition extends the notion of type classes to the twice universal setting in the individual sequence scenario the same simulation scheme is shown to generate sequences which are statistically similar in a strong sense to the training sequence for statistics of any order while essentially maximizing the uncertainty on the output

the ergodic and entropy theorems revisited a new proof of birkhoff s ergodic theorem is given using a sample path covering idea an idea created by ornstein and weiss in their extension of the information convergence theorem to random fields a sketch of the ornstein weiss proof for processes is included in the appendix

fixed slope universal lossy data compression corresponding to any lossless codeword length function l three universal lossy data compression schemes are presented one is with a fixed rate another is with a fixed distortion and a third is with a fixed slope the former two universal lossy data compression schemes are the generalization of yang kieffer s see ibid vol 42 no 1 p 239 45 1995 results to the general case of any lossless codeword length function l whereas the third is new in the case of fixed slope spl lambda 0 our universal lossy data compression scheme works as follows for any source sequence x sup n of length n the encoder first searches for a reproduction sequence y sup n of length n which minimizes a cost function n sup 1 l y sup n spl lambda spl rho sub n x sup n y sup n over all reproduction sequences of length n and then encodes x sup n into the binary codeword of length l y sup n associated with y sup n via the lossless codeword length function l where spl rho sub n x sup n y sup n is the distortion per sample between x sup n and y sup n under some mild assumptions on the lossless codeword length function l it is shown that when this fixed slope data compression scheme is applied to encode a stationary ergodic source the resulting encoding rate per sample and the distortion per sample converge with probability one to r sub spl lambda and d sub spl lambda respectively where d sub spl lambda r sub spl lambda is the point on the rate distortion curve at which the slope of the rate distortion function is spl lambda this result holds particularly for the arithmetic codeword length function and lempel ziv codeword length function the main advantage of this fixed slope universal lossy data compression scheme over the fixed rate fixed distortion universal lossy data compression scheme lies in the fact that it converts the encoding problem to a search problem through a trellis and then permits one to use some sequential search algorithms to implement it simulation results show that this fixed slope universal lossy data compression scheme combined with a suitable search algorithm is promising

data throughputs of noise limited mimo cellular systems we present the closed form approximated expressions of the cumulative distribution function of the spectral efficiency for noise limited multiple input multiple output mimo cellular systems with minimum mean square error based ordered successive interference cancellation mmse osic or zero forcing zf receivers in a composite channel model which includes both rayleigh and log normal fading when the users have independent and identically distributed i i d channels we analyze the spectral efficiency of the mimo time division multiple access tdma systems with maximum data rate scheduler numerical experiments show that these results can consistently produce highly accurate results

a topological field theory for milnor s triple linking number the subject of this work is a topological field theory with a non semisimple gauge group and local observables that are both metric independent and gauge invariant the observables consist of the holonomies of connections around three arbitrary loops and contain not only three dimensional vector fields but also a set of one dimensional scalar fields the gauge invariance is achieved by requiring non trivial gauge transformations in the scalar field sector the importance of theories of this kind in the statistical physics of linked polymer rings is discussed the presence of fields of different dimensions is the novel feature of the proposed model which retains many properties of the non abelian chern simons field theories and has the advantage to be exactly solvable its only one relevant amplitude has been computed here in closed form from its expression it is possible to isolate the milnor s triple linking number a link invariant which has physical applications in the dynamics of fluids and solar magnetohydrodynamics

second order noiseless source coding theorems shannon s celebrated source coding theorem can be viewed as a one sided law of large numbers we formulate second order noiseless source coding theorems for the deviation of the codeword lengths from the entropy for a class of sources that includes markov chains we prove a one sided central limit theorem and a law of the iterated logarithm

universal entropy estimation via block sorting in this correspondence we present a new universal entropy estimator for stationary ergodic sources prove almost sure convergence and establish an upper bound on the convergence rate for finite alphabet finite memory sources the algorithm is motivated by data compression using the burrows wheeler block sorting transform bwt by exploiting the property that the bwt output sequence is close to a piecewise stationary memoryless source we can segment the output sequence and estimate probabilities in each segment experimental results show that our algorithm outperforms lempel ziv lz string matching based algorithms

writing on dirty paper corresp a channel with output y x s z is examined the state s sim n 0 qi and the noise z sim n 0 ni are multivariate gaussian random variables i is the identity matrix the input x in r n satisfies the power constraint l n sum i 1 n x i 2 leq p if s is unknown to both transmitter and receiver then the capacity is frac 1 2 ln 1 p n q nats per channel use however if the state s is known to the encoder the capacity is shown to be c ast frac 1 2 ln 1 p n independent of q this is also the capacity of a standard gaussian channel with signal to noise power ratio p n therefore the state s does not affect the capacity of the channel even though s is unknown to the receiver it is shown that the optimal transmitter adapts its signal to the state s rather than attempting to cancel it

simple universal lossy data compression schemes derived from the lempel ziv algorithm two universal lossy data compression schemes one with fixed rate and the other with fixed distortion are presented based on the well known lempel ziv algorithm in the case of fixed rate r the universal lossy data compression scheme works as follows first pick a codebook b sub n consisting of all reproduction sequences of length n whose lempel ziv codeword length is spl les nr and then use b sub n to encode the entire source sequence n block by n block this fixed rate data compression scheme is universal in the sense that for any stationary ergodic source or for any individual sequence the sample distortion performance as n spl rarr spl infin is given almost surely by the distortion rate function a similar result is shown in the context of fixed distortion lossy source coding

on the redundancy of lossy source coding with abstract alphabets the redundancy problem of lossy source coding with abstract source and reproduction alphabets is considered for coding at a fixed rate level it is shown that for any fixed rate r 0 and any memoryless abstract alphabet source p satisfying some mild conditions there exists a sequence c sub n sub n 1 sup spl infin of block codes at the rate r such that the distortion redundancy of c sub n defined as the difference between the performance of c sub n and the distortion rate function d p r of p is upper bounded by spl part d p r spl part r ln n 2n o ln n n for coding at a fixed distortion level it is demonstrated that for any d 0 and any memoryless abstract alphabet source p satisfying some mild conditions there exists a sequence c sub n sub n 1 sup spl infin of block codes at the fixed distortion d such that the rate redundancy of c sub n defined as the difference between the performance of c sub n and the rate distortion function r p d of p is upper bounded by 7ln n 6n o ln n n these results strengthen the traditional berger s 1968 1971 abstract alphabet source coding theorem and extend the positive redundancy results of zhang yang and wei see ibid vol 43 no 1 p 71 91 1997 and ibid vol 42 p 803 21 1996 on lossy source coding with finite alphabets and the redundancy result of wyner see ibid vol 43 p 1452 64 1997 on block coding of memoryless gaussian sources

sequential codes lossless compression of individual sequences and kolmogorov complexity a general class of sequential codes for lossless compression of individual sequences on a finite alphabet is defined including many types of codes that one would want to implement the principal requirement for membership in the class is that the encoding and decoding operations be performable on a computer the opta function for the class of codes is then considered which is the function that assigns to each individual sequence the infimum of the rates at which the sequence can be compressed over this class of sequential codes two results about the opta function are obtained 1 it is shown that any sequential code in the class compresses some individual sequence at a rate strictly greater than the rate for that sequence given by the opta function and 2 it is shown that the opta function takes a value strictly greater than that of the kolmogorov 1965 complexity rate function for some individual sequences

simplistic universal coding a conceptually simple coding method may be described as follows the source sequence is parsed into fixed length blocks and a list of these blocks is placed in a dictionary in the lossless case the dictionary is transmitted and each successive block is encoded by giving its dictionary location in the lossy case the smallest collection of blocks such that every member of the dictionary is within distortion spl delta of the collection is determined this codebook is transmitted and each successive block is encoded by giving the location of a member of the code within spl delta of it we show that by optimizing on the block length this method is universal that is for any ergodic process it achieves entropy in the limit in the lossless case and the rate distortion function r spl delta in the lossy case

sample converses in source coding theory the rate and distortion performance of a sequence of codes along a sample sequence of symbols generated by a stationary ergodic information source are studied two results are obtained 1 the source sample sequence is encoded by an arbitrary sequence of block codes which operate at a fixed rate level r and a sample converse is obtained which states that with probability one the lower limit of the code sample distortions is lower bounded by d r the value of the distortion rate function at r 2 the source sample sequence is encoded by an arbitrary sequence of variable rate codes which operate at a fixed distortion level d and a sample converse is obtained which states that with probability one the lower limit of the code sample rates is lower bounded by r d the value of the rate distortion function at d a novel ergodic theorem is used to obtain both sample converses

an on line universal lossy data compression algorithm via continuous codebook refinement i basic results a new on line universal lossy data compression algorithm is presented for finite memoryless sources with unknown statistics its performance asymptotically approaches the fundamental rate distortion limit the codebook is generated on the fly and continuously adapted by simple rules there is no separate codebook training or codebook transmission candidate codewords are randomly generated according to an arbitrary and possibly suboptimal distribution through a carefully designed gold washing or information theoretic sieve mechanism good codewords and only good codewords are promoted to permanent status with high probability we also determine the rate at which our algorithm approaches the fundamental limit

critical behavior in lossy source coding the following critical phenomenon was recently discovered when a memoryless source is compressed using a variable length fixed distortion code the fastest convergence rate of the pointwise compression ratio to r d is either o spl radic n or o log n we show it is always o spl radic n except for discrete uniformly distributed sources

the interactions between ergodic theory and information theory information theorists frequently use the ergodic theorem likewise entropy concepts are often used in information theory recently the two subjects have become partially intertwined as deeper results from each discipline find use in the other a brief history of this interaction is presented in this paper together with a more detailed look at three areas of connection namely recurrence theory blowing up bounds and direct sample path methods

how sampling reveals a process a series of observations xi 1 xi 2 xi 3 ldots is presented to us and at each time n when we have observed the first n of them we are called upon to give our guess for what stochastic process produced the data a universal scheme is given which for any bernoulli process not necessarily independent gives a sequence of processes that converges in a strong sense the bar d metric to the real process in addition to this main result many others are given which put it into proper perspective in particular it is shown that in a certain sense the class of bernoulli processes is the largest one for which such a universal scheme is possible

optimal lossless data compression non asymptotics and asymptotics this paper provides an extensive study of the behavior of the best achievable rate and other related fundamental limits in variable length strictly lossless compression in the non asymptotic regime the fundamental limits of fixed to variable lossless compression with and without prefix constraints are shown to be tightly coupled several precise quantitative bounds are derived connecting the distribution of the optimal code lengths to the source information spectrum and an exact analysis of the best achievable rate for arbitrary sources is given fine asymptotic results are proved for arbitrary not necessarily prefix compressors on general mixing sources nonasymptotic explicit gaussian approximation bounds are established for the best achievable rate on markov sources the source dispersion and the source varentropy rate are defined and characterized together with the entropy rate the varentropy rate serves to tightly approximate the fundamental nonasymptotic limits of fixed to variable compression for all but very small block lengths

on the redundancy of the fixed database lempel ziv algorithm for spl phi mixing sources the redundancy problem of the fixed database lempel ziv 1977 algorithm is considered it is demonstrated that for a class of spl phi mixing sources which includes markov sources unifilar sources and finite state sources as special cases the redundancy of the fixed database lempel ziv algorithm with database size n is lower bounded by h loglogn logn o logloglogn logn and upper bounded by 2h loglogn logn o logloglogn logn where h is the entropy rate of the source the method of proof is new and uses the concept of variable length to variable length codes