efficient scalar quantization of exponential and laplacian random variables this paper presents solutions to the entropy constrained scalar quantizer ecsq design problem for two sources commonly encountered in image and speech compression applications sources having the exponential and laplacian probability density functions we use the memoryless property of the exponential distribution to develop a new noniterative algorithm for obtaining the optimal quantizer design we show how to obtain the optimal ecsq either with or without an additional constraint on the number of levels in the quantizer in contrast to prior methods which require a multidimensional iterative solution of a large number of nonlinear equations the new method needs only a single sequence of solutions to one dimensional nonlinear equations in some laplacian cases one additional two dimensional solution is needed as a result the new method is orders of magnitude faster than prior ones we show that as the constraint on the number of levels in the quantizer is relaxed the optimal ecsq becomes a uniform threshold quantizer utq for exponential but not for laplacian sources we then further examine the performance of the utq and optimal ecsq and also investigate some interesting alternatives to the utq including a uniform reconstruction quantizer urq and a constant dead zone ratio quantizer cdzrq

minimax renyi redundancy the redundancy for universal lossless compression of discrete memoryless sources in campbell s setting is characterized as a minimax renyi divergence which is shown to be equal to the maximal alpha mutual information via a generalized redundancy capacity theorem special attention is placed on the analysis of the asymptotics of minimax renyi divergence which is determined up to a term vanishing in blocklength

meta fibonacci codes efficient universal coding of natural numbers in this paper we address the problem of the universal coding of natural numbers a new numeration system is introduced which is based on variable r meta fibonacci sequences and it is a generalization of the zeckendorf numeration system this new numeration system is used to construct binary prefix free uniquely decodable universal codes called meta fibonacci codes the main advantage of these codes is that they are parametrized by a sequence of numbers the sequence o by controlling the growth of the values of this sequence we can control the length of the code word this means that we can provide a general framework for building efficient universal coders for natural numbers such framework is applied to the upper bounds of the code word length defined by leung yan cheong and cover 1978 levenshtein 1968 and ahlswede 1997 there is no other code meeting these bounds in each case we build meta fibonacci codes and demonstrate that the upper bound of their code word length is satisfied up to an additive constant thereby solving these open problems the framework may be applied to other upper bounds that satisfy kraft inequality

embedded image coding using zerotrees of wavelet coefficients the embedded zerotree wavelet algorithm ezw is a simple yet remarkably effective image compression algorithm having the property that the bits in the bit stream are generated in order of importance yielding a fully embedded code the embedded code represents a sequence of binary decisions that distinguish an image from the null image using an embedded coding algorithm an encoder can terminate the encoding at any point thereby allowing a target rate or target distortion metric to be met exactly also given a bit stream the decoder can cease decoding at any point in the bit stream and still produce exactly the same image that would have been encoded at the bit rate corresponding to the truncated bit stream in addition to producing a fully embedded bit stream the ezw consistently produces compression results that are competitive with virtually all known compression algorithms on standard test images yet this performance is achieved with a technique that requires absolutely no training no pre stored tables or codebooks and requires no prior knowledge of the image source the ezw algorithm is based on four key concepts 1 a discrete wavelet transform or hierarchical subband decomposition 2 prediction of the absence of significant information across scales by exploiting the self similarity inherent in images 3 entropy coded successive approximation quantization and 4 universal lossless data compression which is achieved via adaptive arithmetic coding

a strong version of the redundancy capacity theorem of universal coding the capacity of the channel induced by a given class of sources is well known to be an attainable lower bound on the redundancy of universal codes with respect to this class both in the minimax sense and in the bayesian maximin sense we show that this capacity is essentially a lower bound also in a stronger sense that is for most sources in the class this result extends rissanen s 1984 1986 lower bound for parametric families we demonstrate the applicability of this result in several examples e g parametric families with growing dimensionality piecewise fixed sources arbitrarily varying sources and noisy samples of learnable functions finally we discuss implications of our results to statistical inference

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

sublattice synchronization of chaotic networks with delayed couplings synchronization of chaotic units coupled by their time delayed variables is investigated analytically a type of cooperative behavior is found sublattice synchronization although the units of one sublattice are not directly coupled to each other they completely synchronize without time delay the chaotic trajectories of different sublattices are only weakly correlated but not related by generalized synchronization nevertheless the trajectory of one sublattice is predictable from the complete trajectory of the other one the spectra of lyapunov exponents are calculated analytically in the limit of infinite delay times and phase diagrams are derived for different topologies

fast variable run length coding for embedded progressive wavelet based image compression run length coding has attracted much attention in wavelet based image compression because of its simplicity and potentially low complexity the main drawback is the inferior rd performance compared to the state of the art coder spiht in this paper we concentrate on the embedded progressive run length code of tian and wells 1996 1998 we consider significance sequences drawn from the scan in the dominant pass it turns out that self similar curves for scanning the dominant pass increase the compression efficiency significantly this is a consequence of the correlation of direct neighbors in the wavelet domain this dependence can be better exploited by using groups of coefficients similar to the spiht algorithm this results in a new and very fast coding algorithm which shows performance similar to the state of the art coder spiht but with lower complexity and small and fixed memory overhead

minlab minimum noise structure for ladder based biorthogonal filter banks we introduce a minimum noise structure for ladder based biorthogonal minlab filter bank the minimum noise structure ensures that the quantization error has a unity noise gain even though the filter bank is biorthogonal the coder has a very low design and implementation cost the perfect reconstruction property is structurally preserved optimal bit allocation and coding gain formulas are derived we show that the coding gain of the optimal minlab coder is always greater than or equal to unity for both ar 1 process and ma 1 process the minlab coder with two taps has a higher coding gain than the optimal orthonormal coder with an infinite number of taps in addition to its superior decorrelation ability it has many other desired features that make it a potentially valuable and attractive alternative to the orthonormal coder especially for a high fidelity compression

a theoretical high rate analysis of causal versus unitary online transform coding backward adaptive or online transform coding tc of gaussian sources is investigated the karhunen loe spl grave ve transform klt unitary approach and the causal transform ct causal approach are compared in this context when the covariance matrix r sub x of the source is used in the tc scheme klt and ct present similar coding gains at high rates the aim of this study is to model analytically the behavior of these two coding structures when the ideal tc scheme gets perturbed that is when only a perturbed value r sub x spl delta r is known at the encoder in the online tc schemes considered here this estimate is used to compute both the transform and the bit assignment spl delta r is caused by two noise sources estimation noise finite set of available data at the encoder and quantization noise quantized data at the decoder furthermore not only does the transformation itself get perturbed but also the bit assignment does as well in this framework theoretical expressions for the coding gains in both the unitary and the causal cases are derived under the high rate assumption

asymptotic average redundancy of huffman and other block codes we study asymptotically the redundancy of huffman and other codes it has been known from the inception of the huffman 1952 code that in the worst case its redundancy defined as the excess of the code length over the optimal ideal code length is not more than one however to the best of our knowledge no precise asymptotic results have been reported in literature thus far we consider here a memoryless binary source generating a sequence of length n distributed as binomial n p with p being the probability of emitting 0 based on the results of stubley 1994 we prove that for p 1 2 1 m 1 2 sup 1 m 2 sup m o spl rho sup n spl alpha n m rational where spl alpha log sub 2 1 p p and spl beta log sub 2 1 p spl rho x x is the fractional part of x the appearance of the fractal like function explains the erratic behavior of the huffman redundancy and its resistance to succumb to a precise analysis as a side result we prove that the average redundancy of the shannon block code is as n spl rarr spl infin r sub n sup s 1 2 o 1 spl alpha irrational 1 2 1 m 1 2 o spl rho sup n spl alpha n m rational where spl rho 1 finally we derive the redundancy of the golomb 1966 code for the geometric distribution which can be viewed as a special case of the huffman and shannon codes golomb s code redundancy has only oscillating behavior i e there is not convergent mode these findings are obtained by analytic methods such as theory of distribution of sequences modulo 1 and fourier series

the minimum description length principle in coding and modeling we review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling stochastic complexity is formulated as the solution to optimum universal coding problems extending shannon s basic source coding theorem the normalized maximized likelihood mixture and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms we assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference context tree modeling density estimation and model selection in gaussian linear regression serve as examples

universal compression of memoryless sources over unknown alphabets it has long been known that the compression redundancy of independent and identically distributed i i d strings increases to infinity as the alphabet size grows it is also apparent that any string can be described by separately conveying its symbols and its pattern the order in which the symbols appear concentrating on the latter we show that the patterns of i i d strings over all including infinite and even unknown alphabets can be compressed with diminishing redundancy both in block and sequentially and that the compression can be performed in linear time to establish these results we show that the number of patterns is the bell number that the number of patterns with a given number of symbols is the stirling number of the second kind and that the redundancy of patterns can be bounded using results of hardy and ramanujan on the number of integer partitions the results also imply an asymptotically optimal solution for the good turing probability estimation problem

a theory for multiresolution signal decomposition the wavelet representation multiresolution representations are effective for analyzing the information content of images the properties of the operator which approximates a signal at a given resolution were studied it is shown that the difference of information between the approximation of a signal at the resolutions 2 sup j 1 and 2 sup j where j is an integer can be extracted by decomposing this signal on a wavelet orthonormal basis of l sup 2 r sup n the vector space of measurable square integrable n dimensional functions in l sup 2 r a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi x this decomposition defines an orthogonal multiresolution representation called a wavelet representation it is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters wavelet representation lies between the spatial and fourier domains for images the wavelet representation differentiates several spatial orientations the application of this representation to data compression in image coding texture discrimination and fractal analysis is discussed

counting markov types balanced matrices and eulerian graphs the method of types is one of the most popular techniques in information theory and combinatorics two sequences of equal length have the same type if they have identical empirical distributions in this paper we focus on markov types that is sequences generated by a markov source of order one we note that sequences having the same markov type share the same so called balanced frequency matrix that counts the number of distinct pairs of symbols we enumerate the number of markov types for sequences of length over an alphabet of size this turns out to be asymptotically equivalent to estimating the number of the balanced frequency matrices the number of integer solutions of a system of linear diophantine equations and the number of connected eulerian multigraphs for fixed we prove that the number of markov types is asymptotically equal to d m nm2 m m2 m where we give an integral representation for d m for m we conclude that asymptotically the number of types is equivalent to 2m3m 2em2 m2m22m m 2 nm2 m provided that m o n1 4 these findings are derived by analytical techniques ranging from analytic combinatorics to multidimensional generating functions to the saddle point method

minimum expected length of fixed to variable lossless compression without prefix constraints the minimum expected length for fixed to variable length encoding of an n block memoryless source with entropy h grows as nh o 1 where the term o 1 lies between 0 and 1 however this well known performance is obtained under the implicit constraint that the code assigned to the whole n block is a prefix code dropping the prefix constraint which is rarely necessary at the block level we show that the minimum expected length for a finite alphabet memoryless source with known distribution grows as nh 1 2 log n o 1 unless the source is equiprobable we also refine this result up to o 1 for those memoryless sources whose log probabilities do not reside on a lattice

a general framework for codes involving redundancy minimization a framework with two scalar parameters is introduced for various problems of finding a prefix code minimizing a coding penalty function the framework encompasses problems previously proposed by huffman campbell nath and drmota and szpankowski shedding light on the relationships among these problems in particular nath s range of problems can be seen as bridging the minimum average redundancy problem of huffman with the minimum maximum pointwise redundancy problem of drmota and szpankowski using this framework two linear time huffman like algorithms are devised for the minimum maximum pointwise redundancy problem the only one in the framework not previously solved with a huffman like algorithm both algorithms provide solutions common to this problem and a subrange of nath s problems the second algorithm being distinguished by its ability to find the minimum variance solution among all solutions common to the minimum maximum pointwise redundancy and nath problems simple redundancy bounds are also presented

analysis of low bit rate image transform coding calculations based on high resolution quantizations prove that the distortion rate d r of an image transform coding is proportional to 2 sup 2r when r is large enough in wavelet and block cosine bases we show that if r 1 bit pixel then d r varies like r sup 1 2 spl gamma where spl gamma remains of the order of 1 for most natural images the improved performance of embedded codings in wavelet bases is analyzed at low bit rates we show that the compression performance of an orthonormal basis depends mostly on its ability to approximate images with a few nonzero vectors

asymptotic behavior of the lempel ziv parsing scheme and digital search trees the lempel ziv parsing scheme finds a wide range of applications most notably in data compression and algorithms on words it partitions a sequence of length n into variable phrases such that a new phrase is the shortest substring not seen in the past as a phase the parameter of interest is the number mn of phrases that one can construct from a sequence of length n in this paper for the memoryless source with unequal probabilities of symbols generation we derive the limiting distribution of mn which turns out to be normal this proves a long standing open problem in fact to obtain this result we solved another open problem namely that of establishing the limiting distribution of the internal path length in a digital search tree the latter is a consequence of an asymptotic solution of a multiplicative differential functional equation often arising in the analysis of algorithms on words interestingly enough our findings are proved by a combination of probabilistic techniques such as renewal equation and uniform integrability and analytical techniques such as mellin transform differential functional equations de poissonization and so forth in concluding remarks we indicate a possibility of extending our results to markovian models

optimum quantizer performance for a class of non gaussian memoryless sources the performance of optimum quantizers subject to an entropy constraint is studied for a wide class of memoryless sources for a general distortion criterion necessary conditions are developed for optimality and a recursive algorithm is described for obtaining the optimum quantizer under a mean square error criterion the performance of entropy encoded uniform quantization of memoryless gaussian sources is well known to be within 0 255 bits sample of the rate distortion bound at relatively high rates despite claims to the contrary it is demonstrated that similar performance can be expected for a wide range of memoryless sources indeed for the cases considered the worst case performance is observed to be less than 0 3 bits sample from the rate distortion bound and in most cases this disparity is less at iow rates

on universal codeword sets the set of all binary strings terminated by a specific pattern is countably infinite and regular and satisfies the prefix property some properties of such codeword sets in the context of source coding are studied it is shown that some of them are universal and some of them are not but none is asymptotically optimal

the use and usefulness of numeration systems the proper choice of a counting system may solve mathematical problems or lead to improved algorithms this is illustrated by a problem in combinatorial group theory compression of sparse binary strings encoding of contiguous binary strings of unknown lengths ranking of permutations and combinations strategies of games and other examples two abstract counting systems are given from which the concrete ones used for the applications can be derived

optimal prefix codes for infinite alphabets with nonlinear costs let p p i be a measure of strictly positive probabilities on the set of nonnegative integers although the countable number of inputs prevents usage of the huffman algorithm there are nontrivial p for which known methods find a source code that is optimal in the sense of minimizing expected codeword length for some applications however a source code should instead minimize one of a family of nonlinear objective functions beta exponential means those of the form loga sigmaip i an i where n i is the length of the ith codeword and a is a positive constant applications of such minimizations include a novel problem of maximizing the chance of message receipt in single shot communications a 1 this paper introduces methods for finding codes optimal for such exponential means one method applies to geometric distributions while another applies to distributions with lighter tails the latter algorithm is applied to poisson distributions and both are extended to alphabetic codes as well as to minimizing maximum pointwise redundancy the aforementioned application of minimizing the chance of buffer overflow is also considered

on the average redundancy rate of the lempel ziv code in this paper we settle a long standing open problem concerning the average redundancy r sub n of the lempel ziv 78 lz78 code we prove that for a memoryless source the average redundancy rate attains asymptotically er sub n a spl delta n log n o log log n log sup 2 n where a is an explicitly given constant that depends on source characteristics and spl delta x is a fluctuating function with a small amplitude we also derive the leading term for the kth moment of the number of phrases we conclude by conjecturing a precise formula on the expected redundancy for a markovian source the main result of this paper is a consequence of the second order properties of the lempel ziv algorithm obtained by jacquet and szpankowski 1995 these findings have been established by analytical techniques of the precise analysis of algorithms we give a brief survey of these results since they are interesting in their own right and shed some light on the probabilistic behavior of pattern matching based data compression

efficient integer coding for arbitrary probability distributions performance bounds on the average codeword length of golomb codes for arbitrary probability distributions are derived in terms of the mean then based on golomb codes a class of extended gamma codes which are the generalizations of elias gamma code is constructed and proved to be universal optimal decision rules for choosing parameters of golomb codes and extended gamma codes for arbitrary probability distributions are also derived finally the concept of maximum entropy codes is introduced and the existence of such codes is investigated for source classes with linear constraints golomb codes with optimal parameters turn out to be maximum entropy codes for sources with a fixed mean

comments and additions to robust transmission of unbounded strings using fibonacci representations an ambiguous point in the paper by apostolico and fraenkel see ibid vol it 33 no 2 p 238 45 march 1987 is clarified new codes that have better asymptotic performances and better synchronization capability are introduced

theory of optimal orthonormal subband coders the theory of the orthogonal transform coder and methods for its optimal design have been known for a long time we derive a set of necessary and sufficient conditions for the coding gain optimality of an orthonormal subband coder for given input statistics we also show how these conditions can be satisfied by the construction of a sequence of optimal compaction filters one at a time several theoretical properties of optimal compaction filters and optimal subband coders are then derived especially pertaining to behavior as the number of subbands increases significant theoretical differences between optimum subband coders transform coders and predictive coders are summarized finally conditions are presented under which optimal orthonormal subband coders yield as much coding gain as biorthogonal ones for a fixed number of subbands

the minimum average code for finite memoryless monotone sources the problem of selecting a code for finite monotone sources with n symbols is considered the selection criterion is based on minimizing the average redundancy called minave criterion instead of its maximum i e minimax criterion the average probability distribution pnmacr whose associated huffman code has the minimum average redundancy is derived the entropy of the average distribution i e h pnmacr and the average entropy of the monotone distributions i e h pnmacr are studied it is shown that both logn h pnmacr and logn h pnmacr are asymptotically equal to a constant sime0 61 therefore there is only a negligible penalty at most 1 61 bits symbol in using a simple fixed length code with respect to the optimal code an efficient near optimal encoding technique is also proposed the consequences of the two approaches i e minave and minimax are compared in terms of their associated distributions and associated codes in order to evaluate the average performance of the minimax code we prove that the informational divergence of the average distribution and minimax distribution asymptotically grows as 2 275 loglogn

prediction based lower triangular transform a new nonunitary transform called the prediction based lower triangular transform plt is introduced for signal compression the new transform has the same decorrelation property as the kahurnen loeve transform klt but its implementational cost is less than one half of klt compared with the klt the design cost of an m spl times m plt is much lower and is only of the order of o m sup 2 moreover the plt can be factorized into simple building blocks using two different factorizations we introduce two minimum noise structures that have roughly the same complexity as the direct implementation of plt these minimum noise structures have the following properties 1 its noise gain is unity even though the transform is nonunitary 2 perfect reconstruction is structurally guaranteed 3 it can be used for both lossy lossless compression we show that the coding gain of plt implemented using the minimum noise structure is the same as that of klt furthermore universal transform coders using plt are derived for ar 1 process the m spl times m plt has a closed form and needs only m 1 multiplications and additions

suboptimality of the karhunen loeve transform for transform coding we examine the performance of the karhunen loeve transform klt for transform coding applications the klt has long been viewed as the best available block transform for a system that orthogonally transforms a vector source scalar quantizes the components of the transformed vector using optimal bit allocation and then inverse transforms the vector this paper treats fixed rate and variable rate transform codes of non gaussian sources the fixed rate approach uses an optimal fixed rate scalar quantizer to describe the transform coefficients the variable rate approach uses a uniform scalar quantizer followed by an optimal entropy code and each quantized component is encoded separately earlier work shows that for the variable rate case there exist sources on which the klt is not unique and the optimal quantization and coding stage matched to a worst klt yields performance as much as 1 5 db worse than the optimal quantization and coding stage matched to a best klt in this paper we strengthen that result to show that in both the fixed rate and the variable rate coding frameworks there exist sources for which the performance penalty for using a worst klt can be made arbitrarily large further we demonstrate in both frameworks that there exist sources for which even a best klt gives suboptimal performance finally we show that even for vector sources where the klt yields independent coefficients the klt can be suboptimal for fixed rate coding

second order asymptotics in fixed length source coding and intrinsic randomness there is a difference between the optimal rates of fixed length source coding and intrinsic randomness when we care about the second order asymptotics we prove this difference for general information sources and then investigate independent and identically distributed i i d random variables and markovian variables as examples the difference is demonstrated through an investigation of the second order asymptotic behavior of the rates a universal fixed length source code attaining the second order optimal rate is also proposed the difference between the rates of fixed length source coding and intrinsic randomness proves that the outputs of fixed length source codes are not uniformly distributed

a new asymptotically optimal code for the positive integers a new universal binary code for the positive integers is proposed as a modified version of m wang s see ibid vol 34 p 324 6 mar 1988 flag encoding scheme the codeword length of the new scheme is shorter than wang s on an average for large initial segments of the positive integers the performance of the new scheme is also compared with that of other universal schemes furthermore it is shown that an asymptotically optimal code can be achieved by modifying the new flag scheme such that the flag length varies dynamically

mathematical properties of the jpeg2000 wavelet filters the legall 5 3 and daubechies 9 7 filters have risen to special prominence because they were selected for inclusion in the jpeg2000 standard we determine their key mathematical features riesz bounds order of approximation and regularity holder and sobolev we give approximation theoretic quantities such as the asymptotic constant for the l sup 2 error and the angle between the analysis and synthesis spaces which characterizes the loss of performance with respect to an orthogonal projection we also derive new asymptotic error formulae that exhibit bound constants that are proportional to the magnitude of the first nonvanishing moment of the wavelet the daubechies 9 7 stands out because it is very close to orthonormal but this turns out to be slightly detrimental to its asymptotic performance when compared to other wavelets with four vanishing moments

universal coding on infinite alphabets exponentially decreasing envelopes this paper deals with the problem of universal lossless coding on a countable infinite alphabet it focuses on some classes of sources defined by an envelope condition on the marginal distribution namely exponentially decreasing envelope classes with exponent the minimax redundancy of exponentially decreasing envelope classes is proved to be equivalent to then an adaptive algorithm is proposed whose maximum redundancy is equivalent to the minimax redundancy

fisher information and stochastic complexity by taking into account the fisher information and removing an inherent redundancy in earlier two part codes a sharper code length as the stochastic complexity and the associated universal process are derived for a class of parametric processes the main condition required is that the maximum likelihood estimates satisfy the central limit theorem the same code length is also obtained from the so called maximum likelihood code

sparse geometric image representations with bandelets this paper introduces a new class of bases called bandelet bases which decompose the image along multiscale vectors that are elongated in the direction of a geometric flow this geometric flow indicates directions in which the image gray levels have regular variations the image decomposition in a bandelet basis is implemented with a fast subband filtering algorithm bandelet bases lead to optimal approximation rates for geometrically regular images for image compression and noise removal applications the geometric flow is optimized with fast algorithms so that the resulting bandelet basis produces minimum distortion comparisons are made with wavelet image compression and noise removal algorithms

analytic variations on redundancy rates of renewal processes csiszar and shields 1996 proved that the minimax redundancy for a class of stationary renewal processes is spl theta spl radic n where n is the block length this interesting result provides a nontrivial bound on redundancy for a nonparametric family of processes the present paper gives a precise estimate of the redundancy rate for such nonstationary renewal sources namely 2 log2 spl radic spl pi sup 2 6 1 n o log n this asymptotic expansion is derived by complex analytic methods that include generating function representations mellin transforms singularity analysis and saddle point estimates this work places itself within the framework of analytic information theory

variable length prefix codes with multiple delimiters let m 1 m 2 ldots m t be a fixed set of natural integers given in ascending order a multi delimiter code d m 1 ldots m t consists of t words of the form 1 m i 0 and all other binary words with suffixes having one of the forms 01 m i 0 that cannot occur in other places of a word i 1 ldots t completeness universality and density characteristics of multi delimiter codes are proved encoding of integers by multi delimiter codes is considered in detail for these codes fast byte aligned decoding algorithms are also discussed the comparison of compression rate and decoding speed of fibonacci codes s c dense codes and different multi delimiter codes is presented by many useful properties multi delimiter codes are superior to fibonacci codes

on rate distortion models for natural images and wavelet coding performance operational rate distortion rd functions of most natural images when compressed with state of the art wavelet coders exhibit a power law behavior dpropr gamma at moderately high rates with gamma being a constant depending on the input image deviating from the well known exponential form of the rd function dprop2 xir for bandlimited stationary processes this paper explains this intriguing observation by investigating theoretical and operational rd behavior of natural images we take as our source model the fractional brownian motion fbm which is often used to model nonstationary behaviors in natural images we first establish that the theoretical rd function of the fbm process both in 1 d and 2 d indeed follows a power law then we derive operational rd function of the fbm process when wavelet encoded based on water filling principle interestingly both the operational and theoretical rd functions behave as dpropr gamma for natural images the values of gamma are found to be distributed around 1 these results lend an information theoretical support to the merit of multiresolution wavelet compression of self similar processes and in particular natural images that can be modelled by such processes they may also prove useful in predicting performance of rd optimized image coders

the role of gtd in optimizing perfect reconstruction filter banks filter bank optimization for specific input statistics has been of great interest in both theory and practice in many signal processing applications in this paper we propose gtd generalized triangular decomposition filter banks as a subband coder for optimizing the theoretical coding gain we focus on perfect reconstruction orthonormal gtd filter banks and biorthogonal gtd filter banks we show that in both cases there are two fundamental properties in the optimal solutions namely total decorrelation and spectrum equalization the optimal solutions can be obtained by performing the frequency dependent gtd on the cholesky factor of the input power spectrum density matrices we also show that in both theory and numerical simulations the optimal gtd subband coders have superior performance than optimal traditional subband coders in addition the uniform bit loading scheme can with no loss of optimality be used in the optimal biorthogonal gtd coders which solves the granularity problem in the conventional optimum bit loading formula we then extend the use of gtd filter banks to wireless communication systems where linear precoding and zero forcing decision feedback equalization is used in frequency selective channels we consider the quality of service qos problem of minimizing the transmitted power subject to the bit error rate and total bit rate constraints optimal systems with orthonormal precoder and unconstrained precoder are both derived and shown to be related to the frequency dependent gtd of the channel frequency response

pointwise redundancy in lossy data compression and universal lossy data compression we characterize the achievable pointwise redundancy rates for lossy data compression at a fixed distortion level pointwise redundancy refers to the difference between the description length achieved by an nth order block code and the optimal nr d bits for memoryless sources we show that the best achievable redundancy rate is of order o spl radic n in probability this follows from a second order refinement to the classical source coding theorem in the form of a one sided central limit theorem moreover we show that along almost any source realization the description lengths of any sequence of block codes operating at distortion level d exceed nr d by at least as much as c spl radic nloglogn infinitely often corresponding direct coding theorems are also given showing that these rates are essentially achievable the above rates are in sharp contrast with the expected redundancy rates of order o log n reported by various authors our approach is based on showing that the compression performance of an arbitrary sequence of codes is essentially bounded below by the performance of shannon s random code we obtain partial generalizations of the above results for arbitrary sources with memory and we prove lossy analogs of barron s lemma barron 1985

pattern dynamics of a coupled map lattice for open flow abstract the pattern dynamics of the one way coupled logistic lattice which can serve as a phenomenological model for open flow is investigated and shown to be extremely rich for medium and large coupling strengths we find spatially periodic quasiperiodic and chaotic patterns with temporal periodicity and analyze their stability with the help of a newly introduced spatial map criteria are established for predicting the down flow bifurcation behavior of the coupled map lattice and for selecting attractors through modulation of the boundary for smaller coupling strengths we find a novel regime in which chaotic defects form a periodic lattice that is shown to be the result of a boundary crisis

redundancy of the lempel ziv incremental parsing rule the lempel ziv codes are universal variable to fixed length codes that have become virtually standard in practical lossless data compression for any given source output string from a markov or unifilar source we upper bound the difference between the number of binary digits needed to encode the string and the self information of the string we use this result to demonstrate that for unifilar or markov sources the redundancy of encoding the first n letters of the source output with the lempel ziv incremental parsing rule lz 78 the welch modification lzw or a new variant is o ln n sup 1 and we upper bound the exact form of convergence we conclude by considering the relationship between the code length and the empirical entropy associated with a string

a one to one code and its anti redundancy one to one codes are ldquoone shotrdquo codes that assign a distinct codeword to source symbols and are not necessarily prefix codes more generally uniquely decodable interestingly as wyner proved in 1972 for such codes the average code length can be smaller than the source entropy by how much we call this difference the anti redundancy various authors over the years have shown that the anti redundancy can be as big as minus the logarithm of the source entropy however to the best of our knowledge precise estimates do not exist in this note we consider a block code of length n generated for a binary memoryless source and prove that the average anti redundancy is 1 2 log2 n c f n o 1 where c is a constant and either f n 0 if log2 1 p p is irrational where p is the probability of generating a ldquo0rdquo or f n is a fluctuating function as the code length increases this relatively simple finding requires a combination of analytic tools such as precise evaluation of bernoulli sums the saddle point method and theory of distribution of sequences modulo 1

hierarchy and polysynchrony in an adaptive network we describe a simple adaptive network of coupled chaotic maps the network reaches a stationary state frozen topology for all values of the coupling parameter although the dynamics of the maps at the nodes of the network can be nontrivial the structure of the network shows interesting hierarchical properties and in certain parameter regions the dynamics is polysynchronous nodes can be divided in differently synchronized classes but contrary to cluster synchronization nodes in the same class need not be connected to each other these complicated synchrony patterns have been conjectured to play roles in systems biology and circuits the adaptive system we study describes ways whereby this behavior can evolve from undifferentiated nodes

a simple recursive shannon code the shannon code is a very simple suboptimal scheme for computing prefix free codelengths for a given memoryless source a recursive version of the well known shannon code rsh is investigated in this paper it has a redundancy which never exceeds that of the shannon code unlike the huffman code the rsh code and its variations can be directly applied to sources with an infinitely countable alphabet the average redundancy taken over the set of all n tuple distributions is considered as a criterion for code comparisons for n 40 the average redundancy is approximately 0 14 bits for the rsh code compared to approximately 0 5 and 0 03 bits for the shannon and huffman codes respectively for large n a constrained version of the rsh code called crsh has an average redundancy of only 0 07 bits for unsorted symbol probabilities if the symbol probabilities are sorted in descending order then a very simple modification of the rsh algorithm results in a code which is near optimal from the average redundancy point of view

tunstall code khodak variations and random walks a variable to fixed length encoder partitions the source string into variable length phrases that belong to a given and fixed dictionary tunstall and independently khodak designed variable to fixed length codes for memoryless sources that are optimal under certain constraints in this paper we study the tunstall and khodak codes using variety of techniques ranging from stopping times for sums of independent random variables to tauberian theorems and mellin transform after proposing an algebraic characterization of the tunstall and khodak codes we present new results on the variance and a central limit theorem for dictionary phrase lengths this analysis also provides a new argument for obtaining asymptotic results about the mean dictionary phrase length and average redundancy rates

recursive biorthogonal interpolating wavelets and signal adapted interpolating filter banks in this paper by combining the ideas of the recursive wavelets with second generation wavelets a family of recursive biorthogonal interpolating wavelets rbiws is developed the rbiws have simple shape parameter vectors on each level which allows a multichannel decomposition algorithm and provides a flexible structure for designing signal adapted interpolating filter banks in the single level case an efficient approach to design an optimum two channel biorthogonal interpolating filter bank is proposed which maximizes the coding gain under the traditional quantization noise assumption furthermore in the multilevel case using level wise optimization of the shape parameter vectors signal adapted tree structured recursive biorthogonal interpolating filter banks rbifbs are designed which are efficient in computation and can remarkedly improve the coding gain finally numerical results demonstrate the effectiveness of the proposed methods

consensus formation on adaptive networks the structure of a network can significantly influence the properties of the dynamical processes that take place on them while many studies have been paid to this influence much less attention has been devoted to the interplay and feedback mechanisms between dynamical processes and network topology on adaptive networks adaptive rewiring of links can happen in real life systems such as acquaintance networks where people are more likely to maintain a social connection if their views and values are similar in our study we consider different variants of a model for consensus formation our investigations reveal that the adaptation of the network topology fosters cluster formation by enhancing communication between agents of similar opinion although it also promotes the division of these clusters the temporal behavior is also strongly affected by adaptivity while on static networks it is influenced by percolation properties on adaptive networks both the early and late time evolutions of the system are determined by the rewiring process the investigation of a variant of the model reveals that the scenarios of transitions between consensus and polarized states are more robust on adaptive networks

types of markov fields and tilings the method of types is one of the most popular techniques in information theory and combinatorics however thus far the method has been mostly applied to 1 d markov processes and it has not been thoroughly studied for general markov fields markov fields over a finite alphabet of size m ge 2 can be viewed as models for multidimensional systems with local interactions the locality of these interactions is represented by a shape mathcal s while its marking by symbols of the underlying alphabet is called a tile two assignments in a markov field have the same type if they have the same empirical distribution i e if they have the same number of tiles of a given type our goal is to study the growth of the number of possible markov field types in either a d dimensional box of lengths n 1 ldots n d or its cyclic counterpart a d dimensional torus we relate this question to the enumeration of nonnegative integer solutions of a large system of diophantine linear equations called the conservation laws we view a markov type as a vector in a d m mathcal s dimensional space and count the number of such vectors satisfying the conservation laws which turns out to be the number of integer points in a certain polytope for the torus this polytope is of dimension mu d 1 mathrm rk mathrm mathrm c where mathrm rk mathrm mathrm c is the number of linearly independent conservation laws mathrm mathrm c this provides an upper bound on the number of types then we construct a matching lower bound leading to the conclusion that the number of types in the torus markov field is theta n mu where n n 1 ldots n d these results are derived by geometric tools including ideas of discrete and convex multidimensional geometry

patterns of chaos synchronization small networks of chaotic units which are coupled by their time delayed variables are investigated in spite of the time delay the units can synchronize isochronally i e without time shift moreover networks cannot only synchronize completely but can also split into different synchronized sublattices these synchronization patterns are stable attractors of the network dynamics different networks with their associated behaviors and synchronization patterns are presented in particular we investigate sublattice synchronization symmetry breaking spreading chaotic motifs synchronization by restoring symmetry and cooperative pairwise synchronization of a bipartite tree

multilayered image representation application to image compression the main contribution of this work is a new paradigm for image representation and image compression we describe a new multilayered representation technique for images an image is parsed into a superposition of coherent layers piecewise smooth regions layer textures layer etc the multilayered decomposition algorithm consists in a cascade of compressions applied successively to the image itself and to the residuals that resulted from the previous compressions during each iteration of the algorithm we code the residual part in a lossy way we only retain the most significant structures of the residual part which results in a sparse representation each layer is encoded independently with a different transform or basis at a different bitrate and the combination of the compressed layers can always be reconstructed in a meaningful way the strength of the multilayer approach comes from the fact that different sets of basis functions complement each others some of the basis functions will give reasonable account of the large trend of the data while others will catch the local transients or the oscillatory patterns this multilayered representation has a lot of beautiful applications in image understanding and image and video coding we have implemented the algorithm and we have studied its capabilities

regular universal codeword sets corresp proof is given that no regular or synchronizable universal codeword sets can achieve optimality some asymptotic properties of a remarkable class of regular codeword sets related to the fibonacci representation of integers are also obtained

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection

the asymptotic redundancy of bayes rules for markov chains we derive the asymptotics of the redundancy of bayes rules for markov chains of fixed order over a finite alphabet extending the work of barron and clarke 1990 on independent and identically distributed i i d sources the asymptotics are derived when the actual source is the class of spl phi mixing sources which strictly includes markov chains these results can be used to derive minimax asymptotic rates of convergence for universal codes when a markov chain of fixed order is used as a model

the generalized triangular decomposition given a complex matrix h we consider the decomposition h qrp where r is upper triangular and q and p have orthonormal columns special instances of this decomposition include the singular value decomposition svd and the schur decomposition where r is an upper triangular matrix with the eigenvalues of h on the diagonal we show that any diagonal for r can be achieved that satisfies weyl s multiplicative majorization conditions k k k k i 1 r i i 1 1 k k k i 1 r i i 1 i where k is the rank of h i is the i th largest singular value of h and r i is the i th largest in magnitude diagonal element of r given a vector r which satisfies weyl s conditions we call the decomposition h qrp where r is upper triangular with prescribed diagonal r the generalized triangular decomposition gtd a direct nonrecursive algorithm is developed for computing the gtd this algorithm starts with the svd and applies a series of permutations and givens rotations to obtain the gtd the numerical stability of the gtd update step is established the gtd can be used to optimize the power utilization of a communication channel while taking into account quality of service requirements for subchannels another application of the gtd is to inverse eigenvalue problems where the goal is to construct matrices with prescribed eigenvalues and singular values

the performance of universal encoding universal coding theory is surveyed from the viewpoint of the interplay between delay and redundancy the price for universality turns out to be acceptably small

equal diagonal qr decomposition and its application to precoder design for successive cancellation detection in multiple input multiple output mimo multiuser detection theory the qr decomposition of the channel matrix h can be used to form the back cancellation detector in this paper we propose an optimal qr decomposition which we call the equal diagonal qr decomposition or briefly the qrs decomposition we apply the decomposition to precoded successive cancellation detection where we assume that both the transmitter and the receiver have perfect channel knowledge we show that for any channel matrix h there exists a unitary precoder matrix s such that hs qr where the nonzero diagonal entries of the upper triangular matrix r in the qr decomposition of hs are all equal to each other the precoder and the resulting successive cancellation detector have the following properties a the minimum euclidean distance between two signal points at the channel output is equal to the minimum euclidean distance between two constellation points at the precoder input up to a multiplicative factor that equals the diagonal entry in the r factor b the superchannel hs naturally exhibits an optimally ordered column permutation i e the optimal detection order for the vertical bell labs layered space time v blast detector is the natural order c the precoder s minimizes the block error probability of the qr successive cancellation detector d a lower and an upper bound for the free distance at the channel output is expressible in terms of the diagonal entries of the r factor in the qr decomposition of a channel matrix e the precoder s maximizes the lower bound of the channel s free distance subject to a power constraint f for the optimal precoder s the performance of the qr detector is asymptotically at large signal to noise ratios snrs equivalent to that of the maximum likelihood detector mld that uses the same precoder further we consider two multiplexing schemes time division multiple access tdma and orthogonal frequency division multiplexing ofdm we d

on the importance of combining wavelet based nonlinear approximation with coding strategies this paper provides a mathematical analysis of transform compression in its relationship to linear and nonlinear approximation theory contrasting linear and nonlinear approximation spaces we show that there are interesting classes of functions random processes which are much more compactly represented by wavelet based nonlinear approximation these classes include locally smooth signals that have singularities and provide a model for many signals encountered in practice in particular for images however we also show that nonlinear approximation results do not always translate to efficient compress on strategies in a rate distortion sense based on this observation we construct compression techniques and formulate the family of functions stochastic processes for which they provide efficient descriptions in a rate distortion sense we show that this family invariably leads to besov spaces yielding a natural relationship among besov smoothness linear nonlinear approximation order and compression performance in a rate distortion sense the designed compression techniques show similarities to modern high performance transform codecs allowing us to establish relevant rate distortion estimates and identify performance limits

generalized triangular decomposition in transform coding a general family of optimal transform coders tcs is introduced here based on the generalized triangular decomposition gtd developed by jiang this family includes the karhunen loeve transform klt and the generalized version of the prediction based lower triangular transform plt introduced by phoong and lin as special cases the coding gain of the entire family with optimal bit allocation is equal to that of the klt and the plt even though the original plt introduced by phoong is not applicable for vectors that are not blocked versions of scalar wide sense stationary processes the gtd based family includes members that are natural extensions of the plt and therefore also enjoy the so called minlab structure of the plt which has the unit noise gain property other special cases of the gtd tc are the geometric mean decomposition gmd and the bidiagonal decomposition bid transform coders the gmd tc in particular has the property that the optimum bit allocation is a uniform allocation this is because all its transform domain coefficients have the same variance implying thereby that the dynamic ranges of the coefficients to be quantized are identical

coding on countably infinite alphabets this paper describes universal lossless coding strategies for compressing sources on countably infinite alphabets classes of memoryless sources defined by an envelope condition on the marginal distribution provide benchmarks for coding techniques originating from the theory of universal coding over finite alphabets we prove general upper bounds on minimax regret and lower bounds on minimax redundancy for such source classes the general upper bounds emphasize the role of the normalized maximum likelihood nml codes with respect to minimax regret in the infinite alphabet context lower bounds are derived by tailoring sharp bounds on the redundancy of krichevsky trofimov coders for sources over finite alphabets up to logarithmic resp constant factors the bounds are matching for source classes defined by algebraically declining resp exponentially vanishing envelopes effective and almost adaptive coding techniques are described for the collection of source classes defined by algebraically vanishing envelopes those results extend our knowledge concerning universal coding to contexts where the key tools from parametric inference are known to fail

operational rate distortion modeling for wavelet video coders based on our statistical investigation of a typical three dimensional 3 d wavelet codec we present a unified mathematical model to describe its operational rate distortion rd behavior the quantization distortion of the reconstructed video frames is assessed by tracking the quantization noise along the 3 d wavelet decomposition trees the coding bit rate is estimated for a class of embedded video coders experimental results show that the model captures sequence characteristics accurately and reveals the relationship between wavelet decomposition levels and the overall rd performance after being trained with offline rd data the model enables accurate prediction of real rd performance of video codecs and therefore can enable optimal rd adaptation of the encoding parameters according to various network conditions

precise minimax redundancy and regret recent years have seen a resurgence of interest in redundancy of lossless coding the redundancy regret of universal fixed to variable length coding for a class of sources determines by how much the actual code length exceeds the optimal ideal over the class code length in a minimax scenario one finds the best code for the worst source either in the worst case called also maximal minimax or on average we first study the worst case minimax redundancy over a class of stationary ergodic sources and replace shtarkov s bound by an exact formula among others we prove that a generalized shannon code minimizes the worst case redundancy derive asymptotically its redundancy and establish some general properties this allows us to obtain precise redundancy for memoryless markov and renewal sources for example we present the exact constant of the redundancy for memoryless and markov sources by showing that the integer nature of coding contributes log logm m 1 logm o 1 where m is the size of the alphabet then we deal with the average minimax redundancy and regret our approach here is orthogonal to most recent research in this area since we aspire to show that asymptotically the average minimax redundancy is equivalent to the worst case minimax redundancy for some classes of sources after formulating some general bounds relating these two redundancies we prove our assertion for memoryless and markov sources nevertheless we provide evidence that maximal redundancy of renewal processes does not have the same leading term as the average minimax redundancy however our general results show that maximal and average regrets are asymptotically equivalent

markov types and minimax redundancy for markov sources redundancy of universal codes for a class of sources determines by how much the actual code length exceeds the optimal code length in the minimax scenario one designs the best code for the worst source within the class such minimax redundancy comes in two flavors average minimax or worst case minimax we study the worst case minimax redundancy of universal block codes for markovian sources of any order we prove that the maximal minimax redundancy for markov sources of order r is asymptotically equal to 1 2m sup r m 1 log sub 2 n log sub 2 a sub m sup r lnlnm sup 1 m 1 lnm o 1 where n is the length of a source sequence m is the size of the alphabet and a sub m sup r is an explicit constant e g we find that for a binary alphabet m 2 and markov of order r 1 the constant a sub 2 sup 1 16 spl middot g spl ap 14 655449504 where g is the catalan number unlike previous attempts we view the redundancy problem as an asymptotic evaluation of certain sums over a set of matrices representing markov types the enumeration of markov types is accomplished by reducing it to counting eulerian paths in a multigraph in particular we propose exact and asymptotic formulas for the number of strings of a given markov type all of these findings are obtained by analytic and combinatorial tools of analysis of algorithms

joint unitary triangularization for mimo networks this paper considers communication networks where individual links can be described as those based on multiple input multiple output channels unlike orthogonal modulation methods such as the singular value decomposition we allow interference between subchannels which can be removed by the receivers via successive cancellation the degrees of freedom earned by this relaxation are used for obtaining a basis and corresponding decomposition which are simultaneously good for more than one link specifically we derive necessary and sufficient conditions for shaping the ratio vector of subchannel gains of two broadcast channel receivers we then apply this decomposition to two scenarios first in digital multicasting we present a practical capacity achieving scheme which uses only scalar codes and linear processing then we consider the joint source channel problem of transmitting a gaussian source over a two user multiple input multiple output channel where we show the existence of nontrivial cases where the optimal distortion pair which for high signal to noise ratios snrs equals the optimal point to point distortions of the individual users may be achieved by employing a hybrid digital analog scheme over the induced equivalent channel these scenarios demonstrate the advantage of choosing a modulation basis based upon multiple links in the network thus we coin the approach network modulation

biorthogonal and nonuniform lapped transforms for transform coding with reduced blocking and ringing artifacts new lapped transforms are introduced the lapped biorthononal transform lbt and hierarchical lapped biorthogonal transform hlbt are appropriate for image coding and the modulated hlbt biorthogonal transform mmlbt and nonuniform modulated lapped biorthogonal transform nmlbt are appropriate for audio coding the hlbt has a significantly lower computational complexity than the lapped orthogonal transform lot essentially no blocking artifacts and fewer ringing artifacts than the commonly used discrete cosine transform dct the lbt and hlbt have transform coding gains that are typically between 0 5 and 1 2 db higher than that of the dct image coding examples using jpeg and embedded zerotree coders demonstrate the better performance of the let and hlbt the nmlbt has fewer ringing artifacts and better reproduction of transient sounds than the mlt as shown in audio coding examples fast algorithms for both the hlbt and the nmlbt are presented

the most likely optimal symmetric rvlc for each alphabet size up to 30 a symmetric reversible variable length code rvlc is determined which is most likely to be optimal in average codeword length it is shown that these codes are optimal for more than half of all the possible sources

efficient entropy estimation based on doubly stochastic models for quantized wavelet image data under a rate constraint wavelet based image coding involves strategic discarding of information such that the remaining data can be described with a given amount of rate in a practical coding system this task requires knowledge of the relationship between quantization step size and compressed rate for each group of wavelet coefficients the r q curve a common approach to this problem is to fit each subband with a scalar probability distribution and compute entropy estimates based on the model this approach is not effective at rates below 1 0 bits per pixel because the distributions of quantized data do not reflect the dependencies in coefficient magnitudes these dependencies can be addressed with doubly stochastic models which have been previously proposed to characterize more localized behavior though there are tradeoffs between storage computation time and accuracy using a doubly stochastic generalized gaussian model it is demonstrated that the relationship between step size and rate is accurately described by a low degree polynomial in the logarithm of the step size based on this observation an entropy estimation scheme is presented which offers an excellent tradeoff between speed and accuracy after a simple data gathering step estimates are computed instantaneously by evaluating a single polynomial for each group of wavelet coefficients quantized with the same step size these estimates are on average within 3 of a desired target rate for several of state of the art coders

minimax pointwise redundancy for memoryless models over large alphabets we study the minimax pointwise redundancy of universal coding for memoryless models over large alphabets and present two main results we first complete studies initiated in orlitsky and santhanam deriving precise asymptotics of the minimax pointwise redundancy for all ranges of the alphabet size relative to the sequence length second we consider the minimax pointwise redundancy for a family of models in which some symbol probabilities are fixed the latter problem leads to a binomial sum for functions with superpolynomial growth our findings can be used to approximate numerically the minimax pointwise redundancy for various ranges of the sequence length and the alphabet size these results are obtained by analytic techniques such as tree like generating functions and the saddle point method

a wavelet based analysis of fractal image compression why does fractal image compression work what is the implicit image model underlying fractal block coding how can we characterize the types of images for which fractal block coders will work well these are the central issues we address we introduce a new wavelet based framework for analyzing block based fractal compression schemes within this framework we are able to draw upon insights from the well established transform coder paradigm in order to address the issue of why fractal block coders work we show that fractal block coders of the form introduced by jacquin 1992 are haar wavelet subtree quantization schemes we examine a generalization of the schemes to smooth wavelets with additional vanishing moments the performance of our generalized coder is comparable to the best results in the literature for a jacquin style coding scheme our wavelet framework gives new insight into the convergence properties of fractal block coders and it leads us to develop an unconditionally convergent scheme with a fast decoding algorithm our experiments with this new algorithm indicate that fractal coders derive much of their effectiveness from their ability to efficiently represent wavelet zero trees finally our framework reveals some of the fundamental limitations of current fractal compression schemes

image prediction based on neighbor embedding methods this paper describes two new intraimage prediction methods based on two data dimensionality reduction methods nonnegative matrix factorization nmf and locally linear embedding these two methods aim at approximating a block to be predicted in the image as a linear combination of k nearest neighbors determined on the known pixels in a causal neighborhood of the input block variable k can be seen as a parameter controlling some sort of sparsity constraints of the approximation vector the impact of this parameter as well as of the nonnegativity and sum to one constraints for the addressed prediction problem has been analyzed the prediction and rd performances of these two new image prediction methods have then been evaluated in a complete image coding and decoding algorithm simulation results show gains up to 2 db in terms of the psnr of the reconstructed signal after coding and decoding of the prediction residue when compared with h 264 avc intraprediction modes up to 3 db when compared with template matching and up to 1 db when compared with a sparse prediction method

spontaneous structure formation in a network of dynamic elements to discover the generic behaviors of dynamic networks we study a coupled map system with variable coupling strength it is found that this system spontaneously forms various types of network structure according to the parameter values depending on the synchronized or desynchronized motion of unit dynamics the network structure can be either static or dynamic the separation of units into two groups one composed of units with a large number of outgoing connections and the other units with little outgoing connections is observed in dynamic structure it is revealed that the mechanism for such separation is a positive feedback between unit and connection dynamics

average redundancy of the shannon code for markov sources it is known that for memoryless sources the average and maximal redundancy of fixed to variable length codes such as the shannon and huffman codes exhibit two modes of behavior for long blocks it either converges to a limit or it has an oscillatory pattern depending on the irrationality or rationality respectively of certain parameters that depend on the source in this paper we extend these findings concerning the shannon code to the case of a markov source we provide a precise characterization of the convergent versus oscillatory behavior of the shannon code redundancy for a class of irreducible periodic and aperiodic markov sources these findings are obtained by analytic methods such as fourier fejer series analysis and spectral analysis of matrices

mimo transceivers with decision feedback and bit loading theory and optimization this paper considers mimo transceivers with linear precoders and decision feedback equalizers dfes with bit allocation at the transmitter zero forcing zf is assumed considered first is the minimization of transmitted power for a given total bit rate and a specified set of error probabilities for the symbol streams the precoder and dfe matrices are optimized jointly with bit allocation it is shown that the generalized triangular decomposition gtd introduced by jiang li and hager offers an optimal family of solutions the optimal linear transceiver which has a linear equalizer rather than a dfe with optimal bit allocation is a member of this family this shows formally that under optimal bit allocation linear and dfe transceivers achieve the same minimum power the dfe transceiver using the geometric mean decomposition gmd is another member of this optimal family and is such that optimal bit allocation yields identical bits for all symbol streams no bit allocation is necessary when the specified error probabilities are identical for all streams the qr based system used in vblast is yet another member of the optimal family and is particularly well suited when limited feedback is allowed from receiver to transmitter two other optimization problems are then considered a minimization of power for specified set of bit rates and error probabilities the qos problem and b maximization of bit rate for fixed set of error probabilities and power it is shown in both cases that the gtd yields an optimal family of solutions

minimax redundancy for the class of memoryless sources let x sup n x sub 1 x sub n be a memoryless source with unknown distribution on a finite alphabet of size k we identify the asymptotic minimax coding redundancy for this class of sources and provide a sequence of asymptotically minimax codes equivalently we determine the limiting behavior of the minimax relative entropy min sub qxn max sub pxn d p sub xn spl par q sub xn where the maximum is over all independent and identically distributed i i d source distributions and the minimum is over all joint distributions we show in this paper that the minimax redundancy minus k 1 2 log n 2 spl pi e converges to log spl int spl radic det i spl theta d spl theta log spl gamma 1 2 sup k spl gamma k 2 where i spl theta is the fisher information and the integral is over the whole probability simplex the bayes strategy using jeffreys prior is shown to be asymptotically maximin but not asymptotically minimax in our setting the boundary risk using jeffreys prior is higher than that of interior points we provide a sequence of modifications of jeffreys prior that put some prior mass near the boundaries of the probability simplex to pull down that risk to the asymptotic minimax level in the limit

r d optimized tree structured compression algorithms with discrete directional wavelet transform a new image coding method based on discrete directional wavelet transform s wt and quad tree decomposition is proposed here the s wt is a kind of transform proposed in v velisavljevic b beferull lozano m vetterli p l dragotti directionlets anisotropic multidirectional representation with separable filtering ieee trans image process 15 7 2006 which is based on lattice theory and with the difference with the standard wavelet transform is that the former allows more transform directions because the directional property in a small region is more regular than in a big block generally in order to sufficiently make use of the multidirectionality and directional vanishing moment dvm of s wt the input image is divided into many small regions by means of the popular quad tree segmentation and the splitting criterion is on the rate distortion sense after the optimal quad tree is obtained by means of the embedded property of speck a resource bit allocation algorithm is fast implemented utilizing the model proposed in m rajpoot model based optimal bit allocation in ieee data compression conference 2004 proceedings dcc 2004 19 experiment results indicate that our algorithms perform better compared to some state of the art image coders

algebraic theory of optimal filterbanks we introduce an optimality theory for finite impulse response fir filterbanks using a general algebraic point of view we consider an admissible set spl lscr of fir filterbanks and use scalability as the main notion based on which performance of the elements in spl lscr are compared we show that quantification of scalability leads naturally to a partial ordering on the set spl lscr an optimal solution is therefore represented by the greatest element in spl lscr it turns out that a greatest element does not necessarily exist in spl lscr hence one has to settle with one of the maximal elements that exist in spl lscr we provide a systematic way of finding a maximal element by embedding the partial ordering at hand in a total ordering this is done by using a special class of order preserving functions known as schur convex there is however a price to pay for achieving a total ordering there are infinitely many possible choices for schur convex functions and the optimal solution specified in spl lscr depends on this subjective choice an interesting aspect of the presented algebraic theory is that the connection between several concepts namely principal component filterbanks pcfbs filterbanks with maximum coding gain and filterbanks with good scalability is clearly revealed we show that these are simply associated with different extremal elements of the partial ordering induced on spl lscr by scalability

robust transmission of unbounded strings using fibonacci representations families of fibonacci codes and fibonacci representations are defined their main attributes are robustness manifesting itself by the local containment of errors and simple encoding and decoding the main application explored is the transmission of binary strings in which the length is in an unknown range using robust fibonacci representations instead of the conventional error sensitive logarithmic ramp representation though the former is asymptotically longer than the latter the former is actually shorter for very large initial segments of integers

redundancy related bounds for generalized huffman codes this paper presents new lower and upper bounds for the compression rate of binary prefix codes optimized over memoryless sources according to various nonlinear codeword length objectives like the most well known redundancy bounds for minimum average redundancy coding huffman coding these are in terms of a form of entropy and or the probability of an input symbol often the most probable one the bounds here some of which are tight improve on known bounds of the form l h h 1 where h is some form of entropy in bits or in the case of redundancy objectives 0 and l is the length objective also in bits the objectives explored here include exponential average length maximum pointwise redundancy and exponential average pointwise redundancy also called dth exponential redundancy the first of these relates to various problems involving queueing uncertainty and lossless communications the second relates to problems involving shannon coding and universal modeling also explored here for these two objectives is the related matter of individual codeword length