multiple description trellis coded quantization we present a construction of multiple description trellis coded quantizers we use the tensor product of trellises to build a trellis which is applicable to multiple description coding the problems of index assignment and set partitioning for the resulting trellis are considered the viterbi algorithm provides the best path for encoding and the design procedure utilizes a generalized lloyd algorithm the encoding process simultaneously generates all the transmitted sequences furthermore the complexity of the scheme is almost independent of the rate the quantizer provides remarkable performance with little encoding complexity

lagrangian optimization of two description scalar quantizers in this paper we study the problem of optimal design of balanced two description fixed rate scalar quantizer 2dsq under the constraint of convex codecells using a graph based approach to model the problem we show that the minimum expected distortion of the 2dsq is a convex function of the number of codecells in the side quantizers this property allows the problem to be solved by lagrangian minimization for which the optimal lagrangian multiplier exists given a trial multiplier we exploit a monotonicity of the objective function and develop a simple and fast dynamic programming technique to solve the parameterized problem to further improve the algorithm efficiency we propose an rd guided search strategy to find the optimal lagrangian multiplier in our experiments on distributions of interest for signal compression applications the proposed algorithm improves the speed of the fastest algorithm so far by a factor of o k logk where k is the number of codecells in each side quantizer we also assess the impact on the optimality of the convex codecell constraint using a published performance analysis of 2dsq at high rates we show that asymptotically this constraint does not preclude optimality for l 2 distortion measure when channels have a higher than 0 12 loss rate

multiple description coding using pairwise correlating transforms the objective of multiple description coding mdc is to encode a source into multiple bitstreams supporting multiple quality levels of decoding in this paper we only consider the two description case where the requirement is that a high quality reconstruction should be decodable from the two bitstreams together while lower but still acceptable quality reconstructions should be decodable from either of the two individual bitstreams this paper describes techniques for meeting mdc objectives in the framework of standard transform based image coding through the design of pairwise correlating transforms the correlation introduced by the transform helps to reduce the distortion when only a single description is received but it also increases the bit rate beyond that prescribed by the rate distortion function of the source we analyze the relation between the redundancy i e the extra bit rate and the single description distortion using this transform based framework we also describe an image coder that incorporates the pairwise transform and show its redundancy rate distortion performance for real images

optimal quantizer design for noisy channels an approach to combined source channel coding we present an analysis of the zero memory quantization of memoryless sources when the quantizer output is to be encoded and transmitted across a noisy channel necessary conditions for the joint optimization of the quantizer and the encoder decoder pair are presented and an iterative algorithm for obtaining a locally optimum system is developed the performance of this locally optimal system obtained for the class of generalized gaussian distributions and the binary symmetric channel is compared against the optimum performance theoretically attainable using rate distortion theoretic arguments as well as against the performance of lloyd max quantizers encoded using the natural binary code and the folded binary code it is shown that this optimal design could result in substantial performance improvements the performance improvements are more noticeable at high bit rates and for broad tailed densities

stochastic thermodynamics with information reservoirs we generalize stochastic thermodynamics to include information reservoirs such information reservoirs which can be modeled as a sequence of bits modify the second law for example work extraction from a system in contact with a single heat bath becomes possible if the system also interacts with an information reservoir we obtain an inequality and the corresponding fluctuation theorem generalizing the standard entropy production of stochastic thermodynamics from this inequality we can derive an information processing entropy production which gives the second law in the presence of information reservoirs we also develop a systematic linear response theory for information processing machines for a unicyclic machine powered by an information reservoir the efficiency at maximum power can deviate from the standard value of 1 2 for the case where energy is consumed to erase the tape the efficiency at maximum erasure rate is found to be 1 2

on the design of symmetric entropy constrained multiple description scalar quantizer with linear joint decoders this paper addresses the design of symmetric entropy constrained multiple description scalar quantizers ec mdsq with linear joint decoders i e where some of the decoders compute the reconstruction by averaging the reconstructions of individual descriptions thus the use of linear decoders reduces the space complexity at the decoder since only a subset of the codebooks needs to be stored the proposed design algorithm locally minimizes the lagrangian which is a weighted sum of the expected distortion and of the side quantizers rates the algorithm is inspired by the ec mdsq design algorithm proposed by vaishampayan and domaszewicz and it is adapted from two to k descriptions differently from the aforementioned work the optimization of the reconstruction values can no longer be performed separately at the decoder optimization step interestingly we show that the problem is a convex quadratic optimization problem which can be efficiently solved moreover the generalization of the encoder optimization step from two to k descriptions increases drastically the amount of computations we show how to exploit the special form of the cost function conferred by the linear joint decoders to significantly reduce the time complexity at this step we compare the performance of the proposed design with multiple description lattice vector quantizers mdlvq and with the multiple description scheme based on successive refinement and unequal erasure protection uep our experiments show that the proposed approach outperforms mdlvq with dimension 1 quantization as expected additionally when more codebooks are added our scheme even beats mdlvq with quantization dimension approaching infty for rates sufficiently high furthermore the proposed approach is also superior to uep with dimension 1 quantization when the rates are low

optimal subband filter banks for multiple description coding multiple description coding refers to the encoding of an information source into multiple bit streams in a manner that each bit stream independently represents a coarse description of the source while multiple descriptions jointly convey a higher fidelity source representation thus multiple description coding allows enhanced resilience to loss of individual descriptions and can be viewed as a form of source based diversity coding this paper addresses the design of subband filter banks to optimize the rate distortion performance of multiple description subband coding the problem is formulated in a lagrangian optimization framework and is solved directly in the frequency domain to produce the optimal filter spectral responses by varying the lagrangian parameter our design obtains all points on the redundancy distortion curve for gaussian wide sense stationary input processes we consider the cases of both orthonormal and more general biorthogonal filter banks we also analyze the connections between these two solutions and quantify the gap in their optimal performance application of the proposed methods to the popular first order autoregressive ar 1 model yields very interesting and insightful results in the two extreme cases of maximum and minimum redundancy our solutions degenerate to previously known results finally comparisons with popularly deployed filter banks reveal large gaps from the theoretical performance bound

entropy constrained vector quantization an iterative descent algorithm based on a lagrangian formulation for designing vector quantizers having minimum distortion subject to an entropy constraint is discussed these entropy constrained vector quantizers ecvqs can be used in tandem with variable rate noiseless coding systems to provide locally optimal variable rate block source coding with respect to a fidelity criterion experiments on sampled speech and on synthetic sources with memory indicate that for waveform coding at low rates about 1 bit sample under the squared error distortion measure about 1 6 db improvement in the signal to noise ratio can be expected over the best scalar and lattice quantizers when block entropy coded with block length 4 even greater gains are made over other forms of entropy coded vector quantizers for pattern recognition it is shown that the ecvq algorithm is a generalization of the k means and related algorithms for estimating cluster means in that the ecvq algorithm estimates the prior cluster probabilities as well experiments on multivariate gaussian distributions show that for clustering problems involving classes with widely different priors the ecvq outperforms the k means algorithm in both likelihood and probability of error

multiple description coding with randomly and uniformly offset quantizers in this paper two multiple description coding schemes are developed based on prediction induced randomly offset quantizers and unequal deadzone induced near uniformly offset quantizers respectively in both schemes each description encodes one source subset with a small quantization stepsize and other subsets are predictively coded with a large quantization stepsize in the first method due to predictive coding the quantization bins that a coefficient belongs to in different descriptions are randomly overlapped the optimal reconstruction is obtained by finding the intersection of all received bins in the second method joint dequantization is also used but near uniform offsets are created among different low rate quantizers by quantizing the predictions and by employing unequal deadzones by generalizing the recently developed random quantization theory the closed form expression of the expected distortion is obtained for the first method and a lower bound is obtained for the second method the schemes are then applied to lapped transform based multiple description image coding the closed form expressions enable the optimization of the lapped transform an iterative algorithm is also developed to facilitate the optimization theoretical analyzes and image coding results show that both schemes achieve better performance than other methods in this category

allpass delay chain based iir pr filterbank and its application to multiple description subband coding a class of infinite impulse response iir perfect reconstruction pr filterbank is obtained with an allpass delay chain and finite impulse response fir matrices at the analysis side such a formulation leads to a filterbank that can be optimized to any desired response for a first order allpass the synthesis bank becomes fir design examples showing warping of the frequency scale and filters with unequal passband widths are presented the latter part of the work deals with designing a filterbank for a multiple description coding scenario this is achieved in two parts by obtaining the mmse synthesis bank for a given analysis bank and channel state and by optimizing the analysis response to minimize the average distortion in the presence of channel erasures and quantization the proposed iir filterbank as well as the well known fir paraunitary filterbank are optimized for various channels and their performances for the two channel case have been compared with the derived optimal performance

the analysis and design of windowed fourier frame based multiple description source coding schemes in this paper the windowed fourier encoding decoding scheme applied to the multiple description compression problem is analyzed in the general case four window functions are needed to define the encoder and decoder although this number can be reduced to three or two by using time shift or frequency shift division schemes the encoding coefficients are next divided into two groups according to the eveness of either the modulation or translation index the distortion on each channel is analyzed using the zak transform for the optimal windows explicit representation formulas are obtained and nonlocalization results are proved asymptotic formulas of the total distortion and transmission rate are established and the redundancy is shown to trade off between these two

n channel asymmetric entropy constrained multiple description lattice vector quantization this paper is about the design and analysis of an index assignment ia based multiple description coding scheme for the n channel asymmetric case we use entropy constrained lattice vector quantization and restrict attention to simple reconstruction functions which are given by the inverse ia function when all descriptions are received or otherwise by a weighted average of the received descriptions we consider smooth sources with finite differential entropy rate and mse fidelity criterion as in previous designs our construction is based on nested lattices which are combined through a single ia function the results are exact under high resolution conditions and asymptotically as the nesting ratios of the lattices approach infinity for any n the design is asymptotically optimal within the class of ia based schemes moreover in the case of two descriptions and finite lattice vector dimensions greater than one the performance is strictly better than that of existing designs in the case of three descriptions we show that in the limit of large lattice vector dimensions points on the inner bound of pradhan can be achieved furthermore for three descriptions and finite lattice vector dimensions we show that the ia based approach yields in the symmetric case a smaller rate loss than the recently proposed source splitting approach

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

the rate distortion region for multiple descriptions without excess rate during recent years there has been strong interest in a certain source coding problem which some authors call the problem of multiple descriptions old and new wringing techniques enable us to establish a single letter characterization of the rate distortion region in the case of no excess rate for the joint description

optimality and suboptimality of multiple description vector quantization with a lattice codebook the asymptotic analysis of multiple description vector quantization mdvq with a lattice codebook for sources with smooth probability density functions pdfs is considered in this correspondence goyal et al 2002 observed that as the side distortion decreases and the central distortion correspondingly increases the quantizer cells farther away from the coarse lattice points shrink in a spatially periodic pattern in this correspondence two special classes of index assignments are used along strategic groupings of central quantizer cells to derive a straightforward asymptotic analysis which provides an analytical explanation for the aforementioned observation mdvq with a lattice codebook was shown earlier to be asymptotically optimal in high dimensions with a curious converging property that the side quantizers achieve the space filling advantage of an n dimensional sphere instead of an n dimensional optimal polytope the analysis presented here explains this behavior readily while central quantizer cells on a uniform lattice are asymptotically optimal in high dimensions the present authors have shown that by using nonuniform rather than uniform central quantizer cells the central side distortion product in an mdsq can be reduced by 0 4 db at asymptotically high rate the asymptotic analysis derived here partially unifies these previous results in the same framework though a complete characterization is still beyond reach

bit error resilient index assignment for multiple description scalar quantizers this paper addresses the problem of increasing the robustness to bit errors for two description scalar quantizers our approach is to start with an m diagonal index assignment and further apply a permutation to the indexes of each description to increase the minimum hamming distance d rm min of the set of valid index pairs in particular we show how to construct linear permutation pairs achieving d rm min geq 3 and establish a lower bound in terms of the description rate r for the highest value of m for which such permutations exist

resource scalable joint source channel map and mmse estimation of multiple descriptions a joint source channel multiple description jsc md framework for signal estimation and communication in resource constrained lossy networks is presented to keep the encoder complexity at a minimum a signal is coded by a multiple description quantizer mdq with neither entropy nor channel coding the code diversity of mdq and the path diversity of the network are exploited by decoders to combat transmission errors a key design objective is resource scalability powerful nodes in the network can perform jsc md estimation under the criteria of maximum a posteriori probability map or minimum mean square error mmse while primitive nodes resort to simpler md decoding all working with the same mdq code the application of jsc md to distributed estimation of hidden markov models in a sensor network is demonstrated the proposed jsc md map estimator is an algorithm of the longest path in a weighted directed acyclic graph while the jsc md mmse decoder is an extension of the well known forward backward algorithm to multiple descriptions both algorithms simultaneously exploit the source memory the redundancy of the fixed rate mdq and the inter description correlations they outperform the existing hard decision mdq decoders by large margins up to 8 db for gaussian markov sources the complexity of jsc md distributed map sequence estimation can be made as low as that of typical single description viterbi type algorithms

alphabet constrained data compression the optimal data compression problem is posed in terms of an alphabet constraint rather than an entropy constraint solving the optimal alphabet constrained data compression problem yields explicit source encoder decoder designs which is in sharp contrast to other approaches the alphabet constrained approach is shown to have the additional advantages that 1 classical waveform encoding schemes such as pulse code modulation pcm differential pulse code modulation dpcm and delta modulation dm as well as rate distortion theory motivated tree trellis coders fit within this theory 2 the concept of preposterior analysis in data compression is introduced yielding a rich new class of coders and 3 it provides a conceptual framework for the design of joint source channel coders for noisy channel applications examples are presented of single path differential encoding delayed or tree encoding preposterior analysis and source coding over noisy channels

minimum breakdown degradation in binary source encoding a memoryless binary equiprobable source produces one letter per second two people each are provided separately with private information about the source data at a rate of 1 2 bit per second suppose that by pooling their information they can produce a long mn reconstruction of the source output that has arbitrarily small error frequency we prove that then the least common asymptotic error frequency d that each can achieve without the other s help is sqrt 2 1 2 0 207 since it had been shown previously that 0 200 leq d leq 0 207 our result closes the so called 007 gap new analytical techniques introduced to effect the proof are of broader significance in multiuser information theory

nonequilibrium detailed fluctuation theorem for repeated discrete feedback we extend the framework of forward and reverse processes commonly utilized in the derivation and analysis of the nonequilibrium work relations to thermodynamic processes with repeated discrete feedback within this framework we derive a generalization of the detailed fluctuation theorem which is modified by the addition of a term that quantifies the change in uncertainty about the microscopic state of the system upon making measurements of physical observables during feedback as an application we extend two nonequilibrium work relations the nonequilibrium work fluctuation theorem and the relative entropy work relation

optimum quantizer performance for a class of non gaussian memoryless sources the performance of optimum quantizers subject to an entropy constraint is studied for a wide class of memoryless sources for a general distortion criterion necessary conditions are developed for optimality and a recursive algorithm is described for obtaining the optimum quantizer under a mean square error criterion the performance of entropy encoded uniform quantization of memoryless gaussian sources is well known to be within 0 255 bits sample of the rate distortion bound at relatively high rates despite claims to the contrary it is demonstrated that similar performance can be expected for a wide range of memoryless sources indeed for the cases considered the worst case performance is observed to be less than 0 3 bits sample from the rate distortion bound and in most cases this disparity is less at iow rates

ldgm based multiple description coding for finite alphabet sources this work presents an ldgm based practical successive coding scheme for the multiple description md problem for finite alphabet sources the scheme which targets the zhang berger zb rate distortion region is shown to be asymptotically optimal with joint typicality encoding while as a practical encoding solution a message passing algorithm is adopted we further discuss in more detail the application of the coding scheme in three cases of the md problem with the hamming distortion measure 1 no excess sum rate for binary sources 2 successive refinement and 3 no excess marginal rate for the uniform binary source in the no excess sum rate case some progress is made in the characterization of fundamental limits by deriving the analytical expression of the distortion region for general binary sources and of the auxiliary variables needed to achieve its boundary the exact expression of the zhang berger upper bound to the central distortion is also provided for the case of no excess marginal rate for the uniform binary source the proposed ldgm based coding scheme is tested in practice for all three aforementioned cases the experimental results show very good performance demonstrating its ability to approach the theoretical rate distortion limits or the available upper bounds

thermodynamics of genuine nonequilibrium states under feedback control for genuine nonequilibrium states that even at fixed external control parameter exhibit dissipation we extend the hatano sasa equality to processes with feedback control the resulting bound on the maximal extractable work is substantially sharper than what would follow from applying the sagawa ueda equality to transitions involving such states for repeated measurements at short enough intervals the power thus extracted can even exceed the average cost of driving as demonstrated explicitly with a simple analytically solvable example

network vector quantization we present an algorithm for designing locally optimal vector quantizers for general networks we discuss the algorithm s implementation and compare the performance of the resulting network vector quantizers to traditional vector quantizers vqs and to rate distortion r d bounds where available while some special cases of network codes e g multiresolution mr and multiple description md codes have been studied in the literature we here present a unifying approach that both includes these existing solutions as special cases and provides solutions to previously unsolved examples

feature oriented multiple description wavelet based image coding we address the problem of resilient image coding over error prone networks where packet losses occur recent literature highlights the multiple description coding mdc as a promising approach to solve this problem in this paper we introduce a novel wavelet based multiple description image coder referred to as the feature oriented mdc fo mdc the proposed multiple description md coder exploits the statistics of the wavelet coefficients and identifies the subsets of samples that are sensitive to packet loss a joint optimization between tree pruning and quantizer selection in the rate distortion sense is used in order to allocate more bits to these sensitive coefficients when compared with the state of the art md scalar quantization coder the proposed fo mdc yields a more efficient central side distortion tradeoff control mechanism furthermore it proves to be more robust for image transmission even with high packet loss ratios which makes it suitable for protecting multimedia streams over packet erasure channels

n channel entropy constrained multiple description lattice vector quantization in this paper we derive analytical expressions for the central and side quantizers which under high resolution assumptions minimize the expected distortion of a symmetric multiple description lattice vector quantization md lvq system subject to entropy constraints on the side descriptions for given packet loss probabilities we consider a special case of the general n channel symmetric multiple description problem where only a single parameter controls the redundancy tradeoffs between the central and the side distortions previous work on two channel md lvq showed that the distortions of the side quantizers can be expressed through the normalized second moment of a sphere we show here that this is also the case for three channel md lvq furthermore we conjecture that this is true for the general n channel md lvq for given source target rate and packet loss probabilities we find the optimal number of descriptions and construct the md lvq system that minimizes the expected distortion we verify theoretical expressions by numerical simulations and show in a practical setup that significant performance improvements can be achieved over state of the art two channel md lvq by using three channel md lvq

error resilient multiple description coding in the multiple description coding mdc the source is decomposed into two or more descriptions and then transmitted over on off channels at the receiver end each description can be reproduced with acceptable side distortion reconstructed signals with lower central distortion can be accomplished by exploiting more descriptions in order to apply the classical mdc over error prone channels we propose error resilient multiple description coding ermdc to utilize the inherent dependency and redundancy between descriptions to combat bit errors in this paper we develop a pair of the ermdc encoder and central decoder based on the multiple description scalar quantizer mdsq after deriving optimal and suboptimal ermdc decoders a novel index assignment algorithm is provided for the ermdc encoder to improve the error detection capability of the ermdc decoders analytical and experimental results show that the ermdc outperforms the conventional mdsq encoder decoder pair and achieves lower reconstruction distortions than the side distortion within a large range of bit error rates bers

new results in binary multiple descriptions an encoder whose input is a binary equiprobable memoryless source produces one output of rate r 1 and another of rate r 2 let d 1 d 2 and d 0 respectively denote the average error frequencies with which the source data can be reproduced on the basis of the encoder output of rate r l only the encoder output of rate r 2 only and both encoder outputs the two descriptions problem is to determine the region r of all quintuples r 1 r 2 d 1 d 2 d 0 that are achievable in thc usual shannon sense let r d 1 d log 2 d 1 d log 2 1 d denote the error frequency rate distortion function of the source the no excess rate case prevails when r 1 r 2 r d 0 and the excess rate case when r 1 r 2 r d 0 denote the section of r at r 1 r 2 d 0 by d r 1 r 2 d 0 d 1 d 2 r 1 r 2 d 1 d 2 d 0 in r in the no excess rate case we show that a portion of the boundary of d r 1 r 2 d 0 coincides with the curve frac 1 2 d 1 2d 0 frac 1 2 d 2 2d 0 frac 1 2 1 2d 0 2 this curve is an extension of witsenhausen s hyperbola bound to the case d 0 0 it follows that the projection of r onto the d 1 d 2 plane at fixed d 0 consists of all d 1 geq d 0 and d 2 geq d 0 that lie on or above this hyperbola in the excess rate case we show by counterexample that the achievable region of el gamal and cover is not tight

correlation powered information engines and the thermodynamics of self correction information engines can use structured environments as a resource to generate work by randomizing ordered inputs and leveraging the increased shannon entropy to transfer energy from a thermal reservoir to a work reservoir we give a broadly applicable expression for the work production of an information engine generally modeled as a memoryful channel that communicates inputs to outputs as it interacts with an evolving environment the expression establishes that an information engine must have more than one memory state in order to leverage input environment correlations to emphasize this functioning we designed an information engine powered solely by temporal correlations and not by statistical biases as employed by previous engines key to this is the engine s ability to synchronize the engine automatically returns to a desired dynamical phase when thrown into an unwanted dissipative phase by corruptions in the input that is by unanticipated environmental fluctuations this self correcting mechanism is robust up to a critical level of corruption beyond which the system fails to act as an engine we give explicit analytical expressions for both work and critical corruption level and summarize engine performance via a thermodynamic function phase diagram over engine control parameters the results reveal a thermodynamic mechanism based on nonergodicity that underlies error correction as it operates to support resilient engineered and biological systems

an improvement to multiple description transform coding a multiple description transform coding mdtc method has been reported previously the redundancy rate distortion rrd performance of this coding scheme for the independent and identically distributed i i d two dimensional 2 d gaussian source has been analyzed using mean squared error mse as the distortion measure at the small redundancy region the mdtc scheme can achieve excellent rrd performance because a small increase in redundancy can reduce the single description distortion at a rate faster than exponential but the performance of mdtc becomes increasingly poor at larger redundancies this paper describes a generalization of the mdtc gmdtc scheme which introduces redundancy both by transform and through correcting the error resulting from a single description its rrd performance is closer to the theoretical bound in the entire range of redundancy analysis both for a single pair of variables and for multiple variables is presented

achievable rates for multiple descriptions with feed forward the two channel multiple descriptions problem for an independent and identically distributed i i d source with feed forward to one or both side decoders is considered a single letter achievable rate region is derived it enlarges the best known rate region for multiple descriptions without feed forward the proof of the result uses a block markov superposition source coding strategy in point to point source coding feed forward does not decrease the rate distortion function of an i i d source in contrast an example is provided to show that the derived region can be strictly larger than the optimal multiple description rate distortion region without feed forward

optimal index assignment for multiple description scalar quantization with translated lattice codebooks we design a k description scalar quantizer whose construction is based on a structure of translated scalar lattices and a lattice in k 1 dimensional space the use of translated lattices provides a performance advantage by exploiting a so called staggering gain the use of the k 1 dimensional lattice facilitates analytic insight into the performance and significantly speeds up the computation of the index assignment compared to state of the art methods using a common decoding method the proposed index assignment is proven to be optimal for the k description case it is shown that the optimal index assignment is not unique this is illustrated for the two description case where a periodic index assignment is selected from possible optimal assignments and described in detail the performance of the proposed quantizer accurately matches theoretic analysis over the full range of operational redundancies moreover the quantizer outperforms the state of the art md scheme as the redundancy among the description increases

tracking the best quantizer an algorithm is presented for online prediction that allows to track the best expert efficiently even when the number of experts is exponentially large provided that the set of experts has a certain additive structure as an example we work out the case where each expert is represented by a path in a directed graph and the loss of each expert is the sum of the weights over the edges in the path these results are then used to construct universal limited delay schemes for lossy coding of individual sequences in particular we consider the problem of tracking the best scalar quantizer that is adaptively matched to the source sequence with piecewise different behavior a randomized algorithm is presented which can perform on any source sequence asymptotically as well as the best scalar quantization algorithm that is matched to the sequence and is allowed to change the employed quantizer for a given number of times the complexity of the algorithm is quadratic in the sequence length but at the price of some deterioration in performance the complexity can be made linear analogous results are obtained for sequential multiresolution and multiple description scalar quantization of individual sequences

generalized multiple description coding with correlating transforms multiple description md coding is source coding in which several descriptions of the source are produced such that various reconstruction qualities are obtained from different subsets of the descriptions unlike multiresolution or layered source coding there is no hierarchy of descriptions thus md coding is suitable for packet erasure channels or networks without priority provisions generalizing work by orchard wang vaishampayan and reibman see proc ieee int conf image processing vol i santa barbara ca p 608 11 1997 a transform based approach is developed for producing m descriptions of an n tuple source m spl les n the descriptions are sets of transform coefficients and the transform coefficients of different descriptions are correlated so that missing coefficients can be estimated several transform optimization results are presented for memoryless gaussian sources including a complete solution of the n 2 m 2 case with arbitrary weighting of the descriptions the technique is effective only when independent components of the source have differing variances numerical studies show that this method performs well at low redundancies as compared to uniform md scalar quantization

multiple description wavelet based image coding we consider the problem of coding images for transmission over error prone channels the impairments we target are transient channel shutdowns as would occur in a packet network when a packet is lost or in a wireless system during a deep fade when data is delivered it is assumed to be error free but some of the data may never reach the receiver the proposed algorithms are based on a combination of multiple description scalar quantizers with techniques successfully applied to the construction of some of the most efficient subband coders a given image is encoded into multiple independent packets of roughly equal length when packets are lost the quality of the approximation computed at the receiver depends only on the number of packets received but does not depend on exactly which packets are actually received when compared with previously reported results on the performance of robust image coders based on multiple descriptions on standard test images our coders attain similar psnr values using typically about 50 60 of the bit rate required by these other state of the art coders while at the same time providing significantly more freedom in the mechanism for allocation of redundancy among descriptions

quantization as histogram segmentation optimal scalar quantizer design in network systems an algorithm for scalar quantizer design on discrete alphabet sources is proposed the proposed algorithm can be used to design fixed rate and entropy constrained conventional scalar quantizers multiresolution scalar quantizers multiple description scalar quantizers and wyner ziv scalar quantizers the algorithm guarantees globally optimal solutions for conventional fixed rate scalar quantizers and entropy constrained scalar quantizers for the other coding scenarios the algorithm yields the best code among all codes that meet a given convexity constraint in all cases the algorithm run time is polynomial in the size of the source alphabet the algorithm derivation arises from a demonstration of the connection between scalar quantization histogram segmentation and the shortest path problem in a certain directed acyclic graph

asymptotic performance of multiple description transform codes the multiple description transform coder is introduced for sources with memory and an asymptotic analysis is presented for the squared error distortion for stationary gaussian sources the optimal transform and the optimal bit allocation for the multiple description coder are identical to those for the single description coder

on the bandwidth of 3 dimensional hamming graphs this paper presents strategies for improving the known upper and lower bounds for the bandwidth of hamming graphs k n d and 0 1 d our labeling strategy lowers the upper bound on the bandwidth of the continuous hamming graph 0 1 3 from 5 to 4497 a lower bound of 4439 on bw 0 1 3 follows from known isoperimetric inequalities and a related dynamic program is conjectured to raise that lower bound to 4 9 4444 in particular showing the power of our method we prove that the bandwidth of k 6xk 6xk 6 is exactly 101

semiclassical limits for the hyperbolic plane we study the concentration properties of high energy eigenfunctions for the laplace beltrami operator of the hyperbolic plane with special consideration of automorphic eigenfunctions at the center of our investigation is the microlocal lift of an eigenfunction to sl 2 introduced by s zelditch the microlocal lift is based on s helgason s fourier transform and has a straightforward description in terms of the lie algebra sl 2 we begin with an elementary demonstration of zelditch s exact differential equation for the microlocal lift further for a sequence of suitably bounded eigenfunctions with eigenvalues tending to infinity we show that the limit of microlocal lifts is a geodesic flow invariant positive measure our main consideration is the microlocal lifts of the elementary eigenfunctions constructed from the macdonald bessel functions we find that with a scaling of auxiliary parameters the corresponding high energy limit converges to the positive dirac measure for the lift to sl 2 of a single geodesic on the upper half plane in particular at high energy the microlocal lift of a macdonald bessel function is concentrated along the lift of a single geodesic we further find that the convergence is uniform in parameters and thus our considerations can be applied to study automorphic eigenfunctions in particular the maass cusp forms for certain cofinite subgroups of sl 2 a formula is presented for the microlocal lift of a cusp form as an integral of a family of dirac measures and a measure given directly by sums of products of the classical fourier coefficients of the cusp form the formula establishes that questions on quantum chaos for automorphic eigenfunctions are equivalent to classically stated questions on twisted sums of coefficients in particular we find that the uniform distribution for the microlocal limit is equivalent to a uniform distribution for the limit of twisted coefficient sums we extend the considerations to the case of the nonholomorphic eisenstein series for the modular group we find that a result of w luo and p sarnak is equivalent to a limit sum formula involving the riemann zeta function and the elementary divisor function the formula is suggestive of a formula of s ramanujan

multiple description vector quantization with a coarse lattice a multiple description md lattice vector quantization technique for two descriptions was previously introduced in which fine and coarse codebooks are both lattices the encoding begins with quantization to the nearest point in the fine lattice this encoding is an inherent optimization for the decoder that receives both descriptions performance can be improved with little increase in complexity by considering all decoders in the initial encoding step the altered encoding relies only on the symmetries of the coarse lattice this allows us to further improve performance without a significant increase in complexity by replacing the fine lattice codebook with a nonlattice codebook that respects many of the symmetries of the coarse lattice examples constructed with the two dimensional 2 d hexagonal lattice demonstrate large improvement over time sharing between previously known quantizers

successive refinement of information the successive refinement of information consists of first approximating data using a few bits of information then iteratively improving the approximation as more and more information is supplied the goal is to achieve an optimal description at each stage in general an ongoing description which is rate distortion optimal whenever it is interrupted is sought it is shown that in order to achieve optimal successive refinement the necessary and sufficient conditions are that the solutions of the rate distortion problem can be written as a markov chain in particular all finite alphabet signals with hamming distortion satisfy these requirements it is also shown that the same is true for gaussian signals with squared error distortion and for laplacian signals with absolute error distortion a simple counterexample with absolute error distortion and a symmetric source distribution which shows that successive refinement is not always achievable is presented

multiple description coding with prediction compensation a new multiple description coding paradigm is proposed by combining the time domain lapped transform block level source splitting linear prediction and prediction residual encoding the method provides effective redundancy control and fully utilizes the source correlation the joint optimization of all system components and the asymptotic performance analysis are presented image coding results demonstrate the superior performance of the proposed method especially at low redundancies

equidistribution of minimal hypersurfaces for generic metrics for almost all riemannian metrics in the c infty baire sense on a closed manifold m n 1 3 le n 1 le 7 we prove that there is a sequence of closed smooth embedded connected minimal hypersurfaces that is equidistributed in m this gives a quantitative version of the main result of irie et al ann math 187 3 963 972 2018 that established density of minimal hypersurfaces for generic metrics as in irie et al 2018 the main tool is the weyl law for the volume spectrum proven by liokumovich et al ann math 187 3 933 961 2018

multiple description quantization via gram schmidt orthogonalization the multiple description md problem has received considerable attention as a model of information transmission over unreliable channels a general framework for designing efficient md quantization schemes is proposed in this paper we provide a systematic treatment of the el gamal cover egc achievable md rate distortion region and show it can be decomposed into a simplified egc segc region and a superimposed refinement operation furthermore any point in the segc region can be achieved via a successive quantization scheme along with quantization splitting for the quadratic gaussian case the proposed scheme has an intrinsic connection with the gram schmidt orthogonalization which implies that the whole gaussian md rate distortion region is achievable with a sequential dithered lattice based quantization scheme as the dimension of the optimal lattice quantizers becomes large moreover this scheme is shown to be universal for all independent and identically distributed i i d smooth sources with performance no worse than that for an i i d gaussian source with the same variance and asymptotically optimal at high resolution a class of md scalar quantizers in the proposed general framework is also constructed and is illustrated geometrically the performance is analyzed in the high resolution regime which exhibits a noticeable improvement over the existing md scalar quantization schemes

achievable rates for multiple descriptions consider a sequence of independent identically distributed i i d random variables x l x 2 cdots x n and a distortion measure d x i x i on the estimates x i of x i two descriptions i x in 1 2 cdots 2 nr 1 and j x in 1 2 cdots 2 nr 2 are given of the sequence x x 1 x 2 cdots x n from these two descriptions three estimates i x x2 j x and hat x o i x j x are formed with resulting expected distortions e frac 1 n sum n k 1 d x k hat x mk d m m 0 1 2 we find that the distortion constraints d 0 d 1 d 2 are achievable if there exists a probability mass distribution p x p hat x 1 hat x 2 hat x 0 x with ed x hat x m leq d m such that r 1 i x hat x 1 r 2 i x hat x 2 where i cdot denotes shannon mutual information these rates are shown to be optimal for deterministic distortion measures

multiple description trellis coded quantization of sinusoidal parameters a new scheme for sinusoidal audio coding named multiple description spherical trellis coded quantization is proposed and analytic expressions for the point densities and expected distortion of the quantizers are derived based on a high resolution assumption the proposed quantizers are of variable dimensions meaning that any number of sinusoids can be quantized jointly for each audio segment whereby a lower distortion is achieved compared to previously published scalar spherical quantizers the quantizers are designed to minimize a perceptual distortion measure subject to an entropy constraint for a given packet loss probability in experiments the performance of the quantizers is assessed and compared to the corresponding single description spherical quantizer and associated bounds under various conditions and is found to increase robustness towards packet loss

thermodynamics of information processing based on enzyme kinetics an exactly solvable model of an information pump motivated by the recent proposed models of the information engine proc natl acad sci usa 109 11641 2012 and the information refrigerator phys rev lett 111 030602 2013 we propose a minimal model of the information pump and the information eraser based on enzyme kinetics this device can either pump molecules against the chemical potential gradient by consuming the information to be encoded in the bit stream or partially erase the information initially encoded in the bit stream by consuming the gibbs free energy the dynamics of this model is solved exactly and the phase diagram of the operation regimes is determined the efficiency and the power of the information machine is analyzed the validity of the second law of thermodynamics within our model is clarified our model offers a simple paradigm for the investigating of the thermodynamics of information processing involving the chemical potential in small systems

multiple description distribution preserving quantization the notion of a multiple description quantizer mdq includes providing multiple distortion levels if a human observer is involved the design of mdq requires a suitable distortion measure to achieve a graceful quality degradation in the case of description losses while the mean squared error is a ubiquitous distortion measure for many classic mdq schemes it is known to be perceptually relevant only at low distortions we propose a new mdq designed according to an unconventional distortion criterion that combines the mean squared error with a constraint on the probability distribution of the reconstructed signal the performance of the new mdq is shown to approach that of the classic mdq asymptotically as rate increases however once applied in the context of transform audio coding the new mdq significantly outperforms a classic mdq in perceptual tests the new scheme is suitable for a wide range of distortions and renders a seamless transition between coding that preserves signal features and coding of a waveform

symmetric scalable multiple description scalar quantization real time data delivery over best effort error prone packet networks has invigorated the study of robust coding schemes such as scalable multiple description coding smdc in this context the paper introduces a novel generic symmetric scalable multiple description quantizer ssmdsq which generates perfectly balanced source descriptions novel embedded index assignments are proposed which are used to realize high as well as medium to low redundancy ssmdsqs compared to existing designs it is shown that the proposed quantizer constructions exhibit superior distortion rate d r performance moreover this paper describes an innovative extension of the lloyd max algorithm in order to optimize symmetric and asymmetric scalable multiple description quantizers for a family of generalized gaussian gg source distributions the proposed optimization algorithm yields on average a significant d r performance gain over unoptimized quantizers furthermore anchored in the designed ssmdsqs an smdc framework is established to realize packet based transmission over erasure channels in this framework transmission strategies are determined for scenarios wherein the average packet loss rate over the transmission link is a unknown and b can be estimated at the encoder for both scenarios smdc packetized transmission is simulated for a family of gg distributions experimental results confirm that compared to contemporary schemes the designed quantizer constructions with or without optimization account for a significant average gain in signal to noise ratio snr for a wide range of packet loss rates

multiple description vector quantization with lattice codebooks design and analysis the problem of designing a multiple description vector quantizer with lattice codebook spl lambda is considered a general solution is given to a labeling problem which plays a crucial role in the design of such quantizers numerical performance results are obtained for quantizers based on the lattices a sub 2 and z sup i i 1 2 4 8 that make use of this labeling algorithm the high rate squared error distortions for this family of l dimensional vector quantizers are then analyzed for a memoryless source with probability density function pdf p and differential entropy h p spl infin for any a spl epsiv 0 1 and rate pair r r it is shown that the two channel distortion d sub o and the channel 1 or channel 2 distortion d sub s satisfy lim sub r spl rarr spl infin d sub o 2 sup 2r 1 a 1 4 g spl lambda 2 sup 2h p and lim sub r spl rarr spl infin d sub s 2 sup 2r 1 a g s sub l 2 sup 2h p where g spl lambda is the normalized second moment of a voronoi cell of the lattice spl lambda and g s sub l is the normalized second moment of a sphere in l dimensions

trace formula for compact gamma setminus psl_ 2 mathbb r and the equidistribution theory of closed geodesics soit un sous groupe cocompact discret d elements hyperboliques de g psl 2 r et soit x t la surface hyperbolique associee g k le fibre unite co tangent s x t peut s identifier a g des classes de conjugaison d elements hyperboliques determinent des geodesiques fermees dans s x t pour chaque geodesique fermee soit la mesure positive sur c s x definie par p uf soit t ld t ou l est la longueur de on donne des developpements asymptotiques de t f quand f est une fonction propre de casimir de poids m

vector gaussian multiple description with individual and central receivers t multiple descriptions of a vector gaussian source for individual and central receivers are investigated the sum rate of the descriptions with covariance distortion measure constraints in a positive semidefinite ordering is exactly characterized for two descriptions the entire rate region is characterized the key component of the solution is a novel information theoretic inequality that is used to lower bound the achievable multiple description rates jointly gaussian descriptions are optimal in achieving the limiting rates we also show the robustness of this description scheme the distortions achieved are no larger when used to describe any non gaussian source with the same covariance matrix

asymptotic analysis of multiple description quantizers a high rate analysis of multiple description quantizers is presented for rth power distortions and general source densities both fixed length and variable length encoding of the quantizer indices are considered optimal companding functions are shown to be the same as for single channel quantizers as compared to the bound of ozarow 1980 a gap of 8 69 db and 3 07 db exists between the entropy constrained and level constrained cases respectively for a memoryless gaussian source and r 2

fluctuation theorem with information exchange role of correlations in stochastic thermodynamics we establish the fluctuation theorem in the presence of information exchange between a nonequilibrium system and other degrees of freedom such as an observer and a feedback controller where the amount of information exchange is added to the entropy production the resulting generalized second law sets the fundamental limit of energy dissipation and energy cost during the information exchange our results apply not only to feedback controlled processes but also to a much broader class of information exchanges and provide a unified framework of nonequilibrium thermodynamics of measurement and feedback control

uniform distribution of eigenfunctions on compact hyperbolic surfaces on demontre que les fonctions propres k du laplacien sur une surface hyperbolique compacte x devient uniformement distribuee sur x quand k

multiple description image coding based on lagrangian rate allocation in this paper a novel multiple description coding technique is proposed based on optimal lagrangian rate allocation the method assumes the coded data consists of independently coded blocks initially all the blocks are coded at two different rates then blocks are split into two subsets with similar rate distortion characteristics two balanced descriptions are generated by combining code blocks belonging to the two subsets encoded at opposite rates a theoretical analysis of the approach is carried out and the optimal rate distortion conditions are worked out the method is successfully applied to the jpeg 2000 standard and simulation results show a noticeable performance improvement with respect to state of the art algorithms the proposed technique enables easy tuning of the required coding redundancy moreover the generated streams are fully compatible with part 1 of the standard

generalized jarzynski equality under nonequilibrium feedback control the jarzynski equality is generalized to situations in which nonequilibrium systems are subject to a feedback control the new terms that arise as a consequence of the feedback describe the mutual information content obtained by measurement and the efficacy of the feedback control our results lead to a generalized fluctuation dissipation theorem that reflects the readout information and they can be experimentally tested using small thermodynamic systems we illustrate our general results by introducing an information ratchet which can transport a brownian particle in one direction and extract positive work from the particle

combinatorial message sharing and a new achievable region for multiple descriptions this paper presents a new achievable rate distortion region for the general l channel multiple descriptions mds problem a well known general region for this problem is due to venkataramani kramer and goyal vkg their encoding scheme is an extension of the el gamal cover ec and zhang berger zb coding schemes to the l channel case and includes a combinatorial number of refinement codebooks one for each subset of the descriptions as in zb the scheme also allows for a single common codeword to be shared by all descriptions this paper proposes a novel encoding technique involving combinatorial message sharing cms where every subset of the descriptions may share a distinct common message this introduces a combinatorial number of shared codebooks along with the refinement codebooks of these shared codebooks provide a more flexible framework to tradeoff redundancy across the messages for resilience to descriptions loss we derive an achievable rate distortion region for the proposed technique and show that it subsumes the vkg region for general sources and distortion measures we further show that cms provides a strict improvement of the achievable region for any source and distortion measures for which some two description subset is such that zb achieves points outside the ec region we then show a more surprising result cms outperforms vkg for a general class of sources and distortion measures including scenarios where the zb and ec regions coincide for all two description subsets in particular we show that cms strictly improves on vkg for the l channel quadratic gaussian md problem for all l geq 3 despite the fact that the ec region is complete for the corresponding two descriptions problem consequently the correlated quantization scheme an extreme special case of vkg that has been proven to be optimal for several cross sections of the l channel quadratic gaussian md problem is strictly suboptimal in general using the encoding principles derived we show that the cms scheme achieves the complete rate distortion region for several asymmetric cross sections of the l channel quadratic gaussian md problem

n channel symmetric multiple descriptions part i n k source channel erasure codes in this two part paper we present a new achievable rate region for the general n channel symmetric multiple descriptions problem in part i inspired by the concept of maximum distance separable mds erasure channel codes we consider a special case of this rate region where the source is encoded into n descriptions each with rate r these descriptions are transmitted over n bandwidth constrained and errorless channels during transmission a subset of these channels can break down thus erasing the corresponding descriptions the decoder is interested in recovering the source with the reception of at least k descriptions thus the encoder is allowed to sample only one realization of this breakdown process during the entire transmission for gaussian sources we have the following interesting result when any k descriptions arrive the achievable distortion exactly matches the optimal distortion rate performance corresponding to a source rate of kr bits with the reception of any m k descriptions the source reconstruction quality is strictly better the improvement being nearly linear in the number of descriptions received

filter banks for prediction compensated multiple description coding in this paper a prediction compensated multiple description md coding framework for two band filter banks is proposed in which the coefficients in each subband are split into two descriptions each description also includes the prediction residuals of the data in the other description the designs of the optimal orthogonal and biorthogonal filter banks are formulated in a unified framework and both one level and multiple level decompositions are analyzed contrary to the existing md filter banks in the literature the optimal filter banks in the proposed scheme are quite similar to those in single description coding therefore the method can be applied to systems with single description optimized filter banks and still attain near optimal performance image coding results show that this method achieves better performance and lower complexity than the latest jpeg 2000 based mdc

optimal filter banks for multiple description coding analysis and synthesis multiple description md coding is a source coding technique for information transmission over unreliable networks in md coding the coder generates several different descriptions of the same signal and the decoder can produce a useful reconstruction of the source with any received subset of these descriptions in this paper we study the problem of md coding of stationary gaussian sources with memory first we compute an approximate md rate distortion region for these sources which we prove to be asymptotically tight at high rates this region generalizes the md rate distortion region of el gamal and cover 1982 and ozarow 1980 for memoryless gaussian sources then we develop an algorithm for the design of optimal two channel biorthogonal filter banks for md coding of gaussian sources we show that optimal filters are obtained by allocating the redundancy over frequency with a reverse water filling strategy finally we present experimental results which show the effectiveness of our filter banks in the low complexity low rate regime

design of multiple description scalar quantizers the design of scalar quantizers for communication systems that use diversity to overcome channel impairments is considered the design problem is posed as an optimization problem and necessary conditions for optimality are derived a design algorithm a generalization of s p lloyd s 1962 algorithm for quantizer design is developed unlike a single channel scalar quantizer the performance of a multiple description scalar quantizer is dependent on the index assignment the problem of index assignment is addressed good index assignments performance results and sample quantizer designs are presented for a memoryless gaussian source comparisons are made with rate distortion bounds for the multiple description problem

unifying three perspectives on information processing in stochastic thermodynamics so far feedback driven systems have been discussed using i measurement and control ii a tape interacting with a system or iii by identifying an implicit maxwell demon in steady state transport we derive the corresponding second laws from one master fluctuation theorem and discuss their relationship in particular we show that both the entropy production involving mutual information between system and controller and the one involving a shannon entropy difference of an information reservoir like a tape carry an extra term different from the usual current times affinity we thus generalize stochastic thermodynamics to the presence of an information reservoir

nonequilibrium fluctuation theorem for systems under discrete and continuous feedback control in the time reverse process of a feedback manipulated stochastic system we allow performing measurements without violating causality as a result we come across an entropy production due to the measurement process this entropy production in addition to the usual system and medium entropy production constitutes the total entropy production of the combined system of the reservoir the system and the feedback controller we show that this total entropy production of full system satisfies an integrated fluctuation theorem as well as a detailed fluctuation theorem as expected we illustrate and verify this idea through explicit calculation and direct simulation in two examples

tabu search based multi watermarks embedding algorithm with multiple description coding digital watermarking is a useful solution for digital rights management systems and it has been a popular research topic in the last decade most watermarking related literature focuses on how to resist deliberate attacks by applying benchmarks to watermarked media that assess the effectiveness of the watermarking algorithm only a few papers have concentrated on the error resilient transmission of watermarked media in this paper we propose an innovative algorithm for vector quantization vq based image watermarking which is suitable for error resilient transmission over noisy channels by incorporating watermarking with multiple description coding mdc the scheme we propose to embed multiple watermarks can effectively overcome channel impairments while retaining the capability for copyright and ownership protection in addition we employ an optimization technique called tabu search to optimize both the watermarked image quality and the robustness of the extracted watermarks we have obtained promising simulation results that demonstrate the utility and practicality of our algorithm

many roads to synchrony natural time scales and their algorithms we consider two important time scales the markov and cryptic orders that monitor how an observer synchronizes to a finitary stochastic process we show how to compute these orders exactly and that they are most efficiently calculated from the e machine a process s minimal unifilar model surprisingly though the markov order is a basic concept from stochastic process theory it is not a probabilistic property of a process rather it is a topological property and moreover it is not computable from any finite state model other than the e machine via an exhaustive survey we close by demonstrating that infinite markov and infinite cryptic orders are a dominant feature in the space of finite memory processes we draw out the roles played in statistical mechanical spin systems by these two complementary length scales

entropy constrained index assignments for multiple description quantizers multiple description coding has revealed itself highly useful for the transmission of digital signals over packet switched networks or diversity systems the index assignment problem is central in the theory and practice of multiple description quantizers we propose a novel local optimization algorithm that produces index assignments for such quantizers it locally minimizes the distortion due to description losses with constraints on the entropy of each description it works in any vector dimension and is easily generalizable to various cases such as asymmetric descriptions or m description systems we present experimental comparisons with previously known index assignment methods in both scalar and vector cases and show that the proposed algorithm performs significantly better

fast encoder optimization for multi resolution scalar quantizer design the design of optimal multi resolution scalar quantizers using the generalized lloyd method was proposed by brunk and farvardin for the case of squared error distortion since the algorithm details heavily rely on the quadratic expression of the error function its extension to general error functions faces some challenges especially at the encoder optimization step in this work we show how these challenges can be overcome for any convex difference distortion measure under the assumption that all quantizer cells are convex i e intervals and present an efficient algorithm for optimal encoder partition computation the proposed algorithm is faster than the algorithm used by brunk and farvardin moreover it can also be applied to channel optimized and to multiple description scalar quantizer design with squared error distortion and it outperforms in speed the previous encoder optimization algorithms proposed for these problems