mismatch in high rate entropy constrained vector quantization bucklew s 1984 high rate vector quantizer mismatch result is extended from fixed rate coding to variable rate coding using a lagrangian formulation it is shown that if an asymptotically high rate optimal sequence of variable rate codes is designed for a k dimensional probability density function pdf g and then applied to another pdf f for which f g is bounded then the resulting mismatch or loss of performance from the optimal possible is given by the relative entropy or kullback leibler 1968 divergence i f spl par g it is also shown that under the same assumptions an asymptotically optimal code sequence for g can be converted to an asymptotically optimal code sequence for a mismatched source f by modifying only the lossless component of the code applications to quantizer design using uniform and gaussian densities are described including a high rate analog to the shannon rate distortion result of sakrison 1975 and lapidoth 1997 showing that the gaussian is the worst case for lossy compression of a source with known covariance by coupling the mismatch result with composite quantizers the worst case properties of uniform and gaussian densities are extended to conditionally uniform and gaussian densities which provides a lloyd clustering algorithm for fitting mixtures to general densities

distortion bounds for vector quantizers with finite codebook size upper and lower bounds are presented for the distortion of the optimal n point vector quantizer applied to k dimensional signals under certain smoothness conditions on the source distribution the bounds are shown to hold for each and every value of n the codebook size these results extend bounds derived in the high resolution limit which assume that the number of code vectors is arbitrarily large two approaches to the upper bound are presented the first constructive construction achieves the correct asymptotic rate of convergence as well as the correct dependence on the source density although leading to an inferior value for the constant the second construction based on a random coding argument is shown to additionally achieve a value of the constant which is much closer to the best known result derived within the asymptotic theory lower bound results derived in the correspondence are again shown to possess the correct asymptotic form and yield a constant which is almost indistinguishable from the best value achieved in the asymptotic regime finally application of the results to the problem of source coding yields upper bounds on the distortion rate function for a wide class of processes

sample adaptive product quantization asymptotic analysis and examples vector quantization vq is an efficient data compression technique for low bit rate applications however the major disadvantage of vq is that its encoding complexity increases dramatically with bit rate and vector dimension even though one can use a modified vq such as the tree structured vq to reduce the encoding complexity it is practically infeasible to implement such a vq at a high bit rate or for large vector dimensions because of the huge memory requirement for its codebook and for the very large training sequence requirement to overcome this difficulty a structurally constrained vq called the sample adaptive product quantizer sapq has recently been proposed we extensively study the sapq that is based on scalar quantizers in order to exploit the simplicity of scalar quantization through an asymptotic distortion result we discuss the achievable performance and the relationship between distortion and encoding complexity we illustrate that even when sapq is based on scalar quantizers it can provide vq level performance we also provide numerical results that show a 2 3 db improvement over the lloyd max 1982 1960 quantizers for data rates above 4 b point

a pyramid vector quantizer the geometric properties of a memoryless laplacian source are presented and used to establish a source coding theorem motivated by this geometric structure a pyramid vector quantizer pvq is developed for arbitrary vector dimension the pvq is based on the cubic lattice points that lie on the surface of an l dimensional pyramid and has simple encoding and decoding algorithms a product code version of the pvq is developed and generalized to apply to a variety of sources analytical expressions are derived for the pvq mean square error mse and simulation results are presented for pvq encoding of several memoryless sources for large rate and dimension pvq encoding of memoryless laplacian gamma and gaussian sources provides rose improvements of 5 64 8 40 and 2 39 db respectively over the corresponding optimum scalar quantizer although suboptimum in a rate distortion sense because the pvq can encode large dimensional vectors it offers significant reduction in rose distortion compared with the optimum lloyd max scalar quantizer and provides an attractive alternative to currently available vector quantizers

a survey of the theory of source coding with a fidelity criterion the purpose of this paper is threefold 1 to acquaint the reader with the types of problems that have been considered in the area of source coding with a fidelity criterion 2 to survey results that have been obtained on these problems and 3 to outline future research trends in the area

asymptotic order of quantization for cantor distributions in terms of euler characteristic hausdorff and packing measure for homogeneous one dimensional cantor sets which are not necessarily self similar we show under some restrictions that the euler exponent equals the quantization dimension of the uniform distribution on these cantor sets moreover for a special sub class of these sets we present a linkage between the hausdorff and the packing measure of these sets and the high rate asymptotics of the quantization error

on using prototype reduction schemes to optimize kernel based nonlinear subspace methods abstract the subspace method of pattern recognition is a classification technique in which pattern classes are specified in terms of linear subspaces spanned by their respective class based basis vectors to overcome the limitations of the linear methods kernel based nonlinear subspace kns methods have been recently proposed in the literature in kns the kernel principal component analysis kpca has been employed to get principal components not in an input space but in a high dimensional space where the components of the space are nonlinearly related to the input variables the length of projections onto the basis vectors in the kpca are computed using a kernel matrix k whose dimension is equivalent to the number of sample data points clearly this is problematic especially for large data sets in this paper we suggest a computationally superior mechanism to solve the problem rather than define the matrix k with the whole data set and compute the principal components we propose that the data be reduced into a smaller representative subset using a prototype reduction scheme prs since a prs has the capability of extracting vectors that satisfactorily represent the global distribution structure we demonstrate that data points which are ineffective in the classification can be eliminated to obtain a reduced kernel matrix k without degrading the performance our experimental results demonstrate that the proposed mechanism dramatically reduces the computation time without sacrificing the classification accuracy for samples involving real life data sets as well as artificial data sets the results especially demonstrate the computational advantage for large data sets such as those involved in data mining and text categorization applications

high resolution quantization theory and the vector quantizer advantage the authors consider how much performance advantage a fixed dimensional vector quantizer can gain over a scalar quantizer they collect several results from high resolution or asymptotic in rate quantization theory and use them to identify source and system characteristics that contribute to the vector quantizer advantage one well known advantage is due to improvement in the space filling properties of polytopes as the dimension increases others depend on the source s memory and marginal density shape the advantages are used to gain insight into product transform lattice predictive pyramid and universal quantizers although numerical prediction consistently overestimated gains in low rate 1 bit sample experiments the theoretical insights may be useful even at these rates

quantization on the grassmann manifold the problem of quantization in an euclidean space with unitary constraints can be formulated as an unconstrained problem on a grassmann manifold such constraints arise in areas such as wireless communication with multiple antennas at the transmitter and at the receiver due to the constraints the distortion rate analysis developed for euclidean spaces cannot be applied directly this paper extends gersho s asymptotic large rate small distortion distortion bounds to the case when the source is distributed on the complex grassmann manifold the special structure of the grassmann manifold and the distortion measures defined on it differentiate this problem from the traditional vector quantization in euclidean spaces

explicit magnification control of self organizing maps for forbidden data in this paper we examine the scope of validity of the explicit self organizing map som magnification control scheme of bauer 1996 on data for which the theory does not guarantee success namely data that are n dimensional nges2 and whose components in the different dimensions are not statistically independent the bauer algorithm is very attractive for the possibility of faithful representation of the probability density function pdf of a data manifold or for discovery of rare events among other properties since theoretically unsupported data of higher dimensionality and higher complexity would benefit most from the power of explicit magnification control we conduct systematic simulations on forbidden data for the unsupported n 2 cases that we investigate the simulations show that even though the magnification exponent alphaachieved achieved by magnification control is not the same as the desired alphadesired alphaachieved systematically follows alphadesired with a slowly increasing positive offset we show that for simple synthetic higher dimensional data information theoretically optimum pdf matching alphaachieved 1 can be achieved and that negative magnification has the desired effect of improving the detectability of rare classes in addition we further study theoretically unsupported cases with real data

entropy constrained vector quantization an iterative descent algorithm based on a lagrangian formulation for designing vector quantizers having minimum distortion subject to an entropy constraint is discussed these entropy constrained vector quantizers ecvqs can be used in tandem with variable rate noiseless coding systems to provide locally optimal variable rate block source coding with respect to a fidelity criterion experiments on sampled speech and on synthetic sources with memory indicate that for waveform coding at low rates about 1 bit sample under the squared error distortion measure about 1 6 db improvement in the signal to noise ratio can be expected over the best scalar and lattice quantizers when block entropy coded with block length 4 even greater gains are made over other forms of entropy coded vector quantizers for pattern recognition it is shown that the ecvq algorithm is a generalization of the k means and related algorithms for estimating cluster means in that the ecvq algorithm estimates the prior cluster probabilities as well experiments on multivariate gaussian distributions show that for clustering problems involving classes with widely different priors the ecvq outperforms the k means algorithm in both likelihood and probability of error

asymptotically optimal model estimation for quantization using high rate theory approximations we introduce flexible practical quantizers based on possibly non gaussian models in both the constrained resolution cr and the constrained entropy cases we derive model estimation criteria optimizing asymptotic with increasing rate quantizer performance we show that in the cr case the optimal criterion is different from the maximum likelihood criterion commonly used for that purpose and introduce a new criterion that we call constrained resolution minimum description length cr mdl we apply these principles to the generalized gaussian scaled mixture model which is accurate for many real world signals we provide an explanation of the reason why the cr mdl improves quantization performance in the cr case and show that cr mdl can compensate for a possible mismatch between model and data distribution thus this criterion is of a great interest for practical applications our experiments apply the new quantization method to controllable artificial data and to the commonly used modulated lapped transform representation of audio signals we show that both the cr mdl criterion and a non gaussian modeling have significant advantages

adaptive entropy coded subband coding of images the authors describe a design approach called 2 d entropy constrained subband coding ecsbc based upon recently developed 2 d entropy constrained vector quantization ecvq schemes the output indexes of the embedded quantizers are further compressed by use of noiseless entropy coding schemes such as huffman or arithmetic codes resulting in variable rate outputs depending upon the specific configurations of the ecvq and the ecpvq over the subbands many different types of sbc schemes can be derived within the generic 2 d ecsbc framework among these the authors concentrate on three representative types of 2 d ecsbc schemes and provide relative performance evaluations they also describe an adaptive buffer instrumented version of 2 d ecsbc called 2 d ecsbc aec for use with fixed rate channels which completely eliminates buffer overflow underflow problems this adaptive scheme achieves performance quite close to the corresponding ideal 2 d ecsbc system

tradeoff between source and channel coding on a gaussian channel consider a system that quantizes and encodes analog data for transmission across an additive noise gaussian channel to minimize distortion the channel code rate must be chosen to optimally allocate the available transmission rate between lossy source coding and block channel coding we establish tight upper and lower bounds on the channel code rate that minimizes the average distortion of a vector quantizer cascaded with a channel coder and a gaussian channel thus extending some recently obtained results for the binary symmetric channel the upper hounds are obtained by averaging whereas the lower bounds are uniform over all possible index assignments analytic expressions are derived for large and small signal to noise ratios and also for large source vector dimension as in the binary symmetric channel the optimal channel code rate is often substantially smaller than the channel capacity and the distortion decays exponentially with the number of channel uses exact exponents are derived

pyramidal lattice vector quantization for multiscale image coding introduces a new image coding scheme using lattice vector quantization the proposed method involves two steps biorthogonal wavelet transform of the image and lattice vector quantization of wavelet coefficients in order to obtain a compromise between minimum distortion and bit rate we must truncate and scale the lattice suitably to meet this goal we need to know how many lattice points lie within the truncated area we investigate the case of laplacian sources where surfaces of equal probability are spheres for the l sup 1 metric pyramids for arbitrary lattices we give explicit generating functions for the codebook sizes for the most useful lattices like z sup n d sub n e sub s spl and sub 16

upper bounds to the asymptotic performance of block quantizers upper bounds to the asymptotic performance of block quantizers with difference distortion measures are derived in many eases these upper bounds approach known lower bounds as the block length of the quantizer approaches infinity a condition for the optimal point density function of the output levels is derived it is shown to particularize to a known result of gersho the behavior of the bounds for large block lengths is investigated

lattices which are good for almost everything we define an ensemble of lattices and show that for asymptotically high dimension most of its members are simultaneously good as sphere packings sphere coverings additive white gaussian noise awgn channel codes and mean squared error mse quantization codes these lattices are generated by applying construction a to a random linear code over a prime field of growing size i e by lifting the code to spl ropf sup n

on the structure of optimal entropy constrained scalar quantizers the nearest neighbor condition implies that when searching for a mean square optimal fixed rate quantizer it is enough to consider the class of regular quantizers i e quantizers having convex cells and codepoints which lie inside the associated cells in contrast quantizer regularity can preclude optimality in entropy constrained quantization this can be seen by exhibiting a simple discrete scalar source for which the mean square optimal entropy constrained scalar quantizer ecsq has disconnected and hence nonconvex cells at certain rates in this work new results concerning the structure and existence of optimal ecsqs are presented one main result shows that for continuous sources and distortion measures of the form d x y spl rho x y where spl rho is a nondecreasing convex function any finite level ecsq can be regularized so that the resulting regular quantizer has the same entropy and equal or less distortion regarding the existence of optimal ecsqs we prove that under rather general conditions there exists an almost regular optimal ecsq for any entropy constraint for the squared error distortion measure and sources with piecewise monotone and continuous densities the existence of a regular optimal ecsq is shown

rates of convergence for partial mass problems we consider a class of partial mass problems in which a fraction of the mass of a probability measure is allowed to be changed trimmed to maximize fit to a given pattern this includes the problem of optimal partial transportation of mass where a part of the mass need not be transported and also trimming procedures which are often used in statistical data analysis to discard outliers in a sample the data with lowest agreement to a certain pattern this results in a modified trimmed version of the original probability which is closer to the pattern we focus on the case of the empirical measure and analyze to what extent its optimally trimmed version is closer to the true random generator in terms of rates of convergence we deal with probabilities on mathbb r k and measure agreement through probability metrics our choices include transportation cost metrics associated to optimal partial transportation and the kolmogorov distance we show that partial transportation as opposed to classical complete transportation results in a sharp decrease of costs only in low dimension in contrast for the kolmogorov metric this decrease is seen in any dimension

analysis of multiple antenna systems with finite rate feedback using high resolution quantization theory this paper considers the development of a general framework for the analysis of transmit beamforming methods in multiple antenna systems with finite rate feedback inspired by the results of classical high resolution quantization theory the problem of finite rate quantized communication system is formulated as a general fixed rate vector quantization problem with side information available at the encoder or the quantizer but unavailable at the decoder the framework of the quantization problem is sufficiently general to include quantization schemes with general non mean squared distortion functions and constrained source vectors asymptotic distortion analysis of the proposed general quantization problem is provided by extending the vector version of the bennett s integral specifically tight lower and upper bounds of the average asymptotic distortion are proposed sufficient conditions for the achievability of the distortion bounds are also provided and related to corresponding classical fixed rate quantization problems the proposed general methodology provides a powerful analytical tool to study a wide range of finite rate feedback systems to illustrate the utility of the framework we consider the analysis of a finite rate feedback multiple input single output miso beamforming system over independent and identically distributed i i d rayleigh flat fading channels numerical and simulation results are presented that further confirm the accuracy of the analytical results

quantization based on a novel sample adaptive product quantizer sapq in this paper we propose a novel feedforward adaptive quantization scheme called the sample adaptive product quantizer sapq this is a structurally constrained vector quantizer that uses unions of product codebooks sapq is based on a concept of adaptive quantization to the varying samples of the source and is very different from traditional adaptation techniques for nonstationary sources sapq quantizes each source sample using a sequence of quantizers even when using scalar quantization in sapq we can achieve performance comparable to vector quantization with the complexity still close to that of scalar quantization we also show that important lattice based vector quantizers can be constructed using scalar quantization in sapq we mathematically analyze sapq and propose a algorithm to implement it we numerically study sapq for independent and identically distributed gaussian and laplacian sources through our numerical study we find that sapq using scalar quantizers achieves typical gains of 13 db in distortion over the lloyd max quantizer we also show that sapq can he used in conjunction with vector quantizers to further improve the gains

a space quantization method for numerical integration abstract we propose a new method sqm for numerical integration of c functions 0 2 defined on a convex subset c of rd with respect to a continuous distribution it relies on a space quantization of c by a n tuplex x1 xn cn fd is approximated by a weighted sum of the f xi s the integration error bound depends on the distortion en x of the voronoi tessellation of x this notion comes from information theoretists its main properties existence of a minimizing n tuple in cn asymptotics of mincn en as n are presented for a wide class of measures a simple stochastic optimization procedure is proposed to compute in any dimension d x and the characteristics of its voronoi tessellation some new results on the competitive learning vector quantization algorithm when 2 are obtained as a by product some tests simulations and provisional remarks are proposed as a conclusion

lagrangian vector quantization with combined entropy and codebook size constraints in this paper the lagrangian formulation of variable rate vector quantization is extended to quantization with simultaneous constraints on entropy and codebook size including variable and fixed rate quantization as special cases the formulation leads to a lloyd quantizer design algorithm and generalizations of gersho s approximations characterizing optimal performance for asymptotically large rate a variation of gersho s approach is shown to yield rigorous results partially characterizing the asymptotically optimal performance

quantization dimension for condensation systems let be the attracting measure of a condensation system associated with a self similar measure we determine the upper and lower quantization dimension of under the strong separation condition

high rate vector quantization for noisy channels with applications to wideband speech spectrum compression this paper considers the high rate performance of source coding for noisy discrete symmetric channels with random index assignment ia accurate analytical models are developed to characterize the expected distortion performance of vector quantization vq for a large class of distortion measures it is shown that when the point density is continuous the distortion can be approximated as the sum of the source quantization distortion and the channel error induced distortion expressions are also derived for the continuous point density that minimizes the expected distortion next for the case of mean squared error distortion a more accurate analytical model for the distortion is derived by allowing the point density to have a singular component the extent of the singularity is also characterized these results provide analytical models for the expected distortion performance of both conventional vq as well as for channel optimized vq as a practical example compression of the linear predictive coding parameters in the wideband speech spectrum is considered with the log spectral distortion as performance metric the theory is able to correctly predict the channel error rate that is permissible for operation at a particular level of distortion

on the estimation of differential entropy from data located on embedded manifolds estimation of the differential entropy from observations of a random variable is of great importance for a wide range of signal processing applications such as source coding pattern recognition hypothesis testing and blind source separation in this paper we present a method for estimation of the shannon differential entropy that accounts for embedded manifolds the method is based on high rate quantization theory and forms an extension of the classical nearest neighbor entropy estimator the estimator is consistent in the mean square sense and an upper bound on the rate of convergence of the estimator is given because of the close connection between compression and shannon entropy the proposed method has an advantage over methods estimating the renyi entropy through experiments on uniformly distributed data on known manifolds and real world speech data we show the accuracy and usefulness of our proposed method

asymptotic bounds on optimal noisy channel quantization via random coding asymptotically optimal zero delay vector quantization in the presence of channel noise is studied using random coding techniques first an upper bound is derived for the average rth power distortion of channel optimized k dimensional vector quantization at transmission rate r on a binary symmetric channel with bit error probability spl epsiv the upper bound asymptotically equals 2 sup rrg spl epsiv k r where k k r 1 log sub 2 l 2 spl radic spl epsiv 1 spl epsiv spl les g spl epsiv k r spl les 1 for all spl epsiv spl ges 0 lim sub spl epsiv spl rarr 0 g spl epsiv k r 1 and lim sub k spl rarr spl infin g spl epsiv k r 1 numerical computations of g spl epsiv k r are also given this result is analogous to zador s 1982 asymptotic distortion rate of 2 sup rr for quantization on noiseless channels next using a random coding argument on nonredundant index assignments a useful upper bound is derived in terms of point density functions on the minimum mean squared error of high resolution regular vector quantizers in the presence of channel noise the formula provides an accurate approximation to the distortion of a noisy channel quantizer whose codebook is arbitrarily ordered finally it is shown that the minimum mean squared distortion of a regular noisy channel vq with a randomized nonredundant index assignment is in probability asymptotically bounded away from zero

quantization error in hexagonal sensory configurations the authors develop mathematical tools for estimating quantization error in hexagonal sensory configurations these include analytic expressions for the average error and the error distribution of a function of an arbitrary number of independently quantized variables these two quantities are essential for assessing the reliability of a given algorithm they can also be used to compare the relative sensitivity of a particular algorithm to quantization error for hexagonal and other spatial samplings e g square and can have an impact on sensor design furthermore it is shown that the ratio of hexagonal error to square error is bounded between 0 90 and 1 05

vanishing distortion and shrinking cells we establish an asymptotic connection between vanishing rth power distortion and shrinking cell diameters for vector quantizers with convex cells as a consequence a number of shrinking cell conditions may be easily verified by showing that the quantizers in question have distortion tending to zero this also plays an important role in the asymptotic analysis of a common greedy growing scheme for tree structured vector quantizers

on the minimum average distortion of quantizers with index dependent distortion measures in many applications one is interested in optimally deploying a network of unidentical agents to a certain area of interest examples include heterogeneous sensor networks and cellular networks such deployment problems can equivalently be formulated as quantizer design problems where different distortion measures should be associated with different quantization indices in this paper we consider the case where the distortion measure is the r th power r geq 1 of the distance between the reproduction point and the source sample weighted by a factor that varies from one index to another for a uniform distribution of source samples we determine the corresponding optimal scalar quantizers and their average distortions for nonuniform distributions and vector quantization we provide a high resolution analysis of the minimum possible average distortion applications to sensor and cellular networks are also discussed together with numerical design examples

a rigorous approach to high resolution entropy constrained vector quantization the nonnegativity of relative entropy implies that the differential entropy of a random vector x with probability density function pdf f is upper bounded by mathsf e left log g text x right for any arbitrary pdf g using this inequality with a cleverly chosen g we derive a lower bound on the asymptotic excess rate of entropy constrained vector quantization for d dimensional sources and r th power distortion where the asymptotic excess rate is defined as the difference between the smallest output entropy of a vector quantizer satisfying the distortion constraint and the rate distortion function in the limit as the distortion tends to zero specialized to the one dimensional case this lower bound coincides with the asymptotic excess rate achieved by a uniform quantizer thereby recovering the result by gish and pierce that uniform quantizers are asymptotically optimal as the allowed distortion tends to zero furthermore in the one dimensional case the derivation of the lower bound reveals a necessary condition for a sequence of quantizers to be asymptotically optimal this condition implies that any sequence of asymptotically optimal almost regular quantizers must converge to a uniform quantizer as the distortion tends to zero while the obtained lower bound itself is not novel to the best of our knowledge we present the first rigorous derivation that follows the direct approach by gish and pierce without resorting to heuristic high resolution approximations commonly found in the quantization literature furthermore our derivation holds for all d dimensional sources having finite differential entropy and whose integer part has finite entropy in contrast to gish and pierce we do not require additional constraints on the continuity or decay of the source pdf

the root lattices as low bit rate vector quantizers vector quantizers based on the a and d lattices are explored using the mean square error mse fidelity criterion for independent identically distributed sources with normal laplacian and gamma probability density functions code books are studied for uniform coding rates of up to 2 b sample and two and four dimensions the expected mse performance of the quantizers is better than that of optimal nonuniform lloyd max quantizers while the design algorithm presented offers only modest improvements in mse performance for a normally distributed source improvement gains of up to several decibels can be achieved when coding laplacian and gamma sources

a zador like formula for quantizers based on periodic tilings we consider zador s 1963 1966 1982 asymptotic formula for the distortion rate function for a variable rate vector quantizer in the high rate case this formula involves the differential entropy of the source the rate of the quantizer in bits per sample and a coefficient g which depends on the geometry of the quantizer but is independent of the source we give an explicit formula for g in the case when the quantizing regions form a periodic tiling of n dimensional space in terms of the volumes and second moments of the voronoi cells as an application we show extending earlier work of kashyap and neuhoff see ibid vol 47 p 2538 2383 2001 that even a variable rate three dimensional quantizer based on the a15 structure is still inferior to a quantizer based on the body centered cubic lattice we also determine the smallest covering radius of such a structure

comparison of som point densities based on different criteria point densities of model codebook vectors in self organizing maps soms are evaluated in this article for a few one dimensional soms with finite grid lengths and a given probability density function of the input the numerically exact point densities have been computed the point density derived from the som algorithm turned out to be different from that minimizing the som distortion measure showing that the model vectors produced by the basic som algorithm in general do not exactly coincide with the optimum of the distortion measure a new computing technique based on the calculus of variations has been introduced it was applied to the computation of point densities derived from the distortion measure for both the classical vector quantization and the som with general but equal dimensionality of the input vectors and the grid respectively the power laws in the continuum limit obtained in these cases were found to be identical

image compression by vector quantization of multiresolution decompositions abstract an image compression scheme is introduced which involves a multiresolution decomposition derived from the wavelet transform the use of this transformation in image coding is motivated by its similarity to the processing which occurs at the cortical level of the visual system and by the fact that subband coding in general allows the coder to be better matched to the signal statistics in this work each subband is coded separately by vector quantization this work differs from previous wavelet transform coding schemes involving vector quantization in that the quantization parameters for each subband are selected by a nonlinear optimization procedure which involves constraints on the overall bit rate and the encoding complexity the multiplicative visual model is reviewed and a motivation for cascading this system with the wavelet transform to give a more complete description of human vision is provided empirical results are presented which demostrate the effectiveness of the technique for compressing monochromatic images signal to noise ratio measurements are shown which demostrate the system performance for various selections of bit rate and complexity

voronoi regions of lattices second moments of polytopes and quantization if a point is picked at random inside a regular simplex octahedron 600 cell or other polytope what is its average squared distance from the centroid in n dimensional space what is the average squared distance of a random point from the closest point of the lattice a n or d n e n a n ast or d n ast the answers are given here together with a description of the voronoi or nearest neighbor regions of these lattices the results have applications to quantization and to the design of signals for the gaussian channel for example a quantizer based on the eight dimensional lattice e8 has a mean squared error per symbol of 0 0717 cdots when applied to uniformly distributed data compared with 0 08333 cdots for the best one dimensional quantizer

asymptotics of optimal quantizers for some scalar distributions we obtain semi closed forms for the optimal quantizers of some families of one dimensional probability distributions they yield the first examples of non log concave distributions for which uniqueness holds we give two types of applications of these results one is a fast computation of numerical approximations of one dimensional optimal quantizers and their use in a multidimensional framework the other is some asymptotics of the standard empirical measures associated to the optimal quantizers in terms of distribution function laplace transform and characteristic function moreover we obtain the rate of convergence in the bucklew wise theorem and finally the asymptotic size of the voronoi tessels

sequential scalar quantization of vectors an analysis proposes an efficient vector quantization vq technique called sequential scalar quantization ssq the scalar components of the vector are individually quantized in a sequence with the quantization of each component utilizing conditional information from the quantization of previous components unlike conventional independent scalar quantization isq ssq has the ability to exploit intercomponent correlation at the same time since quantization is performed on scalar rather than vector variables ssq offers a significant computational advantage over conventional vq techniques and is easily amenable to a hardware implementation in order to analyze the performance of ssq the authors appeal to asymptotic quantization theory where the codebook size is assumed to be large closed form expressions are derived for the quantizer mean squared error mse these expressions are used to compare the asymptotic performance of ssq with other vq techniques the authors also demonstrate the use of asymptotic theory in designing ssq for a practical application color image quantization where the codebook size is typically small theoretical and experimental results show that ssq far outperforms isq with respect to mse while offering a considerable reduction in computation over conventional vq at the expense of a moderate increase in mse

distortion rate models for entropy coded lattice vector quantization the increasing demand for real time applications requires the use of variable rate quantizers having good performance in the low bit rate domain in order to minimize the complexity of quantization as well as maintaining a reasonably high psnr ratio we propose to use an entropy coded lattice vector quantizer eclvq these quantizers have proven to outperform the well known ezw algorithm s performance in terms of rate distortion tradeoff we focus our attention on the modeling of the mean squared error mse distortion and the prefix code rate for eclvq first we generalize the distortion model of jeong and gibson 1993 on fixed rate cubic quantizers to lattices under a high rate assumption second we derive new rate models for eclvq efficient at low bit rates without any high rate assumptions simulation results prove the precision of our models

fine coarse vector quantization a fast method for searching an unstructured vector quantization vq codebook is introduced and analyzed the method fine coarse vector quantization fcvq operates in two stages a fine structured vq followed by a table lookup coarse unstructured vq its rate distortion arithmetic complexity and storage are investigated using analytical and experimental means optimality condition and an optimizing algorithm are presented the results of experiments with both uniform scalar quantization and tree structured vq tsvq as the first stage are reported comparisons are made with other fast approaches to vector quantization especially tsvq it is found that when rate distortion arithmetic complexity and storage are all taken into account fcvq outperforms tsvq in a number of cases in comparison to full search quantization fcvq has much lower arithmetic complexity at the expense of a slight increase in distortion and a substantial increase in storage the increase in mean squared error over full search decays as a negative power of the available storage

multiple description vector quantization with lattice codebooks design and analysis the problem of designing a multiple description vector quantizer with lattice codebook spl lambda is considered a general solution is given to a labeling problem which plays a crucial role in the design of such quantizers numerical performance results are obtained for quantizers based on the lattices a sub 2 and z sup i i 1 2 4 8 that make use of this labeling algorithm the high rate squared error distortions for this family of l dimensional vector quantizers are then analyzed for a memoryless source with probability density function pdf p and differential entropy h p spl infin for any a spl epsiv 0 1 and rate pair r r it is shown that the two channel distortion d sub o and the channel 1 or channel 2 distortion d sub s satisfy lim sub r spl rarr spl infin d sub o 2 sup 2r 1 a 1 4 g spl lambda 2 sup 2h p and lim sub r spl rarr spl infin d sub s 2 sup 2r 1 a g s sub l 2 sup 2h p where g spl lambda is the normalized second moment of a voronoi cell of the lattice spl lambda and g s sub l is the normalized second moment of a sphere in l dimensions

a simple class of asymptotically optimal quantizers a simple class of quantizers is introduced which are asymptotically optimal as the number of quantization levels increases to infinity with respect to a mean r th power absolute error distortion measure these asymptotically optimal quantizers are very easy to compute their performance is evaluated for several distributions and compares favorably with the performance of the optimal quantizers in all cases for which the latter have been computed in addition their asymptotic robustness is studied under location scale and shape mismatch for several families of distributions

a lower bound on the average error of vector quantizers corresp a lower bound is proposed for the mean squared error of an n dimensional vector quantizer with a large number of output points

estimating the number of clusters in a numerical data set via quantization error modeling in this paper we consider the problem of unsupervised clustering vector quantization of multidimensional numerical data we propose a new method for determining an optimal number of clusters in the data set the method is based on parametric modeling of the quantization error the model parameter can be treated as the effective dimensionality of the data set the proposed method was tested with artificial and real numerical data sets and the results of the experiments demonstrate empirically not only the effectiveness of the method but its ability to cope with difficult cases where other known methods fail a parameterized model for the clustering error is introduced the model parameter is a measure of the data dimension and homogeneity a new cost criterion is derived from the properties of the model the method demonstrates good results for numerical data sets

divergence based vector quantization supervised and unsupervised vector quantization methods for classification and clustering traditionally use dissimilarities frequently taken as euclidean distances in this article we investigate the applicability of divergences instead focusing on online learning we deduce the mathematical fundamentals for its utilization in gradient based online vector quantization algorithms it bears on the generalized derivatives of the divergences known as freechet derivatives in functional analysis which reduces in finite dimensional problems to partial derivatives in a natural way we demonstrate the application of this methodology for widely applied supervised and unsupervised online vector quantization schemes including self organizing maps neural gas and learning vector quantization additionally principles for hyperparameter optimization and relevance learning for parameterized divergences in the case of supervised vector quantization are given to achieve improved classification accuracy

optimum quantization and its applications abstract minimum sums of moments or equivalently distortion of optimum quantizers play an important role in several branches of mathematics fejes toth s inequality for sums of moments in the plane and zador s asymptotic formula for minimum distortion in euclidean d space are the first precise pertinent results in dimension d 2 in this article these results are generalized in the form of asymptotic formulae for minimum sums of moments resp distortion of optimum quantizers on riemannian d manifolds and normed d spaces in addition we provide geometric and analytic information on the structure of optimum configurations our results are then used to obtain information on i the minimum distortion of high resolution vector quantization and optimum quantizers ii the error of best approximation of probability measures by discrete measures and support sets of best approximating discrete measures iii the minimum error of numerical integration formulae for classes of holder continuous functions and optimum sets of nodes iv best volume approximation of convex bodies by circumscribed convex polytopes and the form of best approximating polytopes and v the minimum isoperimetric quotient of convex polytopes in minkowski spaces and the form of the minimizing polytopes

asymptotic level density for a class of vector quantization processes it is shown that for a class of vector quantization processes related to neural modeling that the asymptotic density q x of the quantization levels in one dimension in terms of the input signal distribution p x is a power law q x c p x sup alpha where the exponent alpha depends on the number n of neighbors on each side of a unit and is given by alpha 2 3 1 3n sup 2 3 n 1 sup 2 the asymptotic level density is calculated and monte carlo simulations are presented

neural gas network for vector quantization and its application to time series prediction a neural network algorithm based on a soft max adaptation rule is presented this algorithm exhibits good performance in reaching the optimum minimization of a cost function for vector quantization data compression the soft max rule employed is an extension of the standard k means clustering procedure and takes into account a neighborhood ranking of the reference weight vectors it is shown that the dynamics of the reference weight vectors during the input driven adaptation procedure are determined by the gradient of an energy function whose shape can be modulated through a neighborhood determining parameter and resemble the dynamics of brownian particles moving in a potential determined by the data point density the network is used to represent the attractor of the mackey glass equation and to predict the mackey glass time series with additional local linear mappings for generating output values the results obtained for the time series prediction compare favorably with the results achieved by backpropagation and radial basis function networks

scalar versus vector quantization worst case analysis we study the potential merits of vector quantization and show that there can be an arbitrary discrepancy between the worst case rates required for scalar and vector quantization specifically we describe a random variable and a distortion measure where quantization of a single instance to within a given distortion requires an arbitrarily large number of bits in the worst case but quantization of multiple independent instances to within the same distortion requires an arbitrarily small number of bits per instance in the worst case we relate this discrepancy to expander graphs representation and cover numbers of set systems and a problem studied by slepian wolf and wyner 1973

source and channel rate allocation for channel codes satisfying the gilbert varshamov or tsfasman vladut zink bounds we derive bounds for optimal rate allocation between source and channel coding for linear channel codes that meet the gilbert varshamov or tsfasman vladut zink 1984 bounds formulas giving the high resolution vector quantizer distortion of these systems are also derived in addition we give bounds on how far below the channel capacity the transmission rate should be for a given delay constraint the bounds obtained depend on the relationship between channel code rate and relative minimum distance guaranteed by the gilbert varshamov bound and do not require sophisticated decoding beyond the error correction limit we demonstrate that the end to end mean squared error decays exponentially fast as a function of the overall transmission rate which need not be the case for certain well known structured codes such as hamming codes

bandpass nonlinear systems identification by higher order cross correlation nonlinear systems constituted by a zero memory nonlinearity cascaded with linear filters can be identified by input output cross correlation using a gaussian input signal the method is extended to complex systems through a pair of complex invariance theorems the stated properties allow identifying the linear parts of systems characterized by magnitude phase nonlinearities with the joint use of second and third order input output moments the method can be employed for a wide class of communication bandpass circuits when signals are represented by complex envelopes

uniform and piecewise uniform lattice vector quantization for memoryless gaussian and laplacian sources lattice vector quantizer design procedures for nonuniform sources are presented the procedures yield lattice vector quantizers with excellent performance and retaining the structure required for fast quantization analytical methods for truncating and scaling lattices to be used in vector quantizations are given and their utility is demonstrated for independent and identically distributed i i d gaussian and laplacian sources an analytical technique for piecewise linear multidimensional compandor designs is evaluated for i i d gaussian and laplacian sources by comparing its performance to that of the other vector quantizers

asymptotic quantization error of continuous signals and the quantization dimension extensions of the limiting qnanfizafion error formula of bennet are proved these are of the form d s k n f n beta b where n is the number of output levels d s k n f is the s th moment of the metric distance between quantizer input and output beta b 0 k s beta is the signal space dimension and f is the signal distribution if a suitably well behaved k dimensional signal density f x exists b b s k int f rho x dx 1 rho rho k s k and b s k does not depend on f for k 1 s 2 this reduces to bennett s formula if f is the cantor distribution on 0 1 0 and this k equals the fractal dimension of the cantor set 12 13 random quantization optimal quantization in the presence of an output information constraint and quantization noise in high dimensional spaces are also investigated

the progressive transmission disadvantage the disadvantage of progressive transmission compared to transmission in a single step is analyzed for the high rate case using a polytope that is self similar under binary subdivision this disadvantage is calculated to be 0 3424 db or 0 057 bit compared to scalar quantization obtained with a step size of 1 bit per ensemble

joint bit allocation and dimensions optimization for vector transform quantization in vector transform quantization vtq vectors consisting of m consecutive samples of a waveform are transformed into a set of m coefficients that are quantized by m

image vector quantizer based on a classification in the dct domain a classification algorithm in the discrete cosine transform dct domain for the classified vector quantization cvq technique is proposed the classifier employs four dct coefficients of 4 4 subblock as edge oriented features the classifier is designed using a cluster seeking algorithm to ensure that the centroid of a set of vectors in a class always belong to that class since the classification is performed in the dct domain this approach can be easily extended to the dct transform coding technique simulation results show that a good visual quality of the coded image at fixed rates in the 0 625 0 813 b pixel bpp range is obtained with comparable complexity the weighted mse wmse analysis in conjunction with the proposed classifier is discussed

zones of influence in a two dimensional unsteady hypersonic boundary layer an asymptotic structure is developed for a linear high frequency unsteady disturbance superimposed upon a steady possibly separated nonlinear flow the unsteady viscous sublayer is found to split into a two region structure the leading order flowfield is driven primarily by the upper region which coincides with the region of non parallel flow in the original steady viscous sublayer it is found that introducing a viscous inviscid interaction into the unsteady problem drastically alters the domain of dependence of the unsteady flow throughout the entire viscous sublayer the determination of the correct domain of dependence is found to involve a subtle interplay between the convective terms the pressure displacement interaction and the non parallel base flow preliminary extensions to fully nonlinear unsteady interactive boundary layers are noted

adaptive entropy coded predictive vector quantization of images the authors consider 2 d predictive vector quantization pvq of images subject to an entropy constraint and demonstrate the substantial performance improvements over existing unconstrained approaches they describe a simple adaptive buffer instrumented implementation of this 2 d entropy coded pvq scheme which can accommodate the associated variable length entropy coding while completely eliminating buffer overflow underflow problems at the expense of only a slight degradation in performance this scheme called 2 d pvq aecq adaptive entropy coded quantization is shown to result in excellent rate distortion performance and impressive quality reconstructions of real world images indeed the real world coding results shown demonstrate little distortion at rates as low as 0 5 b pixel