information theoretic novelty detection we present a novel approach to online change detection problems when the training sample size is small the proposed approach is based on estimating the expected information content of a new data point and allows an accurate control of the false positive rate even for small data sets in the case of the gaussian distribution our approach is analytically tractable and closely related to classical statistical tests we then propose an approximation scheme to extend our approach to the case of the mixture of gaussians we evaluate extensively our approach on synthetic data and on three real benchmark data sets the experimental validation shows that our method maintains a good overall accuracy but significantly improves the control over the false positive rate

a tight upper bound of the average absolute error in a constant step size sign algorithm a direct performance index of the adaptive filtering sign algorithm sa is the average absolute error aae at the output of the filter adopting this performance index an easy analysis of sa is achieved under a weak assumption it is proved for both deterministic and random inputs to the filter that the aae has a tight upper bound that exceeds the minimum aae by half the product of the step size and power of the filter input the assumption used is existence of average squared and average absolute values of filter input signals a practical interest of the result is that it provides a formula for the biggest step size as a function of tolerable adaptation noise to desired signal ratio

subdifferentials faces and dual matrices abstract a characterization of the dual matrices for the unitarily invariant norms is given moreover the connection between the dual matrices the subdifferentials of matrix norms and the faces of the unit ball of matrices is presented

an mcmc approach to universal lossy compression of analog sources motivated by the markov chain monte carlo mcmc approach to the compression of discrete sources developed by jalali and weissman we propose a lossy compression algorithm for analog sources that relies on a finite reproduction alphabet which grows with the input length the algorithm achieves in an appropriate asymptotic sense the optimum shannon theoretic tradeoff between rate and distortion universally for stationary ergodic continuous amplitude sources we further propose an mcmc based algorithm that resorts to a reduced reproduction alphabet when such reduction does not prevent achieving the shannon limit the latter algorithm is advantageous due to its reduced complexity and improved rates of convergence when employed on sources with a finite and small optimum reproduction alphabet

density estimation and random variate generation using multilayer networks in this paper we consider two important topics density estimation and random variate generation we present a framework that is easily implemented using the familiar multilayer neural network first we develop two new methods for density estimation a stochastic method and a related deterministic method both methods are based on approximating the distribution function the density being obtained by differentiation in the second part of the paper we develop new random number generation methods our methods do not suffer from some of the restrictions of existing methods in that they can be used to generate numbers from any density provided that certain smoothness conditions are satisfied one of our methods is based on an observed inverse relationship between the density estimation process and random number generation we present two variants of this method a stochastic and a deterministic version we propose a second method that is based on a novel control formulation of the problem where a controller network is trained to shape a given density into the desired density we justify the use of all the methods that we propose by providing theoretical convergence results in particular we prove that the l sub spl infin convergence to the true density for both the density estimation and random variate generation techniques occurs at a rate o log log n n sup 1 spl epsi 2 where n is the number of data points and spl epsi can be made arbitrarily small for sufficiently smooth target densities this bound is very close to the optimally achievable convergence rate under similar smoothness conditions also for comparison the l sub 2 root mean square rms convergence rate of a positive kernel density estimator is o n sup 2 5 when the optimal kernel width is used we present numerical simulations to illustrate the performance of the proposed density estimation and random variate generation methods in addition we present an extended introduction and bibliography that serves as an overview and reference for the practitioner

variable rate vector quantization for medical image compression three techniques for variable rate vector quantizer design are applied to medical images the first two are extensions of an algorithm for optimal pruning in tree structured classification and regression due to breiman et al the code design algorithms find subtrees of a given tree structured vector quantizer tsvq each one optimal in that it has the lowest average distortion of all subtrees of the tsvq with the same or lesser average rate since the resulting subtrees have variable depth natural variable rate coders result the third technique is a joint optimization of a vector quantizer and a noiseless variable rate code this technique is relatively complex but it has the potential to yield the highest performance of all three techniques

a greedy tree growing algorithm for the design of variable rate vector quantizers image compression a technique for directly designing a variable rate tree structured vector quantizer by growing the tree one node at a time rather than one layer at time is presented the technique is a natural extension of a tree growing method for decision trees when the tree is pruned with a generalized algorithm for optimally pruning trees improvement is measured in the signal to noise ratio at high rates over pruning a fixed rate tree structured vector quantizer of the same initial rate the growing algorithm can be interpreted as a constrained inverse of the pruning algorithm

convergence of the sign algorithm for adaptive filtering with correlated data convergence of a decreasing gain sign algorithm sa for adaptive filtering is analyzed the presence of the hard limiter in the algorithm makes a rigorous analysis difficult therefore there are few results available such results normally include restrictive assumptions such as the assumptions that successive observation vectors are independent and the new error signal of the adaptive filter has a time invariant probability density function the former assumption is not valid in the context of adaptive filtering since two successive observation vectors share most of their components while the latter assumption is a restriction on the adaptive weights whose evolution is a priori unknown in lieu of using these assumptions an almost sure convergence of the sa is proved under the assumption that the sequence of observation vectors is m dependent this assumption allows strong correlation between successive observations

a perturbative approach to novelty detection in autoregressive models we propose a new method to perform novelty detection in dynamical systems governed by linear autoregressive models the method is based on a perturbative expansion to a statistical test whose leading term is the classical f test and whose o 1 n correction can be approximated as a function of the number of training points and the model order alone the method can be justified as an approximation to an information theoretic test we demonstrate on several synthetic examples that the first correction to the f test can dramatically improve the control over the false positive rate of the system we also test the approach on some real time series data demonstrating that the method still retains a good accuracy in detecting novelties

a new class of nonlinear filters neural filters a class of nonlinear filters based on threshold decomposition and neural networks is defined it is shown that these neural filters include all filters defined either by continuous functions such as linear finite impulse response fir filters or by boolean functions such as generalized stack filters adaptive least mean absolute error and adaptive least mean square error algorithms are derived for determining optimal neural filters as special cases adaptive generalized stack and adaptive generalized weighted order statistic filtering algorithms under both error criteria are derived experimental results in 1d and 2d signal processing are presented to compare the performances of the adaptive neural filters and other widely used filters

tracking analysis of the sign algorithm in nonstationary environments a tracking analysis of the adaptive filters equipped with the sign algorithm and operating in nonstationary environments is presented under the assumption that the nonstationary can be modeled using a random disturbance it is shown that the long term time average of the mean absolute error is bounded and that there exists an optimal choice of the convergence constant mu which minimizes this quality applying the commonly used independence assumption and under the assumption that the nonstationarity is solely due to the time varying behavior of the optimal coefficients it is shown that the distributions of the successive coefficient misalignment vectors converge to a limiting distribution when the adaptive filter is used in the system identification mode finally under the additional assumption that the signals involved are zero mean and gaussian a set of nonlinear difference equations is derived that characterizes the mean and mean squared behavior of the filter coefficients and the mean squared estimation error during adaptation and tracking results of several experiments that show very good correlation with the theoretical analyses are presented

termination and continuity of greedy growing for tree structured vector quantizers tree structured vector quantizers tsvq provide a computationally efficient variable rate method of compressing vector valued data in applications the problem of designing a tsvq from empirical training data is critical greedy growing algorithms are a common and effective approach to the design problem they are recursive procedures that produce a tsvq one node at a time by optimizing a simple splitting criterion at each step while unsupervised greedy growing algorithms are well understood from an experimental point of view there has been little theory to support their use or to examine their behavior on large training sets the authors present a rigorous analysis of a greedy growing algorithm proposed by riskin 1990 riskin and gray 1991 and balakrishnan 1991 the first part of the paper is a description of the algorithm and an examination of its asymptotic behavior as it applies to a fixed absolutely continuous distribution the second part of the paper establishes the structural consistency of the algorithm with respect to a convergent sequence of distributions as an application the authors obtain results concerning the large sample empirical behavior of the algorithm when it is applied to stationary ergodic training vectors

distortion rate bounds for fixed and variable rate multiresolution source codes the source coding theorem for stationary sources describes the optimal performance theoretically achievable by fixed and variable rate block quantizers the source coding theorem may be generalized by considering the problem of multiresolution or successive refinement source coding which is the topic of this work given a distortion vector d sub 1 d sub l this work describes the family of achievable rate vectors r sub 1 r sub l for describing a stationary source at l resolutions where the description at the first resolution is given at rate r sub 1 and achieves an expected distortion no greater than d sub 1 the description at the second resolution includes both the first description and a refining description of rate r sub 2 and achieves expected distortion no greater than d sub 2 and so on the work includes performance bounds for both fixed and variable rate source codes on discrete time stationary ergodic sources and discrete time stationary nonergodic sources for any integer number of resolutions l spl ges 1 for l 1 the source coding theorems for stationary sources result for l 1 the results extend previous theorems for discrete alphabet memoryless sources

faces of the unit ball of a unitarily invariant norm abstract let be a unitarily invariant norm on the space of all n m complex matrices and let g r n r be the symmetric gauge function associated with that is to say we have a g s a for any a where s a is the nonincreasing sequence of singular values of a in this article we consider the relationship between the facial structures of the closed unit balls of g and which we denote by bg and b our main result gives a complete characterization of the faces of b in terms of the faces of bg for with each face e of b we associate a standard face f e of bg i e f e is a face of bg whose barycenter is a nonnegative nonincreasing n vector conversely each standard face f of bg originates a set of n m matrices say the set m f which is proven to be a face of b our main result also asserts that any face e of b is of the form q m fr where q and r are unitary matrices of appropriate orders and f f e we then explore the main result and draw some of its consequences for example we show that the singular values of the barycenter of a face e of b have the same multiplicities as the barycenter of f e formulas are given which relate the dimensions of e and f e the relative interior and the relative boundary of e are characterized in terms of the relative interior and the relative boundary of f e we completely describe the face of b generated by a matrix a our results also cover the case of real matrices

combining image compression and classification using vector quantization we describe a method of combining classification and compression into a single vector quantizer by incorporating a bayes risk term into the distortion measure used in the quantizer design algorithm once trained the quantizer can operate to minimize the bayes risk weighted distortion measure if there is a model providing the required posterior probabilities or it can operate in a suboptimal fashion by minimizing the squared error only comparisons are made with other vector quantizer based classifiers including the independent design of quantization and minimum bayes risk classification and kohonen s lvq a variety of examples demonstrate that the proposed method can provide classification ability close to or superior to learning vq while simultaneously providing superior compression performance

predictive multistage vector quantizer design using asymptotic closed loop optimization this correspondence builds on the asymptotic closed loop approach to predictive vector quantizer design khalil et al 2001 and extends it to the design of predictive multistage vector quantizers for low bit rate video coding the design approach resolves longstanding shortcomings in particular design stability and empty cell problems simulation results show substantial gains over traditional design approaches

multiresolution multiresource progressive image transmission this paper presents a new progressive image transmission pit design algorithm in which the resolution and resources rate or distortion and storage size at each transmission stage are allowed to be prespecified this algorithm uses the wavelet transform and tree structured vector quantizer tsvq techniques the wavelet transform is used to obtain a pyramid structure representation of an image the vector quantizer technique is used to design a tsvq for each subimage so that all the subimages that constitute the image at the current stage can be successively refined according to the resources available at that stage the resources assigned to each subimage for the successive refinement is determined to optimize the performance at the current stage under the resource constraints normally the resource constraints at each stage are determined by the specification of the transmission time or distortion for image data and the storage complexity of the tsvq the resolution at each stage is determined specified according to the application or as part of the design process to optimize the visual effect

network vector quantization we present an algorithm for designing locally optimal vector quantizers for general networks we discuss the algorithm s implementation and compare the performance of the resulting network vector quantizers to traditional vector quantizers vqs and to rate distortion r d bounds where available while some special cases of network codes e g multiresolution mr and multiple description md codes have been studied in the literature we here present a unifying approach that both includes these existing solutions as special cases and provides solutions to previously unsolved examples

image compression using binary space partitioning trees for low bit rate compression applications segmentation based coding methods provide in general high compression ratios when compared with traditional e g transform and subband coding approaches in this paper we present a new segmentation based image coding method that divides the desired image using binary space partitioning bsp the bsp approach partitions the desired image recursively by arbitrarily oriented lines in a hierarchical manner this recursive partitioning generates a binary tree which is referred to as the bsp tree representation of the desired image the most critical aspect of the bsp tree method is the criterion used to select the partitioning lines of the bsp tree representation in previous works we developed novel methods for selecting the bsp tree lines and showed that the bsp approach provides efficient segmentation of images in this paper we describe a hierarchical approach for coding the partitioning lines of the bsp tree representation we also show that the image signal within the different regions resulting from the recursive partitioning can be represented using low order polynomials furthermore we employ an optimum pruning algorithm to minimize the bit rate of the bsp tree representation for a given budget constraint while minimizing distortion simulation results and comparisons with other compression methods are also presented

codeword distribution for frequency sensitive competitive learning with one dimensional input data we study the codeword distribution for a conscience type competitive learning algorithm frequency sensitive competitive learning fscl using one dimensional input data we prove that the asymptotic codeword density in the limit of large number of codewords is given by a power law of the form q x c spl middot p x sup spl alpha where p x is the input data density and spl alpha depends on the algorithm and the form of the distortion measure to be minimized we further show that the algorithm can be adjusted to minimize any l sub p distortion measure with p ranging in 0 2

neural tree density estimation for novelty detection in this paper a neural competitive learning tree is introduced as a computationally attractive scheme for adaptive density estimation and novelty detection the learning rule yields equiprobable quantization of the input space and provides an adaptive focusing mechanism capable of tracking time varying distributions it is shown by simulation that the neural tree performs reasonably well while being much faster than any of the other competitive learning algorithms

optimal pruning with applications to tree structured source coding and modeling an algorithm introduced by l breiman et al 1984 in the context of classification and regression trees is reinterpreted and extended to cover a variety of applications in source coding and modeling in which trees are involved these include variable rate and minimum entropy tree structured vector quantization minimum expected cost decision trees variable order markov modeling optimum bit allocation and computer graphics and image processing using quadtrees a concentration on the first of these and a detailed analysis of variable rate tree structured vector quantization are provided it is found that variable rate tree structured vector quantization outperforms not only the fixed rate variety but also full search vector quantization the successive approximation character of variable rate tree structured vector quantization permits it to degrade gracefully if the rate is reduced at the encoder this has applications to the problem of buffer overflow

optimal bit allocation via the generalized bfos algorithm p h westerink et al 1988 developed an optimal bit allocation algorithm that is simplified when all operational distortion rate functions which are referred to as quantizer functions are convex this algorithm is restated using the generalized breiman friedman olshen and stone bfos algorithm a recently developed technique for variable rate vector quantizer design for both cases of convex and nonconvex quantizer functions qfs and its complexity is analyzed the use of the generalized bfos algorithm for optimal bit allocation is analysed it is shown that if each source has a convex quantizer function then the complexity of the algorithm is low

topographic map formation by maximizing unconditional entropy a plausible strategy for online unsupervised competitive learning and nonparametric density estimation an unsupervised competitive learning rule called the vectorial boundary adaptation rule vbar is introduced for topographic map formation since vbar is aimed at producing an equiprobable quantization of the input space it yields a nonparametric model of the input probability density function furthermore since equiprobable quantization is equivalent to unconditional entropy maximization we argue that this is a plausible strategy for maximizing mutual information shannon information rate in the case of online learning we use mutual information as a tool for comparing the performance of our rule with kohonen s self organizing feature map algorithm

vector quantization of image subbands a survey subband and wavelet decompositions are powerful tools in image coding because of their decorrelating effects on image pixels the concentration of energy in a few coefficients their multirate multiresolution framework and their frequency splitting which allows for efficient coding matched to the statistics of each frequency band and to the characteristics of the human visual system vector quantization vq provides a means of converting the decomposed signal into bits in a manner that takes advantage of remaining inter and intraband correlation as well as of the more flexible partitions of higher dimensional vector spaces since 1988 a growing body of research has examined the use of vq for subband wavelet transform coefficients we present a survey of these methods

variable rate tree structured vector quantizers in general growth algorithms for optimal tree structured vector quantizers do not exist in this paper we show that if the source satisfies certain conditions namely that of diminishing marginal returns optimal growth algorithms do exist we present such an algorithm and compare its performance with that of other tree growth algorithms even for sources that do not meet the necessary conditions for the growth algorithm to be optimal such as for speech with unknown statistics it is seen by simulation that the algorithm outperforms other known growth algorithms for sources that do not satisfy the required conditions the algorithm presented here can also be used to grow the initial tree for the pruning process the performance of such pruned trees is superior to that of trees pruned from full trees of the same rate

the asymptotic closed loop approach to predictive vector quantizer design with application in video coding the basic vector quantization vq technique employed in video coding belongs to the category of predictive vector quantization pvq as it involves quantization of the motion compensated frame prediction error it is well known that the design of pvq suffers from fundamental difficulties due to the prediction loop which have an impact on the convergence and the stability of the design procedure we propose an approach to pvq design that enjoys the stability of open loop design while it ensures ultimate optimization of the closed loop system the method is derived for general predictive quantization and we demonstrate it on video compression at low bit rates where it provides substantial improvement over standard open and closed loop design techniques further the approach outperforms standard dct based video coding

pencils of complex and real symmetric and skew matrices abstract this expository paper establishes the canonical forms under congruence for pairs of complex or real symmetric or skew matrices the treatment is in the spirit of the well known book of gantmacher on matrix theory and may be regarded as a supplement to gantmacher s chapters on pencils of matrices

variable rate vector quantization for speech image and video compression the performance of a vector quantizer can be improved by using a variable rate code three variable rate vector quantization systems are applied to speech image and video sources and compared to standard vector quantization and noiseless variable rate coding approaches the systems range from a simple and flexible tree based vector quantizer to a high performance but complex jointly optimized vector quantizer and noiseless code the systems provide significant performance improvements for subband speech coding predictive image coding and motion compensated video but provide only marginal improvements for vector quantization of linear predictive coefficients in speech and direct vector quantization of images criteria are suggested for determining when variable rate vector quantization may provide significant performance improvement over standard approaches

quiver diagrams and signed adaptive filters a graphical technique for exploring the behavior of deterministic discrete time adaptive algorithms excited by periodic inputs is examined the constraint to periodic inputs is convenient the underlying concepts used also apply to stationary inputs drawn from finite alphabets the resulting graph is based on plotting single period parameter trajectories in the parameter error space and is essentially a discrete version of flow diagrams associated with continuous time systems for the least mean square lms adaptive fir filter with the algorithm step size sufficiently small the change of adapted parameters from a particular initial value is given by the total update over the input period with the parameterization frozen at its initial value this results in the familiar elliptical hyperparaboloid surface steepest descent interpretation this method can be extended to the lms adaptive infinite input response iir filter with suitable approximations based on a small step size

singularities of bi hamiltonian systems we study the relationship between singularities of bi hamiltonian systems and algebraic properties of compatible poisson brackets as the main tool we introduce the notion of linearization of a poisson pencil from the algebraic viewpoint a linearized poisson pencil can be understood as a lie algebra with a fixed 2 cocycle in terms of such linearizations we give a criterion for non degeneracy of singular points of bi hamiltonian systems and describe their types

adaptive threshold nonlinear algorithm for adaptive filters with robustness against impulse noise a new adaptation algorithm for adaptive filters is proposed introducing adaptive threshold in the nonlinear correlation function the adaptive threshold controlled by the long term average of the error signal power makes the filtering system highly robust against impulse noise which is demonstrated by the results of simulation and theoretical calculation

entropy constrained tree structured vector quantizer design current methods for the design of pruned or unbalanced tree structured vector quantizers such as the generalized breiman friedman olshen stone gbfos algorithm proposed in 1980 are effective but suffer from several shortcomings we identify and clarify issues of suboptimality including greedy growing the suboptimal encoding rule and the need for time sharing between quantizers to achieve arbitrary rates we then present the leaf optimal tree design lotd method which with a modest increase in design complexity alters and reoptimizes tree structures obtained from conventional procedures there are two main advantages over existing methods first the optimal entropy constrained nearest neighbor rule is used for encoding at the leaves second explicit quantizer solutions are obtained at all rates without recourse to time sharing we show that performance improvement is theoretically guaranteed simulation results for image coding demonstrate that close to 1 db reduction of distortion for a given rate can be achieved by this technique relative to the gbfos method

some remarks about mishchenko fomenko subalgebras abstract we discuss and compare two different approaches to the notion of mishchenko fomenko subalgebras in poisson lie algebras of finite dimensional lie algebras one of them commonly accepted by the algebraic community uses polynomial ad invariants the other is based on formal ad invariants and allows one to deal with arbitrary lie algebras not necessarily algebraic in this sense the latter is more universal

multistage storage and entropy constrained tree structured vector quantization this paper proposes a new algorithm for the design of a tree structured vector quantizer that operates under storage and entropy constraints hence it is called the storage and entropy constrained tree structured vector quantizer sectsvq the algorithm uses the tree growing approach and designs the tree one stage layer at a time the constraints on the rate and storage size i e the number of nodes or codewords at each stage are specified prior to the design procedure while growing the tree at each stage the algorithm optimally allocates the rate and the number of nodes available for the current stage to the nodes of the previous stage using the dynamic programming technique the nodes of the current stage are then determined based on the allocations in addition to being useful as a tree structured vq sectsvq is particularly suited for application in progressive transmission moreover the optimal allocation technique used in the design can be effectively applied to other optimal resource allocation problems the sectsvq algorithm is implemented for various sources and is shown to compare favorably with other vqs

bayes risk weighted vector quantization with posterior estimation for image compression and classification classification and compression play important roles in communicating digital information their combination is useful in many applications including the detection of abnormalities in compressed medical images in view of the similarities of compression and low level classification it is not surprising that there are many similar methods for their design because some of these methods are useful for designing vector quantizers it seems natural that vector quantization vq is explored for the combined goal we investigate several vq based algorithms that seek to minimize both the distortion of compressed images and errors in classifying their pixel blocks these algorithms are investigated with both full search and tree structured codes we emphasize a nonparametric technique that minimizes both error measures simultaneously by incorporating a bayes risk component into the distortion measure used for the design and encoding we introduce a tree structured posterior estimator to produce the class posterior probabilities required for the bayes risk computation in this design for two different image sources we demonstrate that this system provides superior classification while maintaining compression close or superior to that of several other vq based designs including kohonen s 1992 learning vector quantizer and a sequential quantizer classifier design

adaptive filtering with binary reinforcement recently there has been increased interest in high speed adaptive filtering where the usual stochastic gradient or least mean square lms algorithm is replaced with the simpler algorithm where adaptation is guided only by the polarity of the error signal in this paper the convergence of this binary reinforcement br algorithm is proved under the usual independence assumption and the surprising observation is made that unlike the lms algorithm convergence occurs for any positive value of the step size parameter while the stochastic gradient algorithm attempts to minimize a mean square error cost function the binary reinforcement algorithm in fact attempts to minimize a mean absolute error cost function it is proved for the binary reinforcement algorithm that the tap weight vector converges in distribution to a random vector that is suitably concentrated about the optimal value based on a least mean absolute error cost function for a sufficiently small step size the expected cost of the asymptotic weight vector can be made as close as desired to thc minimum cost attain ed by the optimum weight vector

comparison of the convergence of two algorithms for adaptive fir digital filters the convergence properties of two different algorithms for the updating of the coefficients of an adaptive fir digital filter are investigated and compared with one another these algorithms are the stochastic iteration algorithm and the sign algorithm in this latter algorithm a one bit gradient estimation is used which makes its implementation very simple the convergence is characterized by the residual echo variance after convergence and a parameter that indicates the speed of the convergence it is shown that the convergence of the sign algorithm can always be assured but is much slower than that of the stochastic iteration algorithm if the same variance of the residual echo is to be obtained

statistical independence and novelty detection with information preserving nonlinear maps according to barlow 1989 feature extraction can be understood as finding a statistically independent representation of the probability distribution underlying the measured signals the search for a statistically independent representation can be formulated by the criterion of minimal mutual information which reduces to decorrelation in the case of gaussian distributions if nongaussian distributions are to be considered minimal mutual information is the appropriate generalization of decorrelation as used in linear principal component analyses pca we also generalize to nonlinear transformations by only demanding perfect transmission of information this leads to a general class of nonlinear transformations namely symplectic maps conservation of information allows us to consider only the statistics of single coordinates the resulting factorial representation of the joint probability distribution gives a density estimation we apply this concept to the real world problem of electrical motor fault detection treated as a novelty detection task

the covering problem and spl mu dependent adaptive algorithms the paper presents a family of techniques called adaptive covering algorithms which solve a particular covering problem how to best cover a target shape using a set of simply parameterized elements the algorithms inspired by adaptive filtering techniques provide a computationally simple robust and efficient alternative to more traditional methods such as bayesian approaches convex hulls and multi layer perceptrons the paper develops a theoretical understanding of the adaptive covering algorithms by relating their behavior via weak convergence techniques to the evolution of a deterministic ordinary differential equation ode in the process the authors give new convergence results for a class of step size dependent recursive algorithms stability and instability of the ode can be interpreted in terms of the local stability instability of the algorithm in terms of the covering problem candidate coverings tend to improve as more data is gathered whenever the ode is stable several examples are given which demonstrate the ideas and which verify that the analysis accurately predicts the true behavior of the algorithms

three dimensional subband coding of video we describe and show the results of video coding based on a three dimensional 3 d spatio temporal subband decomposition the results include a 1 mbps coder based on a new adaptive differential pulse code modulation scheme adpcm and adaptive bit allocation this rate is useful for video storage on cd rom coding results are also shown for a 384 kbps rate that are based on adpcm for the lowest frequency band and a new form of vector quantization geometric vector quantization gvq for the data in the higher frequency bands gvq takes advantage of the inherent structure and sparseness of the data in the higher bands results are also shown for a 128 kbps coder that is based on an unbalanced tree structured vector quantizer utsvq for the lowest frequency band and gvq for the higher frequency bands the results are competitive with traditional video coding techniques and provide the motivation for investigating the 3 d subband framework for different coding schemes and various applications

a differential index assignment scheme for tree structured vector quantization a differential index di assignment scheme is proposed for the image encoding system in which a variable length tree structured vector quantizer vltsvq is adopted each source vector is quantized into a terminal node of vltsvq and each terminal node is represented as a unique binary vector the proposed index assignment scheme utilizes the correlation between interblocks of the image to increase the compression ratio with the image quality maintained simulation results show that the proposed scheme achieves a much higher compression ratio than the conventional one does and that the amount of the bit rate reduction of the proposed scheme becomes large as the correlation of the image becomes large the proposed encoding scheme can be effectively used to encode mr images whose pixel values are in general highly correlated with those of the neighbor pixels

the poisson center and polynomial maximal poisson commutative subalgebras especially for nilpotent lie algebras of dimension at most seven abstract let g be a finite dimensional lie algebra over an algebraically closed field k of characteristic zero we collect some general results on the poisson center of s g including some simple criteria regarding its polynomiality and also on certain poisson commutative subalgebras of s g these facts are then used to complete our previous work on the subject ooms 2009 o5 5 i e to give an explicit description for the poisson center of all indecomposable nilpotent lie algebras of dimension at most seven among other things we also provide a polynomial maximal poisson commutative subalgebra of s g enjoying additional properties as a by product we show that a conjecture by milovanov is valid in this situation these results easily carry over to the enveloping algebra u g

the stability of the unit balls of symmetric and unitarily invariant norms abstract a compact convex set k is called stable if the midpoint mapping k k k x y x y 2 is open the main result asserts that the stability of the closed unit ball of a unitarily invariant norm is equivalent to the stability of the closed unit ball of the associated symmetric gauge function this result as well as other pointwise related results are obtained using a recently found close relationship between the facial structures of those two kinds of unit balls

image coding using vector quantization a review a review of vector quantization techniques used for encoding digital images is presented first the concept of vector quantization is introduced then its application to digital images is explained spatial predictive transform hybrid binary and subband vector quantizers are reviewed the emphasis is on the usefulness of the vector quantization when it is combined with conventional image coding techniques or when it is used in different domains

neural gas network for vector quantization and its application to time series prediction a neural network algorithm based on a soft max adaptation rule is presented this algorithm exhibits good performance in reaching the optimum minimization of a cost function for vector quantization data compression the soft max rule employed is an extension of the standard k means clustering procedure and takes into account a neighborhood ranking of the reference weight vectors it is shown that the dynamics of the reference weight vectors during the input driven adaptation procedure are determined by the gradient of an energy function whose shape can be modulated through a neighborhood determining parameter and resemble the dynamics of brownian particles moving in a potential determined by the data point density the network is used to represent the attractor of the mackey glass equation and to predict the mackey glass time series with additional local linear mappings for generating output values the results obtained for the time series prediction compare favorably with the results achieved by backpropagation and radial basis function networks

performance analysis of adaptive filters equipped with the dual sign algorithm a convergence analysis for stochastic gradient adaptive filters equipped with the dual sign algorithm dsa is presented expressions for the mean and mean squared values of the coefficient misalignment vector are derived under the assumption that the input signal is gaussian the main differences between the current analysis and previous analyses are 1 the present analysis is valid for arbitrary autocorrelation matrices and 2 the analysis does not approximate the behavior for the dsa as switching between two sign algorithms a certain amount of time after the dsa is initialized a simulation example comparing the analytical results with empirical ones is presented and the two curves show excellent agreement with each other

multiple cost constraints for the design of tree structured vector quantizers minimizing the distortion subject to a cost constraint is fundamental in the design of tree structured vector quantizers because of various competing cost measures the use of single cost constraints has led to undesirable results the author studies the relationships among several cost functions and shows how multiple cost constraints can be used to significantly improve tree design

hierarchical unsupervised learning with growing via phase transitions we address unsupervised learning subject to structural constraints with particular emphasis placed on clustering with an imposed decision tree structure most known methods are greedy optimizing one node of the tree at a time to minimize a local cost by constrast we develop a joint optimization method derived based on information theoretic principles and closely related to known methods in statistical physics the approach is inspired by the deterministic annealing algorithm for unstructured data clustering which was based on maximum entropy inference the new approach is founded on the principle of minimum cross entropy using informative priors to approximate the unstructured clustering solution while imposing the structural constraint the resulting method incorporates supervised learning principles applied in an unsupervised problem setting in our approach the tree grows by a sequence of bifurcations that occur while optimizing an effective free energy cost at decreasing temperature scales thus estimates of the tree size and structure are naturally obtained at each temperature in the process examples demonstrate considerable improvement over known methods

variable branch tree structured vector quantization tree structured vector quantizers tsvq and their variants have recently been proposed all trees used are fixed m ary tree structured such that the training samples in each node must be artificially divided into a fixed number of clusters this paper proposes a variable branch tree structured vector quantizer vbtsvq based on a genetic algorithm which searches for the number of child nodes of each splitting node for optimal coding in vbtsvq moreover one disadvantage of tsvq is that the searched codeword usually differs from the full searched codeword briefly the searched codeword in tsvq sometimes is not the closest codeword to the input vector this paper proposes the multiclassification encoding method to select many classified components to represent each cluster and the codeword encoded in the vbtsvq is usually the same as that of the full search vbtsvq outperforms other tsvqs in the experiments presented here

deterministic annealing for entropy constrained vector quantizer design typical vector quantizer vq design algorithms such as generalized lloyd methods are susceptible to local minima in addition for entropy constrained vq it is not obvious what the right number of codevectors is for any given source and target rate one method that has shown promising results for solving both problems is deterministic annealing where randomness is added to the encoding process the uncertainty of which is gradually reduced until a deterministic coder is found this method has been used for fixed rate and entropy constrained unstructured vq as well as for a variety of structurally constrained vqs for the unstructured entropy constrained case we generalize the existing methods to include new cost functions and we argue that depending on the chosen cost function some splitting criteria are more suitable than others we give analytical advantages and disadvantages to several cost functions and several splitting criteria then give performance evaluations by testing on synthetic and natural sources

recursive partitioning to reduce distortion adaptive partitioning of a multidimensional feature space plays a fundamental role in the design of data compression schemes most partition based design methods operate in an iterative fashion seeking to reduce distortion at each stage of their operation by implementing a linear split of a selected cell the operation and eventual outcome of such methods is easily described in terms of binary tree structured vector quantizers this paper considers a class of simple growing procedures for tree structured vector quantizers of primary interest is the asymptotic distortion of quantizers produced by the unsupervised implementation of the procedures it is shown that application of the procedures to a convergent sequence of distributions with a suitable limit yields quantizers whose distortion tends to zero analogous results are established for tree structured vector quantizers produced from stationary ergodic training data the analysis is applicable to procedures employing both axis parallel and oblique splitting and a variety of distortion measures the results of the paper apply directly to unsupervised procedures that may be efficiently implemented on a digital computer