approximate maximum entropy joint feature inference consistent with arbitrary lower order probability constraints application to statistical classification we propose a new learning method for discrete space statistical classifiers similar to chow and liu 1968 and cheeseman 1983 we cast classification inference within the more general framework of estimating the joint probability mass function p m f for the feature vector class label pair cheeseman s proposal to build the maximum entropy me joint p m f consistent with general lower order probability constraints is in principle powerful allowing general dependencies between features however enormous learning complexity has severely limited the use of this approach alternative models such as bayesian networks bns require explicit determination of conditional independencies these may be difficult to assess given limited data here we propose an approximate me method which like previous methods incorporates general constraints while retaining quite tractable learning the new method restricts joint p m f support during learning to a small subset of the full feature space classification gains are realized over dependence trees tree augmented naive bayes networks bns trained by the kutato algorithm and multilayer perceptrons extensions to more general inference problems are indicated we also propose a novel exact inference method when there are several missing features

statistical mechanics and phase transitions in clustering a new approach to clustering based on statistical physics is presented the problem is formulated as fuzzy clustering and the association probability distribution is obtained by maximizing the entropy at a given average variance the corresponding lagrange multiplier is related to the temperature and motivates a deterministic annealing process where the free energy is minimized at each temperature critical temperatures are derived for phase transitions when existing clusters split it is a hierarchical clustering estimating the most probable cluster parameters at various average variances

robust model predictive control of constrained linear systems with bounded disturbances this paper provides a novel solution to the problem of robust model predictive control of constrained linear discrete time systems in the presence of bounded disturbances the optimal control problem that is solved online includes uniquely the initial state of the model employed in the problem as a decision variable the associated value function is zero in a disturbance invariant set that serves as the origin when bounded disturbances are present and permits a strong stability result namely robust exponential stability of the disturbance invariant set for the controlled system with bounded disturbances to be obtained the resultant online algorithm is a quadratic program of similar complexity to that required in conventional model predictive control

advances in residual vector quantization a review advances in residual vector quantization rvq are surveyed definitions of joint encoder optimality and joint decoder optimality are discussed design techniques for rvqs with large numbers of stages and generally different encoder and decoder codebooks are elaborated and extended fixed rate rvqs and variable rate rvqs that employ entropy coding are examined predictive and finite state rvqs designed and integrated into neural network based source coding structures are revisited successive approximation rvqs that achieve embedded and refinable coding are reviewed a new type of successive approximation rvq that varies the instantaneous block rate by using different numbers of stages on different blocks is introduced and applied to image waveforms and a scalar version of the new residual quantizer is applied to image subbands in an embedded wavelet transform coding system

combined source channel vector quantization using deterministic annealing the authors present a new approach to combined source channel vector quantization the method derived within information theory and probability theory utilizes deterministic annealing to avoid some local minima that trap conventional descent algorithms such as the generalized lloyd algorithm the resulting vector quantizers satisfy the necessary conditions for local optimality for the noisy channel case they tested the method against several versions of the noisy channel generalized lloyd algorithm on stationary first order gauss markov sources with a binary symmetric channel the method outperformed other methods under all test conditions with the gains over noisy channel gla growing with the codebook size the quantizers designed using deterministic annealing are also shown to behave robustly under channel mismatch conditions as a comparison with a separate source channel system over a large range of test channel conditions the method outperformed a bandwidth equivalent system incorporating a hamming code also for severe channel conditions the method produces solutions with explicit error control coding

a survey of the theory of source coding with a fidelity criterion the purpose of this paper is threefold 1 to acquaint the reader with the types of problems that have been considered in the area of source coding with a fidelity criterion 2 to survey results that have been obtained on these problems and 3 to outline future research trends in the area

optimal quantizer design for noisy channels an approach to combined source channel coding we present an analysis of the zero memory quantization of memoryless sources when the quantizer output is to be encoded and transmitted across a noisy channel necessary conditions for the joint optimization of the quantizer and the encoder decoder pair are presented and an iterative algorithm for obtaining a locally optimum system is developed the performance of this locally optimal system obtained for the class of generalized gaussian distributions and the binary symmetric channel is compared against the optimum performance theoretically attainable using rate distortion theoretic arguments as well as against the performance of lloyd max quantizers encoded using the natural binary code and the folded binary code it is shown that this optimal design could result in substantial performance improvements the performance improvements are more noticeable at high bit rates and for broad tailed densities

unsupervised texture segmentation in a deterministic annealing framework we present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity texture segmentation is formulated as a data clustering problem based on sparse proximity data dissimilarities of pairs of textured regions are computed from a multiscale gabor filter image representation we discuss and compare a class of clustering objective functions which is systematically derived from invariance principles as a general optimization framework we propose deterministic annealing based on a mean field approximation the canonical way to derive clustering algorithms within this framework as well as an efficient implementation of mean field annealing and the closely related gibbs sampler are presented we apply both annealing variants to brodatz like microtexture mixtures and real word images

constrained clustering as an optimization method a deterministic annealing approach to clustering is derived on the basis of the principle of maximum entropy this approach is independent of the initial state and produces natural hierarchical clustering solutions by going through a sequence of phase transitions it is modified for a larger class of optimization problems by adding constraints to the free energy the concept of constrained clustering is explained and three examples are are given in which it is used to introduce deterministic annealing the previous clustering method is improved by adding cluster mass variables and a total mass constraint the traveling salesman problem is reformulated as constrained clustering yielding the elastic net en approach to the problem more insight is gained by identifying a second lagrange multiplier that is related to the tour length and can also be used to control the annealing process the open path constraint formulation is shown to relate to dimensionality reduction by self organization in unsupervised learning a similar annealing procedure is applicable in this case as well

survey constrained model predictive control stability and optimality model predictive control is a form of control in which the current control action is obtained by solving at each sampling instant a finite horizon open loop optimal control problem using the current state of the plant as the initial state the optimization yields an optimal control sequence and the first control in this sequence is applied to the plant an important advantage of this type of control is its ability to cope with hard constraints on controls and states it has therefore been widely applied in petro chemical and related industries where satisfaction of constraints is particularly important because efficiency demands operating points on or close to the boundary of the set of admissible states and controls in this review we focus on model predictive control of constrained systems both linear and nonlinear and discuss only briefly model predictive control of unconstrained nonlinear and or time varying systems we concentrate our attention on research dealing with stability and optimality in these areas the subject has developed in our opinion to a stage where it has achieved sufficient maturity to warrant the active interest of researchers in nonlinear control we distill from an extensive literature essential principles that ensure stability and use these to present a concise characterization of most of the model predictive controllers that have been proposed in the literature in some cases the finite horizon optimal control problem solved on line is exactly equivalent to the same problem with an infinite horizon in other cases it is equivalent to a modified infinite horizon optimal control problem in both situations known advantages of infinite horizon optimal control accrue

an axiomatic approach to soft learning vector quantization and clustering this paper presents an axiomatic approach to soft learning vector quantization lvq and clustering based on reformulation the reformulation of the fuzzy c means fcm algorithm provides the basis for reformulating entropy constrained fuzzy clustering ecfc algorithms according to the proposed approach the development of specific algorithms reduces to the selection of a generator function linear generator functions lead to the fcm and fuzzy learning vector quantization algorithms while exponential generator functions lead to ecfc and entropy constrained learning vector quantization algorithms the reformulation of lvq and clustering algorithms also provides the basis for developing uncertainty measures that can identify feature vectors equidistant from all prototypes these measures are employed by a procedure developed to make soft lvq and clustering algorithms capable of identifying outliers in the data set this procedure is evaluated by testing the algorithms generated by linear and exponential generator functions on speech data

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

fuzzy sets and membership functions abstract in this paper we try to provide an analytical basis for the theory of fuzzy sets the main idea is to introduce the the notions of consensus sets and subjective variables which strongly resemble respectively the events and random variables in the probability theory the resemblance is a natural consequence from the fact that one of the most important roles of the fuzzy set theory is to present a method for the analysis of the uncertainty originated in human subjectivity while the probability theory for that of randomness

r fuzzy sets abstract the theory of fuzzy sets was founded by zadeh as an approach to cope with the pressing need to deal with phenomena which cannot be modelled properly by conventional mathematics because they contain factors which are fuzzy in nature however in this frame of the theory of fuzzy sets we cannot discern two conflicting fuzzy conceptions properly and therefore the excluded middle law is violated in this theory one does not differ fuzzy sets from their membership functions that is a fuzzy set in the sense of zadeh is equivalent to its membership function this may be the main reason why the excluded middle law was violated in this paper we try to provide a mathematical frame of fuzzy sets theory such that it can not only eliminate the suspicion for the objective reality of membership functions of fuzzy sets but also avoid violating the excluded middle law in our frame fuzzy sets and their membership functions are not equivalent conceptions

output feedback robust mpc with one free control move for the linear polytopic uncertain system with bounded disturbance in this paper a previous approach for the robust model predictive control mpc for a linear polytopic uncertain system is extended to the case with bounded disturbance and unmeasurable state the controller on line optimizes a free control move followed by an output feedback control law based on the pre specified state estimator a key technique for this controller is an appropriate formulation of the estimation error bound which accounts for recursive feasibility of the optimization problem the quadratic boundedness qb of the augmented state is guaranteed by the proposed approach a numerical example is given to illustrate the effectiveness of the proposed controller

on estimation error bounds for receding horizon filters using quadratic boundedness quadratic boundedness is used to deal with stability and design of receding horizon estimators upper bounds on the norm of the estimation error have been found by means of invariant sets that can be constructed by using quadratic boundedness moreover these bounds are expressed in terms of linear matrix inequalities and are well suited to being minimized for the purpose of design

information cut for clustering using a gradient descent approach we introduce a new graph cut for clustering which we call the information cut it is derived using parzen windowing to estimate an information theoretic distance measure between probability density functions we propose to optimize the information cut using a gradient descent based approach our algorithm has several advantages compared to many other graph based methods in terms of determining an appropriate affinity measure computational complexity memory requirements and coping with different data scales we show that our method may produce clustering and image segmentation results comparable or better than the state of the art graph based methods

a stochastic self organizing map for proximity data we derive an efficient algorithm for topographic mapping of proximity data tmp which can be seen as an extension of kohonen s self organizing map to arbitrary distance measures the tmp cost function is derived in a baysian framework of folded markov chains for the description of autoencoders it incorporates the data by a dissimilarity matrix and the topographic neighborhood by a matrix of transition probabilities from the principle of maximum entropy a nonfactorizing gibbs distribution is obtained which is approximated in a mean field fashion this allows for maximum likelihood estimation using an expectation maximization algorithm in analogy to the transition from topographic vector quantization to the self organizing map we suggest an approximation to tmp that is computationally more efficient in order to prevent convergence to local minima an annealing scheme in the temperature parameter is introduced for which the critical temperature of the first phase transition is calculated in terms o

unsupervised texture classification using vector quantization and deterministic relaxation neural network this paper describes the use of a neural network architecture for classifying textured images in an unsupervised manner using image specific constraints the texture features are extracted by using two dimensional 2 d gabor filters arranged as a set of wavelet bases the classification model comprises feature quantization partition and competition processes the feature quantization process uses a vector quantizer to quantize the features into codevectors where the probability of grouping the vectors is modeled as gibbs distribution a set of label constraints for each pixel in the image are provided by the partition and competition processes an energy function corresponding to the a posteriori probability is derived from these processes and a neural network is used to represent this energy function the state of the network and the codevectors of the vector quantizer are iteratively adjusted using a deterministic relaxation procedure until a stable state is reached the final equilibrium state of the vector quantizer gives a classification of the textured image a cluster validity measure based on modified hubert index is used to determine the optimal number of texture classes in the image

on dynamic output feedback robust mpc for constrained quasi lpv systems this paper considers the dynamic output feedback robust model predictive control mpc of a quasi linear parameter varying quasi lpv system with bounded noise in our previous works for the unknown true state either its ellipsoidal bounds or its polyhedral bounds were solely applied in the main optimisation problem the recursive feasibility of the main optimisation problem was guaranteed by a simple refreshment of the ellipsoidal bound but might be lost by applying the polyhedral bounds this paper shows how and to what extent the recursive feasibility can be restored when the polyhedral bounds are still utilised first we propose a new approach which at each sampling time utilises either the ellipsoidal bound or the polyhedral bound in the main optimisation problem the latter being used if and only if it is contained in the former then we show the sufficient conditions under which the approaches based on polyhedral bounds preserve the property of recursive feasibility a numerical example is

model based clustering by probabilistic self organizing maps in this paper we consider the learning process of a probabilistic self organizing map pbsom as a model based data clustering procedure that preserves the topological relationships between data clusters in a neural network based on this concept we develop a coupling likelihood mixture model for the pbsom that extends the reference vectors in kohonen s self organizing map som to multivariate gaussian distributions we also derive three expectation maximization em type algorithms called the socem soem and sodaem algorithms for learning the model pbsom based on the maximum likelihood criterion socem is derived by using the classification em cem algorithm to maximize the classification likelihood soem is derived by using the em algorithm to maximize the mixture likelihood and sodaem is a deterministic annealing da variant of socem and soem moreover by shrinking the neighborhood size socem and soem can be interpreted respectively as da variants of the cem and em algorithms for gaussian model based clustering the experimental results show that the proposed pbsom learning algorithms achieve comparable data clustering performance to that of the deterministic annealing em daem approach while maintaining the topology preserving property

the design of joint source and channel trellis waveform coders the generalized lloyd algorithm is applied to the design of joint source and channel trellis waveform coders to encode discrete time continuous amplitude stationary and ergodic sources operating over discrete memoryless noisy channels experimental results are provided for independent and autoregressive gaussian sources binary symmetric channels and absolute error and squared error distortion measures performance of the joint codes is compared with the tandem combination of a trellis source code and a trellis channel code on the independent gaussian source using the squared error distortion measure operating over an additive white gaussian noise channel it is observed that the jointly optimized codes achieve performance close to or better than that of separately optimized tandem codes of the same constraint length performance improvement via a predictive joint source and channel trellis code is demonstrated for the autoregressive gaussian source using the squared error distortion measure

deterministic annealing gustafson kessel fuzzy clustering algorithm abstract the gustafson and kessel gk fuzzy clustering algorithm proposed by gustafson and kessel in 1979 was the first important extension to the fuzzy c means fcm algorithm up to now the gk algorithm had become one of the most commonly used fuzzy clustering algorithms where the mahalanobis distance is used as a dissimilarity measure to provide more effectiveness and robustness than the fcm algorithm recently chaomurilige et al 2015 proposed a theoretical analysis on the parameter selection for the gk algorithm in which they indicated that the parameter of the fuzziness index heavily influences the performance of the gk algorithm in this paper we propose a novel gk fuzzy clustering algorithm based on the deterministic annealing approach for decreasing the effect of parameters we first consider maximizing the shannon s entropy of membership functions to the gk objective function and then use deterministic annealing to adjust the annealing parameter we also mathematically provide a theoretical initialization lower bound for the annealing parameter of the proposed deterministic annealing gk da gk algorithm comparisons between the da gk algorithm and other methods are made the computational complexity of the proposed method is also provided experimental results and comparisons actually verify theoretical results and also indicate the superiority and effectiveness of the proposed da gk algorithm

a fast search algorithm for vector quantization using mean pyramids of codewords one of the most serious problems for vector quantization especially for high dimensional vectors is the high computational complexity of searching for the closest codeword in the codebook design and encoding phases although quantizing high dimensional vectors rather than low dimensional vectors results in better performance the computation time needed for vector quantization grows exponentially with the vector dimension this makes high dimensional vectors unsuitable for vector quantization to overcome this problem a fast search algorithm under the assumption that the distortion is measured by the squared euclidean distance is proposed using the mean pyramids of codewords the algorithm ran reject many codewords that are impossible matches and hence save a great deal of computation time the algorithm is efficient for high dimensional codeword searches experimental results confirm the effectiveness of the proposed method

network vector quantization we present an algorithm for designing locally optimal vector quantizers for general networks we discuss the algorithm s implementation and compare the performance of the resulting network vector quantizers to traditional vector quantizers vqs and to rate distortion r d bounds where available while some special cases of network codes e g multiresolution mr and multiple description md codes have been studied in the literature we here present a unifying approach that both includes these existing solutions as special cases and provides solutions to previously unsolved examples

dynamic output feedback predictive control for nonlinear systems represented by a takagi sugeno model this paper addresses the output feedback predictive control for a takagi sugeno t s fuzzy system with bounded noise the controller optimizes an infinite horizon objective function respecting the input and state constraints the control law is parameterized as a dynamic output feedback that is dependent on the membership functions and the closed loop stability is specified by the notion of quadratic boundedness online algorithms that guarantee the recursive feasibility of the convex optimization problem and the convergence of the augmented state to a neighborhood of the equilibrium point are proposed in this paper a numerical example is given to illustrate the effectiveness of the proposed output feedback controllers

stability based validation of clustering solutions data clustering describes a set of frequently employed techniques in exploratory data analysis to extract natural group structure in data such groupings need to be validated to separate the signal in the data from spurious structure in this context finding an appropriate number of clusters is a particularly important model selection question we introduce a measure of cluster stability to assess the validity of a cluster model this stability measure quantifies the reproducibility of clustering solutions on a second sample and it can be interpreted as a classification risk with regard to class labels produced by a clustering algorithm the preferred number of clusters is determined by minimizing this classification risk as a function of the number of clusters convincing results are achieved on simulated as well as gene expression data sets comparisons to other methods demonstrate the competitive performance of our method and its suitability as a general validation tool for clustering solutions in real world problems

resolution based complexity control for gaussian mixture models in the domain of unsupervised learning mixtures of gaussians have become a popular tool for statistical modeling for this class of generative models we present a complexity control scheme which provides an effective means for avoiding the problem of overfitting usually encountered with unconstrained mixtures of gaussians in high dimensions according to some prespecified level of resolution as implied by a fixed variance noise model the scheme provides an automatic selection of the dimensionalities of some local signal subspaces by maximum likelihood estimation together with a resolution based control scheme for adjusting the number of mixture components we arrive at an incremental model refinement procedure within a common deterministic annealing framework which enables an efficient exploration of the model space the advantages of the resolution based framework are illustrated by experimental results on synthetic and high dimensional real world data

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

general statistical inference for discrete and mixed spaces by an approximate application of the maximum entropy principle we propose a method for learning a general statistical inference engine operating on discrete and mixed discrete continuous feature spaces such a model allows inference on any of the discrete features given values for the remaining features applications are e g to medical diagnosis with multiple possible diseases fault diagnosis information retrieval and imputation in databases bayesian networks bns are versatile tools that possess this inference capability however bns require explicit specification of conditional independencies which may be difficult to assess given limited data alternatively cheeseman 1983 proposed finding the maximum entropy me joint probability mass function pmf consistent with arbitrary lower order probability constraints this approach is in principle powerful and does not require explicit expression of conditional independence however until now the huge learning complexity has severely limited the use of this approach here we propose an approximate me method which also encodes arbitrary low order constraints but while retaining quite tractable learning our method uses a restriction of joint pmf support during learning to a subset of the feature space results on the university of california irvine repository reveal performance gains over several bn approaches and over multilayer perceptrons

vector quantization by deterministic annealing a deterministic annealing approach is suggested to search for the optimal vector quantizer given a set of training data the problem is reformulated within a probabilistic framework no prior knowledge is assumed on the source density and the principle of maximum entropy is used to obtain the association probabilities at a given average distortion the corresponding lagrange multiplier is inversely related to the temperature and is used to control the annealing process in this process as the temperature is lowered the system undergoes a sequence of phase transitions when existing clusters split naturally without use of heuristics the resulting codebook is independent of the codebook used to initialize the iterations

trellis coded quantization designed for noisy channels trellis coded quantization tcq of memoryless sources is developed for transmission over a binary symmetric channel the optimized tcq coder can achieve essentially the same performance as ayanoglu and gray s 1987 unconstrained trellis coding optimized for the binary symmetric channel but with a much lower implementation complexity for transmission rates above 1 b sample in most cases the optimized tcq coder also provides larger signal to noise ratio than farvardin and vaishampayan s 1991 channel optimized vector quantization algorithms are developed for the joint design of trellis coded quantization modulation tcq tcm the jointly designed tcq tcm system outperforms the straightforward cascade of separately designed tcq and tcm systems the improvement is most significant at low channel signal to noise ratio for a first order gauss markov source the predictive tcq tcm performance can exceed that of optimum pulse amplitude modulation

stochastic organization of output codes in multiclass learning problems the best known decomposition schemes of multiclass learning problems are one per class coding opc and error correcting output coding ecoc both methods perform a prior decomposition that is before training of the classifier takes place the impact of output codes on the inferred decision rules can be experienced only after learning therefore we present a novel algorithm for the code design of multiclass learning problems this algorithm applies a maximum likelihood objective function in conjunction with the expectation maximization em algorithm minimizing the augmented objective function yields the optimal decomposition of the multiclass learning problem in two class problems experimental results show the potential gain of the optimized output codes over opc or ecoc methods

brief paper mpc for tracking piecewise constant references for constrained linear systems in this paper a novel model predictive control mpc for constrained non square linear systems to track piecewise constant references is presented this controller ensures constraint satisfaction and asymptotic evolution of the system to any target which is an admissible steady state therefore any sequence of piecewise admissible setpoints can be tracked without error if the target steady state is not admissible the controller steers the system to the closest admissible steady state these objectives are achieved by i adding an artificial steady state and input as decision variables ii using a modified cost function to penalize the distance from the artificial to the target steady state iii considering an extended terminal constraint based on the notion of invariant set for tracking the control law is derived from the solution of a single quadratic programming problem which is feasible for any target furthermore the proposed controller provides a larger domain of attraction for a given control horizon than the standard mpc and can be explicitly computed by means of multiparametric programming tools on the other hand the extra degrees of freedom added to the mpc may cause a loss of optimality that can be arbitrarily reduced by an appropriate weighting of the offset cost term

on the performance and complexity of channel optimized vector quantizers the performance and complexity of channel optimized vector quantizers are studied for the gauss markov source observations on the geometric structure of these quantizers are made which have an important implication on the encoding complexity for the squared error distortion measure it is shown that an operation equivalent to a euclidean distance measurement with respect to an appropriately defined set of points used to identify the encoding regions can be used to perform the encoding this implies that the encoding complexity is proportional to the number of encoding regions it is then demonstrated that for very noisy channels and a heavily correlated source when the codebook size is large the number of encoding regions is considerably smaller than the codebook size implying a reduction in encoding complexity

a sequence based extension of mean field annealing using the forward backward algorithm application to image segmentation optimization based on mean field theory including mean field annealing mfa is widely used for discrete optimization label assignment problems defined on the pixel sites of an image one formulation of mfa is via maximum entropy where one seeks the joint distribution over the random assignments subject to an average level of cost mfa is obtained by assuming the assignments at each pixel are statistically independent given the observed image alternatively we make the less restrictive assumption of independent row labelings the independence assumption means that at each step mfa optimizes over only one pixel whereas our method jointly optimizes over an entire row i e our method is less greedy in principle an mfa extension could be developed that explicitly re estimates the row labeling distributions but such an approach is in practice infeasible even so we can indirectly implement this re estimation by re estimating quantities that determine the row labeling distributions these quantities are the a posteriori site probabilities re estimable via the well known forward backward f b algorithm thus our algorithm which descends in the me lagrangian free energy consists of iterative application of f b to the image rows columns at convergence maximum a posteriori site labeling is performed our method was applied to segmentation of both real and synthetic noise corrupted images it achieved lower markov random field model potentials and better segmentations compared with other methods and in high noise standard mfa

robust constrained model predictive control using linear matrix inequalities abstract the primary disadvantage of current design techniques for model predictive control mpc is their inability to deal explicitly with plant model uncertainty in this paper we present a new approach for robust mpc synthesis that allows explicit incorporation of the description of plant uncertainty in the problem formulation the uncertainty is expressed in both the time and frequency domains the goal is to design at each time step a state feedback control law that minimizes a worst case infinite horizon objective function subject to constraints on the control input and plant output using standard techniques the problem of minimizing an upper bound on the worst case objective function subject to input and output constraints is reduced to a convex optimization involving linear matrix inequalities lmis it is shown that the feasible receding horizon state feedback control design robustly stabilizes the set of uncertain plants several extensions such as application to systems with time delays problems involving constant set point tracking trajectory tracking and disturbance rejection which follow naturally from our formulation are discussed the controller design is illustrated with two examples

mixture of experts regression modeling by deterministic annealing we propose a new learning algorithm for regression modeling the method is especially suitable for optimizing neural network structures that are amenable to a statistical description as mixture models these include mixture of experts hierarchical mixture of experts hme and normalized radial basis functions nrbf unlike recent maximum likelihood ml approaches we directly minimize the squared regression error we use the probabilistic framework as means to define an optimization method that avoids many shallow local minima on the complex cost surface our method is based on deterministic annealing da where the entropy of the system is gradually reduced with the expected regression cost energy minimized at each entropy level the corresponding lagrangian is the system s free energy and this annealing process is controlled by variation of the lagrange multiplier which acts as a temperature parameter the new method consistently and substantially outperformed the competing methods for training nrbf and hme regression functions over a variety of benchmark regression examples

the multitone channel the maximum bit rate of multitone qam quadrature amplitude modulation over a general linear channel is found first the overall bit rate for an awgn channel with a two level transfer function is maximized using a multitone qam system the power distribution between the tones and the number of bits symbol per tone is optimized for a given symbol error rate extending these results to the general channel it is shown that the optimum power division for multitone signals is similar to the water pouring solution of information theory furthermore multitone qam performance is about 9 db worse than the channel capacity independent of the channel characteristics the multitone results throughout are compared to those of an equivalent single tone linearly equalized system the comparison shows that the multitone system is useful for some channels e g those with deep nulls in the transfer function the maximum bit error rate over a twisted pair channel which is performance dominated by near end crosstalk next is also found

a mapping approach to rate distortion computation and analysis in rate distortion theory results are often derived and stated in terms of the optimizing density over the reproduction space in this paper the problem is reformulated in terms of the optimal mapping from the unit interval with lebesgue measure that would induce the desired reproduction probability density this results in optimality conditions that are random relatives of the known lloyd 1982 optimality conditions for deterministic quantizers the validity of the mapping approach is assured by fundamental isomorphism theorems for measure spaces we show that for the squared error distortion the optimal reproduction random variable is purely discrete at supercritical distortion where the shannon 1948 lower bound is not tight the gaussian source is thus the only source that produces continuous reproduction variables for the entire range of positive rate to analyze the evolution of the optimal reproduction distribution we use the mapping formulation and establish an analogy to statistical mechanics the solutions are given by the distribution at isothermal statistical equilibrium and are parameterized by the temperature in direct correspondence to the parametric solution of the variational equations in rate distortion theory the analysis of an annealing process shows how the number of symbols grows as the system undergoes phase transitions thus an algorithm based on the mapping approach often needs but a few variables to find the exact solution while the blahut 1972 algorithm would only approach it at the limit of infinite resolution finally a quick deterministic annealing algorithm to generate the rate distortion curve is suggested the resulting curve is exact as long as continuous phase transitions in the process are accurately followed

a deterministic annealing approach for parsimonious design of piecewise regression models a new learning algorithm is proposed for piecewise regression modeling it employs the technique of deterministic annealing to design space partition regression functions while the performance of traditional space partition regression functions such as cart and mars is limited by a simple tree structured partition and by a hierarchical approach for design the deterministic annealing algorithm enables the joint optimization of a more powerful piecewise structure based on a voronoi partition the new method is demonstrated to achieve consistent performance improvements over regular cart as well as over its extension to allow arbitrary hyperplane boundaries comparison tests on several benchmark data sets from the regression literature are provided

technical communique design of state estimators for uncertain linear systems using quadratic boundedness the notion of quadratic boundedness which allows one to address the stability of a dynamic system in the presence of bounded disturbances is applied to the design of state estimators for discrete time linear systems with polytopic uncertainties necessary and sufficient stability conditions are stated and upper bounding sequences on the estimation error are derived for the purpose of design such conditions can be expressed in terms of linear matrix inequalities lmis thus guaranteeing the numerical tractability simulation results are reported to show the effectiveness of the approach

regularized discriminant entropy analysis in this paper we propose the regularized discriminant entropy rde which considers both class information and scatter information on original data based on the results of maximizing the rde we develop a supervised feature extraction algorithm called regularized discriminant entropy analysis rdea rdea is quite simple and requires no approximation in theoretical derivation the experiments with several publicly available data sets show the feasibility and effectiveness of the proposed algorithm with encouraging results we propose the regularized discriminant entropy rde the rde is based on the within class entropy and robust estimation we design a supervised algorithm regularized discriminant entropy analysis rdea rdea can be regarded as a framework for supervised feature extraction

entropy constrained tree structured vector quantizer design current methods for the design of pruned or unbalanced tree structured vector quantizers such as the generalized breiman friedman olshen stone gbfos algorithm proposed in 1980 are effective but suffer from several shortcomings we identify and clarify issues of suboptimality including greedy growing the suboptimal encoding rule and the need for time sharing between quantizers to achieve arbitrary rates we then present the leaf optimal tree design lotd method which with a modest increase in design complexity alters and reoptimizes tree structures obtained from conventional procedures there are two main advantages over existing methods first the optimal entropy constrained nearest neighbor rule is used for encoding at the leaves second explicit quantizer solutions are obtained at all rates without recourse to time sharing we show that performance improvement is theoretically guaranteed simulation results for image coding demonstrate that close to 1 db reduction of distortion for a given rate can be achieved by this technique relative to the gbfos method

technical communique a synthesis approach for output feedback robust constrained model predictive control this paper addresses the synthesis approach for output feedback robust model predictive control for systems with polytopic description bounded state disturbance and measurement noise off line it calculates a sequence of output feedback laws based on the state estimators by solving linear matrix inequality lmi optimization problems on line at each sampling time it chooses an appropriate output feedback law from this sequence the primary contribution is to present a rigorous method to guarantee satisfaction of input state constraints a numerical example is given to illustrate the effectiveness of the controller

brief paper constrained robust model predictive control via parameter dependent dynamic output feedback this paper considers output feedback robust model predictive control for the quasi linear parameter varying quasi lpv system with bounded disturbance the so called quasi lpv means that the varying parameters of the linear system are known at the current time but unknown in the future the control law is parameterized as a parameter dependent dynamic output feedback and the closed loop stability is specified by the notion of quadratic boundedness an iterative algorithm is proposed for the on line synthesis of the control law via convex optimization a numerical example is given to illustrate the effectiveness of the controller

voronoi regions of lattices second moments of polytopes and quantization if a point is picked at random inside a regular simplex octahedron 600 cell or other polytope what is its average squared distance from the centroid in n dimensional space what is the average squared distance of a random point from the closest point of the lattice a n or d n e n a n ast or d n ast the answers are given here together with a description of the voronoi or nearest neighbor regions of these lattices the results have applications to quantization and to the design of signals for the gaussian channel for example a quantizer based on the eight dimensional lattice e8 has a mean squared error per symbol of 0 0717 cdots when applied to uniformly distributed data compared with 0 08333 cdots for the best one dimensional quantizer

linear codes interpolation from noisy patterns by means of a vector quantization process abstract an algorithm inferring a boolean linear code from noisy patterns received by a noisy channel under the assumption of uniform occurrence distribution over the codewords and an upper bound to the amount of data are presented a vector quantizer is designed from the noisy patterns choosing the obtained codebook as code approximation it is shown both theoretically and experimentally that when the data are affected by independent random errors this strategy requires a small number of patterns to obtain a good identification with high probability of the code from the noisy data

deterministic annealing based optimization for zero delay source channel coding in networks this paper studies the problem of global optimization of zero delay source channel codes that map between the source space and the channel space under a given transmission power constraint and for the mean square error distortion particularly we focus on two well known network settings the wyner ziv setting where only a decoder has access to side information and the distributed setting where independent encoders transmit over independent channels to a central decoder prior work derived the necessary conditions for optimality of the encoder and decoder mappings along with a greedy optimization algorithm that imposes these conditions iteratively in conjunction with the heuristic noisy channel relaxation method to mitigate poor local minima while noisy channel relaxation is arguably effective in simple settings it fails to provide accurate global optimization in more complicated settings considered in this paper we propose a powerful nonconvex optimization method based on the concept of deterministic annealing which is derived from information theoretic principles and was successfully employed in several problems including vector quantization classification and regression we present comparative numerical results that show strict superiority of the proposed method over greedy optimization methods as well as prior approaches in literature

an analysis of the elastic net approach to the traveling salesman problem this paper analyzes the elastic net approach durbin and willshaw 1987 to the traveling salesman problem of finding the shortest path through a set of cities the elastic net approach jointly minimizes the length of an arbitrary path in the plane and the distance between the path points and the cities the tradeoff between these two requirements is controlled by a scale parameter k a global minimum is found for large k and is then tracked to a small value in this paper we show that 1 in the small k limit the elastic path passes arbitrarily close to all the cities but that only one path point is attracted to each city 2 in the large k limit the net lies at the center of the set of cities and 3 at a critical value of k the energy function bifurcates we also show that this method can be interpreted in terms of extremizing a probability distribution controlled by k the minimum at a given k corresponds to the maximum a posteriori map bayesian estimate of the tour under a natural statistical interpretation the analysis presented in this paper gives us a better understanding of the behavior of the elastic net allows us to better choose the parameters for the optimization and suggests how to extend the underlying ideas to other domains

concepts of probabilistic sets abstract in the field of pattern recognition or decision making theory the important subjects are as follows 1 ambiguity of property of objects 2 variety of character of objects 3 subjectivity of observers 4 evolution of knowledge of observers i e learning considering these points the concept of probabilistic set is proposed it is based on both probability theory and fuzzy concepts a probabilistic set on a total space is defined by a point wise measurable function from a parameter space which is a probability space to a characteristic space which is a measurable space it is shown that the family of all probabilistic sets constitutes a complete pseudo boolean algebra moment analysis is possible by using a probability measure of the parameter space other useful concepts are also mentioned such as probabilistic mappings and expected cardinal numbers

transmission of analog signals using multicarrier modulation a combined source channel coding approach we have proposed and analyzed a combined source channel coding scheme using multicarrier modulation by changing the power and modulation of subchannels carrying different bits of the quantized signal the channel induced distortion can be minimized we derive an algorithm for the allocation of power among subchannels analog sources quantized by scalar and vector quantizers are considered for a gaussian source scalar quantized to 8 bit multicarrier transmission yields a 13 db improvement of signal to distortion ratio over a single carrier system quantizers having a smaller number of bits yield smaller improvements we consider both full search and binary tree search vector quantizers using a natural binary coding scheme for a vector quantizer using 2 b sample multicarrier transmission yields an improvement in signal to distortion ratio that lies between 0 3 and 0 8 db depending on the quantizer design

min max control of constrained uncertain discrete time linear systems for discrete time uncertain linear systems with constraints on inputs and states we develop an approach to determine state feedback controllers based on a min max control formulation robustness is achieved against additive norm bounded input disturbances and or polyhedral parametric uncertainties in the state space matrices we show that the finite horizon robust optimal control law is a continuous piecewise affine function of the state vector and can be calculated by solving a sequence of multiparametric linear programs when the optimal control law is implemented in a receding horizon scheme only a piecewise affine function needs to be evaluated on line at each time step the technique computes the robust optimal feedback controller for a rather general class of systems with modest computational effort without needing to resort to gridding of the state space

on finite block length quantization distortion we investigate the upper and lower bounds on the quantization distortions for independent and identically distributed continuous sources in the finite block length regime we derive a lower bound on the quantization distortion as well as an upper bound on the quantization distortion based on random quantization codebooks moreover we apply the obtained bounds to the continuous gaussian source for the gaussian source we propose a computationally tractable method to numerically compute the upper and lower bounds for both bounded and unbounded quantization codebooks numerical results show that the gap between the upper and lower bounds is small for the block length of several hundreds and the upper and lower bounds are tighter than those proposed in the existing works

systems with persistent disturbances predictive control with restricted constraints predictive regulation of linear discrete time systems subject to unknown but bounded disturbances and to state control constraints is addressed an algorithm based on constraint restrictions is presented and its stability properties are analysed

gradient algorithms for designing predictive vector quantizers a predictive vector quantizer pvq is a vector extension of a predictive quantizer it consists of two parts a conventional memoryless vector quantizer vq and a vector predictor two gradient algorithms for designing a pvq are developed in this paper the steepest descent sd algorithm and the stochastic gradient sg algorithm both have the property of improving the quantizer and the predictor in the sense of minimizing the distortion as measured by the average mean squared error the differences between the two design approaches are the period and the step size used in each iteration to update the codebook and predictor the sg algorithm updates once for each input training vector and uses a small step size while the sd updates only once for a long period possibly one pass over the entire training sequence and uses a relatively large step size code designs and tests are simulated for both gauss markov sources and for sampled speech waveforms and the results are compared to codes designed using techniques that attempt to optimize only the quantizer for the predictor and not vice versa

vector quantization with transmission energy allocation for time varying channels this work is concerned with the problem of designing robust vector quantizer vq based communication systems for operation over time varying gaussian channels transmission energy allocation to vq codeword bits according to their error sensitivities is a powerful tool for improving robustness to channel noise the power of this technique can be further enhanced by appropriately combining it with index assignment methods we pose the corresponding joint optimization problem and suggest a simple iterative algorithm for finding a locally optimal solution the susceptibility of the solution to poor local minima is significantly reduced by an enhanced version of the algorithm which invokes the method of noisy channel relaxation whereby the vq system is optimized while gradually decreasing the assumed level of channel noise in a series of experiments the resulting combined technique is shown to outperform standard pseudo gray coding by up to 3 5 db and to exhibit graceful degradation at mismatched channel conditions finally we extend these ideas to the case where both the transmitter and the receiver have information on the current state of a time varying channel the proposed method is based on switched encoding and adaptive decoding experimental results show that the proposed system achieves close to optimal performance

output feedback robust mpc for lpv system with polytopic model parametric uncertainty and bounded disturbance abstractthe output feedback robust model predictive control mpc for the linear parameter varying lpv system with norm bounded disturbance is addressed where the model parametric matrices are only known to be bounded within a polytope the previous techniques of norm bounding technique quadratic boundedness qb dynamic output feedback and ellipsoid true state bound tsb refreshment formula for guaranteeing recursive feasibility are fused into the newly proposed approaches in the notion of qb the full lyapunov matrix is applied for the first time in this context the single step dynamic output feedback robust mpc where the infinite horizon control moves are parameterised as a dynamic output feedback law is the main topic of this paper while the multi step method is also suggested in order to strictly guarantee the physical constraints the outer bound of the true state replaces the true state itself so tightness of this bound has a major effect on the control performance in order to ti

a study of vector quantization for noisy channels several issues related to vector quantization for noisy channels are discussed an algorithm based on simulated annealing is developed for assigning binary codewords to the vector quantizer code vectors it is shown that this algorithm could result in dramatic performance improvements as compared to randomly selected codewords a modification of the simulated annealing algorithm for binary codeword assignment is developed for the case where the bits in the codeword are subjected to unequal error probabilities resulting from unequal levels of error protection an algorithm for the design of an optimal vector quantizer for a noisy channel is briefly discussed and its robustness under channel mismatch conditions is studied numerical results for a stationary first order gauss markov source and a binary symmetric channel are provided it is concluded that the channel optimized vector quantizer design algorithm if used carefully can result in a fairly robust system with no additional delay the case in which the communication channel is nonstationary as in mobile radio channels is studied and some preliminary ideas for quantizer design are presented

pseudo gray coding a pseudo gray code is an assignment of n bit binary indexes to 2 points in a euclidean space so that the hamming distance between two points corresponds closely to the euclidean distance pseudo gray coding provides a redundancy free error protection scheme for vector quantization vq of analog signals when the binary indexes are used as channel symbols on a discrete memoryless channel and the points are signal codevectors binary indexes are assigned to codevectors in a way that reduces the average quantization distortion introduced in the reproduced source vectors when a transmitted index is corrupted by channel noise a globally optimal solution to this problem is generally intractable due to an inherently large computational complexity a locally optimal solution the binary switching algorithm is introduced based on the objective of minimizing a useful upper bound on the average system distortion the algorithm yields a significant reduction in average distortion and converges in reasonable running times the sue of pseudo gray coding is motivated by the increasing need for low bit rate vq based encoding systems that operate on noisy channels such as in mobile radio speech communications

predictive vector quantizer design using deterministic annealing a new approach is proposed for predictive vector quantizer pvq design which is inherently probabilistic and is based on ideas from information theory and analogies to statistical physics the approach effectively resolves three longstanding fundamental shortcomings of standard pvq design the first complication is due to the pvq prediction loop which has a detrimental impact on the convergence and the stability of the design procedure the second shortcoming is due to the piecewise constant nature of the quantizer function which makes it difficult to optimize the predictor with respect to the overall reconstruction error finally a shortcoming inherited from standard vq design is the tendency of the design algorithm to terminate at a locally rather than the globally optimal solution we propose a new pvq design approach that embeds our previous asymptotic closed loop acl approach within a deterministic annealing da framework the overall da acl method profits from its two main components in a complementary way acl is used to overcome the first difficulty and offers the means for stable quantizer design as it provides an open loop design platform yet allows the pvq design algorithm to asymptotically converge to optimization of the closed loop performance objective da simultaneously mitigates or eliminates the remaining design shortcomings its probabilistic framework replaces hard quantization with a differentiable expected cost function that can be jointly optimized for the predictor and quantizer parameters and its annealing schedule allows the avoidance of many poor local optima substantial performance gains over traditional methods have been achieved in the simulations

a unified approach to tree structured and multistage vector quantization for noisy channels the large encoding complexity and sensitivity to channel errors of vector quantization vq are discussed the performance of two low complexity vqs the tree structured vq tsvq and the multistage vq msvq when used over noisy channels are analyzed an algorithm is developed for the design of channel matched tsvq cm tsvq and channel matched msvq cm msvq under the squared error criterion extensive numerical results are given for the correlation coefficient 0 9 comparisons with the ordinary tsvq and msvq designed for the noiseless channel show substantial improvements when the channel is very noisy the cm msvq which can be regarded as a block structured combined source channel coding scheme is compared with a block structured tandem source channel coding scheme with the same block length as the cm msvq for the gauss markov source the cm msvq outperforms the tandem scheme in all cases that the authors have considered it is demonstrated that the cm msvq is fairly robust to channel mismatch

a generalized interpolative vector quantization method for jointly optimal quantization interpolation and binarization of text images this paper presents an approach for the effective combination of interpolation with binarization of gray level text images to reconstruct a high resolution binary image from a lower resolution gray level one we study two nonlinear interpolative techniques for text image interpolation these nonlinear interpolation methods map quantized low dimensional 2 spl times 2 image blocks to higher dimensional 4 spl times 4 possibly binary blocks using a table lookup operation the first method performs interpolation of text images using context based nonlinear interpolative vector quantization nlivq this system has a simple training procedure and has performance for gray level high resolution images that is comparable to our more sophisticated generalized interpolative vq givq approach which is the second method in it we jointly optimize the quantizer and interpolator to find matched codebooks for the low and high resolution images then to obtain the binary codebook that incorporates binarization with interpolation we introduce a binary constrained optimization method using givq in order to incorporate the nearest neighbor constraint on the quantizer while minimizing the distortion in the interpolated image a deterministic annealing based optimization technique is applied with a few interpolation examples we demonstrate the superior performance of this method over the nlivq method especially for binary outputs and other standard techniques e g bilinear interpolation and pixel replication

a theory of proximity based clustering structure detection by optimization abstract in this paper a systematic optimization approach for clustering proximity or similarity data is developed starting from fundamental invariance and robustness properties a set of axioms is proposed and discussed to distinguish different cluster compactness and separation criteria the approach covers the case of sparse proximity matrices and is extended to nested partitionings for hierarchical data clustering to solve the associated optimization problems a rigorous mathematical framework for deterministic annealing and mean field approximation is presented efficient optimization heuristics are derived in a canonical way which also clarifies the relation to stochastic optimization by gibbs sampling similarity based clustering techniques have a broad range of possible applications in computer vision pattern recognition and data analysis as a major practical application we present a novel approach to the problem of unsupervised texture segmentation which relies on statistical tests as a measure of homogeneity the quality of the algorithms is empirically evaluated on a large collection of brodatz like micro texture mondrians and on a set of real word images to demonstrate the broad usefulness of the theory of proximity based clustering the performances of different criteria and algorithms are compared on an information retrieval task for a document database the superiority of optimization algorithms for clustering is supported by extensive experiments

efficient robust predictive control predictive constrained control of time varying and or uncertain linear systems has been effected through the use of ellipsoidal invariant sets kothare et al 1996 linear matrix inequalities lmis have been used to design a state dependent state feedback law that maintains the state vector inside invariant feasible sets for the purposes of prediction however at each time instant the state feedback law is assumed constant in addition due to the large number of lmis involved online computation becomes intractable for anything other than small dimensional systems here we propose an approach that deploys a fixed state feedback law but introduces extra degrees of freedom through the use of perturbations on the fixed state feedback law the problem is so formulated that all demanding computations can be performed offline leaving only a simple optimization problem to be solved online over and above the very significant reduction in computational cost the extra degrees of freedom allow for better performance and wider applicability

modified fuzzy c means algorithm for cellular manufacturing several studies have used the fuzzy c means fcm algorithm for part machine grouping in cellular manufacturing however the application of the standard fcm algorithm to this problem has a number of drawbacks this paper proposes a modified fcm mfcm algorithm that groups components and machines in parallel and through an annealing process with the weighting exponent arrives at a crisp solution and an objective function value which can be interpreted in terms of the number of voids and intercellular movements of the part machine grouping obtained computational experiences show that although mfcm may sometimes require slightly more computing time than other methods not only is it able to find better solutions but also it has higher discriminating power for determining the number of cells

hierarchical unsupervised learning with growing via phase transitions we address unsupervised learning subject to structural constraints with particular emphasis placed on clustering with an imposed decision tree structure most known methods are greedy optimizing one node of the tree at a time to minimize a local cost by constrast we develop a joint optimization method derived based on information theoretic principles and closely related to known methods in statistical physics the approach is inspired by the deterministic annealing algorithm for unstructured data clustering which was based on maximum entropy inference the new approach is founded on the principle of minimum cross entropy using informative priors to approximate the unstructured clustering solution while imposing the structural constraint the resulting method incorporates supervised learning principles applied in an unsupervised problem setting in our approach the tree grows by a sequence of bifurcations that occur while optimizing an effective free energy cost at decreasing temperature scales thus estimates of the tree size and structure are naturally obtained at each temperature in the process examples demonstrate considerable improvement over known methods

deterministic annealing for entropy constrained vector quantizer design typical vector quantizer vq design algorithms such as generalized lloyd methods are susceptible to local minima in addition for entropy constrained vq it is not obvious what the right number of codevectors is for any given source and target rate one method that has shown promising results for solving both problems is deterministic annealing where randomness is added to the encoding process the uncertainty of which is gradually reduced until a deterministic coder is found this method has been used for fixed rate and entropy constrained unstructured vq as well as for a variety of structurally constrained vqs for the unstructured entropy constrained case we generalize the existing methods to include new cost functions and we argue that depending on the chosen cost function some splitting criteria are more suitable than others we give analytical advantages and disadvantages to several cost functions and several splitting criteria then give performance evaluations by testing on synthetic and natural sources

design of discrete coefficient fir filters by a fast entropy directed deterministic annealing algorithm we present an entropy directed deterministic annealing optimization algorithm and show its applicability to the problem of designing digital filters with discrete coefficients each implemented as a sum of signed power of two terms and additional general hardware constraints the algorithm is based on analogies from statistical mechanics and is related to the well known mean field annealing algorithm it utilizes estimates of conditional entropy to prune the problem during the optimization thereby reducing the computational time by 30 to 50 in conjunction with a scheme to compute the value of the objective function as a sequence of updates this approach leads to a very fast algorithm as an application example demonstrating the potential of the new method we consider the design of digital filters with discrete coefficients consisting of a minimum number of signed power of two terms

bayesian motion estimation and interpolation in interlaced video sequences motion estimation in video is a very important tool for such applications as interpolation motion compensation and segmentation however in interlaced video it is often desirable to perform interpolation prior to the motion estimation in this correspondence a novel technique is presented that rather than minimizing the interpolation error in each of the frames used for motion estimation seeks to make these errors equal the new method is shown to perform deinterlacing better than several other approaches also based on bayesian estimation