estimation of generalized multisensor hidden markov chains and unsupervised image segmentation this paper attacks the problem of generalized multisensor mixture estimation a distribution mixture is said to be generalized when the exact nature of components is not known but each of them belongs to a finite known set of families of distributions estimating such a mixture entails a supplementary difficulty one must label for each class and each sensor the exact nature of the corresponding distribution such generalized mixtures have been studied assuming that the components lie in the pearson system we propose a more general procedure with applications to estimating generalized multisensor hidden markov chains our proposed method is applied to the problem of unsupervised image segmentation the method proposed allows one to 1 identify the conditional distribution for each class and each sensor 2 estimate the unknown parameters in this distribution 3 estimate priors and 4 estimate the true class image

covariance regularization by thresholding this paper considers regularizing a covariance matrix of p variables estimated from n observations by hard thresholding we show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense the variables are gaussian or sub gaussian and log p n 0 and obtain explicit rates the results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity we discuss an intuitive resampling scheme for threshold selection and prove a general cross validation result that justifies this approach we also compare thresholding to other covariance estimators in simulations and on an example from climate data 1 introduction estimation of covariance matrices is important in a number of areas of statistical analysis including dimension reduction by principal component analysis pca classification by linear or quadratic discriminant analysis lda and qda establishing independence and conditional independence relations in the context of graphical models and setting confidence intervals on linear functions of the means of the components in recent years many application areas where these tools are used have been dealing with very high dimensional datasets and sample sizes can be very small relative to dimension examples include genetic data brain imaging spectroscopic imaging climate data and many others it is well known by now that the empirical covariance matrix for samples of size n from a p variate gaussian distribution np p is not a good estimator of the population covariance if p is large many results in random matrix theory illustrate this from the classical mary law 29 to the more recent work of johnstone and his students on the theory of the largest eigenvalues 12 23 30 and associated eigenvectors 24 however with the exception of a method for estimating the covariance spectrum 11 these probabilistic results do not offer alternatives to the sample covariance matrix alternative estimators for large covariance matrices have therefore attracted a lot of attention recently two broad classes of covariance estimators have emerged those that rely on a natural ordering among variables and assume that variables

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

time varying autoregressive moving average models for covariance estimation we consider large scale covariance estimation using a small number of samples in applications where there is a natural ordering between the random variables the two classical approaches to this problem rely on banded covariance and banded inverse covariance structures corresponding to time varying moving average ma and autoregressive ar models respectively motivated by this analogy to spectral estimation and the well known modeling power of autoregressive moving average arma processes we propose a novel time varying arma covariance structure similarly to known results in the context of ar and ma we address the completion of an arma covariance matrix from its main band and its estimation based on random samples finally we examine the advantages of our proposed methods using numerical experiments

diffusions for global optimizations we seek a global minimum of u 0 1 n to r the solution to d dt x t nabla u x t will find local minima the solution to dx t nabla u x t dt sqrt 2t dw t where w is standard n dimensional brownian motion and the boundaries are reflecting will concentrate near the global minima of u at least when temperature t is small the equilibrium distribution for x t is gibbs with density pi t x alpha exp u x t this suggests setting t t t downarrow 0 to find the global minima of u we give conditions on u x and t t such that the solution to dx t nabla u x t dt sqrt 2t dw t converges weakly to a distribution concentrated on the global minima of u

estimation of generalized mixtures and its application in image segmentation we introduce the notion of a generalized mixture and propose some methods for estimating it along with applications to unsupervised statistical image segmentation a distribution mixture is said to be generalized when the exact nature of the components is not known but each belongs to a finite known set of families of distributions for instance we can consider a mixture of three distributions each being exponential or gaussian the problem of estimating such a mixture contains thus a new difficulty we have to label each of three components there are eight possibilities we show that the classical mixture estimation algorithms expectation maximization em stochastic em sem and iterative conditional estimation ice can be adapted to such situations once as we dispose of a method of recognition of each component separately that is when we know that a sample proceeds from one family of the set considered we have a decision rule for what family it belongs to considering the pearson system which is a set of eight families the decision rule above is defined by the use of skewness and kurtosis the different algorithms so obtained are then applied to the problem of unsupervised bayesian image segmentation we propose the adaptive versions of sem em and ice in the case of blind i e pixel by pixel segmentation global segmentation methods require modeling by hidden random markov fields and we propose adaptations of two traditional parameter estimation algorithms gibbsian em gem and ice allowing the estimation of generalized mixtures corresponding to pearson s system the efficiency of different methods is compared via numerical studies and the results of unsupervised segmentation of three real radar images by different methods are presented

adaptive relaxation labeling current implementation of probabilistic relaxation labeling prl is based on stationary compatibility coefficients scc s such labeling frequently diverges from an achieved minimum labeling error in this correspondence it is shown that by having nonstationary compatibility coefficients nscc s the prl stabilizes about the minimum error which is obtained during the early iterations also a noniterative labeling algorithm which uses nscc and has a performance similar to that of the modified prl is suggested

properties of minimum mean squared error block quantizers corresp two results in minimum mean square error quantization theory are presented the first section gives a simplified derivation of a well known upper bound to the distortion introduced by a k dimensional optimum quantizer it is then shown that an optimum multidimensional quantizer preserves the mean vector of the input and that the mean square quantization error is given by the sum of the component variances of the input minus the sum of the variances of the output

confidence and conflict in multivariate calibration summary multivariate calibration involves using an estimated relationship between a multivariate response y and an explanatory vector x to predict unknown x in future from further observed responses in controlled calibration with multivariate normal errors the profile likelihood function for the unknown x denoted 4 displays a term which measures the mutual inconsistency of the given response vector denoted z in predicting 4 this inconsistency diagnostic fundamentally differentiates the behaviour of likelihood based and bayes confidence regions from those of the unconditional sampling approach in addition the diagnostic serves to pinpoint an inadequate response vector z

estimation of generalized mixture in the case of correlated sensors this paper deals with unsupervised bayesian classification of multidimensional data we propose an extension of a previous method of generalized mixture estimation to the correlated sensors case the method proposed is valid in the independent data case as well as in the hidden markov chain or field model case with known applications in signal processing particularly speech or image processing the efficiency of the method proposed is shown via some simulations concerning hidden markov fields with application to unsupervised image segmentation

a class of fast exact bayesian filters in dynamical models with jumps we address the statistical filtering problem in dynamical models with jumps when a particular application is adequately modeled by linear and gaussian probability density functions with jumps a usual method consists in approximating the optimal bayesian estimate in the sense of the minimum mean square error mmse in a linear and gaussian jump markov state space system jmss practical solutions include algorithms based on numerical approximations or on sequential monte carlo smc methods in this paper we propose a class of alternative methods which consists in building statistical models which locally similarly model the problem of interest but in which the computation of the mmse estimate can be be computed exactly without numerical nor smc approximations and at a computational cost which is linear in the number of observations

shrinkage to tapering estimation of large covariance matrices in this paper we introduce a shrinkage to tapering approach for estimating large covariance matrices when the number of samples is substantially fewer than the number of variables i e n p and p n the proposed estimator improves upon both shrinkage and tapering estimators by shrinking the sample covariance matrix to its tapered version we first show that under both normalized frobenius and spectral risks the minimum mean squared error mmse shrinkage to identity estimator is inconsistent and outperformed by a minimax tapering estimator for a class of high dimensional and diagonally dominant covariance matrices motivated by this observation we propose a shrinkage to tapering oracle sto estimator for efficient estimation of general large covariance matrices a closed form formula of the optimal coefficient of the proposed sto estimator is derived under the minimum frobenius risk since the true covariance matrix is to be estimated we further propose a sto approximating stoa algorithm with a data driven bandwidth selection procedure to iteratively estimate the coefficient and the covariance matrix we study the finite sample performances of different estimators and our simulation results clearly show the improved performances of the proposed sto estimators finally the proposed stoa method is applied to a real breast cancer gene expression data set

unsupervised restoration of hidden nonstationary markov chains using evidential priors this paper addresses the problem of unsupervised bayesian hidden markov chain restoration when the hidden chain is stationary the classical hidden markov chain hmc model is quite efficient and associated unsupervised bayesian restoration methods using the expectation maximization em algorithm work well when the hidden chain is non stationary on the other hand the unsupervised restoration results using the hmc model can be poor due to a bad match between the real and estimated models the novelty of this paper is to offer a more appropriate model for hidden nonstationary markov chains via the theory of evidence using recent results relating to triplet markov chains tmcs we show via simulations that the classical restoration results can be improved by the use of the theory of evidence and dempster shafer fusion the latter improvement is performed in an entirely unsupervised way using an original parameter estimation method some application examples to unsupervised image segmentation are also provided

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

on the asymptotic distribution of the errors in vector quantization in a recent paper lee and neuhoff see ibid vol 42 no 2 p446 60 1996 found an asymptotic formula for the distribution of the length of the errors produced by a vector quantizer with many quantization points this distribution depends on the source probability density the quantizer point density and the quantizer shape profile the latter characterizes the shapes of the quantization cells as a function of position the purpose of this paper is to give a rigorous derivation of this formula by identifying precise conditions under which it is shown that if a sequence of vector quantizers with a given dimension and an increasing number of points has specific point densities and specific shape profiles converging to a model point density and a model shape profile respectively then the distribution of the length of the quantization error suitably normalized converges to the aforementioned formula with the model point density and the model shape profile substituted

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

comments on application of the conditional population mixture model to image segmentation in the above correspondence1 a maximum likelihood method is proposed for estimating class memberships and underlying statistical parameters within the context of distribution mixtures in the present comment it is pointed out that biases are incurred in parameter estimation that the class memberships and parameters are conceptually different and therefore that the so called standard mixture likelihood is to be preferred also in the correspondence 1 akaike s information criterion aic is used to choose the number of classes in the mixture here a brief theoretical caveat is issued

covariance matrix estimation and classification with limited training data a new covariance matrix estimator useful for designing classifiers with limited training data is developed in experiments this estimator achieved higher classification accuracy than the sample covariance matrix and common covariance matrix estimates in about half of the experiments it achieved higher accuracy than regularized discriminant analysis but required much less computation

regularized estimation of large covariance matrices this paper considers estimating a covariance matrix of p variables from n observations by either banding the sample covariance matrix or estimating a banded version of the inverse of the covariance we show that these estimates are consistent in the operator norm as long as logp 2 n 0 and obtain explicit rates the results are uniform over some fairly natural well conditioned families of covariance matrices we also introduce an analogue of the gaussian white noise model and show that if the population covariance is embeddable in that model and well conditioned then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix the results can be extended to smooth versions of banding and to non gaussian distributions with su ciently short tails a resampling approach is proposed for choosing the banding parameter in practice this approach is illustrated numerically on both simulated and real data

maximum likelihood reconstruction for emission tomography previous models for emission tomography et do not distinguish the physics of et from that of transmission tomography we give a more accurate general mathematical model for et where an unknown emission density x y z generates and is to be reconstructed from the number of counts n d in each of d detector units d within the model we give an algorithm for determining an estimate of which maximizes the probability p n of observing the actual detector count data n over all possible densities let independent poisson variables n b with unknown means b b 1 b represent the number of unobserved emissions in each of b boxes pixels partitioning an object containing an emitter suppose each emission in box b is detected in detector unit d with probability p b d d 1 d with p b d a one step transition matrix assumed known we observe the total number n n d of emissions in each detector unit d and want to estimate the unknown b b 1 b for each the observed data n has probability or likelihood p n the em algorithm of mathematical statistics starts with an initial estimate 0 and gives the following simple iterative procedure for obtaining a new estimate new from an old estimate old to obtain k k 1 2 new b old b dd 1 n d p b d old b p b d b 1 b

bandwidth selection for local linear regression smoothers summary the paper presents a general strategy for selecting the bandwidth of nonparametric regression estimators and specializes it to local linear regression smoothers the procedure requires the sample to be divided into a training sample and a testing sample using the training sample we first compute a family of regression smoothers indexed by their bandwidths next we select the bandwidth by minimizing the empirical quadratic prediction error on the testing sample the resulting bandwidth satisfies a finite sample oracle inequality which holds for all bounded regression functions this permits asymptotically optimal estimation for nearly any regression function the practical performance of the method is illustrated by a simulation study which shows good finite sample behaviour of our method compared with other bandwidth selection procedures

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

making use of partial knowledge about hidden states in hmms an approach based on belief functions this paper addresses the problem of parameter estimation and state prediction in hidden markov models hmms based on observed outputs and partial knowledge of hidden states expressed in the belief function framework the usual hmm model is recovered when the belief functions are vacuous parameters are learned using the evidential expectation maximization algorithm a recently introduced variant of the expectation maximization algorithm for maximum likelihood estimation based on uncertain data the inference problem i e finding the most probable sequence of states based on observed outputs and partial knowledge of states is also addressed experimental results demonstrate that partial information about hidden states when available may substantially improve the estimation and prediction performances

pairwise markov chains we propose a model called a pairwise markov chain pmc which generalizes the classical hidden markov chain hmc model the generalization which allows one to model more complex situations in particular implies that in pmc the hidden process is not necessarily a markov process however pmc allows one to use the classical bayesian restoration methods like maximum a posteriori map or maximal posterior mode mpm so akin to hmc pmc allows one to restore hidden stochastic processes with numerous applications to signal and image processing such as speech recognition image segmentation and symbol detection or classification among others furthermore we propose an original method of parameter estimation which generalizes the classical iterative conditional estimation ice valid for a classical hidden markov chain model and whose extension to possibly non gaussian and correlated noise is briefly treated some preliminary experiments validate the interest of the new model

bayesian multi object filtering for pairwise markov chains random finite sets rfs are recent tools for addressing the multi object filtering problem the probability hypothesis density phd filter is an approximation of the multi object bayesian filter which results from the rfs formulation of the problem and has been used in many applications in the rfs framework it is assumed that each target and associated observation follow a hidden markov chain hmc model hmcs conveniently describe some physical properties of practical interest for practitioners but they also implicitly imply restrictive independence properties which in practice may not be satisfied by data in this paper we show that these structural limitations of hmc models can somehow be relaxed by embedding them into the more general class of pairwise markov chain pmc models we thus focus on the computation of the phd filter in a pmc framework and we propose a practical implementation of the phd filter for a particular class of pmc models

bayes smoothing algorithms for segmentation of binary images modeled by markov random fields a new image segmentation algorithm is presented based on recursive bayes smoothing of images modeled by markov random fields and corrupted by independent additive noise the bayes smoothing algorithm yields the a posteriori distribution of the scene value at each pixel given the total noisy image in a recursive way the a posteriori distribution together with a criterion of optimality then determine a bayes estimate of the scene the algorithm presented is an extension of a 1 d bayes smoothing algorithm to 2 d and it gives the optimum bayes estimate for the scene value at each pixel computational concerns in 2 d however necessitate certain simplifying assumptions on the model and approximations on the implementation of the algorithm in particular the scene noiseless image is modeled as a markov mesh random field a special class of markov random fields and the bayes smoothing algorithm is applied on overlapping strips horizontal vertical of the image consisting of several rows columns it is assumed that the signal scene values vector sequence along the strip is a vector markov chain since signal correlation in one of the dimensions is not fully used along the edges of the strip estimates are generated only along the middle sections of the strips the overlapping strips are chosen such that the union of the middle sections of the strips gives the whole image the bayes smoothing algorithm presented here is valid for scene random fields consisting of multilevel discrete or continuous random variables

an equivalence of the em and ice algorithm for exponential family we compare the expectation maximization em algorithm with another iterative approach namely the iterative conditional estimation ice algorithm which was formally introduced in the field of statistical segmentation of images we show that in case the probability density function pdf belongs to the exponential family the em algorithm is one particular case of the ice algorithm

joint parameter estimation and symbol detection for linear or nonlinear unknown channels we present an iterative method for joint channel parameter estimation and symbol selection via the baum welch algorithm or equivalently the expectation maximization em algorithm channel parameters including noise variance are estimated using a maximum likelihood criterion the markovian properties of the channel state sequence enable us to calculate the required likelihood using a forward backward algorithm the calculated likelihood functions can easily give optimum decisions on information symbols which minimize the symbol error probability the proposed receiver can be used for both linear and nonlinear channels it improves the system throughput by making saving in the transmission of known symbols usually employed for channel identification simulation results which show fast convergence are presented

dempster shafer fusion of evidential pairwise markov chains hidden markov models have been extended in many directions leading to pairwise markov models triplet markov models or discriminative random fields all of which have been successfully applied in many fields covering signal and image processing the dempster shafer theory of evidence has also shown its interest in a wide range of situations involving reasoning under uncertainty and or information fusion there are however only few works dealing with both of these modeling tools simultaneously the aim of this paper which falls under this category of works is to propose a general evidential markov model offering wide modeling and processing possibilities regarding information imprecision sensor unreliability and data fusion the main interest of the proposed model relies in the possibility of achieving easily the dempster shafer fusion without destroying the markovianity

signal and image segmentation using pairwise markov chains the aim of this paper is to apply the recent pairwise markov chain model which generalizes the hidden markov chain one to the unsupervised restoration of hidden data the main novelty is an original parameter estimation method that is valid in a general setting where the form of the possibly correlated noise is not known several experimental results are presented in both gaussian and generalized mixture contexts they show the advantages of the pairwise markov chain model with respect to the classical hidden markov chain one for supervised and unsupervised restorations

the sparse matrix transform for covariance estimation and analysis of high dimensional signals covariance estimation for high dimensional signals is a classically difficult problem in statistical signal analysis and machine learning in this paper we propose a maximum likelihood ml approach to covariance estimation which employs a novel non linear sparsity constraint more specifically the covariance is constrained to have an eigen decomposition which can be represented as a sparse matrix transform smt the smt is formed by a product of pairwise coordinate rotations known as givens rotations using this framework the covariance can be efficiently estimated using greedy optimization of the log likelihood function and the number of givens rotations can be efficiently computed using a cross validation procedure the resulting estimator is generally positive definite and well conditioned even when the sample size is limited experiments on a combination of simulated data standard hyperspectral data and face image sets show that the smt based covariance estimates are consistently more accurate than both traditional shrinkage estimates and recently proposed graphical lasso estimates for a variety of different classes and sample sizes an important property of the new covariance estimate is that it naturally yields a fast implementation of the estimated eigen transformation using the smt representation in fact the smt can be viewed as a generalization of the classical fast fourier transform fft in that it uses butterflies to represent an orthonormal transform however unlike the fft the smt can be used for fast eigen signal analysis of general non stationary signals

hyperspectral image segmentation using a new spectral unmixing based binary partition tree representation the binary partition tree bpt is a hierarchical region based representation of an image in a tree structure the bpt allows users to explore the image at different segmentation scales often the tree is pruned to get a more compact representation and so the remaining nodes conform an optimal partition for some given task here we propose a novel bpt construction approach and pruning strategy for hyperspectral images based on spectral unmixing concepts linear spectral unmixing consists of finding the spectral signatures of the materials present in the image endmembers and their fractional abundances within each pixel the proposed methodology exploits the local unmixing of the regions to find the partition achieving a global minimum reconstruction error results are presented on real hyperspectral data sets with different contexts and resolutions

bayesian estimation of karhunen loeve expansions a random subspace approach one of the most widely used procedures for dimensionality reduction of high dimensional data is principal component analysis pca more broadly low dimensional stochastic representation of random fields with finite variance is provided via the well known karhunen loeve expansion kle the kle is analogous to a fourier series expansion for a random process where the goal is to find an orthogonal transformation for the data such that the projection of the data onto this orthogonal subspace is optimal in the l 2 sense i e which minimizes the mean square error in practice this orthogonal transformation is determined by performing an svd singular value decomposition on the sample covariance matrix or on the data matrix itself sampling error is typically ignored when quantifying the principal components or equivalently basis functions of the kle furthermore it is exacerbated when the sample size is much smaller than the dimension of the random field in this paper we introduce a bayesian kle procedure allowing one to obtain a probabilistic model on the principal components which can account for inaccuracies due to limited sample size the probabilistic model is built via bayesian inference from which the posterior becomes the matrix bingham density over the space of orthonormal matrices we use a modified gibbs sampling procedure to sample on this space and then build probabilistic karhunen loeve expansions over random subspaces to obtain a set of low dimensional surrogates of the stochastic process we illustrate this probabilistic procedure with a finite dimensional stochastic process inspired by brownian motion

on the foundations of relaxation labeling processes a large class of problems can be formulated in terms of the assignment of labels to objects frequently processes are needed which reduce ambiguity and noise and select the best label among several possible choices relaxation labeling processes are just such a class of algorithms they are based on the parallel use of local constraints between labels this paper develops a theory to characterize the goal of relaxation labeling the theory is founded on a definition of con sistency in labelings extending the notion of constraint satisfaction in certain restricted circumstances an explicit functional exists that can be maximized to guide the search for consistent labelings this functional is used to derive a new relaxation labeling operator when the restrictions are not satisfied the theory relies on variational cal culus it is shown that the problem of finding consistent labelings is equivalent to solving a variational inequality a procedure nearly identical to the relaxation operator derived under restricted circum stances serves in the more general setting further a local convergence result is established for this operator the standard relaxation labeling formulas are shown to approximate our new operator which leads us to conjecture that successful applications of the standard methods are explainable by the theory developed here observations about con vergence and generalizations to higher order compatibility relations are described

asymptotic distribution of the errors in scalar and vector quantizers high rate or asymptotic quantization theory has found formulas for the average squared length more generally the qth moment of the length of the error produced by various scalar and vector quantizers with many quantization points in contrast this paper finds an asymptotic formula for the probability density of the length of the error and in certain special cases for the probability density of the multidimensional error vector itself the latter can be used to analyze the distortion of two stage vector quantization the former permits one to learn about the point density and cell shapes of a quantizer from a histogram of quantization error lengths histograms of the error lengths in simulations agree well with the derived formulas also presented are a number of properties of the error density including the relationship between the error density the point density and the cell shapes the fact that its qth moment equals bennett s integral a formula for the average distortion of a scalar or vector quantizer and the fact that for stationary sources the marginals of the multidimensional error density of an optimal vector quantizer with large dimension are approximately i i d gaussian

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

discrete markov image modeling and inference on the quadtree noncasual markov or energy based models are widely used in early vision applications for the representation of images in high dimensional inverse problems due to their noncausal nature these models generally lead to iterative inference algorithms that are computationally demanding in this paper we consider a special class of nonlinear markov models which allow one to circumvent this drawback these models are defined as discrete markov random fields mrf attached to the nodes of a quadtree the quadtree induces causality properties which enable the design of exact noniterative inference algorithms similar to those used in the context of markov chain models we first introduce an extension of the viterbi algorithm which enables exact maximum a posteriori map estimation on the quadtree two other algorithms related to the mpm criterion and to bouman and shapiro s 1994 sequential map smap estimator are derived on the same hierarchical structure the estimation of the model hyper parameters is also addressed two expectation maximization em type algorithms allowing unsupervised inference with these models are defined the practical relevance of the different models and inference algorithms is investigated in the context of image classification problem on both synthetic and natural images

shrinkage estimators for covariance matrices estimation of covariance matrices in small samples has been studied by many authors standard estimators like the unstructured maximum likelihood estimator ml or restricted maximum likelihood reml estimator can be very unstable with the smallest estimated eigenvalues being too small and the largest too big a standard approach to more stably estimating the matrix in small samples is to compute the ml or reml estimator under some simple structure that involves estimation of fewer parameters such as compound symmetry or independence however these estimators will not be consistent unless the hypothesized structure is correct if interest focuses on estimation of regression coefficients with correlated or longitudinal data a sandwich estimator of the covariance matrix may be used to provide standard errors for the estimated coefficients that are robust in the sense that they remain consistent under misspecification of the covariance structure with large matrices however the inefficiency of the sandwich estimator becomes worrisome we consider here two general shrinkage approaches to estimating the covariance matrix and regression coefficients the first involves shrinking the eigenvalues of the unstructured ml or reml estimator the second involves shrinking an unstructured estimator toward a structured estimator for both cases the data determine the amount of shrinkage these estimators are consistent and give consistent and asymptotically efficient estimates for regression coefficients simulations show the improved operating characteristics of the shrinkage estimators of the covariance matrix and the regression coefficients in finite samples the final estimator chosen includes a combination of both shrinkage approaches i e shrinking the eigenvalues and then shrinking toward structure we illustrate our approach on a sleep eeg study that requires estimation of a 24 x 24 covariance matrix and for which inferences on mean parameters critically depend on the covariance estimator chosen we recommend making inference using a particular shrinkage estimator that provides a reasonable compromise between structured and unstructured estimators

markov random field texture models we consider a texture to be a stochastic possibly periodic two dimensional image field a texture model is a mathematical procedure capable of producing and describing a textured image we explore the use of markov random fields as texture models the binomial model where each point in the texture has a binomial distribution with parameter controlled by its neighbors and number of tries equal to the number of gray levels was taken to be the basic model for the analysis a method of generating samples from the binomial model is given followed by a theoretical and practical analysis of the method s convergence examples show how the parameters of the markov random field control the strength and direction of the clustering in the image the power of the binomial model to produce blurry sharp line like and blob like textures is demonstrated natural texture samples were digitized and their parameters were estimated under the markov random field model a hypothesis test was used for an objective assessment of goodness of fit under the markov random field model overall microtextures fit the model well the estimated parameters of the natural textures were used as input to the generation procedure the synthetic microtextures closely resembled their real counterparts while the regular and inhomogeneous textures did not

multiscale bayesian restoration in pairwise markov trees an important problem in multiresolution analysis of signals and images consists in estimating continuous hidden random variables x x sub s sub s spl isin s from observed ones y y sub s sub s spl isin s this is done classically in the context of hidden markov trees hmts in this note we deal with the recently introduced pairwise markov trees pmts we first show that pmts are more general than hmts we then deal with the linear gaussian case and we extend from hmts with independent noise hmt in to pmt a smoothing kalman like recursive estimation algorithm which was proposed by chou et al as well as an algorithm for computing the likelihood

maximum likelihood discriminant analysis on the plane using a markovian model of spatial context abstract a method is presented for utilising spatial information in performing discriminant analysis on multivariate data at each point on a regular lattice as for example with landsat the spatial information is modelled as a particularly simple markov random field the discriminant analysis is based on the maximum likelihood method the results are compared with alternatives by monte carlo simulation and seem to be encouraging

bennett s integral for vector quantizers this paper extends bennett s 1948 integral from scalar to vector quantizers giving a simple formula that expresses the rth power distortion of a many point vector quantizer in terms of the number of points point density function inertial profile and the distribution of the source the inertial profile specifies the normalized moment of inertia of quantization cells as a function of location the extension is formulated in terms of a sequence of quantizers whose point density and inertial profile approach known functions as the number of points increase precise conditions are given for the convergence of distortion suitably normalized to bennett s integral previous extensions did not include the inertial profile and consequently provided only bounds or applied only to quantizers with congruent cells such as lattice and optimal quantizers the new version of bennett s integral provides a framework for the analysis of suboptimal structured vector quantizers it is shown how the loss in performance of such quantizers relative to optimal unstructured ones can be decomposed into point density and cell shape losses as examples these losses are computed for product quantizers and used to gain further understanding of the performance of scalar quantizers applied to stationary memoryless sources and of transform codes applied to gaussian sources with memory it is shown that the short coming of such quantizers is that they must compromise between point density and cell shapes

interval estimation in discrimination the multivariate normal equal covariance case summary a general approach to discrimination problems is described which emphasizes the viewpoint of individual cases and expresses the uncertainty with which individual decisions are made it is applied to the particular case of two multivariate normal distributions with a common covariance matrix an affine transformation to which the key quantities are invariant simplifies discussion of the distribution of the estimated log odds ratio and suggests a practically useful plot which generalizes the usual display of discriminant scores and has several attractive features an illustration is given two approximate interval estimation techniques for the true log odds ratio are described and evaluated in a range of practical situations by a simulation study some further applications of our general approach are briefly indicated

on the covariance of two noncentral f random variables and the variance of the estimated liner discriminant function summary the covariance of two noncentral f random variables with a common covariance matrix is obtained from which the variance of their difference follows easily an application of this latter result in discriminant analysis is then given

two results on the asymptotic performance of quantizers a necessary and sufficient condition is presented for the normalized asymptotic r th power distortion of a mismatched multidimensional quantizer to converge to a certain optimum constant known as bennett s integral the dimensionality of our results is allowed to approach infinity in order to make some universal source coding comparisons to quantization theory

analysis of field experiments by least squares smoothing summary assuming a smooth trend plus independent error model for the environmental effects in the yields of a field plot experiment least squares smoothing methods are developed to estimate both the treatment effects and the unknown trend treatment estimates are closely related to those resulting from a generalized least squares analysis in which the covariance structure for the environmental effects has a particular form however the main emphases are on the accuracy of treatment estimates under a fixed smooth trend plus error model and the exploratory power of the basic method to isolate trend effects of unknown form although the detailed development is for the one dimensional case generalizations of the smoothness concept and extensions to two dimensions are also discussed application of the basic method is illustrated on three data sets and the results compared with other analyses

inference and learning in evidential discrete latent markov models we present the evidential hidden markov models evhmm an extension of probabilistic hmm for time series modeling where conditional belief functions are used in place of probabilities to manage uncertainty on discrete latent variables inference and learning mechanisms are described and allow to solve the three problems initially defined for hmm namely the classification problem find the most plausible model the decoding problem finding the best sequence of hidden states and the learning problem based on incomplete and uncertain data estimate the parameters exact inference mechanisms based on the gbt are proposed which allows one to recover standard hmm when probabilities are considered an em like procedure is developed for parameter learning relying on some approximations suggested to make the solutions tractable relationships are discussed with both the learning criterion conjectured by vannoorenberghe and smets and the formulation of emcs by pieczynski et al a comparison with standard hmm on simulated data confirms the interest of considering random disjunctive sets to represent data incompleteness in evidential temporal graphical models

a simple class of asymptotically optimal quantizers a simple class of quantizers is introduced which are asymptotically optimal as the number of quantization levels increases to infinity with respect to a mean r th power absolute error distortion measure these asymptotically optimal quantizers are very easy to compute their performance is evaluated for several distributions and compares favorably with the performance of the optimal quantizers in all cases for which the latter have been computed in addition their asymptotic robustness is studied under location scale and shape mismatch for several families of distributions

causal recursive parameter estimation for discrete time hidden bivariate markov chains an algorithm for causal recursive parameter estimation of a discrete time hidden bivariate markov chain is developed in this model a discrete time bivariate markov chain is observed through a discrete time memoryless channel the algorithm relies on the em based recursive approach developed by stiller and radons for hidden markov models a distinct advantage of the discrete time hidden bivariate markov chain model is that the sojourn time distribution of its observable process in each state is phase type rather than geometric as in the hidden markov model phase type distributions can approximate any desired sojourn time distribution particular phase type distributions include mixtures and convolutions of geometric distributions the parameter estimation algorithm requires causal recursive estimation of the relevant statistics in each em step these statistics include the number of jumps of the bivariate markov chain from one state to another including self transitions in the given observation interval and first and second order statistical averages of the observable process in each state when the memoryless channel is gaussian we develop the explicit recursions and demonstrate the performance of the algorithm in estimating the model s parameter and its sojourn time distribution in a numerical example

an analytic solution for estimating two dimensional hidden markov models two dimensional hidden markov model 2 d hmm is an extension of 1 d hmm to 2 d it provides a reasonable statistical method to model matrix data this paper presents some new strict definitions of 2 d hmm and proves the equivalence between them and gives a study of the three basic problems for 2 d hmm namely probability evaluation optimal state matrix and parameter estimation by using the ideal that the sequences of states on columns or rows of a 2 d hmm can be seen as states of a 1 d hmm several new formulae solving these problems are theoretically derived and further demonstrated by computer simulations

applications of hidden markov chains in image analysis in image analysis two dimensional markov models i e markov field models have been applied for segmentation purposes but except for the area of text recognition the application of hidden markov chains has been rare through four very different examples this paper demonstrates the applicability also for hidden markov chains in image analysis and shows that the problems of image analysis often may have one dimensional characteristics even though the images are two dimensional

generalized thresholding of large covariance matrices we propose a new class of generalized thresholding operators that combine thresholding with shrinkage and study generalized thresholding of the sample covariance matrix in high dimensions generalized thresholding of the covariance matrix has good theoretical properties and carries almost no computational burden we obtain an explicit convergence rate in the operator norm that shows the tradeoff between the sparsity of the true model dimension and the sample size and shows that generalized thresholding is consistent over a large class of models as long as the dimension p and the sample size n satisfy log p n 0 in addition we show that generalized thresholding has the sparsistency property meaning it estimates true zeros as zeros with probability tending to 1 and under an additional mild condition is sign consistent for nonzero elements we show that generalized thresholding covers as special cases hard and soft thresholding smoothly clipped absolute deviation and adaptive lasso and comp

cs decomposition based bayesian subspace estimation in numerous applications it is required to estimate the principal subspace of the data possibly from a very limited number of samples additionally it often occurs that some rough knowledge about this subspace is available and could be used to improve subspace estimation accuracy in this case this is the problem we address herein and in order to solve it a bayesian approach is proposed the main idea consists of using the cs decomposition of the semi orthogonal matrix whose columns span the subspace of interest this parametrization is intuitively appealing and allows for non informative prior distributions of the matrices involved in the cs decomposition and very mild assumptions about the angles between the actual subspace and the prior subspace the posterior distributions are derived and a gibbs sampling scheme is presented to obtain the minimum mean square distance estimator of the subspace of interest numerical simulations and an application to real hyperspectral data assess the validity and the performances of the estimator

a statistical model for positron emission tomography abstract positron emission tomography pet still in its research stages is a technique that promises to open new medical frontiers by enabling physicians to study the metabolic activity of the body in a pictorial manner much as in x ray transmission tomography and other modes of computerized tomography the quality of the reconstructed image in pet is very sensitive to the mathematical algorithm to be used for reconstruction in this article we tailor a mathematical model to the physics of positron emissions and we use the model to describe the basic image reconstruction problem of pet as a standard problem in statistical estimation from incomplete data we describe various estimation procedures such as the maximum likelihood ml method using the em algorithm the method of moments and the least squares method a computer simulation of a pet experiment is then used to demonstrate the ml and the least squares reconstructions the main purposes of this article are to report on what we believe is an

graphical and recursive models for contingency tables summary we discuss two classes of models for contingency tables graphical and recursive models both of which arise from restrictions that are expressible as conditional independencies of variable pairs the first of these is a subclass of hierarchical log linear models each of its models can be represented by an undirected graph in the second class each model corresponds to a particular kind of a directed graph instead and can be characterized by a nontrivial factorization of the joint distribution in terms of response variables we derive decomposable or multiplicative models as the intersecting class this result has useful consequences for exploratory types of analysis as well as for the model interpretation we can give an aid for detecting well fitting decomposable models in a transformation of the observed contingency table and each decomposable model may be interpreted with the help of an undirected or directed graph

modeling and unsupervised classification of multivariate hidden markov chains with copulas parametric modeling and estimation of non gaussian multidimensional probability density function is a difficult problem whose solution is required by many applications in signal and image processing a lot of efforts have been devoted to escape the usual gaussian assumption by developing perturbed gaussian models such as spherically invariant random vectors sirvs in this work we introduce an alternative solution based on copulas that enables theoretically to represent any multivariate distribution estimation procedures are proposed for some mixtures of copula based densities and are compared in the hidden markov chain setting in order to perform statistical unsupervised classification of signals or images useful copulas and sirv for multivariate signal classification are particularly studied through experiments

vector quantizers and predictive quantizers for gauss markov sources low rate vector quantizers are designed and simulated for highly correlated gauss markov sources and the resulting performance is compared with arnstein s optimized predictive quantizer and with huang and schultheiss optimized transform coder two implementations of vector quantizers are considered full search vector quantizers which are optimal but require large codebook searches and tree searched vector quantizers which are suboptimal but require far less searching the various systems are compared on the basis of performance complexity and generality of design techniques

noniterative map reconstruction using sparse matrix representations we present a method for noniterative maximum a posteriori map tomographic reconstruction which is based on the use of sparse matrix representations our approach is to precompute and store the inverse matrix required for map reconstruction this approach has generally not been used in the past because the inverse matrix is typically large and fully populated i e not sparse in order to overcome this problem we introduce two new ideas the first idea is a novel theory for the lossy source coding of matrix transformations which we refer to as matrix source coding this theory is based on a distortion metric that reflects the distortions produced in the final matrix vector product rather than the distortions in the coded matrix itself the resulting algorithms are shown to require orthonormal transformations of both the measurement data and the matrix rows and columns before quantization and coding the second idea is a method for efficiently storing and computing the required orthonormal transformations which we call a sparse matrix transform smt the smt is a generalization of the classical fft in that it uses butterflies to compute an orthonormal transform but unlike an fft the smt uses the butterflies in an irregular pattern and is numerically designed to best approximate the desired transforms we demonstrate the potential of the noniterative map reconstruction with examples from optical tomography the method requires offline computation to encode the inverse transform however once these offline computations are completed the noniterative map algorithm is shown to reduce both storage and computation by well over two orders of magnitude as compared to a linear iterative reconstruction methods