the em algorithm an old folk song sung to a fast new tune celebrating the 20th anniversary of the presentation of the paper by dempster laird and rubin which popularized the em algorithm we investigate after a brief historical account strategies that aim to make the em algorithm converge faster while maintaining its simplicity and stability e g automatic monotone convergence in likelihood first we introduce the idea of a working parameter to facilitate the search for efficient data augmentation schemes and thus fast em implementations second summarizing various recent extensions of the em algorithm we formulate a general alternating expectation conditional maximization algorithm aecm that couples flexible data augmentation schemes with model reduction schemes to achieve efficient computations we illustrate these methods using multivariate t models with known or unknown degrees of freedom and poisson models for image reconstruction we show through both empirical and theoretical evidence the potential for a dramatic reduction in computational time with little increase in human effort we also discuss the intrinsic connection between em type algorithms and the gibbs sampler and the possibility of using the techniques presented here to speed up the latter the main conclusion of the paper is that with the help of statistical considerations it is possible to construct algorithms that are simple stable and fast

tree approximations to markov random fields methods for approximately computing the marginal probability mass functions and means of a markov random field mrf by approximating the lattice by a tree are described applied to the a posteriori mrf these methods solve bayesian spatial pattern classification and image restoration problems the methods are described several theoretical results concerning fixed point problems are proven and four numerical examples are presented including comparison with optimal estimators and the iterated conditional mode estimator and including two agricultural optical remote sensing problems

document ink bleed through removal with two hidden markov random fields and a single observation field we present a new method for blind document bleed through removal based on separate markov random field mrf regularization for the recto and for the verso side where separate priors are derived from the full graph the segmentation algorithm is based on bayesian maximum a posteriori map estimation the advantages of this separate approach are the adaptation of the prior to the contents creation process e g superimposing two handwritten pages and the improvement of the estimation of the recto pixels through an estimation of the verso pixels covered by recto pixels moreover the formulation as a binary labeling problem with two hidden labels per pixels naturally leads to an efficient optimization method based on the minimum cut maximum flow in a graph the proposed method is evaluated on scanned document images from the 18th century showing an improvement of character recognition results compared to other restoration methods

fast training of recurrent networks based on the em algorithm in this work a probabilistic model is established for recurrent networks the expectation maximization em algorithm is then applied to derive a new fast training algorithm for recurrent networks through mean field approximation this new algorithm converts training a complicated recurrent network into training an array of individual feedforward neurons these neurons are then trained via a linear weighted regression algorithm the training time has been improved by five to 15 times on benchmark problems

simultaneous parameter estimation and segmentation of gibbs random fields using simulated annealing an adaptive segmentation algorithm is developed which simultaneously estimates the parameters of the underlying gibbs random field grf and segments the noisy image corrupted by additive independent gaussian noise the algorithm which aims at obtaining the maximum a posteriori map segmentation is a simulated annealing algorithm that is interrupted at regular intervals for estimating the grf parameters maximum likelihood ml estimates of the parameters based on the current segmentation are used to obtain the next segmentation it is proven that the parameter estimates and the segmentations converge in distribution to the ml estimate of the parameters and the map segmentation with those parameter estimates respectively due to computational difficulties however only an approximate version of the algorithm is implemented the approximate algorithm is applied on several two and four region images with different noise levels and with first order and second order neighborhoods

tensor splines for interpolation and approximation of dt mri with applications to segmentation of isolated rat hippocampi in this paper we present novel algorithms for statistically robust interpolation and approximation of diffusion tensors which are symmetric positive definite spd matrices and use them in developing a significant extension to an existing probabilistic algorithm for scalar field segmentation in order to segment diffusion tensor magnetic resonance imaging dt mri datasets using the riemannian metric on the space of spd matrices we present a novel and robust higher order cubic continuous tensor product of splines algorithm to approximate the spd diffusion tensor fields the resulting approximations are appropriately dubbed tensor splines next we segment the diffusion tensor field by jointly estimating the label assigned to each voxel field which is modeled by a gauss markov measure field gmmf and the parameters of each smooth tensor spline model representing the labeled regions results of interpolation approximation and segmentation are presented for synthetic data and real diffusion tensor fields from an isolated rat hippocampus along with validation we also present comparisons of our algorithms with existing methods and show significantly improved results in the presence of noise as well as outliers

gauss markov measure field models for low level vision we present a class of models derived from classical discrete markov random fields that may be used for the solution of ill posed problems in image processing and in computational vision they lead to reconstruction algorithms that are flexible computationally efficient and biologically plausible to illustrate their use we present their application to the reconstruction of the dominant orientation and direction fields to the classification of multiband images and to image quantization and filtering

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

the application of the gibbs bogoliubov feynman inequality in mean field calculations for markov random fields the gibbs bogoliubov feynman gbf inequality of statistical mechanics is adopted with an information theoretic interpretation as a general optimization framework for deriving and examining various mean field approximations for markov random fields mrf s the efficacy of this approach is demonstrated through the compound gauss markov cgm model comparisons between different mean field approximations and experimental results in image restoration

physical consideration of an image in image restoration using bayes formula abstract we consider image restoration by bayes formula and investigate the relationship between an image and a prior probability from the following two viewpoints hyperparameter estimation and the accuracy of a restored image the q ising model is adopted as a prior probability in bayes formula not the q ising energy but the potts energy plays an important role in the hyperparameter estimation from the viewpoint of the hyperparameter estimation the relationship between a natural image and a prior probability is characterized through the potts energy and magnetization of an image the potts energy and magnetization of an image are defined by a set of pixels state of an image the closer to the average potts energy and magnetization over a prior probability the potts energy and magnetization of a natural image is the closer to the true value of a hyperparameter the estimated value of a hyperparameter from a degraded image is for the accuracy of a restored image the image which has a smaller q ising energy is better restored by bayes formula composed of the q ising prior the consideration for the relationship between an image and a prior probability is expected to be valid for a more complicated prior probability

entropy controlled quadratic markov measure field models for efficient image segmentation we present a new markov random field mrf based model for parametric image segmentation instead of directly computing a label map our method computes the probability that the observed data at each pixel is generated by a particular intensity model prior information about segmentation smoothness and low entropy of the probability distribution maps is codified in the form of a mrf with quadratic potentials so that the optimal estimator is obtained by solving a quadratic cost function with linear constraints although for segmentation purposes the mode of the probability distribution at each pixel is naturally used as an optimal estimator our method permits the use of other estimators such as the mean or the median which may be more appropriate for certain applications numerical experiments and comparisons with other published schemes are performed using both synthetic images and real data of brain mri for which expert hand made segmentations are available finally we show that the proposed methodology may be easily extended to other problems such as stereo disparity estimation

an extension of the standard mixture model for image segmentation standard gaussian mixture modeling gmm is a well known method for image segmentation however the pixels themselves are considered independent of each other making the segmentation result sensitive to noise to reduce the sensitivity of the segmented result with respect to noise markov random field mrf models provide a powerful way to account for spatial dependences between image pixels however their main drawback is that they are computationally expensive to implement and require large numbers of parameters based on these considerations we propose an extension of the standard gmm for image segmentation which utilizes a novel approach to incorporate the spatial relationships between neighboring pixels into the standard gmm the proposed model is easy to implement and compared with mrf models requires lesser number of parameters we also propose a new method to estimate the model parameters in order to minimize the higher bound on the data negative log likelihood based on the gradient method experimental results obtained on noisy synthetic and real world grayscale images demonstrate the robustness accuracy and effectiveness of the proposed model in image segmentation as compared to other methods based on standard gmm and mrf models

the mean field theory in em procedures for markov random fields in many signal processing and pattern recognition applications the hidden data are modeled as markov processes and the main difficulty of using the maximisation em algorithm for these applications is the calculation of the conditional expectations of the hidden markov processes it is shown how the mean field theory from statistical mechanics can be used to calculate the conditional expectations for these problems efficiently the efficacy of the mean field theory approach is demonstrated on parameter estimation for one dimensional mixture data and two dimensional unsupervised stochastic model based image segmentation experimental results indicate that in the 1 d case the mean field theory approach provides results comparable to those obtained by baum s 1987 algorithm which is known to be optimal in the 2 d case where baum s algorithm can no longer be used the mean field theory provides good parameter estimates and image segmentation for both synthetic and real world images

adaptive segmentation of speckled images using a hierarchical random field model a two level hierarchical random field model developed for speckled images and in particular for synthetic aperture radar sar imagery is described at the higher level is a gibbs random field governing the grouping of image pixels into regions while at the lower level are speckle processes for the different regions which are also modeled as random fields in accordance with the physical phenomena that cause speckle the speckle processes are modeled as circularly symmetric complex gaussian random fields as with real gaussian fields certain forms of autocovariance of complex gaussian field lead to markovianity properties with the assumption of a separable autocovariance for the complex gaussian random fields local joint statistics of the resulting speckle intensity fields magnitude squared of the complex field are determined the speckle model concurs with the known marginal statistics and also accounts for the autocorrelation observed in actual speckled images a map segmentation using simulated annealing and based on the hierarchical model is presented the algorithm is adaptive in that it recursively segments the image and estimates the model parameters necessary for the segmentation

hyperparameter estimation in image restoration the hyperparameter in image restoration by the bayes formula is an important quantity this communication shows a physical method for the estimation of the hyperparameter without approximation for artificially generated images by prior probability the hyperparameter is computed accurately for practical images accuracy of the estimated hyperparameter depends on the magnetization and energy of the images we discuss the validity of prior probability for an original image

application of the em algorithm for the multitarget multisensor tracking problem an important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies a number of approaches have been proposed for this multitarget multisensor tracking problem ranging from reasonably simple though ad hoc schemes to fairly complex but theoretically optimum approaches in this paper we describe an iterative procedure for time recursive multitarget multisensor tracking based on use of the expectation maximization em algorithm more specifically we pose the multitarget multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data whereas the target associated sensor outputs constitute the complete data target updates at each time use an em based approach that calculates the maximum a posteriori map estimate of the target states under the assumption of appropriate motion models based on the outputs of disparate sensors the approach uses a markov random field mrf model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration the advantage of this em based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities we provide selected results illustrating the performance complexity characteristics of this em based approach compared with competing schemes

multiresolution gauss markov random field models for texture segmentation this paper presents multiresolution models for gauss markov random fields gmrfs with applications to texture segmentation coarser resolution sample fields are obtained by subsampling the sample field at fine resolution although the markov property is lost under such resolution transformation coarse resolution non markov random fields can be effectively approximated by markov fields we present two techniques to estimate the gmrf parameters at coarser resolutions from the fine resolution parameters one by minimizing the kullback leibler distance and another based on local conditional distribution invariance we also allude to the fact that different gmrf parameters at the fine resolution can result in the same probability measure after subsampling and present the results for first and second order cases we apply this multiresolution model to texture segmentation different texture regions in an image are modeled by gmrfs and the associated parameters are assumed to be known parameters at lower resolutions are estimated from the fine resolution parameters the coarsest resolution data is first segmented and the segmentation results are propagated upward to the finer resolution we use the iterated conditional mode icm minimization at all resolutions our experiments with synthetic brodatz texture and real satellite images show that the multiresolution technique results in a better segmentation and requires lesser computation than the single resolution algorithm

approximate maximum likelihood hyperparameter estimation for gibbs priors the parameters of the prior the hyperparameters play an important role in bayesian image estimation of particular importance for the case of gibbs priors is the global hyperparameter spl beta which multiplies the hamiltonian here we consider maximum likelihood ml estimation of spl beta from incomplete data i e problems in which the image which is drawn from a gibbs prior is observed indirectly through some degradation or blurring process important applications include image restoration and image reconstruction from projections exact ml estimation of spl beta from incomplete data is intractable for most image processing here we present an approximate ml estimator that is computed simultaneously with a maximum a posteriori map image estimate the algorithm is based on a mean field approximation technique through which multidimensional gibbs distributions are approximated by a separable function equal to a product of one dimensional 1 d densities we show how this approach can be used to simplify the ml estimation problem we also show how the gibbs bogoliubov feynman gbf bound can be used to optimize the approximation for a restricted class of problems we present the results of a monte carlo study that examines the bias and variance of this estimator when applied to image restoration

bayesian clustering for unsupervised estimation of surface and texture models a method of calculating the maximum likelihood clustering for the unsupervised estimation of polynomial models for the data in images of smooth surfaces or for range data for such surfaces is presented an image or a depth map of a region of smooth 3 d surface is modeled as a polynomial plus white noise a region of physically meaningful textured image such as the image of foliage grass or road in outdoor scenes or conductor or lintburn on a thick film substrate is modeled as a colored gaussian markov random field mrf with a polynomial mean value function unsupervised model parameter estimation is accomplished by determining the segmentation and model parameter values that maximize the likelihood of the data or a more general bayesian performance functional agglomerative clustering is used for this purpose

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

uniform strategies rational relations and jumping automata a general concept of uniform strategies has recently been proposed as a relevant notion in game theory for computer science which subsumes various notions from the literature it relies on properties involving sets of plays in two player turn based arenas equipped with arbitrary binary relations between plays these properties are expressed in a language based on ctl with a quantifier over related plays there are two semantics for our quantifier a strict one and a full one that we study separately regarding the strict semantics the existence of a uniform strategy is undecidable for rational binary relations but introducing jumping tree automata and restricting attention to recognizable relations allows us to establish a 2 exptime complete complexity and still capture a class of two player imperfect information games with epistemic temporal objectives regarding the full semantics relying on information set automata we establish that the existence of a uniform strategy is decidable for rational relations and we provide a nonelementary synthesis procedure we also exhibit an essentially optimal subclass of rational relations for which the problem becomes 2 exptime complete considering rich classes of relations makes the theory of uniform strategies powerful it directly entails various results in logics of knowledge and time some of them already known and others new

probabilistic image processing by means of the bethe approximation for the q ising model the framework of bayesian image restoration for multi valued images by means of the q ising model with nearest neighbour interactions is presented hyperparameters in the probabilistic model are determined so as to maximize the marginal likelihood a practical algorithm is described for multi valued image restoration based on the bethe approximation the algorithm corresponds to loopy belief propagation in artificial intelligence we conclude that in real world grey level images the q ising model can give us good results

mixture of regression models for large spatial datasets abstractwhen a spatial regression model that links a response variable to a set of explanatory variables is desired it is unlikely that the same regression model holds throughout the domain when t

bayesian decision feedback for segmentation of binary images we present real time algorithms for the segmentation of binary images modeled by markov mesh random fields mmrfs and corrupted by independent noise the goal is to find a recursive algorithm to compute the maximum a posteriori map estimate of each pixel of the scene using a fixed lookahead of d rows and d columns of the observations first this map fixed lag estimation problem is set up and the corresponding optimal recursive but computationally complex estimator is derived then both hard and soft conditional decision feedbacks are introduced at appropriate stages of the optimal estimator to reduce the complexity the algorithm is applied to several synthetic and real images the results demonstrate the viability of the algorithm both complexity wise and performance wise and show its subjective relevance to the image segmentation problem

a sequence based extension of mean field annealing using the forward backward algorithm application to image segmentation optimization based on mean field theory including mean field annealing mfa is widely used for discrete optimization label assignment problems defined on the pixel sites of an image one formulation of mfa is via maximum entropy where one seeks the joint distribution over the random assignments subject to an average level of cost mfa is obtained by assuming the assignments at each pixel are statistically independent given the observed image alternatively we make the less restrictive assumption of independent row labelings the independence assumption means that at each step mfa optimizes over only one pixel whereas our method jointly optimizes over an entire row i e our method is less greedy in principle an mfa extension could be developed that explicitly re estimates the row labeling distributions but such an approach is in practice infeasible even so we can indirectly implement this re estimation by re estimating quantities that determine the row labeling distributions these quantities are the a posteriori site probabilities re estimable via the well known forward backward f b algorithm thus our algorithm which descends in the me lagrangian free energy consists of iterative application of f b to the image rows columns at convergence maximum a posteriori site labeling is performed our method was applied to segmentation of both real and synthetic noise corrupted images it achieved lower markov random field model potentials and better segmentations compared with other methods and in high noise standard mfa

the application of mean field theory to image motion estimation previously markov random field mrf model based techniques have been proposed for image motion estimation since motion estimation is usually an ill posed problem various constraints are needed to obtain a unique and stable solution the main advantage of the mrf approach is its capacity to incorporate such constraints for instance motion continuity within an object and motion discontinuity at the boundaries between objects in the mrf approach motion estimation is often formulated as an optimization problem and two frequently used optimization methods are simulated annealing sa and iterative conditional mode icm although the sa is theoretically optimal in the sense of finding the global optimum it usually takes many iterations to converge the icm on the other hand converges quickly but its results are often unsatisfactory due to its hard decision nature previously the authors have applied the mean field theory to image segmentation and image restoration problems it provides results nearly as good as sa but with much faster convergence the present paper shows how the mean field theory can be applied to mrf model based motion estimation this approach is demonstrated on both synthetic and real world images where it produced good motion estimates

faster and simpler approximation algorithms for mixed packing and covering problems we propose an algorithm for approximately solving the mixed packing and covering problem given a convex compact set 0 b r n either compute x b such that f x 1 e a and g x 1 e b or decide that x b f x a g x b 0 here f g b r m are vectors whose components are m non negative convex and concave functions respectively and a b r m are constant positive vectors our algorithm requires an efficient feasibility oracle or block solver which given vectors c d r m and a r computes x b such that c tf x d tg x a or correctly decides that no such x b exists our algorithm which is based on the lagrangian or price directive decomposition method generalizes the result from k jansen approximation algorithm for the mixed fractional packing and covering problem in proceedings of 3rd ifip conference on theoretical computer science ifip tcs 2004 kluwer 2004 pp 223 236 siam journal on optimization 17 2006 331 352 and needs only o m lnm e 2ln e 1 iterations or calls to the feasibility oracle furthermore we show that a more general block solver can be used to obtain a more general approximation within the same runtime bound

ml parameter estimation for markov random fields with applications to bayesian tomography markov random fields mrfs have been widely used to model images in bayesian frameworks for image reconstruction and restoration typically these mrf models have parameters that allow the prior model to be adjusted for best performance however optimal estimation of these parameters sometimes referred to as hyperparameters is difficult in practice for two reasons i direct parameter estimation for mrfs is known to be mathematically and numerically challenging ii parameters can not be directly estimated because the true image cross section is unavailable we propose a computationally efficient scheme to address both these difficulties for a general class of mrf models and we derive specific methods of parameter estimation for the mrf model known as generalized gaussian mrf ggmrf we derive methods of direct estimation of scale and shape parameters for a general continuously valued mrf for the ggmrf case we show that the ml estimate of the scale parameter spl sigma has a simple closed form solution and we present an efficient scheme for computing the ml estimate of the shape parameter p by an off line numerical computation of the dependence of the partition function on p we present a fast algorithm for computing ml parameter estimates when the true image is unavailable to do this we use the expectation maximization em algorithm we develop a fast simulation method to replace the e step and a method to improve the parameter estimates when the simulations are terminated prior to convergence experimental results indicate that our fast algorithms substantially reduce the computation and result in good scale estimates for real tomographic data sets

a markov random field regulated pitman yor process prior for spatially constrained data clustering in this work we propose a markov random field regulated pitman yor process mrf pyp prior for nonparametric clustering of data with spatial interdependencies the mrf pyp is constructed by imposing a pitman yor process over the distribution of the latent variables that allocate data points to clusters model states the discount hyperparameter of which is regulated by an additionally postulated simplified pointwise markov random field gibbsian distribution with a countably infinite number of states further based on the stick breaking construction of the pitman yor process we derive an efficient truncated variational bayesian algorithm for model inference we examine the efficacy of our approach by considering an unsupervised image segmentation application using a real world dataset we show that our approach completely outperforms related methods from the field of bayesian nonparametrics including the recently proposed infinite hidden markov random field model and the dirichlet process prior

statistical mechanics approach to the phase unwrapping problem the use of mean field theory to unwrap principal phase patterns has been recently proposed in this paper we generalize the mean field approach to process phase patterns with arbitrary degree of undersampling the phase unwrapping problem is formulated as that of finding the ground state of a locally constrained finite size spin l ising model under a non uniform magnetic field the optimization problem is solved by the mean field annealing technique synthetic experiments show the effectiveness of the proposed algorithm

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

cluster expansions for the deterministic computation of bayesian estimators based on markov random fields we describe a family of approximations denoted by cluster approximations for the computation of the mean of a markov random field mrf this is a key computation in image processing when applied to the a posteriori mrf the approximation is to account exactly for only spatially local interactions application of the approximation requires the solution of a nonlinear multivariable fixed point equation for which we prove several existence uniqueness and convergence of algorithm results four numerical examples are presented including comparison with monte carlo calculations

statistical mechanical approach to image processing the basic frameworks and techniques of the bayesian approach to image restoration are reviewed from the statistical mechanical point of view first a few basic notions in digital image processing are explained to convince the reader that statistical mechanics has a close formal similarity to this problem second the basic formulation of the statistical estimation from the observed degraded image by using the bayes formula is demonstrated the relationship between bayesian statistics and statistical mechanics is also listed particularly it is explained that some correlation inequalities on the nishimori line of the random spin model also play an important role in bayesian image restoration third the framework of bayesian image restoration for binary images by means of the ising model is reviewed some practical algorithms for binary image restoration are given by employing the mean field and the bethe approximations finally bayesian image restoration for a grey level image using the gaussian model is reviewed and the gaussian model is extended to a more practical probabilistic model by introducing the line state to treat the effects of edges the line state is also extended to quantized values

map based stochastic diffusion for stereo matching and line fields estimation this paper proposes a stochastic approach to estimate the disparity field combined with line field in the maximum a posteriori map method based on markov random field mrf model it is important to optimize and converge the gibbs potential function corresponding to the perturbed disparity field the proposed optimization method stochastic diffusion takes advantage of the probabilistic distribution of the neighborhood fields to diffuse the gibbs potential space iteratively by using the neighborhood distribution in the non random and non deterministic diffusion both the estimation accuracy and the convergence speed are improved in the paper the hierarchical stochastic diffusion is also applied to the disparity field the hierarchical approach reduces the memory and computational load and increases the convergence speed of the potential space the paper also proposes an effective configuration of the neighborhood to be suitable for the hierarchical disparity structure according to the experiments the stochastic diffusion shows good estimation performance the line field improves the estimation at the object boundary and coincides with the object boundary with the useful contours the stochastic diffusion is applicable to any kind of field estimation given the appropriate definition of the field and mrf models

simple parallel hierarchical and relaxation algorithms for segmenting noncausal markovian random fields the modeling and segmentation of images by mrf s markov random fields is treated these are two dimensional noncausal markovian stochastic processes two conceptually new algorithms are presented for segmenting textured images into regions in each of which the data are modeled as one of c mrf s the algorithms are designed to operate in real time when implemented on new parallel computer architectures that can be built with present technology a doubly stochastic representation is used in image modeling here a gaussian mrf is used to model textures in visible light and infrared images and an autobinary or autoternary etc mrf to model a priori information about the local geometry of textured image regions for image segmentation the true texture class regions are treated either as a priori completely unknown or as a realization of a binary or ternary etc mrf in the former case image segmentation is realized as true maximum likelihood estimation in the latter case it is realized as true maximum a posteriori likelihood segmentation in addition to providing a mathematically correct means for introducing geometric structure the autobinary or ternary etc mrf can be used in a generative mode to generate image geometries and artificial images and such simulations constitute a very powerful tool for studying the effects of these models and the appropriate choice of model parameters the first segmentation algorithm is hierarchical and uses a pyramid like structure in new ways that exploit the mutual dependencies among disjoint pieces of a textured region

the convergence of mean field procedures for mrfs image processing convergence of the mean field equations is analyzed for the ising model with a nonhomogeneous external field when spl beta is small the equations are a contraction mapping and thereby convergent when spl beta is large however they can be noncontracting and in this case contraction is restored by a slight modification of the equations

a fuzzy clustering approach toward hidden markov random field models for enhanced spatially constrained image segmentation hidden markov random field hmrf models have been widely used for image segmentation as they appear naturally in problems where a spatially constrained clustering scheme taking into account the mutual influences of neighboring sites is asked for fuzzy c means fcm clustering has also been successfully applied in several image segmentation applications in this paper we combine the benefits of these two approaches by proposing a novel treatment of hmrf models formulated on the basis of a fuzzy clustering principle we approach the hmrf model treatment problem as an fcm type clustering problem effected by introducing the explicit assumptions of the hmrf model into the fuzzy clustering procedure our approach utilizes a fuzzy objective function regularized by kullback leibler divergence information and is facilitated by application of a mean field like approximation of the mrf prior we experimentally demonstrate the superiority of the proposed approach over competing methodologies considering a series of synthetic and real world image segmentation applications

em procedures using mean field like approximations for markov model based image segmentation this paper deals with markov random field model based image segmentation this involves parameter estimation in hidden markov models for which one of the most widely used procedures is the em algorithm in practice difficult ies arise due to the dependence structure in the models and approximations are required to make the algorithm tractable we propose a class of algorithms in which the idea is to deal with systems of independent variables this corresponds to approximations of the pixels interactions similar to the mean field approximation it follows algorithms that have the advantage of taking the markovian structure into account while preserving the good features of em in addition this class that includes new and already known procedures is presented in a unified framework showing that apparently distant algorithms come from similar approximation principles we illustrate the algorithms performance on synthetic and real images these experiments point out the ability of our procedures to take the spatial information into account our algorithms often show significant improvement when comparing with the em algorithm applied with no account of the spatial structure and with the icm algorithm based on maximization of the pseudo likelihood and commonly used in image segmentation

object based estimation of dense motion fields motion estimation belongs to key techniques in image sequence processing segmentation of the motion fields such that ideally each independently moving object uniquely corresponds to one region is one of the essential elements in object based image processing this paper is concerned with unsupervised simultaneous estimation of dense motion fields and their segmentations it is based on a stochastic model relating image intensities to motion information based on the analysis of natural images a region based model of motion compensated prediction error is proposed in each region the error is modeled by a white stationary generalized gaussian random process the motion field and its segmentation are themselves modeled by a compound gibbs markov random field accounting for statistical bindings in spatial direction and along the direction of motion trajectories the a posteriori distribution of the motion field for a given image sequence is formulated as an objective function such that its maximization results in the map estimate a deterministic multiscale relaxation technique with regular structure is employed for optimization of the objective function simulation results are in a good agreement with human perception for both the motion fields and their segmentations

irgs image segmentation using edge penalties and region growing this paper proposes an image segmentation method named iterative region growing using semantics irgs which is characterized by two aspects first it uses graduated increased edge penalty giep functions within the traditional markov random field mrf context model in formulating the objective functions second irgs uses a region growing technique in searching for the solutions to these objective functions the proposed irgs is an improvement over traditional mrf based approaches in that the edge strength information is utilized and a more stable estimation of model parameters is achieved moreover the irgs method provides the possibility of building a hierarchical representation of the image content and allows various region features and even domain knowledge to be incorporated in the segmentation process the algorithm has been successfully tested on several artificial images and synthetic aperture radar sar images

robust student s t mixture model with spatial constraints and its application in medical image segmentation finite mixture model based on the student s t distribution which is heavily tailed and more robust than gaussian has recently received great attention for image segmentation a new finite student s t mixture model smm is proposed in this paper existing models do not explicitly incorporate the spatial relationships between pixels first our model exploits dirichlet distribution and dirichlet law to incorporate the local spatial constrains in an image secondly we directly deal with the student s t distribution in order to estimate the model parameters whereas the student s t distributions in previous models are represented as an infinite mixture of scaled gaussians that lead to an increase in complexity finally instead of using expectation maximization em algorithm the proposed method adopts the gradient method to minimize the higher bound on the data negative log likelihood and to optimize the parameters the proposed model is successfully compared to the state of the art finite mixture models numerical experiments are presented where the proposed model is tested on various simulated and real medical images

three validation metrics for automated probabilistic image segmentation of brain tumours the validity of brain tumour segmentation is an important issue in image processing because it has a direct impact on surgical planning we examined the segmentation accuracy based on three two sample validation metrics against the estimated composite latent gold standard which was derived from several experts manual segmentations by an em algorithm the distribution functions of the tumour and control pixel data were parametrically assumed to be a mixture of two beta distributions with different shape parameters we estimated the corresponding receiver operating characteristic curve dice similarity coefficient and mutual information over all possible decision thresholds based on each validation metric an optimal threshold was then computed via maximization we illustrated these methods on mr imaging data from nine brain tumour cases of three different tumour types each consisting of a large number of pixels the automated segmentation yielded satisfactory accuracy with varied optimal thresholds the performances of these validation metrics were also investigated via monte carlo simulation extensions of incorporating spatial correlation structures using a markov random field model were considered

automatic segmentation and reconstruction of the cortex from neonatal mri abstract segmentation and reconstruction of cortical surfaces from magnetic resonance mr images are more challenging for developing neonates than adults this is mainly due to the dynamic changes in the contrast between gray matter gm and white matter wm in both t1 and t2 weighted images t1w and t2w during brain maturation in particular in neonatal t2w images wm typically has higher signal intensity than gm this causes mislabeled voxels during cortical segmentation especially in the cortical regions of the brain and in particular at the interface between gm and cerebrospinal fluid csf we propose an automatic segmentation algorithm detecting these mislabeled voxels and correcting errors caused by partial volume effects our results show that the proposed algorithm corrects errors in the segmentation of both gm and wm compared to the classic expectation maximization em scheme quantitative validation against manual segmentation demonstrates good performance the mean dice value 0 758 0 037 for gm and 0 794 0 078 for wm the inner central and outer cortical surfaces are then reconstructed using implicit surface evolution a landmark study is performed to verify the accuracy of the reconstructed cortex the mean surface reconstruction error 0 73 mm for inner surface and 0 63 mm for the outer both segmentation and reconstruction have been tested on 25 neonates with the gestational ages ranging from 27 to 45 weeks this preliminary analysis confirms previous findings that cortical surface area and curvature increase with age and that surface area scales to cerebral volume according to a power law while cortical thickness is not related to age or brain growth

a markov model for blind image separation by a mean field em algorithm this paper deals with blind separation of images from noisy linear mixtures with unknown coefficients formulated as a bayesian estimation problem this is a flexible framework where any kind of prior knowledge about the source images and the mixing matrix can be accounted for in particular we describe local correlation within the individual images through the use of markov random field mrf image models these are naturally suited to express the joint pdf of the sources in a factorized form so that the statistical independence requirements of most independent component analysis approaches to blind source separation are retained our model also includes edge variables to preserve intensity discontinuities mrf models have been proved to be very efficient in many visual reconstruction problems such as blind image restoration and allow separation and edge detection to be performed simultaneously we propose an expectation maximization algorithm with the mean field approximation to derive a procedure for estimating the mixing matrix the sources and their edge maps we tested this procedure on both synthetic and real images in the fully blind case i e no prior information on mixing is exploited and found that a source model accounting for local autocorrelation is able to increase robustness against noise even space variant furthermore when the model closely fits the source characteristics independence is no longer a strict requirement and cross correlated sources can be separated as well

hidden markov measure field models for image segmentation parametric image segmentation consists of finding a label field that defines a partition of an image into a set of nonoverlapping regions and the parameters of the models that describe the variation of some property within each region a new bayesian formulation for the solution of this problem is presented based on the key idea of using a doubly stochastic prior model for the label field which allows one to find exact optimal estimators for both this field and the model parameters by the minimization of a differentiable function an efficient minimization algorithm and comparisons with existing methods on synthetic images are presented as well as examples of realistic applications to the segmentation of magnetic resonance volumes and to motion segmentation

distributed em algorithm for gaussian mixtures in sensor networks this paper presents a distributed expectation maximization em algorithm over sensor networks in the e step of this algorithm each sensor node independently calculates local sufficient statistics by using local observations a consensus filter is used to diffuse local sufficient statistics to neighbors and estimate global sufficient statistics in each node by using this consensus filter each node can gradually diffuse its local information over the entire network and asymptotically the estimate of global sufficient statistics is obtained in the m step of this algorithm each sensor node uses the estimated global sufficient statistics to update model parameters of the gaussian mixtures which can maximize the log likelihood in the same way as in the standard em algorithm because the consensus filter only requires that each node communicate with its neighbors the distributed em algorithm is scalable and robust it is also shown that the distributed em algorithm is a stochastic approximation to the standard em algorithm thus it converges to a local maximum of the log likelihood several simulations of sensor networks are given to verify the proposed algorithm

combining monte carlo and mean field like methods for inference in hidden markov random fields issues involving missing data are typical settings where exact inference is not tractable as soon as nontrivial interactions occur between the missing variables approximations are required and most of them are based either on simulation methods or on deterministic variational methods while variational methods provide fast and reasonable approximate estimates in many scenarios simulation methods offer more consideration of important theoretical issues such as accuracy of the approximation and convergence of the algorithms but at a much higher computational cost in this work we propose a new class of algorithms that combine the main features and advantages of both simulation and deterministic methods and consider applications to inference in hidden markov random fields hmrfs these algorithms can be viewed as stochastic perturbations of variational expectation maximization vem algorithms which are not tractable for hmrf we focus more specifically on one of these perturbations and we prove their almost sure convergence to the same limit set as the limit set of vem in addition experiments on synthetic and real world images show that the algorithm performance is very close and sometimes better than that of other existing simulation based and variational em like algorithms

on the adequacy of variational lower bound functions for likelihood based inference in markovian models with missing values summary variational methods have been proposed for obtaining deterministic lower bounds for log likelihoods within missing data problems but with little formal justification or investigation of the worth of the lower bound surfaces as tools for inference we provide within a general markovian context sufficient conditions under which estimators from the variational approximations are asymptotically equivalent to maximum likelihood estimators and we show empirically for the simple example of a first order autoregressive model with missing values that the lower bound surface can be very similar in shape to the true log likelihood in non asymptotic situations

a spatially constrained generative model and an em algorithm for image segmentation in this paper we present a novel spatially constrained generative model and an expectation maximization em algorithm for model based image segmentation the generative model assumes that the unobserved class labels of neighboring pixels in the image are generated by prior distributions with similar parameters where similarity is defined by entropic quantities relating to the neighboring priors in order to estimate model parameters from observations we derive a spatially constrained em algorithm that iteratively maximizes a lower bound on the data log likelihood where the penalty term is data dependent our algorithm is very easy to implement and is similar to the standard em algorithm for gaussian mixtures with the main difference that the labels posteriors are smoothed over pixels between each e and m step by a standard image filter experiments on synthetic and real images show that our algorithm achieves competitive segmentation results compared to other markov based methods and is in general faster