map estimation via agreement on trees message passing and linear programming we develop and analyze methods for computing provably optimal maximum a posteriori probability map configurations for a subclass of markov random fields defined on graphs with cycles by decomposing the original distribution into a convex combination of tree structured distributions we obtain an upper bound on the optimal value of the original problem i e the log probability of the map assignment in terms of the combined optimal values of the tree problems we prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common an important implication is that any such shared configuration must also be a map configuration for the original distribution next we develop two approaches to attempting to obtain tight upper bounds a a tree relaxed linear program lp which is derived from the lagrangian dual of the upper bounds and b a tree reweighted max product message passing algorithm that is related to but distinct from the max product algorithm in this way we establish a connection between a certain lp relaxation of the mode finding problem and a reweighted form of the max product min sum message passing algorithm

the numerical solution of fifth order boundary value problems by the variational iteration method this paper applies the variational iteration method to solve fifth order boundary value problems just one iteration results in highly accurate solutions a comparison of the results with exact ones is made to confirm the validity and efficiency

variational iteration method for autonomous ordinary differential systems in this paper a new iteration technique is proposed to solve autonomous ordinary differential systems in this method general lagrange multipliers are introduced to construct correction functionals for the systems the multipliers in the functionals can be identified by the variational theory the initial approximations can be freely chosen with possible unknown constants which can be determined by imposing the boundary initial conditions some examples are given the results reveal that the method is very effective and convenient

bayesian regularization and pruning using a laplace prior standard techniques for improved generalization from neural networks include weight decay and pruning weight decay has a bayesian interpretation with the decay function corresponding to a prior over weights the method of transformation groups and maximum entropy suggests a laplace rather than a gaussian prior after training the weights then arrange themselves into two classes 1 those with a common sensitivity to the data error and 2 those failing to achieve this sensitivity and that therefore vanish since the critical value is determined adaptively during training pruning in the sense of setting weights to exact zeros becomes an automatic consequence of regularization alone the count of free parameters is also reduced automatically as weights are pruned a comparison is made with results of mackay using the evidence framework and a gaussian regularizer

strang splitting method for burgers huxley equation we derive an analytical approach to the strang splitting method for the burgers huxley equation bhe u t u u x u x x 1 u u u we proved that srtang splitting method has a second order convergence in h s r where h s r is the sobolev space and s is an arbitrary nonnegative integer we numerically solve the bhe by strang splitting method and compare the results with the reference solution

variational message passing bayesian inference is now widely established as one of the principal foundations for machine learning in practice exact inference is rarely possible and so a variety of approximation techniques have been developed one of the most widely used being a deterministic framework called variational inference in this paper we introduce variational message passing vmp a general purpose algorithm for applying variational inference to bayesian networks like belief propagation vmp proceeds by sending messages between nodes in the network and updating posterior beliefs using local operations at each node each such update increases a lower bound on the log evidence unless already at a local maximum in contrast to belief propagation vmp can be applied to a very general class of conjugate exponential models because it uses a factorised variational approximation furthermore by introducing additional variational parameters vmp can be applied to models containing non conjugate distributions the vmp framework also allows the lower bound to be evaluated and this can be used both for model comparison and for detection of convergence variational message passing has been implemented in the form of a general purpose inference engine called vibes variational inference for bayesian networks which allows models to be specified graphically and then solved variationally without recourse to coding

triplet markov fields for the classification of complex structure data we address the issue of classifying complex data we focus on three main sources of complexity namely the high dimensionality of the observed data the dependencies between these observations and the general nature of the noise model underlying their distribution we investigate the recent triplet markov fields and propose new models in this class designed for such data and in particular allowing very general noise models in addition our models can handle the inclusion of a learning step in a consistent way so that they can be used in a supervised framework one advantage of our models is that whatever the initial complexity of the noise model parameter estimation can be carried out using state of the art bayesian clustering techniques under the usual simplifying assumptions as generative models they can be seen as an alternative in the supervised case to discriminative conditional random fields identifiability issues underlying the models in the nonsupervised case are discussed while the models performance is illustrated on simulated and real data exhibiting the mentioned various sources of complexity

variational iteration method new development and applications variational iteration method has been favourably applied to various kinds of nonlinear problems the main property of the method is in its flexibility and ability to solve nonlinear equations accurately and conveniently in this paper recent trends and developments in the use of the method are reviewed major applications to nonlinear wave equation nonlinear fractional differential equations nonlinear oscillations and nonlinear problems arising in various engineering applications are surveyed the confluence of modern mathematics and symbol computation has posed a challenge to developing technologies capable of handling strongly nonlinear equations which cannot be successfully dealt with by classical methods variational iteration method is uniquely qualified to address this challenge the flexibility and adaptation provided by the method have made the method a strong candidate for approximate analytical solutions this paper outlines the basic conceptual framework of variational iteration technique with application to nonlinear problems both achievements and limitations are discussed with direct reference to approximate solutions for nonlinear equations a new iteration formulation is suggested to overcome the shortcoming a very useful formulation for determining approximately the period of a nonlinear oscillator is suggested examples are given to illustrate the solution procedure

learning probabilistic models for contour completion in natural images using a large set of human segmented natural images we study the statistics of region boundaries we observe several power law distributions which likely arise from both multi scale structure within individual objects and from arbitrary viewing distance accordingly we develop a scale invariant representation of images from the bottom up using a piecewise linear approximation of contours and constrained delaunay triangulation to complete gaps we model curvilinear grouping on top of this graphical geometric structure using a conditional random field to capture the statistics of continuity and different junction types quantitative evaluations on several large datasets show that our contour grouping algorithm consistently dominates and significantly improves on local edge detection

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

on learning conditional random fields for stereo until recently the lack of ground truth data has hindered the application of discriminative structured prediction techniques to the stereo problem in this paper we use ground truth data sets that we have recently constructed to explore different model structures and parameter learning techniques to estimate parameters in markov random fields mrfs via maximum likelihood one usually needs to perform approximate probabilistic inference conditional random fields crfs are discriminative versions of traditional mrfs we explore a number of novel crf model structures including a crf for stereo matching with an explicit occlusion model crfs require expensive inference steps for each iteration of optimization and inference is particularly slow when there are many discrete states we explore belief propagation variational message passing and graph cuts as inference methods during learning and compare with learning via pseudolikelihood to accelerate approximate inference we have developed a new method called sparse variational message passing which can reduce inference time by an order of magnitude with negligible loss in quality learning using sparse variational message passing improves upon previous approaches using graph cuts and allows efficient learning over large data sets when energy functions violate the constraints imposed by graph cuts

delineating buildings by grouping lines with mrfs traditionally markov random field mrf models have been used in low level image analysis the article presents an mrf based scheme to perform object delineation the proposed edge based approach involves extracting straight lines from the edge map of an image then an mrf model is used to group these lines to delineate buildings in aerial images

discriminative random fields in this research we address the problem of classification and labeling of regions given a single static natural image natural images exhibit strong spatial dependencies and modeling these dependencies in a principled manner is crucial to achieve good classification accuracy in this work we present discriminative random fields drfs to model spatial interactions in images in a discriminative framework based on the concept of conditional random fields proposed by lafferty et al 2001 the drfs classify image regions by incorporating neighborhood spatial interactions in the labels as well as the observed data the drf framework offers several advantages over the conventional markov random field mrf framework first the drfs allow to relax the strong assumption of conditional independence of the observed data generally used in the mrf framework for tractability this assumption is too restrictive for a large number of applications in computer vision second the drfs derive their classification power by exploiting the probabilistic discriminative models instead of the generative models used for modeling observations in the mrf framework third the interaction in labels in drfs is based on the idea of pairwise discrimination of the observed data making it data adaptive instead of being fixed a priori as in mrfs finally all the parameters in the drf model are estimated simultaneously from the training data unlike the mrf framework where the likelihood parameters are usually learned separately from the field parameters we present preliminary experiments with man made structure detection and binary image restoration tasks and compare the drf results with the mrf results

fast approximate energy minimization via graph cuts many tasks in computer vision involve assigning a label such as disparity to every pixel a common constraint is that the labels should vary smoothly almost everywhere while preserving sharp discontinuities that may exist e g at object boundaries these tasks are naturally stated in terms of energy minimization the authors consider a wide class of energies with various smoothness constraints global minimization of these energy functions is np hard even in the simplest discontinuity preserving case therefore our focus is on efficient approximation algorithms we present two algorithms based on graph cuts that efficiently find a local minimum with respect to two types of large moves namely expansion moves and swap moves these moves can simultaneously change the labels of arbitrarily large sets of pixels in contrast many standard algorithms including simulated annealing use small moves where only one pixel changes its label at a time our expansion algorithm finds a labeling within a known factor of the global minimum while our swap algorithm handles more general energy functions both of these algorithms allow important cases of discontinuity preserving energies we experimentally demonstrate the effectiveness of our approach for image restoration stereo and motion on real data with ground truth we achieve 98 percent accuracy

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

spectral galerkin methods for the primary two point boundary value problem in modelling viscoelastic flows the essential character of the primary elliptic hyperbolic operator encountered in the modelling of viscoelastic flows is encaptured in a non linear fifth order two point boundary value problem in one dimension expansions in terms of beam functions and chebyshev polynomials are used to compute solutions to the primary two point boundary value problem within a spectral galerkin formulation an investigation of the performance of the methods with respect to the level of non linearity is carried out accurate results are obtained with chebyshev polynomials for high levels of non linearity whereas the behaviour of beam function expansions proves far more sensitive to the level of non linearity

correctness prediction accuracy improvement and generalization of stereo matching using supervised learning machine learning has been instrumental in most areas of computer vision but has not been applied to the problem of stereo matching with similar frequency or success in this paper we present a supervised learning approach by defining a set of features that capture various forms of information about each pixel and then by using them to predict the correctness of stereo matches based on a random forest we show highly competitive results in predicting the correctness of matches and in confidence estimation which allows us to rank pixels according to the reliability of their assigned disparities moreover we show how these confidence values can be used to improve the accuracy of disparity maps by integrating them with an mrf based stereo algorithm this is an important distinction from current literature that has mainly focused on sparsification by removing potentially erroneous disparities to generate quasi dense disparity maps finally we demonstrate domain generalization of our method by applying classifiers to datasets different than those they were trained on with minimal loss of accuracy

stereo matching using belief propagation in this paper we formulate the stereo matching problem as a markov network and solve it using bayesian belief propagation the stereo markov network consists of three coupled markov random fields that model the following a smooth field for depth disparity a line process for depth discontinuity and a binary process for occlusion after eliminating the line process and the binary process by introducing two robust functions we apply the belief propagation algorithm to obtain the maximum a posteriori map estimation in the markov network other low level visual cues e g image segmentation can also be easily incorporated in our stereo model to obtain better stereo results experiments demonstrate that our methods are comparable to the state of the art stereo algorithms for many test cases

a pixel dissimilarity measure that is insensitive to image sampling because of image sampling traditional measures of pixel dissimilarity can assign a large value to two corresponding pixels in a stereo pair even in the absence of noise and other degrading effects we propose a measure of dissimilarity that is provably insensitive to sampling because it uses the linearly interpolated intensity functions surrounding the pixels experiments on real images show that our measure alleviates the problem of sampling with little additional computational overhead

stochastic stereo matching over scale a stochastic optimization approach to stereo matching is presented unlike conventional correlation matching and feature matching the method provides a dense array of disparities eliminating the need for interpolation first the stereo matching problem is defined in terms of finding a disparity map that satisfies two competing constraints 1 matched points should have similar image intensity and 2 the disparity map should vary as slowly as possible these constraints are interpreted as specifying the potential energy of a system of oscillators ground states are approximated by a new variant of simulated annealing which has two important features first the microcanonical ensemble is simulated using a new algorithm that is more efficient and more easily implemented than the familiar metropolis algorithm which simulates the canonical ensemble secondly it uses a hierarchical coarse to fine control structure employing gaussian or laplacian pyramids of the stereo images in this way quickly computed results at low resolutions are used to initialize the system at higher resolutions

training products of experts by minimizing contrastive divergence it is possible to combine multiple latent variable models of the same data by multiplying their probability distributions together and then renormalizing this way of combining individual expert models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data a product of experts poe is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary training a poe by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule fortunately a poe can be trained using a different objective function called contrastive divergence whose derivatives with regard to the parameters can be approximated accurately and efficiently examples are presented of contrastive divergence learning using several types of expert on several types of data

the numerical solution of a generalized burgers huxley equation through a conditionally bounded and symmetry preserving method in this article we propose a non standard finite difference scheme to approximate the solutions of a generalized burgers huxley equation from fluid dynamics our numerical method preserves the skew symmetry of the partial differential equation under study and under some analytical constraints of the model constants and the computational parameters involved it is capable of preserving the boundedness and the positivity of the solutions in the linear regime the scheme is consistent to first order in time due partially to the inclusion of a tuning parameter in the approximation of a temporal derivative and to second order in space we compare the results of our computational technique against the exact solutions of some particular initial boundary value problems our simulations indicate that the method presented in this work approximates well the theoretical solutions and moreover that the method preserves the boundedness of solutions within the analytical constraints derived here in the problem of approximating solitary wave solutions of the model under consideration we present numerical evidence on the existence of an optimum value of the tuning parameter of our technique for which a minimum relative error is achieved finally we linearly perturb a steady state solution of the partial differential equation under investigation and show that our simulations still converge to the same constant solution establishing thus robustness of our method in this sense

structural matching in computer vision using probabilistic relaxation in this paper we develop the theory of probabilistic relaxation for matching features extracted from 2d images derive as limiting cases the various heuristic formulae used by researchers in matching problems and state the conditions under which they apply we successfully apply our theory to the problem of matching and recognizing aerial road network images based on road network models and to the problem of edge matching in a stereo pair for this purpose each line network is represented by an attributed relational graph where each node is a straight line segment characterized by certain attributes and related with every other node via a set of binary relations

a comparative study of energy minimization methods for markov random fields with smoothness based priors among the most exciting advances in early vision has been the development of efficient energy minimization algorithms for pixel labeling tasks such as depth or texture computation it has been known for decades that such problems can be elegantly expressed as markov random fields yet the resulting energy minimization problems have been widely viewed as intractable algorithms such as graph cuts and loopy belief propagation lbp have proven to be very powerful for example such methods form the basis for almost all the top performing stereo methods however the trade offs among different energy minimization algorithms are still not well understood in this paper we describe a set of energy minimization benchmarks and use them to compare the solution quality and runtime of several common energy minimization algorithms we investigate three promising methods graph cuts lbp and tree reweighted message passing in addition to the well known older iterated conditional mode icm algorithm our benchmark problems are drawn from published energy functions used for stereo image stitching interactive segmentation and denoising we also provide a general purpose software interface that allows vision researchers to easily switch between optimization methods the benchmarks code images and results are available at http vision middlebury edu mrf

modeling visual patterns by integrating descriptive and generative methods this paper presents a class of statistical models that integrate two statistical modeling paradigms in the literature i descriptive methods such as markov random fields and minimax entropy learning zhu s c wu y n and mumford d 1997 neural computation 9 8 and ii generative methods such as principal component analysis independent component analysis bell a j and sejnowski t j 1997 vision research 37 3327 3338 transformed component analysis frey b and jojic n 1999 iccv wavelet coding mallat s and zhang z 1993 ieee trans on signal processing 41 3397 3415 chen s donoho d and saunders m a 1999 journal on scientific computing 20 1 33 61 and sparse coding olshausen b a and field d j 1996 nature 381 607 609 lewicki m s and olshausen b a 1999 josa a 16 7 1587 1601 in this paper we demonstrate the integrated framework by constructing a class of hierarchical models for texton patterns the term texton was coined by psychologist julesz in the early 80s at the bottom level of the model we assume that an observed texture image is generated by multiple hidden texton maps and textons on each map are translated scaled stretched and oriented versions of a window function like mini templates or wavelet bases the texton maps generate the observed image by occlusion or linear superposition this bottom level of the model is generative in nature at the top level of the model the spatial arrangements of the textons in the texton maps are characterized by minimax entropy principle which leads to embellished versions of gibbs point process models stoyan d kendall w s and mecke j 1985 stochastic geometry and its applications the top level of the model is descriptive in nature we demonstrate the integrated model by a set of experiments

the numerical solution of fifth order boundary value problemsby the decomposition method in this paper we present a fast and accurate numerical scheme for the solution of fifth order boundary value problems with two point boundary conditions the adomian decomposition method and a modified form of this method are applied to construct the numerical solution the new approach provides the solution in the form of a rapidly convergent series and not at grid points two numerical illustrations are given to show the pertinent features of technique

tree based reparameterization framework for analysis of sum product and related algorithms we present a tree based reparameterization trp framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles this class includes the belief propagation bp or sum product algorithm as well as variations and extensions of bp algorithms in this class can be formulated as a sequence of reparameterization updates each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph i e a tree or more generally a hypertree the ultimate goal is to obtain an alternative but equivalent factorization using functions that represent exact or approximate marginal distributions on cliques of the graph our framework highlights an important property of the sum product algorithm and the larger class of reparameterization algorithms the original distribution on the graph with cycles is not changed the perspective of tree based updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency we develop interpretations of these results in terms of information geometry the invariance of the distribution in conjunction with the fixed point characterization enables us to derive an exact expression for the difference between the true marginals on an arbitrary graph with cycles and the approximations provided by belief propagation more broadly our analysis applies to any algorithm that minimizes the bethe free energy we also develop bounds on the approximation error which illuminate the conditions that govern their accuracy finally we show how the reparameterization perspective extends naturally to generalizations of bp e g kikuchi 1951 approximations and variants via the notion of hypertree reparameterization

approximate analytical solution for seepage flow with fractional derivatives in porous media abstract in this paper a new and more exact model for seepage flow in porous media with fractional derivatives has been proposed which has modified the well known darcy law and overcome the continuity assumption of seepage flow a new kind of analytical method of nonlinear problem called the variational iteration method is described and used to give approximate solutions of the problem the results show that the proposed iteration method requiring no linearization or small perturbation is very effective and convenient

combining belief networks and neural networks for scene segmentation we are concerned with the problem of image segmentation in which each pixel is assigned to one of a predefined finite number of labels in bayesian image analysis this requires fusing together local predictions for the class labels with a prior model of label images following the work of bouman and shapiro 1994 we consider the use of tree structured belief networks tsbns as prior models the parameters in the tsbn are trained using a maximum likelihood objective function with the em algorithm and the resulting model is evaluated by calculating how efficiently it codes label images a number of authors have used gaussian mixture models to connect the label field to the image data we compare this approach to the scaled likelihood method of smyth 1994 and morgan and bourlard 1995 where local predictions of pixel classification from neural networks are fused with the tsbn prior our results show a higher performance is obtained with the neural networks we evaluate the classification results obtained and emphasize not only the maximum a posteriori segmentation but also the uncertainty as evidenced e g by the pixelwise posterior marginal entropies we also investigate the use of conditional maximum likelihood training for the tsbn and find that this gives rise to improved classification performance over the ml trained tsbn

operator splitting for the kdv equation we provide a new analytical approach to operator splitting for equations of the type u t au b u where a is a linear operator and b is quadratic a particular example is the korteweg de vries kdv equation u t uu x u xxx 0 we show that the godunov and strang splitting methods converge with the expected rates if the initial data are sufficiently regular

b spline collocation for solution of two point boundary value problems a numerical method based on b spline is developed to solve the general nonlinear two point boundary value problems up to order 6 the standard formulation of sextic spline for the solution of boundary value problems leads to non optimal approximations in order to derive higher orders of accuracy high order perturbations of the problem are generated and applied to construct the numerical algorithm the error analysis and convergence properties of the method are studied via green s function approach o h 6 global error estimates are obtained for numerical solution of these classes of problems numerical results are given to illustrate the efficiency of the proposed method results of numerical experiments verify the theoretical behavior of the orders of convergence

approximate solution of nonlinear differential equations with convolution product nonlinearities in this paper a new iteration method is proposed to solve nonlinear problems special attention is paid to nonlinear differential equations with convolution product nonlinearities the results reveal the approximations obtained by the proposed method are uniformly valid for both small and large parameters in nonlinear problems furthermore the first order of approximations are more accurate than perturbation solutions at high order of approximation