tree approximations to markov random fields methods for approximately computing the marginal probability mass functions and means of a markov random field mrf by approximating the lattice by a tree are described applied to the a posteriori mrf these methods solve bayesian spatial pattern classification and image restoration problems the methods are described several theoretical results concerning fixed point problems are proven and four numerical examples are presented including comparison with optimal estimators and the iterated conditional mode estimator and including two agricultural optical remote sensing problems

statistical analysis of loopy belief propagation in random fields loopy belief propagation lbp which is equivalent to the bethe approximation in statistical mechanics is a message passing type inference method that is widely used to analyze systems based on markov random fields mrfs in this paper we propose a message passing type method to analytically evaluate the quenched average of lbp in random fields by using the replica cluster variation method the proposed analytical method is applicable to general pairwise mrfs with random fields whose distributions differ from each other and can give the quenched averages of the bethe free energies over random fields which are consistent with numerical results the order of its computational cost is equivalent to that of standard lbp in the latter part of this paper we describe the application of the proposed method to bayesian image restoration in which we observed that our theoretical results are in good agreement with the numerical results for natural images

a bayesian approach for the estimation and transmission of regularization parameters for reducing blocking artifacts with block based compression approaches for both still images and sequences of images annoying blocking artifacts are exhibited primarily at high compression ratios they are due to the independent processing quantization of the block transformed values of the intensity or the displaced frame difference we propose the application of the hierarchical bayesian paradigm to the reconstruction of block discrete cosine transform bdct compressed images and the estimation of the required parameters we derive expressions for the iterative evaluation of these parameters applying the evidence analysis within the hierarchical bayesian paradigm the proposed method allows for the combination of parameters estimated at the coder and decoder the performance of the proposed algorithms is demonstrated experimentally

recognizing multiple overlapping objects in image an optimal formulation a statistically optimal formulation is presented for recognizing multiple partially occluded objects the optimality in terms of the maximum a posteriori map principle is with respect to all rather than just individual modeled objects various constraints are incorporated into the posterior distribution a two stage map estimation approach is proposed to reduce the computational cost

simultaneous parameter estimation and segmentation of gibbs random fields using simulated annealing an adaptive segmentation algorithm is developed which simultaneously estimates the parameters of the underlying gibbs random field grf and segments the noisy image corrupted by additive independent gaussian noise the algorithm which aims at obtaining the maximum a posteriori map segmentation is a simulated annealing algorithm that is interrupted at regular intervals for estimating the grf parameters maximum likelihood ml estimates of the parameters based on the current segmentation are used to obtain the next segmentation it is proven that the parameter estimates and the segmentations converge in distribution to the ml estimate of the parameters and the map segmentation with those parameter estimates respectively due to computational difficulties however only an approximate version of the algorithm is implemented the approximate algorithm is applied on several two and four region images with different noise levels and with first order and second order neighborhoods

lago a computationally efficient approach for statistical detection we study a general class of statistical detection problems where the underlying objective is to detect items belonging to a rare class from a very large database we propose a computationally efficient method to achieve this goal our method consists of two steps in the first step we estimate the density function of the rare class alone with an adaptive bandwidth kernel density estimator the adaptive choice of the bandwidth is inspired by the ancient chinese board game known today as go in the second step we adjust this density locally depending on the density of the background class nearby we show that the amount of adjustment needed in the second step is approximately equal to the adaptive bandwidth from the first step which gives us additional computational savings we name the resulting method lago for locally adjusted go kernel density estimator we then apply lago to a real drug discovery dataset and compare its performance with a number of existing and popular methods

image restoration using the q ising spin glass we investigate static and dynamic properties of gray scale image restoration by making use of the q ising spin glass model whose ladder symmetry allows us to take in account the distance between two spins we thus give an explicit expression of the hamming distance between the original and restored images as a function of the hyperparameters in the mean field limit finally numerical simulations for real world pictures are carried out to prove the efficiency of our model

finite size effects in bayesian model selection and generalization we show that in supervised learning from a supplied data set bayesian model selection based on the evidence does not optimize generalization performance even for a learnable linear problem this is demonstrated by examining the finite size effects in hyperparameter assignment from the evidence procedure and the resultant generalization performance our approach demonstrates the weakness of average case and asymptotic analyses using simulations we corroborate our analytic results and examine an alternative model selection criterion namely cross validation this numerical study shows that the cross validation hyperparameter estimates correlate more strongly than those of the evidence with optimal performance however we show that for a sufficiently large input dimension the evidence procedure could provide a reliable alternative to the more computationally expensive cross validation

inference in alpha rhythm phase and amplitude modeled on markov random field using belief propagation from electroencephalograms alpha rhythm is a major component of spontaneous electroencephalographic eeg data we develop a novel method that can be used to estimate the instantaneous phases and amplitudes of the alpha rhythm with high accuracy by modeling the alpha rhythm phase and amplitude as markov random field mrf models by using a belief propagation technique we construct an exact inference algorithm that can be used to estimate instantaneous phases and amplitudes and calculate the marginal likelihood maximizing the marginal likelihood enables us to estimate the hyperparameters on the basis of type ii maximum likelihood estimation we prove that the instantaneous phase and amplitude estimation by our method is consistent with that by the hilbert transform which has been commonly used to estimate instantaneous phases and amplitudes of a signal filtered from observed data in the limited case that the observed data consist of only one frequency signal whose amplitude is constant and a gaussian noise comparison of the performances of observation noise reduction by our method and by a gaussian mrf model of alpha rhythm signal indicates that our method reduces observation noise more efficiently moreover the instantaneous phase and amplitude estimates obtained using our method are more accurate than those obtained by the hilbert transform application of our method to experimental eeg data also demonstrates that the relationship between the alpha rhythm phase and the reaction time emerges more clearly by using our method than the hilbert transform this indicates our method s practical usefulness therefore applying our method to experimental eeg data will enable us to estimate the instantaneous phases and amplitudes of the alpha rhythm more precisely

solvable markov random field model in color image restoration we propose a scheme for image restoration of full color images by means of a solvable probabilistic model in the red green blue space a special case of our solvable probabilistic model is equivalent to a multicomponent gaussian model in the statistical mechanics exact closed expressions of the evidence and the expectation value of intensity at each pixel in our solvable probabilistic model can be obtained by using multidimensional gaussian integral formulas and a discrete fourier transform in the present paper the degradation process is assumed to be an additive white gaussian noise hyperparameters are determined so as to maximize the evidence that is expressed in terms of the partition function in our solvable probabilistic model this work is a pioneering work for the bayesian approach to the color image restoration by means of the statistical mechanical technique

optimized wiener filter we consider using a wiener filter for denoising a set of input signals that is degraded by additive white gaussian noise the wiener filter is designed to minimize the mean square error and it requires the knowledge of covariance of input signals and variance of the noise the mean square error is defined as a distance between input signals and input signals estimated by the wiener filter from noisy output signals although we have to infer input signals from only noisy output signals the input signals are required for the evaluation of the mean square error we reformulate the mean square error using noisy output signals and the covariance of the input signals the covariance of the input signals is estimated by minimizing the mean square error we apply our method to the case when an input signal is not generated by the assumed prior probability in particular we apply our method to image restoration and obtain good estimation results

mean field annealing a formalism for constructing gnc like algorithms optimization problems are approached using mean field annealing mfa which is a deterministic approximation using mean field theory and based on peierls s inequality to simulated annealing the mfa mathematics are applied to three different objective function examples in each case mfa produces a minimization algorithm that is a type of graduated nonconvexity when applied to the weak membrane objective mfa results in an algorithm qualitatively identical to the published gnc algorithm one of the examples mfa applied to a piecewise constant objective function is then compared experimentally with the corresponding gnc weak membrane algorithm the mathematics of mfa are shown to provide a powerful and general tool for deriving optimization algorithms

accuracy of the bethe approximation for hyperparameter estimation in probabilistic image processing we investigate the accuracy of statistical mechanical approximations for the estimation of hyperparameters from observable data in probabilistic image processing which is based on bayesian statistics and maximum likelihood estimation hyperparameters in statistical science correspond to interactions or external fields in the statistical mechanics context in this paper hyperparameters in the probabilistic model are determined so as to maximize a marginal likelihood a practical algorithm is described for grey level image restoration based on a gaussian graphical model and the bethe approximation the algorithm corresponds to loopy belief propagation in artificial intelligence we examine the accuracy of hyperparameter estimation when we use the bethe approximation it is well known that a practical algorithm for probabilistic image processing can be prescribed analytically when a gaussian graphical model is adopted as a prior probabilistic model in bayes formula we are therefore able to compare in a numerical study results obtained through mean field type approximations with those based on exact calculation

the application of the gibbs bogoliubov feynman inequality in mean field calculations for markov random fields the gibbs bogoliubov feynman gbf inequality of statistical mechanics is adopted with an information theoretic interpretation as a general optimization framework for deriving and examining various mean field approximations for markov random fields mrf s the efficacy of this approach is demonstrated through the compound gauss markov cgm model comparisons between different mean field approximations and experimental results in image restoration

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

physical consideration of an image in image restoration using bayes formula abstract we consider image restoration by bayes formula and investigate the relationship between an image and a prior probability from the following two viewpoints hyperparameter estimation and the accuracy of a restored image the q ising model is adopted as a prior probability in bayes formula not the q ising energy but the potts energy plays an important role in the hyperparameter estimation from the viewpoint of the hyperparameter estimation the relationship between a natural image and a prior probability is characterized through the potts energy and magnetization of an image the potts energy and magnetization of an image are defined by a set of pixels state of an image the closer to the average potts energy and magnetization over a prior probability the potts energy and magnetization of a natural image is the closer to the true value of a hyperparameter the estimated value of a hyperparameter from a degraded image is for the accuracy of a restored image the image which has a smaller q ising energy is better restored by bayes formula composed of the q ising prior the consideration for the relationship between an image and a prior probability is expected to be valid for a more complicated prior probability

the mean field theory in em procedures for markov random fields in many signal processing and pattern recognition applications the hidden data are modeled as markov processes and the main difficulty of using the maximisation em algorithm for these applications is the calculation of the conditional expectations of the hidden markov processes it is shown how the mean field theory from statistical mechanics can be used to calculate the conditional expectations for these problems efficiently the efficacy of the mean field theory approach is demonstrated on parameter estimation for one dimensional mixture data and two dimensional unsupervised stochastic model based image segmentation experimental results indicate that in the 1 d case the mean field theory approach provides results comparable to those obtained by baum s 1987 algorithm which is known to be optimal in the 2 d case where baum s algorithm can no longer be used the mean field theory provides good parameter estimates and image segmentation for both synthetic and real world images

statistical trajectory of an approximate em algorithm for probabilistic image processing we calculate analytically a statistical average of trajectories of an approximate expectation maximization em algorithm with generalized belief propagation gbp and a gaussian graphical model for the estimation of hyperparameters from observable data in probabilistic image processing a statistical average with respect to observed data corresponds to a configuration average for the random field ising model in spin glass theory in the present paper hyperparameters which correspond to interactions and external fields of spin systems are estimated by an approximate em algorithm a practical algorithm is described for gray level image restoration based on a gaussian graphical model and gbp the gbp approach corresponds to the cluster variation method in statistical mechanics our main result in the present paper is to obtain the statistical average of the trajectory in the approximate em algorithm by using loopy belief propagation and gbp with respect to degraded images generated from a probability density function with true values of hyperparameters the statistical average of the trajectory can be expressed in terms of recursion formulas derived from some analytical calculations

hyperparameter estimation in image restoration the hyperparameter in image restoration by the bayes formula is an important quantity this communication shows a physical method for the estimation of the hyperparameter without approximation for artificially generated images by prior probability the hyperparameter is computed accurately for practical images accuracy of the estimated hyperparameter depends on the magnetization and energy of the images we discuss the validity of prior probability for an original image

statistical mechanics of image restoration we develop the statistical mechanics formulation of the image restoration problem pioneered by geman and geman 1984 using bayesian methods we establish the posterior probability distribution for restored images for given data corrupted image and prior assumptions about source and corruption process in the simplest cases studied here the posterior is controlled by a cost function analagous to the configurational energy of an ising model with local fields whose sense is defined by the data through a combination of monte carlo simulation and mean field theory we address three key issues first we explore the sensitivity of the posterior distribution to the choice of prior parameters we find phase transitions separating regions in which the distribution is effective data dominated from regions in which it is ineffective prior dominated second we examine the question of how best to use the posterior distribution to prescribe a single optimal restored image we argue that the mean of the posterior is in general to be preferred over the mode both in principle and in practice finally borrowing from monte carlo techniques for free energy calculations we address the question of prior parameter estimation within the evidence framework of gull 1989 and mackay 1992 our results suggest that parameters identified by this framework provide effective priors leading to optimal restoration only to the extent that the forms of the priors are well matched to the processes they claim to represent

asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications in this paper we extend our previous work on the stochastic block model a commonly used generative model for social and biological networks and the problem of inferring functional groups or communities from the topology of the network we use the cavity method of statistical physics to obtain an asymptotically exact analysis of the phase diagram we describe in detail properties of the detectability undetectability phase transition and the easy hard phase transition for the community detection problem our analysis translates naturally into a belief propagation algorithm for inferring the group memberships of the nodes in an optimal way i e that maximizes the overlap with the underlying group memberships and learning the underlying parameters of the block model finally we apply the algorithm to two examples of real world networks and discuss its performance

approximate maximum likelihood hyperparameter estimation for gibbs priors the parameters of the prior the hyperparameters play an important role in bayesian image estimation of particular importance for the case of gibbs priors is the global hyperparameter spl beta which multiplies the hamiltonian here we consider maximum likelihood ml estimation of spl beta from incomplete data i e problems in which the image which is drawn from a gibbs prior is observed indirectly through some degradation or blurring process important applications include image restoration and image reconstruction from projections exact ml estimation of spl beta from incomplete data is intractable for most image processing here we present an approximate ml estimator that is computed simultaneously with a maximum a posteriori map image estimate the algorithm is based on a mean field approximation technique through which multidimensional gibbs distributions are approximated by a separable function equal to a product of one dimensional 1 d densities we show how this approach can be used to simplify the ml estimation problem we also show how the gibbs bogoliubov feynman gbf bound can be used to optimize the approximation for a restricted class of problems we present the results of a monte carlo study that examines the bias and variance of this estimator when applied to image restoration

boundary detection by constrained optimization a statistical framework is used for finding boundaries and for partitioning scenes into homogeneous regions the model is a joint probability distribution for the array of pixel gray levels and an array of labels in boundary finding the labels are binary zero or one representing the absence or presence of boundary elements in partitioning the label values are generic two labels are the same when the corresponding scene locations are considered to belong to the same region the distribution incorporates a measure of disparity between certain spatial features of block pairs of pixel gray levels using the kolmogorov smirnov nonparametric measures of difference between the distributions of these features the number of model parameters is minimized by forbidding label configurations which are assigned probability zero the maximum a posteriori estimator of boundary placements and partitionings is examined the forbidden states introduce constraints into the calculation of these configurations stochastic relaxation methods are extended to accommodate constrained optimization

determination of parameters in an image recovery by statistical mechanical means abstract a method of an image recovery was proposed in a preceding paper where the determination of a pair of parameters was not definite methods of determining the parameters are proposed

a practical bayesian framework for backpropagation networks a quantitative and practical bayesian framework is described for learning of mappings in feedforward networks the framework makes possible 1 objective comparisons between solutions using alternative network architectures 2 objective stopping rules for network pruning or growing procedures 3 objective choice of magnitude and type of weight decay terms or additive regularizers for penalizing large weights etc 4 a measure of the effective number of well determined parameters in a model 5 quantified estimates of the error bars on network parameters and on network output and 6 objective comparisons with alternative learning and interpolation models such as splines and radial basis functions the bayesian evidence automatically embodies occam s razor penalizing overflexible and overcomplex models the bayesian approach helps detect poor underlying assumptions in learning models for learning models well matched to a problem a good correlation between generalization ability and the bayesian evidence is obtained

bayesian and regularization methods for hyperparameter estimation in image restoration in this paper we propose the application of the hierarchical bayesian paradigm to the image restoration problem we derive expressions for the iterative evaluation of the two hyperparameters applying the evidence and maximum a posteriori map analysis within the hierarchical bayesian paradigm we show analytically that the analysis provided by the evidence approach is more realistic and appropriate than the map approach for the image restoration problem we furthermore study the relationship between the evidence and an iterative approach resulting from the set theoretic regularization approach for estimating the two hyperparameters or their ratio defined as the regularization parameter finally the proposed algorithms are tested experimentally

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

bayes smoothing algorithms for segmentation of binary images modeled by markov random fields a new image segmentation algorithm is presented based on recursive bayes smoothing of images modeled by markov random fields and corrupted by independent additive noise the bayes smoothing algorithm yields the a posteriori distribution of the scene value at each pixel given the total noisy image in a recursive way the a posteriori distribution together with a criterion of optimality then determine a bayes estimate of the scene the algorithm presented is an extension of a 1 d bayes smoothing algorithm to 2 d and it gives the optimum bayes estimate for the scene value at each pixel computational concerns in 2 d however necessitate certain simplifying assumptions on the model and approximations on the implementation of the algorithm in particular the scene noiseless image is modeled as a markov mesh random field a special class of markov random fields and the bayes smoothing algorithm is applied on overlapping strips horizontal vertical of the image consisting of several rows columns it is assumed that the signal scene values vector sequence along the strip is a vector markov chain since signal correlation in one of the dimensions is not fully used along the edges of the strip estimates are generated only along the middle sections of the strips the overlapping strips are chosen such that the union of the middle sections of the strips gives the whole image the bayes smoothing algorithm presented here is valid for scene random fields consisting of multilevel discrete or continuous random variables

probabilistic image processing by means of the bethe approximation for the q ising model the framework of bayesian image restoration for multi valued images by means of the q ising model with nearest neighbour interactions is presented hyperparameters in the probabilistic model are determined so as to maximize the marginal likelihood a practical algorithm is described for multi valued image restoration based on the bethe approximation the algorithm corresponds to loopy belief propagation in artificial intelligence we conclude that in real world grey level images the q ising model can give us good results

stable propagation algorithm for the minimization of the bethe free energy we propose a new propagation algorithm for the minimization of the bethe free energy for a generic lattice model with pair interactions the algorithm turns out to be more stable than belief propagation as it reaches a fixed point also for highly frustrated systems such as spin glasses and faster than the provably convergent double loop algorithms

the application of mean field theory to image motion estimation previously markov random field mrf model based techniques have been proposed for image motion estimation since motion estimation is usually an ill posed problem various constraints are needed to obtain a unique and stable solution the main advantage of the mrf approach is its capacity to incorporate such constraints for instance motion continuity within an object and motion discontinuity at the boundaries between objects in the mrf approach motion estimation is often formulated as an optimization problem and two frequently used optimization methods are simulated annealing sa and iterative conditional mode icm although the sa is theoretically optimal in the sense of finding the global optimum it usually takes many iterations to converge the icm on the other hand converges quickly but its results are often unsatisfactory due to its hard decision nature previously the authors have applied the mean field theory to image segmentation and image restoration problems it provides results nearly as good as sa but with much faster convergence the present paper shows how the mean field theory can be applied to mrf model based motion estimation this approach is demonstrated on both synthetic and real world images where it produced good motion estimates

bayesian estimation for homogeneous and inhomogeneous gaussian random fields this paper investigates bayesian estimation for gaussian markov random fields in particular a new class of compound model is proposed which describes the observed intensities using an inhomogeneous model and the degree of spatial variation described by a second random field the coupled markov random fields are used as prior distributions and combined with gaussian noise models to produce posterior distributions on which estimation is based all model parameters are estimated in a fully bayesian setting using the metropolis hasting algorithm the full posterior estimation procedures are illustrated and compared using various artificial examples for these examples the inhomogeneous model performs very favorably when compared to the homogeneous model allowing differential degrees of smoothing and varying local textures

maximum likelihood parameter estimation for unsupervised stochastic model based image segmentation an unsupervised stochastic model based approach to image segmentation is described and some of its properties investigated in this approach the problem of model parameter estimation is formulated as a problem of parameter estimation from incomplete data and the expectation maximization em algorithm is used to determine a maximum likelihood ml estimate previously the use of the em algorithm in this application has encountered difficulties since an analytical expression for the conditional expectations required in the em procedure is generally unavailable except for the simplest models in this paper two solutions are proposed to solve this problem a monte carlo scheme and a scheme related to besag s 1986 iterated conditional mode icm method both schemes make use of markov random field modeling assumptions examples are provided to illustrate the implementation of the em algorithm for several general classes of image models experimental results on both synthetic and real images are provided

constrained restoration and the recovery of discontinuities the linear image restoration problem is to recover an original brightness distribution x sup 0 given the blurred and noisy observations y kx sup 0 b where k and b represent the point spread function and measurement error respectively this problem is typical of ill conditioned inverse problems that frequently arise in low level computer vision a conventional method to stabilize the problem is to introduce a priori constraints on x sup 0 and design a cost functional h x over images x which is a weighted average of the prior constraints regularization term and posterior constraints data term the reconstruction is then the image x which minimizes h a prominent weakness in this approach especially with quadratic type stabilizers is the difficulty in recovering discontinuities the authors therefore examine prior smoothness constraints of a different form which permit the recovery of discontinuities without introducing auxiliary variables for marking the location of jumps and suspending the constraints in their vicinity in this sense discontinuities are addressed implicitly rather than explicitly

cluster expansions for the deterministic computation of bayesian estimators based on markov random fields we describe a family of approximations denoted by cluster approximations for the computation of the mean of a markov random field mrf this is a key computation in image processing when applied to the a posteriori mrf the approximation is to account exactly for only spatially local interactions application of the approximation requires the solution of a nonlinear multivariable fixed point equation for which we prove several existence uniqueness and convergence of algorithm results four numerical examples are presented including comparison with monte carlo calculations

statistical mechanical approach to image processing the basic frameworks and techniques of the bayesian approach to image restoration are reviewed from the statistical mechanical point of view first a few basic notions in digital image processing are explained to convince the reader that statistical mechanics has a close formal similarity to this problem second the basic formulation of the statistical estimation from the observed degraded image by using the bayes formula is demonstrated the relationship between bayesian statistics and statistical mechanics is also listed particularly it is explained that some correlation inequalities on the nishimori line of the random spin model also play an important role in bayesian image restoration third the framework of bayesian image restoration for binary images by means of the ising model is reviewed some practical algorithms for binary image restoration are given by employing the mean field and the bethe approximations finally bayesian image restoration for a grey level image using the gaussian model is reviewed and the gaussian model is extended to a more practical probabilistic model by introducing the line state to treat the effects of edges the line state is also extended to quantized values

statistical mechanical evaluation of a spread spectrum watermarking model with image restoration in cases in which an original image is blind a decoding method where both the image and the messages can be estimated simultaneously is desirable we propose a spread spectrum watermarking model with image restoration based on bayes estimation we therefore need to assume some prior probabilities the probability for estimating the messages is given by the uniform distribution and the ones for the image are given by the infinite range model and two dimensional 2d ising model any attacks from unauthorized users can be represented by channel models we can obtain the estimated messages and image by maximizing the posterior probability we analyzed the performance of the proposed method by the replica method in the case of the infinite range model we first calculated the theoretical values of the bit error rate from obtained saddle point equations and then verified them by computer simulations for this purpose we assumed that the image is binary and is generated from a given prior probability we also assume that attacks can be represented by the gaussian channel the computer simulation retults agreed with the theoretical values in the case of prior probability given by the 2d ising model in which each pixel is statically connected with four neighbors we evaluated the decoding performance by computer simulations since the replica theory could not be applied results using the 2d ising model showed that the proposed method with image restoration is as effective as the infinite range model for decoding messages we compared the performances in a case in which the image was blind and one in which it was informed the difference between these cases was small as long as the embedding and attack rates were small this demonstrates that the proposed method with simultaneous estimation is effective as a watermarking decoder

compound gauss markov random fields for image estimation algorithms for obtaining approximations to statistically optimal estimates for images modeled as compound gauss markov random fields are discussed the authors consider both the maximum a posteriori probability map estimate and the minimum mean squared error mmse estimate for image estimation and image restoration compound image models consist of several submodels having different characteristics along with an underlying structure model which govern transitions between these image submodels two different compound random field models are employed the doubly stochastic gaussian dsg random field and a compound gauss markov cgm random field the authors present map estimators for dsg and cgm random fields using simulated annealing a fast converging algorithm called deterministic relaxation which however converges to only a locally optimal map estimate is also presented as an alternative for reducing computational loading on sequential machines for comparison purposes the authors include results on the fixed lag smoothing mmse estimator for the dsg field and its suboptimal m algorithm approximation

parameter estimation in markov random field contextual models using geometric models of objects we present a new scheme for the estimation of markov random field line process parameters which uses geometric cad models of the objects in the scene the models are used to generate synthetic images of the objects from random view points the edge maps computed from the synthesized images are used as training samples to estimate the line process parameters using a least squares method we show that this parameter estimation method is useful for detecting edges in range as well as intensity edges the main contributions of the paper are 1 use of cad models to obtain true edge labels which are otherwise not available and 2 use of canonical markov random field representation to reduce the number of parameters

non monotonic behaviour in relaxation dynamics of image restoration we have investigated the relaxation dynamics of image restoration through a bayesian approach the relaxation dynamics is much faster at zero temperature than at the nishimori temperature where the pixel wise error rate is minimized in equilibrium at low temperature we observed non monotonic development of the overlap we suggest that the optimal performance is realized through premature termination in the relaxation processes in the case of the infinite range model we also performed markov chain monte carlo simulations to clarify the underlying mechanism of non trivial behaviour at low temperature by checking the local field distributions of each pixel

simulated annealing in compound gaussian random fields image processing recently a stochastic relaxation technique called simulated annealing has been developed to search for a globally optimal solution in image estimation and restoration problems the convergence of simulated annealing has been proved only for random fields with a compact range space because of this images were modeled as random fields with bounded discrete or continuous values however in most image processing problems it is more natural to model the image as a random field with values in a noncompact space e g conditional gaussian models the proof of convergence of the stochastic relaxation method is extended to a class of compound gauss markov random fields simulation results are provided to show the power of these methods

distribution estimation of hyperparameters in markov random field models we developed a method of distribution estimation of hyperparameters in markov random field mrf models this study was motivated by the growing quantity of image data in natural sciences owing to recent advances in measurement techniques mrf models are used to restore images in information science and the hyperparameters of these models can be adjusted to improve restoration performance the parameters appearing in data analysis represent physical quantities such as diffusion coefficients indeed many frameworks of hyperparameter estimation have been proposed but most are point estimation that is susceptible to stochastic fluctuations distribution estimation can be used to evaluate the confidence one has in point estimates of hyperparameters in a similar way to physicists using error bars when they evaluate important physical quantities we use a solvable mrf model to investigate the performance of distribution estimation in simulations

the nishimori line and bayesian statistics the nishimori line is a line or hypersurface in the parameter space of systems with quenched disorder where simple expressions of the averages of physical quantities over the quenched random variables are obtained it has been playing an important role in the theoretical studies of the random frustrated systems since its discovery in around 1980 in this paper an interpretation of the nishimori line from the viewpoint of statistical information processing is developed our main aim is the reconstruction of the whole theory of the nishimori line from the viewpoint of bayesian statistics or almost equivalently from the viewpoint of the theory of error correcting codes as a byproduct of the interpretation counterparts of the nishimori line in models without gauge invariance are given we also discussed the issues on the finite temperature decoding of error correcting codes and clarify the role of gauge invariance in this topic

mean field annealing using compound gauss markov random fields for edge detection and image estimation the authors consider the problem of edge detection and image estimation in nonstationary images corrupted by additive gaussian noise the noise free image is represented using the compound gauss markov random field developed by f c jeng and j w woods 1990 and the problem of image estimation and edge detection is posed as a maximum a posteriori estimation problem since the a posteriori probability function is nonconvex computationally intensive stochastic relaxation algorithms are normally required a deterministic relaxation method based on mean field annealing with a compound gauss markov random cgmrf field model is proposed the authors present a set of iterative equations for the mean values of the intensity and both horizontal and vertical line processes with or without taking into account some interaction between them the relationship between this technique and two other methods is considered edge detection and image estimation results on several noisy images are included

cluster variation method in statistical physics and probabilistic graphical models the cluster variation method cvm is a hierarchy of approximate variational techniques for discrete ising like models in equilibrium statistical mechanics improving on the mean field approximation and the bethe peierls approximation which can be regarded as the lowest level of the cvm in recent years it has been applied both in statistical physics and to inference and optimization problems formulated in terms of probabilistic graphical models the foundations of the cvm are briefly reviewed and the relations with similar techniques are discussed the main properties of the method are considered with emphasis on its exactness for particular models and on its asymptotic properties the problem of the minimization of the variational free energy which arises in the cvm is also addressed and recent results about both provably convergent and message passing algorithms are discussed

mean field theory of em algorithm for bayesian grey scale image restoration the em algorithm for the bayesian grey scale image restoration is investigated in the framework of the mean field theory our model system is identical to the infinite range random field ising model the maximum marginal likelihood method is applied to the determination of hyper parameters we calculate both the data averaged mean square error between the original image and its maximizer of posterior marginal estimate and the data averaged marginal likelihood function exactly after evaluating the hyper parameter dependence of the data averaged marginal likelihood function we derive the em algorithm which updates the hyper parameters to obtain the maximum likelihood estimate analytically the time evolutions of the hyper parameters and so called q function are obtained the relation between the speed of convergence of the hyper parameters and the shape of the q function is explained from the viewpoint of dynamics

parallel and deterministic algorithms from mrfs surface reconstruction deterministic approximations to markov random field mrf models are derived one of the models is shown to give in a natural way the graduated nonconvexity gnc algorithm proposed by a blake and a zisserman 1987 this model can be applied to smooth a field preserving its discontinuities a class of more complex models is then proposed in order to deal with a variety of vision problems all the theoretical results are obtained in the framework of statistical mechanics and mean field techniques a parallel iterative algorithm to solve the deterministic equations of the two models is presented together with some experiments on synthetic and real images

infinitely imbalanced logistic regression in binary classification problems it is common for the two classes to be imbalanced one case is very rare compared to the other in this paper we consider the infinitely imbalanced case where one class has a finite sample size and the other class s sample size grows without bound for logistic regression the infinitely imbalanced case often has a useful solution under mild conditions the intercept diverges as expected but the rest of the coefficient vector approaches a non trivial and useful limit that limit can be expressed in terms of exponential tilting and is the minimum of a convex objective function the limiting form of logistic regression suggests a computational shortcut for fraud detection problems

uniform coverage designs for molecule selection in screening for drug discovery chemists often select a large subset of molecules from a very large database e g select 1 000 molecules from 100 000 to generate diverse leads for drug optimization highly active compounds in several structurally different chemical classes are sought molecules can be characterized by numerical descriptors and the chosen subset should cover the descriptor space or subspaces formed by several descriptors we propose a method that concentrates on low dimensional subspaces a criterion for uniformity of coverage and a fast exchange algorithm to optimize the criterion these methods are illustrated by using a national cancer institute database

examples of renormalization group transformations for image sets using the example of configurations generated with the worm algorithm for the two dimensional ising model we propose renormalization group rg transformations inspired by the tensor rg that can be applied to sets of images we relate criticality to the logarithmic divergence of the largest principal component we discuss the changes in link occupation under the rg transformation suggest ways to obtain data collapse and compare with the two state tensor rg approximation near the fixed point

dynamics of the maximum marginal likelihood hyperparameter estimation in image restoration gradient descent versus expectation and maximization algorithm dynamical properties of image restoration and hyperparameter estimation are investigated by means of statistical mechanics we introduce an exactly solvable model for image restoration and derive differential equations with respect to macroscopic quantities from these equations we evaluate relaxation processes of the system to the equilibrium state our statistical mechanical approach also enables us to investigate the hyperparameter estimation by means of maximization of the marginal likelihood by using gradient descent and the expectation and maximization algorithm from the dynamical point of view

maximum likelihood estimation methods for multispectral random field image models considers the problem of estimating parameters of multispectral random field rf image models using maximum likelihood ml methods for images with an assumed gaussian distribution analytical results are developed for multispectral simultaneous autoregressive msar and markov random field mmrf models which lead to practical procedures for calculating ml estimates although previous work has provided least squares methods for parameter estimation the superiority of the ml method is evidenced by experimental results provided in this work the effectiveness of multispectral rf models using ml estimates in modeling color texture images is also demonstrated

sufficient conditions for convergence of the sum product algorithm novel conditions are derived that guarantee convergence of the sum product algorithm also known as loopy belief propagation or simply belief propagation bp to a unique fixed point irrespective of the initial messages for parallel synchronous updates the computational complexity of the conditions is polynomial in the number of variables in contrast with previously existing conditions our results are directly applicable to arbitrary factor graphs with discrete variables and are shown to be valid also in the case of factors containing zeros under some additional conditions the conditions are compared with existing ones numerically and if possible analytically for binary variables with pairwise interactions sufficient conditions are derived that take into account local evidence i e single variable factors and the type of pair interactions attractive or repulsive it is shown empirically that this bound outperforms existing bounds

replica inference approach to unsupervised multiscale image segmentation we apply a replica inference based potts model method to unsupervised image segmentation on multiple scales this approach was inspired by the statistical mechanics problem of community detection and its phase diagram specifically the problem is cast as identifying tightly bound clusters communities or solutes against a background or solvent within our multiresolution approach we compute information theory based correlations among multiple solutions replicas of the same graph over a range of resolutions significant multiresolution structures are identified by replica correlations manifest by information theory overlaps we further employ such information theory measures such as normalized mutual information and variation of information thermodynamic quantities such as the system entropy and energy and dynamic measures monitoring the convergence time to viable solutions as metrics for transitions between various solvable and unsolvable phases within the solvable phase transitions between contending solutions such as those corresponding to segmentations on different scales may also appear with the aid of these correlations as well as thermodynamic measures the phase diagram of the corresponding potts model is analyzed at both zero and finite temperatures optimal parameters corresponding to a sensible unsupervised segmentations appear within the easy phase of the potts model our algorithm is fast and shown to be at least as accurate as the best algorithms to date and to be especially suited to the detection of camouflaged images

fuzziness in valuing financial instruments by certainty equivalents abstract our aim is to use the utility theory of von neumann and morgenstern and the approach of certainty equivalent c e in a fuzzy framework for pricing a future uncertain amount the fuzzy theory is more adaptable to represent this problem because the financial evaluations depend on perspectives which are fuzzy known then we define a seller and buyer fuzzy financial c e

statistical mechanics of image restoration and error correcting codes we develop a statistical mechanical formulation for image restoration and error correcting codes these problems are shown to be equivalent to the ising spin glass with ferromagnetic bias under random external fields we prove that the quality of restoration decoding is maximized at a specific set of parameter values determined by the source and channel properties for image restoration in a mean field system a line of optimal performance is shown to exist in the parameter space these results are illustrated by solving exactly the infinite range model the solutions enable us to determine how precisely one should estimate unknown parameters monte carlo simulations are carried out to see how far the conclusions from the infinite range model are applicable to the more realistic two dimensional case in image restoration