a polynomial time approximation algorithm for the permanent of a matrix with nonnegative entries we present a polynomial time randomized algorithm for estimating the permanent of an arbitrary n n matrix with nonnegative entries this algorithm technically a fully polynomial randomized approximation scheme computes an approximation that is with high probability within arbitrarily small specified relative error of the true value of the permanent

on the wave front solution of a competition diffusion system in population dynamics abstract in this paper we consider a competition diffusion system of two equations zhou and pao asymptotic behavior of a competition diffusion system in population dynamics nonlinear anal 6 11 1982 1163 1184 the diffusion coefficients of the system are not equal we prove existence of a wave front solution which connects two nonzero restpoints of the system in the proof we rely essentially on the results of kolmogorov et al a study of diffusion with increase in the quantity of matter and its application to a biological problem bull moscow state univ 17 1937 1 72 we also estimate the wave speed

the bethe permanent of a nonnegative matrix it has recently been observed that the permanent of a nonnegative square matrix i e of a square matrix containing only nonnegative real entries can very well be approximated by solving a certain bethe free energy function minimization problem with the help of the sum product algorithm we call the resulting approximation of the permanent the bethe permanent in this paper we give reasons why this approach to approximating the permanent works well namely we show that the bethe free energy function is convex and that the sum product algorithm finds its minimum efficiently we then discuss the fact that the permanent is lower bounded by the bethe permanent and we comment on potential upper bounds on the permanent based on the bethe permanent we also present a combinatorial characterization of the bethe permanent in terms of permanents of so called lifted versions of the matrix under consideration moreover we comment on possibilities to modify the bethe permanent so that it approximates the permanent even better and we conclude the paper with some observations and conjectures about permanent based pseudocodewords and permanent based kernels

belief propagation and loop calculus for the permanent of a non negative matrix we consider computation of the permanent of a positive n n non negative matrix p pji i j 1 n or equivalently the problem of weighted counting of the perfect matchings over the complete bipartite graph kn n the problem is known to be of likely exponential complexity stated as the partition function z of a graphical model the problem allows for exact loop calculus representation chertkov m and chernyak v 2006 phys rev e 72 065102 in terms of an interior minimum of the bethe free energy functional over non integer doubly stochastic matrix of marginal beliefs ji i j 1 n also correspondent to a fixed point of the iterative message passing algorithm of the belief propagation bp type our main result is an explicit expression of the exact partition function permanent in terms of the matrix of bp marginals as z perm p zbp perm ji 1 ji i j 1 ji where zbp is the bp expression for the permanent stated explicitly in terms of we give two derivations of the formula a direct one based on the bethe free energy and an alternative one combining the ihara graph function and the loop calculus approaches assuming that the matrix of the bp marginals is calculated we provide two lower bounds and one upper bound to estimate the multiplicative term two complementary lower bounds are based on the gurvits van der waerden theorem and on a relation between the modified permanent and determinant respectively

linear estimation of boundary value stochastic processes part i the role and construction of complementary models this paper presents a substantial extension of the method of complementary models for minimum variance linear estimation introduced by weinert and desai in their important paper 1 specifically the method of complementary models is extended to solve estimation problems for both discrete and continuous parameter linear boundary value stochastic processes in one and higher dimensions a major contribution of this paper is an application of green s identity in deriving a differential operator representation of the estimator to clarify the development and to illustrate the range of applications of our approach two brief examples are provided one is a 1 d discrete two point boundary value process and the other is a 2 d process governed by poisson s equation on the unit disk

a monte carlo algorithm for estimating the permanent let a be an n times n matrix with 0 1 valued entries and let operatorname per a be the permanent of a this paper describes a monte carlo algorithm that produces a good in the relative sense estimate of operatorname per a and has running time operatorname poly n 2 n 2 where operatorname poly n denotes a function that grows polynomially with n

factor graphs for quantum probabilities a factor graph representation of quantum mechanical probabilities involving any number of measurements is proposed unlike standard statistical models the proposed representation uses auxiliary variables state variables that are not random variables all joint probability distributions are marginals of some complex valued function q and it is demonstrated how the basic concepts of quantum mechanics relate to factorizations and marginals of q

on the number of matrices and a random matrix with prescribed row and column sums and 0 1 entries abstract we consider the set r c of all m n matrices having 0 1 entries and prescribed row sums r r 1 r m and column sums c c 1 c n we prove an asymptotic estimate for the cardinality r c via the solution to a convex optimization problem we show that if r c is sufficiently large then a random matrix d r c sampled from the uniform probability measure in r c with high probability is close to a particular matrix z z r c that maximizes the sum of entropies of entries among all matrices with row sums r column sums c and entries between 0 and 1 similar results are obtained for 0 1 matrices with prescribed row and column sums and assigned zeros in some positions

smoothing for random fields modeled by partial differential equations the authors present an efficient numerically reliable smoothing algorithm for random fields modeled by linear constant coefficient partial differential equations the estimate is computed from discrete measurements by using the discrete fourier transform to convert the two dimensional 2d problem to a collection of uncoupled one dimensional 1d problems which are then solved using stable iterations

norm product belief propagation primal dual message passing for approximate inference inference problems in graphical models can be represented as a constrained optimization of a free energy function in this paper we treat both forms of probabilistic inference estimating marginal probabilities of the joint distribution and finding the most probable assignment through a unified message passing algorithm architecture in particular we generalize the belief propagation bp algorithms of sum product and max product and tree reweighted trw sum and max product algorithms trbp and introduce a new set of convergent algorithms based on convex free energy and linear programming lp relaxation as a zero temperature of a convex free energy the main idea of this work arises from taking a general perspective on the existing bp and trbp algorithms while observing that they all are reductions from the basic optimization formula of f ihi where the function f is an extended valued strictly convex but nonsmooth and the functions hi are extended valued functions not necessarily convex we use tools from convex duality to present the primal dual ascent algorithm which is an extension of the bregman successive projection scheme and is designed to handle optimization of the general type f ihi we then map the fractional free energy variational principle for approximate inference onto the optimization formula above and introduce the norm product message passing algorithm special cases of the norm product include sum product and max product bp algorithms trbp and nmplp algorithms when the fractional free energy is set to be convex convex free energy the norm product is globally convergent for the estimation of marginal probabilities and for approximating the lp relaxation we also introduce another branch of the norm product which arises as the zero temperature of the convex free energy which we refer to as the convex max product the convex max product is convergent unlike max product and aims at solving the lp relaxation

constructing free energy approximations and generalized belief propagation algorithms important inference problems in statistical physics computer vision error correcting coding theory and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs the belief propagation bp algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree but only approximate when the factor graph has cycles we show that bp fixed points correspond to the stationary points of the bethe approximation of the free energy for a factor graph we explain how to obtain region based free energy approximations that improve the bethe approximation and corresponding generalized belief propagation gbp algorithms we emphasize the conditions a free energy approximation must satisfy in order to be a valid or maxent normal approximation we describe the relationship between four different methods that can be used to generate valid approximations the bethe method the junction graph method the cluster variation method and the region graph method finally we explain how to tell whether a region based approximation and its corresponding gbp algorithm is likely to be accurate and describe empirical results showing that gbp can significantly outperform bp

estimating a probability mass function with unknown labels in the context of a species sampling problem we discuss a nonparametric maximum likelihood estimator for the underlying probability mass function the estimator is known in the computer science literature as the high profile estimator we prove strong consistency and derive the rates of convergence for an extended model version of the estimator we also study a sieved estimator for which similar consistency results are derived numerical computation of the sieved estimator is of great interest for practical problems such as forensic dna analysis and we present a computational algorithm based on the stochastic approximation of the expectation maximisation algorithm as an interesting byproduct of the numerical analyses we introduce an algorithm for bounded isotonic regression for which we also prove convergence less

internal models and recursive estimation for 2 d isotropic random fields efficient recursive smoothing algorithms are developed for isotropic random fields that can be obtained by passing white noise through rational filters the estimation problem is shown to be equivalent to a countably infinite set of 1 d separable two point boundary value smoothing problems the 1 d smoothing problems are solved using a markovianization approach followed by a standard 1 d smoothing algorithm the desired field estimate is then obtained as properly weighted sum of the 1 d smoothed estimates the 1 d two point boundary value problems are also shown to have the same asymptotic properties and yield a stable spectral factorization of the power spectrum of the isotropic random fields

loop calculus for nonbinary alphabets using concepts from information geometry the bethe approximation is a well known approximation of the partition function used in statistical physics recently an equality relating the partition function and its bethe approximation was obtained for graphical models with binary variables by chertkov and chernyak in this equality the multiplicative error in the bethe approximation is represented as a weighted sum over all generalized loops in the graphical model in this paper the equality is generalized to graphical models with nonbinary alphabet using concepts from information geometry

on the optimality of solutions of the max product belief propagation algorithm in arbitrary graphs graphical models such as bayesian networks and markov random fields mrfs represent statistical dependencies of variables by a graph the max product belief propagation algorithm is a local message passing algorithm on this graph that is known to converge to a unique fixed point when the graph is a tree furthermore when the graph is a tree the assignment based on the fixed point yields the most probable values of the unobserved variables given the observed ones good empirical performance has been obtained by running the max product algorithm or the equivalent min sum algorithm on graphs with loops for applications including the decoding of turbo codes except for two simple graphs cycle codes and single loop graphs there has been little theoretical understanding of the max product algorithm on graphs with loops here we prove a result on the fixed points of max product on a graph with arbitrary topology and with arbitrary probability distributions discrete or continuous valued nodes we show that the assignment based on a fixed point is a neighborhood maximum of the posterior probability the posterior probability of the max product assignment is guaranteed to be greater than all other assignments in a particular large region around that assignment the region includes all assignments that differ from the max product assignment in any subset of nodes that form no more than a single loop in the graph in some graphs this neighborhood is exponentially large we illustrate the analysis with examples

lp decoding of regular ldpc codes in memoryless channels we study error bounds for linear programming decoding of regular low density parity check ldpc codes for memoryless binary input output symmetric channels we prove bounds on the word error probability that are inverse doubly exponential in the girth of the factor graph for memoryless binary input awgn channel we prove lower bounds on the threshold for regular ldpc codes whose factor graphs have logarithmic girth under lp decoding specifically we prove a lower bound of 0 735 upper bound of eb n0 2 67 db on the threshold of 3 6 regular ldpc codes whose factor graphs have logarithmic girth our proof is an extension of a recent paper of arora daskalakis and steurer stoc 2009 who presented a novel probabilistic analysis of lp decoding over a binary symmetric channel their analysis is based on the primal lp representation and has an explicit connection to message passing algorithms we extend this analysis to any mbios channel

maxentropic markov chains corresp the markov chain that has maximum entropy for given first and second moments is determined the solution provides a discrete analog to the continuous gauss markov process

walk sums and belief propagation in gaussian graphical models we present a new framework based on walks in a graph for analysis and inference in gaussian graphical models the key idea is to decompose the correlation between each pair of variables as a sum over all walks between those variables in the graph the weight of each walk is given by a product of edgewise partial correlation coefficients this representation holds for a large class of gaussian graphical models which we call walk summable we give a precise characterization of this class of models and relate it to other classes including diagonally dominant attractive non frustrated and pairwise normalizable we provide a walk sum interpretation of gaussian belief propagation in trees and of the approximate method of loopy belief propagation in graphs with cycles the walk sum perspective leads to a better understanding of gaussian belief propagation and to stronger results for its convergence in loopy graphs

counting in graph covers a combinatorial characterization of the bethe entropy function we present a combinatorial characterization of the bethe entropy function of a factor graph such a characterization being in contrast to the original analytical definition of this function we achieve this combinatorial characterization by counting valid configurations in finite graph covers of the factor graph analogously we give a combinatorial characterization of the bethe partition function whose original definition was also of an analytical nature as we point out our approach has similarities to the replica method but also stark differences the above findings are a natural backdrop for introducing a decoder for graph based codes that we will call symbolwise graph cover decoding a decoder that extends our earlier work on blockwise graph cover decoding both graph cover decoders are theoretical tools that help toward a better understanding of message passing iterative decoding namely blockwise graph cover decoding links max product min sum algorithm decoding with linear programming decoding and symbolwise graph cover decoding links sum product algorithm decoding with bethe free energy function minimization at temperature one in contrast to the gibbs entropy function which is a concave function the bethe entropy function is in general not concave everywhere in particular we show that every code picked from an ensemble of regular low density parity check codes with minimum hamming distance growing with high probability linearly with the block length has a bethe entropy function that is convex in certain regions of its domain

approximating the permanent with fractional belief propagation we discuss schemes for exact and approximate computations of permanents and compare them with each other specifically we analyze the belief propagation bp approach and its fractional belief propagation fbp generalization for computing the permanent of a non negative matrix known bounds and conjectures are verified in experiments and some new theoretical relations bounds and conjectures are proposed the fractional free energy ffe function is parameterized by a scalar parameter 1 1 where 1 corresponds to the bp limit and 1 corresponds to the exclusion principle but ignoring perfect matching constraints mean field mf limit ffe shows monotonicity and continuity with respect to for every non negative matrix we define its special value 1 0 to be the for which the minimum of the parameterized ffe function is equal to the permanent of the matrix where the lower and upper bounds of the interval corresponds to respective bounds for the permanent our experimental analysis suggests that the distribution of varies for different ensembles but always lies within the 1 1 2 interval moreover for all ensembles considered the behavior of is highly distinctive offering an empirical practical guidance for estimating permanents of non negative matrices via the ffe approach

max product for maximum weight matching convergence correctness and lp duality max product belief propagation bp is an iterative message passing algorithm for finding the maximum a posteriori map assignment of a discrete probability distribution specified by a graphical model despite the spectacular success of the algorithm in many application areas such as iterative decoding and combinatorial optimization which involve graphs with many cycles theoretical results about both the correctness and convergence of the algorithm are known in only a few cases see section i for references in this paper we prove the correctness and convergence of max product for finding the maximum weight matching mwm in bipartite graphs even though the underlying graph of the mwm problem has many cycles somewhat surprisingly we show that the max product algorithm converges to the correct mwm as long as the mwm is unique we provide a bound on the number of iterations required and show that for a graph of size n the computational cost of the algorithm scales as o n3 which is the same as the computational cost of the best known algorithms for finding the mwm we also provide an interesting relation between the dynamics of the max product algorithm and the auction algorithm which is a well known distributed algorithm for solving the mwm problem

belief propagation and lp relaxation for weighted matching in general graphs loopy belief propagation has been employed in a wide variety of applications with great empirical success but it comes with few theoretical guarantees in this paper we analyze the performance of the max product form of belief propagation for the weighted matching problem on general graphs we show that the performance of max product is exactly characterized by the natural linear programming lp relaxation of the problem in particular we first show that if the lp relaxation has no fractional optima then max product always converges to the correct answer this establishes the extension of the recent result by bayati shah and sharma which considered bipartite graphs to general graphs perhaps more interestingly we also establish a tight converse namely that the presence of any fractional lp optimum implies that max product will fail to yield useful estimates on some of the edges we extend our results to the weighted b matching and r edge cover problems we also demonstrate how to simplify the max product message update equations for weighted matching making it easily deployable in distributed settings like wireless or sensor networks

a scalable algorithm for tracking an unknown number of targets using multiple sensors we propose an algorithm for tracking an unknown number of targets based on measurements provided by multiple sensors our algorithm achieves low computational complexity and excellent scalability by running belief propagation on a suitably devised factor graph a redundant formulation of data association uncertainty and the use of augmented target states including binary target indicators make it possible to exploit statistical independencies for a drastic reduction of complexity an increase in the number of targets sensors or measurements leads to additional variable nodes in the factor graph but not to higher dimensions of the messages as a consequence the complexity of our method scales only quadratically in the number of targets linearly in the number of sensors and linearly in the number of measurements per sensor the performance of the method compares well with that of previously proposed methods including methods with a less favorable scaling behavior in particular our method can outperform multisensor versions of the probability hypothesis density phd filter the cardinalized phd filter and the multi bernoulli filter

a randomized algorithm for the capacity of finite state channels inspired by ideas from the field of stochastic approximation we propose a randomized algorithm to compute the capacity of a finite state channel with a markovian input when the mutual information rate of the channel is concave with respect to the chosen parameterization the proposed algorithm proves to be convergent to the capacity of the channel almost surely with the derived convergence rate we also discuss the convergence behavior of the algorithm without the concavity assumption

on the uniqueness of loopy belief propagation fixed points we derive sufficient conditions for the uniqueness of loopy belief propagation fixed points these conditions depend on both the structure of the graph and the strength of the potentials and naturally extend those for convexity of the bethe free energy we compare them with a strengthened version of conditions derived elsewhere for pairwise potentials we discuss possible implications for convergent algorithms as well as for other approximate free energies

improved lower bounds on the size of balls over permutations with the infinity metric we study the size or volume of balls in the metric space of permutations s n under the infinity metric we focus on the regime of balls with radius r rho cdot n 1 rho in 0 1 i e a radius that is a constant fraction of the maximum possible distance we provide new lower bounds on the size of such balls these new lower bounds reduce the asymptotic gap to the known upper bounds to at most 0 029 bits per symbol additionally they imply an improved ball packing bound for error correcting codes and an improved upper bound on the size of optimal covering codes

an efficient tree decomposition method for permanents and mixed discriminants abstract we present an efficient algorithm to compute permanents mixed discriminants and hyperdeterminants of structured matrices and multidimensional arrays tensors we describe the sparsity structure of an array in terms of a graph and we assume that its treewidth denoted as is small our algorithm requires o n 2 arithmetic operations to compute permanents and o n 2 n 3 for mixed discriminants and hyperdeterminants we finally show that mixed volume computation continues to be hard under bounded treewidth assumptions

on traveling wave solutions in general reaction diffusion systems with time delays abstract we study the existence of traveling wave solutions in a general class of mixed quasi monotone reaction diffusion systems with time delays first by applying the schauder fixed point theorem we prove the existence of a traveling wave solution between classically defined upper and lower solutions for better applications of the upper lower solution method on various real life models the existence result is further extended under weak form or piecewise smooth upper lower solutions in several reaction diffusion systems with time delays single species logistic growth n species competition and ratio dependent predator prey with gompertz growth we apply our main result to establish the existence of traveling wave solutions flowing towards the positive or coexistent states under reasonable conditions on ecological parameters

sufficient conditions for convergence of the sum product algorithm novel conditions are derived that guarantee convergence of the sum product algorithm also known as loopy belief propagation or simply belief propagation bp to a unique fixed point irrespective of the initial messages for parallel synchronous updates the computational complexity of the conditions is polynomial in the number of variables in contrast with previously existing conditions our results are directly applicable to arbitrary factor graphs with discrete variables and are shown to be valid also in the case of factors containing zeros under some additional conditions the conditions are compared with existing ones numerically and if possible analytically for binary variables with pairwise interactions sufficient conditions are derived that take into account local evidence i e single variable factors and the type of pair interactions attractive or repulsive it is shown empirically that this bound outperforms existing bounds

a new class of upper bounds on the log partition function we introduce a new class of upper bounds on the log partition function of a markov random field mrf this quantity plays an important role in various contexts including approximating marginal distributions parameter estimation combinatorial enumeration statistical decision theory and large deviations bounds our derivation is based on concepts from convex duality and information geometry in particular it exploits mixtures of distributions in the exponential domain and the legendre mapping between exponential and mean parameters in the special case of convex combinations of tree structured distributions we obtain a family of variational problems similar to the bethe variational problem but distinguished by the following desirable properties i they are convex and have a unique global optimum and ii the optimum gives an upper bound on the log partition function this optimum is defined by stationary conditions very similar to those defining fixed points of the sum product algorithm or more generally any local optimum of the bethe variational problem as with sum product fixed points the elements of the optimizing argument can be used as approximations to the marginals of the original model the analysis extends naturally to convex combinations of hypertree structured distributions thereby establishing links to kikuchi approximations and variants

an analysis of belief propagation on the turbo decoding graph with gaussian densities motivated by its success in decoding turbo codes we provide an analysis of the belief propagation algorithm on the turbo decoding graph with gaussian densities in this context we are able to show that under certain conditions the algorithm converges and that somewhat surprisingly though the density generated by belief propagation may differ significantly from the desired posterior density the means of these two densities coincide since computation of posterior distributions is tractable when densities are gaussian use of belief propagation in such a setting may appear unwarranted indeed our primary motivation for studying belief propagation in this context stems from a desire to enhance our understanding of the algorithm s dynamics in a non gaussian setting and to gain insights into its excellent performance in turbo codes nevertheless even when the densities are gaussian belief propagation may sometimes provide a more efficient alternative to traditional inference methods