approximating bayesian belief networks by arc removal i propose a general framework for approximating bayesian belief networks through model simplification by arc removal given an upper bound on the absolute error allowed on the prior and posterior probability distributions of the approximated network a subset of arcs is removed thereby speeding up probabilistic inference

improving variational methods via pairwise linear response identities inference methods are often formulated as variational approximations these approximations allow easy evaluation of statistics by marginalization or linear response but these estimates can be inconsistent we show that by introducing constraints on covariance one can ensure consistency of linear response with the variational parameters and in so doing inference of marginal probability distributions is improved for the bethe approximation and its generalizations improvements are achieved with simple choices of the constraints the approximations are presented as variational frameworks iterative procedures related to message passing are provided for finding the minima

dynamical complexity detection in short term physiological series using base scale entropy physiological systems generate complex fluctuations in their output signals that reflect the underlying dynamics the base scale entropy method was proposed as a complexity measure to investigate the complexity of time series the advantages of this method are simplicity and extremely fast calculation for very short data sets this method enables analyzing very short nonstationary and noisy data sets we employed this method for short term physiological time series for analysis of heart rate variability signals the results show that the simple and easily calculated measure can effectively detect the complexity dissimilarity of physiological time series in different physiological or pathological states which is convenient for clinical applications

approximate counting the lovasz local lemma and inference in graphical models in this article we introduce a new approach to approximate counting in bounded degree systems with higher order constraints our main result is an algorithm to approximately count the number of solutions to a cnf formula when the width is logarithmic in the maximum degree this closes an exponential gap between the known upper and lower bounds moreover our algorithm extends straightforwardly to approximate sampling which shows that under lovasz local lemma like conditions it is not only possible to find a satisfying assignment it is also possible to generate one approximately uniformly at random from the set of all satisfying assignments our approach is a significant departure from earlier techniques in approximate counting and is based on a framework to bootstrap an oracle for computing marginal probabilities on individual variables finally we give an application of our results to show that it is algorithmically possible to sample from the posterior distribution in an interesting class of graphical models

an efficient numerical scheme for 1d parabolic singularly perturbed problems with an interior and boundary layers in this paper we consider a 1d parabolic singularly perturbed reaction convection diffusion problem which has a small parameter in both the diffusion term multiplied by the parameter e 2 and the convection term multiplied by the parameter in the differential equation e 0 1 0 1 e moreover the convective term degenerates inside the spatial domain and also the source term has a discontinuity of first kind on the degeneration line in general for sufficiently small values of the diffusion and the convection parameters the exact solution exhibits an interior layer in a neighborhood of the interior degeneration point and also a boundary layer in a neighborhood of both end points of the spatial domain we study the asymptotic behavior of the exact solution with respect to both parameters and we construct a monotone finite difference scheme which combines the implicit euler method defined on a uniform mesh to discretize in time together with the classical upwind finite difference scheme defined on an appropriate nonuniform mesh of shishkin type to discretize in space the numerical scheme converges in the maximum norm uniformly in e and having first order in time and almost first order in space illustrative numerical results corroborating in practice the theoretical results are showed

uniform convergence difference schemes for singularly perturbed mixed boundary problems in this paper we consider the conservative form of singularly perturbed ordinary differential equations with mixed boundary conditions a fitted mesh finite difference scheme is constructed for these problems the scheme is shown to be uniformly convergent with respect to the perturbed parameter a class of conservative difference schemes with uniform mesh are also considered these difference schemes are proved to be first order uniformly convergent the computed results for both cases are in good agreement with the exact solutions

approximating maps for belief networks is np hard and other theorems abstract finding maximum a posteriori map assignments also called most probable explanations is an important problem on bayesian belief networks shimony has shown that finding maps is np hard in this paper we show that approximating maps with a constant ratio bound is also np hard in addition we examine the complexity of two related problems which have been mentioned in the literature we show that given the map for a belief network and evidence set or the family of maps if the optimal is not unique it remains np hard to find or approximate alternative next best explanations furthermore we show that given the map or maps for a belief network and an initial evidence set it is also np hard to find or approximate the map assignment for the same belief network with a modified evidence set that differs from the initial set by the addition or removal of even a single node assignment finally we show that our main result applies to networks with constrained in degree and out degree applies to randomized approximation and even still applies if the ratio bound instead of being constant is allowed to be a polynomial function of various aspects of the network topology

uniformly convergent difference schemes for singularly perturbed parabolic diffusion convection problems without turning points we consider first the initial boundary value problem for the parabolic equation varepsilon y xx a x t y x b x t y d x t y t f x t on 0 1 0 t where 0 0 b 0 andd 0 bounds ony and its derivatives are obtained under certain compatibility assumptions on the data we examine a family of difference schemes for this problem which is exponentially fitted in thex variable and uses classical differencing in thet variable on rectangular grids which are arbitrarily spaced in both thex andt directions under a courant friedrichs lewy type condition the errors at the grid points are shown to be bounded byc h k whereh k is the maximum mesh width in thex t direction andc is a constant independent of h andk the corresponding result for a two point boundary value problem is also derived

approximating probabilistic inference in bayesian belief networks is np hard abstract it is known that exact computation of conditional probabilities in belief networks is np hard many investigators in the ai community have tacitly assumed that algorithms for performing approximate inference with belief networks are of polynomial complexity indeed special cases of approximate inference can be performed in time polynomial in the input size however we have discovered that the general problem of approximating conditional probabilities with belief networks like exact inference resides in the np hard complexity class we develop a complexity analysis to elucidate the difficulty of approximate probabilistic inference more specifically we show that the existence of a polynomial time relative approximation algorithm for major classes of problem instances implies that np p we present our proof and explore the implications of the result

singularly perturbed convection diffusion problems with boundary and weak interior layers in this paper a singularly perturbed convection diffusion equation with a discontinuous source term is examined boundary and weak interior layers appear in the solution a numerical method is constructed for this problem which involves an appropriate piecewise uniform mesh the method is shown to be uniformly convergent with respect to the singular perturbation parameter

an optimal approximation algorithm for bayesian inference abstract approximating the inference probability pr x x e e in any sense even for a single evidence node e is np hard this result holds for belief networks that are allowed to contain extreme conditional probabilities that is conditional probabilities arbitrarily close to 0 nevertheless all previous approximation algorithms have failed to approximate efficiently many inferences even for belief networks without extreme conditional probabilities we prove that we can approximate efficiently probabilistic inference in belief networks without extreme conditional probabilities we construct a randomized approximation algorithm the bounded variance algorithm that is a variant of the known likelihood weighting algorithm the bounded variance algorithm is the first algorithm with provably fast inference approximation on all belief networks without extreme conditional probabilities from the bounded variance algorithm we construct a deterministic approximation algorithm using current advances in the theory of pseudorandom generators in contrast to the exponential worst case behavior of all previous deterministic approximations the deterministic bounded variance algorithm approximates inference probabilities in worst case time that is subexponential 2 log n d for some integer d that is a linear function of the depth of the belief network

the computational complexity of probabilistic inference using bayesian belief networks research note abstract bayesian belief networks provide a natural efficient method for representing probabilistic dependencies among a set of variables for these reasons numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks more general classes of belief networks however have eluded efforts to develop efficient inference algorithms we show that probabilistic inference using belief networks is np hard therefore it seems unlikely that an exact algorithm can be developed to perform probabilistic inference efficiently over all classes of belief networks this result suggests that research should be directed away from the search for a general efficient probabilistic inference algorithm and toward the design of efficient special case average case and approximation algorithms

a review on singularly perturbed differential equations with turning points and interior layers singular perturbation problems with turning points arise as mathematical models for various physical phenomena the problem with interior turning point represent one dimensional version of stationary convection diffusion problems with a dominant convective term and a speed field that changes its sign in the catch basin boundary turning point problems on the other hand arise in geophysics and in modeling thermal boundary layers in laminar flow in this paper we review some existing literature on asymptotic and numerical analysis of singularly perturbed turning point and interior layer problems the purpose is to find out what problems are treated and what numerical asymptotic methods are employed with an eye towards the goal of developing general methods to solve such problems since major work in this area started after 1970 so this paper limits its coverage to the work done by numerous researchers between 1970 and 2011

and or branch and bound search for combinatorial optimization in graphical models this is the first of two papers presenting and evaluating the power of a new framework for combinatorial optimization in graphical models based on and or search spaces we introduce a new generation of depth first branch and bound algorithms that explore the and or search tree using static and dynamic variable orderings the virtue of the and or representation of the search space is that its size may be far smaller than that of a traditional or representation which can translate into significant time savings for search algorithms the focus of this paper is on linear space search which explores the and or search tree in the second paper we explore memory intensive and or search algorithms in conjunction with the and or search space we investigate the power of the mini bucket heuristics in both static and dynamic setups we focus on two most common optimization problems in graphical models finding the most probable explanation in bayesian networks and solving weighted csps in extensive empirical evaluations we demonstrate that the new and or branch and bound approach improves considerably over the traditional or search strategy and show how various variable ordering schemes impact the performance of the and or search scheme

on the hardness of approximate reasoning abstract many ai problems when formalized reduce to evaluating the probability that a propositional expression is true in this paper we show that this problem is computationally intractable even in surprisingly restricted cases and even if we settle for an approximation to this probability we consider various methods used in approximate reasoning such as computing degree of belief and bayesian belief networks as well as reasoning techniques such as constraint satisfaction and knowledge compilation that use approximation to avoid computational difficulties and reduce them to model counting problems over a propositional domain we prove that counting satisfying assignments of propositional languages is intractable even for horn and monotone formulae and even when the size of clauses and number of occurrences of the variables are extremely limited this should be contrasted with the case of deductive reasoning where horn theories and theories with binary clauses are distinguished by the existence of linear time satisfiability algorithms what is even more surprising is that as we show even approximating the number of satisfying assignments i e approximating approximate reasoning is intractable for most of these restricted theories we also identify some restricted classes of propositional formulae for which efficient algorithms for counting satisfying assignments can be given

dynamic uncertain causality graph for knowledge representation and probabilistic reasoning statistics base matrix and application graphical models for probabilistic reasoning are now in widespread use many approaches have been developed such as bayesian network a newly developed approach named as dynamic uncertain causality graph ducg is initially presented in a previous paper in which only the inference algorithm in terms of individual events and probabilities is addressed in this paper we first explain the statistic basis of ducg then we extend the algorithm to the form of matrices of events and probabilities it is revealed that the representation of ducg can be incomplete and the exact probabilistic inference may still be made a real application of ducg for fault diagnoses of a generator system of a nuclear power plant is demonstrated which involves variables most inferences take with a laptop computer the causal logic between inference result and observations is graphically displayed to users so that they know not only the result but also why the result obtained

finding maps for belief networks is np hard given a probabilistic world model an important problem is to find the maximum a posteriori probability map instantiation of all the random variables given the evidence numerous researchers using such models employ some graph representation for the distributions such as a bayesian belief network this representation simplifies the complexity of specifying the distributions from exponential in n the number of variables in the model to linear in n in many interesting cases we show however that finding the map is np hard in the general case when these representations are used even if the size of the representation happens to be linear in n furthermore minor modifications to the proof show that the problem remains np hard for various restrictions of the topology of the graphs the same technique can be applied to the results of a related paper by cooper to further restrict belief network topology in the proof that probabilistic inference is np hard

a random polynomial time algorithm for approximating the volume of convex bodies a randomized polynomial time algorithm for approximating the volume of a convex body k in n dimensional euclidean space is presented the proof of correctness of the algorithm relies on recent theory of rapidly mixing markov chains and isoperimetric inequalities to show that a certain random walk can be used to sample nearly uniformly from within k

a singularly perturbed parabolic problem with a layer in the initial condition abstract singularly perturbed reaction diffusion parabolic problems with non smooth initial conditions are studied a finite difference method is constructed on a piecewise uniform mesh for a parabolic problem with an initial condition containing a layer the parameter uniform convergence of this numerical method is analyzed this numerical method is used to generate an approximation to the solution of a singularly perturbed parabolic problem with a discontinuous initial condition

a brief survey on numerical methods for solving singularly perturbed problems in the present paper a brief survey on computational techniques for the different classes of singularly perturbed problems is given this survey is a continuation of work performed earlier by the first author and contains the literature of the work done by the researchers during the years 2000 2009 however some older important relevant papers are also included in this survey we also mentioned those papers which are not surveyed in the previous survey papers by the first author of this paper see appl math comput 30 1989 223 259 130 2002 457 510 134 2003 371 429 for details thus this survey paper contains a surprisingly large amount of literature on singularly perturbed problems and indeed can serve as an introduction to some of the ideas and methods for the singular perturbation problems

on the complexity of inference about probabilistic relational models abstract we investigate the complexity of probabilistic inference from knowledge bases that encode probability distributions on finite domain relational structures our interest here lies in the complexity in terms of the domain under consideration in a specific application instance we obtain the result that assuming netime etime this problem is not polynomial for reasonably expressive representation systems the main consequence of this result is that it is unlikely to find inference techniques with a better worst case behavior than the commonly employed strategy of constructing standard bayesian networks over ground atoms knowledge based model construction

approximating probabilistic inference in bayesian belief networks a belief network comprises a graphical representation of dependencies between variables of a domain and a set of conditional probabilities associated with each dependency unless rho np an efficient exact algorithm does not exist to compute probabilistic inference in belief networks stochastic simulation methods which often improve run times provide an alternative to exact inference algorithms such a stochastic simulation algorithm d bnras which is a randomized approximation scheme is presented to analyze the run time belief networks are parameterized by the dependence value d sub xi which is a measure of the cumulative strengths of the belief network dependencies given background evidence xi this parameterization defines the class of f dependence networks the run time of d bnras is polynomial when f is a polynomial function thus the results prove the existence of a class of belief networks for which inference approximation is polynomial and hence provably faster than any exact algorithm

approximate belief updating in max 2 connected bayes networks is np hard a max 2 connected bayes network is one where there are at most 2 distinct directed paths between any two nodes we show that even for this restricted topology null evidence belief updating is hard to approximate

mini buckets a general scheme for bounded inference this article presents a class of approximation algorithms that extend the idea of bounded complexity inference inspired by successful constraint propagation algorithms to probabilistic inference and combinatorial optimization the idea is to bound the dimensionality of dependencies created by inference algorithms this yields a parameterized scheme called mini buckets that offers adjustable trade off between accuracy and efficiency the mini bucket approach to optimization problems such as finding the most probable explanation mpe in bayesian networks generates both an approximate solution and bounds on the solution quality we present empirical results demonstrating successful performance of the proposed approximation scheme for the mpe task both on randomly generated problems and on realistic domains such as medical diagnosis and probabilistic decoding

singularly perturbed parabolic problems with non smooth data in this paper a class of singularly perturbed parabolic equations with discontinuous data is examined interior layers appear in the solutions of problems in this class a numerical method is constructed for this class of problems which involves an appropriate piecewise uniform mesh the method is shown to be uniformly convergent with respect to the singular perturbation parameter

singularly perturbed reaction diffusion problems with discontinuities in the initial and or the boundary data abstract numerical approximations to the solutions of three different problem classes of singularly perturbed parabolic reaction diffusion problems each with a discontinuity in the boundary initial data are generated for each problem class an analytical function associated with the discontinuity in the data is identified parameter uniform numerical approximations to the difference between the analytical function and the solution of the singularly perturbed problem are generated using piecewise uniform shishkin meshes numerical results are given to illustrate all the theoretical error bounds established in the paper