a graph theoretic approach to automata minimality the paper presents a graph theoretic approach to test the minimality of a deterministic automaton in particular we focus on problems concerning the dependence of the minimality of an automaton on the choice of the set f of final states or on the cardinality of the set f we introduce different minimality conditions of an automaton and show that such conditions can be characterized in graph theoretic terms

approximating bayesian belief networks by arc removal i propose a general framework for approximating bayesian belief networks through model simplification by arc removal given an upper bound on the absolute error allowed on the prior and posterior probability distributions of the approximated network a subset of arcs is removed thereby speeding up probabilistic inference

learning equivalence classes of bayesian network structures two bayesian network structures are said to be equivalent if the set of distributions that can be represented with one of those structures is identical to the set of distributions that can be represented with the other many scoring criteria that are used to learn bayesian network structures from data are score equivalent that is these criteria do not distinguish among networks that are equivalent in this paper we consider using a score equivalent criterion in conjunction with a heuristic search algorithm to perform model selection or model averaging we argue that it is often appropriate to search among equivalence classes of network structures as opposed to the more common approach of searching among individual bayesian network structures we describe a convenient graphical representation for an equivalence class of structures and introduce a set of operators that can be applied to that representation by a search algorithm to move among equivalence classes we show that our equivalence class operators can be scored locally and thus share the computational efficiency of traditional operators defined for individual structures we show experimentally that a greedy model selection algorithm using our representation yields slightly higher scoring structures than the traditional approach without any additional time overhead and we argue that more sophisticated search algorithms are likely to benefit much more

evolution of a multimodal map induced by an equivariant vector field it has been shown that the topological characterization of an equivariant system should preferably be achieved by working in a fundamental domain generated by the symmetry properties appearing in the phase space in this paper we discuss the case when the equivariance of the studied system is taken into account to study the evolution of the population of periodic orbits when a control parameter is varied the burke shaw system is considered here as an example it is shown that the equivariance of this system may be used to reduce the multimodal first return map in a poincare section to a unimodal map a relationship between four symbol sequences and two symbol sequences is given the non trivial evolution of the orbit spectrum of a multimodal map is then predicted from the much simpler unimodal map to which the multimodal map reduces

an efficient algorithm for computing bisimulation equivalence we propose an efficient algorithmic solution to the problem of determining a bisimulation relation on a finite structure working both on the explicit and on the implicit symbolic representation as far as the explicit case is concerned starting from a set theoretic point of view we propose an algorithm that optimizes the solution to the relational coarsest partition problem given by paige and tarjan siam j comput 16 6 1987 973 its use in model checking packages is also discussed and tested for well structured graphs our algorithm reaches a linear worst case behaviour the proposed algorithm is then re elaborated to produce a symbolic version

a probabilistic plan recognition algorithm based on plan tree grammars we present the phatt algorithm for plan recognition unlike previous approaches to plan recognition phatt is based on a model of plan execution we show that this clarifies several difficult issues in plan recognition including the execution of multiple interleaved root goals partially ordered plans and failing to observe actions we present the phatt algorithm s theoretical basis and an implementation based on tree structures we also investigate the algorithm s complexity both analytically and empirically finally we present phatt s integrated constraint reasoning for parametrized actions and temporal constraints

decision boundary for discrete bayesian network classifiers bayesian network classifiers are a powerful machine learning tool in order to evaluate the expressive power of these models we compute families of polynomials that sign represent decision functions induced by bayesian network classifiers we prove that those families are linear combinations of products of lagrange basis polynomials in absence of v structures in the predictor sub graph we are also able to prove that this family of polynomials does indeed characterize the specific classifier considered we then use this representation to bound the number of decision functions representable by bayesian network classifiers with a given structure

understanding the role of noise in stochastic local search analysis and experiments stochastic local search sls algorithms have recently been proven to be among the best approaches to solving computationally hard problems sls algorithms typically have a number of parameters optimized empirically that characterize and determine their performance in this article we focus on the noise parameter the theoretical foundation of sls including an understanding of how to the optimal noise varies with problem difficulty is lagging compared to the strong empirical results obtained using these algorithms a purely empirical approach to understanding and optimizing sls noise as problem instances vary can be very computationally intensive to complement existing experimental results we formulate and analyze several markov chain models of sls in this article in particular we compute expected hitting times and show that they are rational functions for individual problem instances as well as their mixtures expected hitting time curves are analytical counterparts to noise response curves reported in the experimental literature hitting time analysis using polynomials and convex functions is also discussed in addition we present examples and experimental results illustrating the impact of varying noise probability on sls run time in experiments where most probable explanations in bayesian networks are computed we use synthetic problem instances as well as problem instances from applications we believe that our results provide an improved theoretical understanding of the role of noise in stochastic local search thereby providing a foundation for further progress in this area

using action based hierarchies for real time diagnosis abstract an intelligent agent diagnoses perceived problems so that it can respond to them appropriately basically the agent performs a series of tests whose results discriminate among competing hypotheses given a specific diagnosis the agent performs the associated action using the traditional information theoretic heuristic to order diagnostic tests in a decision tree the agent can maximize the information obtained from each successive test and thereby minimize the average time number of tests required to complete a diagnosis and perform the appropriate action however in real time domains even the optimal sequence of tests cannot always be performed in the time available nonetheless the agent must respond for agents operating in real time domains we propose an alternative action based approach in which a each node in the diagnosis tree is augmented to include an ordered set of actions each of which has positive utility for all of its children in the tree and b the tree is structured to maximize the expected utility of the action available at each node upon perceiving a problem the agent works its way through the tree performing tests that discriminate among successively smaller subsets of potential faults when a deadline occurs the agent performs the best available action associated with the most specific node it has reached so far although the action based approach does not minimize the time required to complete a specific diagnosis it provides positive utility responses with step wise improvements in expected utility throughout the diagnosis process we present theoretical and empirical results contrasting the advantages and disadvantages of the information theoretic and action based approaches

fundamental concepts of qualitative probabilistic networks abstract graphical representations for probabilistic relationships have recently received considerable attention in ai qualitative probabilistic networks abstract from the usual numeric representations by encoding only qualitative relationships which are inequality constraints on the joint probability distribution over the variables although these constraints are insufficient to determine probabilities uniquely they are designed to justify the deduction of a class of relative likelihood conclusions that imply useful decision making properties two types of qualitative relationship are defined each a probabilistic form of monotonicity constraint over a group of variables qualitative influences describe the direction of the relationship between two variables qualitative synergies describe interactions among influences the probabilistic definitions chosen justify sound and efficient inference procedures based on graphical manipulations of the network these procedures answer queries about qualitative relationships among variables separated in the network and determine structural properties of optimal assignments to decision variables

controlled generation of hard and easy bayesian networks impact on maximal clique size in tree clustering this article presents and analyzes algorithms that systematically generate random bayesian networks of varying difficulty levels with respect to inference using tree clustering the results are relevant to research on efficient bayesian network inference such as computing a most probable explanation or belief updating since they allow controlled experimentation to determine the impact of improvements to inference algorithms the results are also relevant to research on machine learning of bayesian networks since they support controlled generation of a large number of data sets at a given difficulty level our generation algorithms called bpart and mpart support controlled but random construction of bipartite and multipartite bayesian networks the bayesian network parameters that we vary are the total number of nodes degree of connectivity the ratio of the number of non root nodes to the number of root nodes regularity of the underlying graph and characteristics of the conditional probability tables the main dependent parameter is the size of the maximal clique as generated by tree clustering this article presents extensive empirical analysis using the hugin tree clustering approach as well as theoretical analysis related to the random generation of bayesian networks using bpart and mpart

bounds for the loss in probability of correct classification under model based approximation in many pattern recognition classification problem the true class conditional model and class probabilities are approximated for reasons of reducing complexity and or of statistical estimation the approximated classifier is expected to have worse performance here measured by the probability of correct classification we present an analysis valid in general and easily computable formulas for estimating the degradation in probability of correct classification when compared to the optimal classifier an example of an approximation is the naive bayes classifier we show that the performance of the naive bayes depends on the degree of functional dependence between the features and labels we provide a sufficient condition for zero loss of performance too

the complexity of counting cuts and of computing the probability that a graph is connected several enumeration and reliability problems are shown to be p complete and hence at least as hard as np complete problems included are important problems in network reliability analysis namely computing the probability that a graph is connected and counting the number of minimum cardinality s t cuts or directed network cuts also shown to be p complete are counting vertex covers in a bipartite graph counting antichains in a partial order and approximating the probability that a graph is connected and the probability that a pair of vertices is connected

complexity results for explanations in the structural model approach we analyze the computational complexity of halpern and pearl s causal explanations in the structural model approach which are based on their notions of weak and actual cause in particular we give a precise picture of the complexity of deciding explanations partial explanations and partial explanations and of computing the explanatory power of partial explanations moreover we analyze the complexity of deciding whether all explanation or an partial explanation over certain variables exists we also analyze the complexity of deciding explanations and partial explanations in the case of succinctly represented context sets the complexity of deciding explanations in the general case of situations and the complexity of deciding subsumption and equivalence between causal models all complexity results are derived for the general case as well as for the restriction to the case of binary causal models in which all endogenous variables may take only two values to our knowledge no complexity results for explanations in the structural model approach have been derived so far our results give insight into the computational structure of halpern and pearl s explanations and pave the way for efficient algorithms and implementations

a sufficiently fast algorithm for finding close to optimal clique trees abstract we offer an algorithm that finds a clique tree such that the size of the largest clique is at most 2 1 k where k is the size of the largest clique in a clique tree in which this size is minimized and is the approximation ratio of an approximation algorithm for the 3 way vertex cut problem when 4 3 our algorithm s complexity is o 2 4 67k n poly n and it errs by a factor of 3 67 where poly n is the running time of linear programming this algorithm is extended to find clique trees in which the state space of the largest clique is bounded when k o log n our algorithm yields a polynomial inference algorithm for bayesian networks

a comfort measure for diagnostic problem solving abstract in order to apply bayes theorem for diagnostic problem solving when multiple disorders can occur simultaneously several previous proposals have suggested using the ratio of posterior probabilities to rank diagnostic hypotheses approaches using such relative likelihoods lose the measure of absolute strengths of hypotheses and thus are incapable of evaluating the quality of a problem solution in this paper we propose to impose a quantity called a comfort measure on the solution a solution of a diagnostic problem is a minimal size set of hypotheses such that the sum of their posterior probabilities exceeds a given comfort measure based on a probabilistic causal model developed previously a problem solving strategy is presented which does not require the manifestation independence assumption required with direct bayesian classification and which is applicable to multimembership classification problems this strategy selectively generates diagnostic hypotheses and calculates both their relative likelihood and the lower and upper bounds of their posterior probabilities these bounds are successively refined as more hypotheses are generated using these bounds not the real posterior probabilities the problem solving strategy identifies a solution satisfying the given comfort measure usually after only a small portion of all possible hypotheses have been generated

context specific sign propagation in qualitative probabilistic networks qualitative probabilistic networks are qualitative abstractions of probabilistic networks summarising probabilistic influences by qualitative signs as qualitative networks model influences at the level of variables knowledge about probabilistic influences that hold only for specific values cannot be expressed the results computed from a qualitative network as a consequence can be weaker than strictly necessary and may in fact be rather uninformative we extend the basic formalism of qualitative probabilistic networks by providing for the inclusion of context specific information about influences and show that exploiting this information upon reasoning has the ability to forestall unnecessarily weak results

optimal canonization of all substrings of a string abstract any word can be decomposed uniquely into lexicographically nonincreasing factors each one of which is a lyndon word this paper addresses the relationship between the lyndon decomposition of a word x and a canonical rotation of x i e a rotation w of x that is lexicographically smallest among all rotations of x the main combinatorial result is a characterization of the lyndon factor of x with which w must start as an application faster on line algorithms for finding the canonical rotation s of x are developed by nontrivial extension of known lyndon factorization strategies unlike their predecessors the new algorithms lend themselves to incremental variants that compute in linear time the canonical rotations of all prefixes of x the fastest such variant represents the main algorithmic contribution of the paper it performs within the same 3 x character comparisons bound as that of the fastest previous on line algorithms for the canonization of a single string this leads to the canonization of all substrings of a string in optimal quadratic time within less than 3 x 2 character comparisons and using linear auxiliary space

continuant polynomials and worst case behavior of hopcroft s minimization algorithm this paper is concerned with the analysis of the worst case behavior of hopcroft s algorithm for minimizing deterministic finite state automata we extend a result of castiglione restivo and sciortino they show that hopcroft s algorithm has a worst case behavior for the automata recognizing fibonacci words in a previous paper we have proved that this holds for all standard sturmian words having an ultimately periodic directive sequence the directive sequence for fibonacci words is 1 1 we prove here that the same conclusion holds for all standard sturmian words having a directive sequence with bounded elements more precisely we obtain in fact a characterization of those directive sequences for which hopcroft s algorithm has worst case running time these are the directive sequences d 1 d 2 d 3 for which the sequence of geometric means d 1d 2 d n 1 n is bounded as a consequence we easily show that there exist directive sequences for which the worst case for the running time is not attained

approximating maps for belief networks is np hard and other theorems abstract finding maximum a posteriori map assignments also called most probable explanations is an important problem on bayesian belief networks shimony has shown that finding maps is np hard in this paper we show that approximating maps with a constant ratio bound is also np hard in addition we examine the complexity of two related problems which have been mentioned in the literature we show that given the map for a belief network and evidence set or the family of maps if the optimal is not unique it remains np hard to find or approximate alternative next best explanations furthermore we show that given the map or maps for a belief network and an initial evidence set it is also np hard to find or approximate the map assignment for the same belief network with a modified evidence set that differs from the initial set by the addition or removal of even a single node assignment finally we show that our main result applies to networks with constrained in degree and out degree applies to randomized approximation and even still applies if the ratio bound instead of being constant is allowed to be a polynomial function of various aspects of the network topology

the complexity of bayesian networks specified by propositional and relational languages abstract we examine the complexity of inference in bayesian networks specified by logical languages we consider representations that range from fragments of propositional logic to function free first order logic with equality in doing so we cover a variety of plate models and of probabilistic relational models we study the complexity of inferences when network query and domain are the input the inferential and the combined complexity when the network is fixed and query and domain are the input the query data complexity and when the network and query are fixed and the domain is the input the domain complexity we draw connections with probabilistic databases and liftability results and obtain complexity classes that range from polynomial to exponential levels we identify new languages with tractable inference and we relate our results to languages based on plates and probabilistic relational models

average complexity of moore s and hopcroft s algorithms in this paper we prove that for the uniform distribution on complete deterministic automata the average time complexity of moore s state minimization algorithm is o nloglogn where n is the number of states in the input automata and the number of letters in the alphabet is fixed then an unusual family of implementations of hopcroft s algorithm is characterized for which the algorithm will be proved to be always faster than moore s algorithm finally we present experimental results on the usual implementations of hopcroft s algorithm

planar formulae and their uses we define the set of planar boolean formulae and then show that the set of true quantified planar formulae is polynomial space complete and that the set of satisfiable planar formulae is np complete using these results we are able to provide simple and nearly uniform proofs of np completeness for planar node cover planar hamiltonian circuit and line geometric connected dominating set and of polynomial space completeness for planar generalized geography the np completeness of planar node cover and planar hamiltonian circuit and line were first proved elsewhere m r garey and d s johnson the rectilinear steiner tree is np complete siam j appl math 32 1977 pp 826 834 and m r garey d s johnson and r e tarjan the planar hamilton circuit problem is np complete siam j comp 5 1976 pp 704 714

the complexity of logic based abduction abduction is an important form of nonmonotonic reasoning allowing one to find explanations for certain symptoms or manifestations when the application domain is described by a logical theory we speak about logic based abduction candidates for abductive explanations are usually subjected to minimality criteria such as subset minimality minimal cardinality minimal weight or minimality under prioritization of individual hypotheses this paper presents a comprehensive complexity analysis of relevant decision and search problems related to abduction on propositional theories our results indicate that abduction is harder than deduction in particular we show that with the most basic forms of abduction the relevant decision problems are complete for complexity classes at the second level of the polynomial hierarchy while the use of prioritization raises the complexity to the third level in certain cases

complexity results for structure based causality we give a precise picture of the computational complexity of causal relationships in pearl s structural models where we focus on causality between variables event causality and probabilistic causality as for causality between variables we consider the notions of causal irrelevance cause cause in a context direct cause and indirect cause as for event causality we analyze the complexity of the notions of necessary and possible cause and of the sophisticated notions of weak and actual cause by halpern and pearl in the course of this we also prove an open conjecture by halpern and pearl and establish other semantic results we then analyze the complexity of the probabilistic notions of probabilistic causal irrelevance likely causes of events and occurrences of events despite other events moreover we consider decision and optimization problems involving counterfactual formulas to our knowledge no complexity aspects of causal relationships in the structural model approach have been considered so far and our results shed light on this issue

an algorithm for probabilistic planning abstract we define the probabilistic planning problem in terms of a probability distribution over initial world states a boolean combination of propositions representing the goal a probability threshold and actions whose effects depend on the execution time state of the world and on random chance adopting a probabilistic model complicates the definition of plan success instead of demanding a plan that provably achieves the goal we seek plans whose probability of success exceeds the threshold in this paper we present buridan an implemented least commitment planner that solves problems of this form we prove that the algorithm is both sound and complete we then explore buridan s efficiency by contrasting four algorithms for plan evaluation using a combination of analytic methods and empirical experiments we also describe the interplay between generating plans and evaluating them and discuss the role of search control in probabilistic planning

the complexity of approximating maps for belief networks with bounded probabilities abstract probabilistic inference and maximum a posteriori map explanation are two important and related problems on bayesian belief networks both problems are known to be np hard for both approximation and exact solution in 1997 dagum and luby showed that efficiently approximating probabilistic inference is possible for belief networks in which all probabilities are bounded away from 0 in this paper we show that the corresponding result for map explanation does not hold finding or approximating maps for belief networks remains np hard for belief networks with probabilities bounded within the range l u for any 0 l our results cover both deterministic and randomized approximation

an axiomatic treatment of three qualitative decision criteria the need for computationally efficient decision making techniques together with the desire to simplify the processes of knowledge acquisition and agent specification have led various researchers in artificial intelligence to examine qualitative decision tools however the adequacy of such tools is not clear this paper investigates the foundations of maximin minmax regret and competitive ratio three central qualitative decision criteria by characterizing those behaviors that could result from their use this characterizaton provides two important insights 1 under what conditions can we employ an agent model based on these basic qualitative decision criteria and 2 how rational are these decision procedures for the competitive ratio criterion in particular this latter issue is of central importance to our understanding of current work on on line algorithms our main result is a constructive representation theorem that uses two choice axioms to characterize maximin minmax regret and competitive ratio

an efficient parallel algorithm for the single function coarsest partition problem we describe an efficient parallel algorithm to solve the single function coarsest partition problem the algorithm runs in o log n time using o n log log n operations on the arbitrary crcw pram the previous best known algorithms run in o log2 n time using o n log2n operations on the crew pram and o log n time using o n log n operations on the arbitrary crcw pram our solution is based on efficient algorithms for solving several subproblems that are of independent interest in particular we present efficient parallel algorithms to find a minimal starting point of a circular string with respect to lexicographic ordering and to sort lexicographically a list of strings of different lengths

cortical circuitry implementing graphical models in this letter we develop and simulate a large scale network of spiking neurons that approximates the inference computations performed by graphical models unlike previous related schemes which used sum and product operations in either the log or linear domains the current model uses an inference scheme based on the sum and maximization operations in the log domain simulations show that using these operations a large scale circuit which combines populations of spiking neurons as basic building blocks is capable of finding close approximations to the full mathematical computations performed by graphical models within a few hundred milliseconds the circuit is general in the sense that it can be wired for any graph structure it supports multistate variables and it uses standard leaky integrate and fire neuronal units following previous work which proposed relations between graphical models and the large scale cortical anatomy we focus on the cortical microcircuitry and propose how anatomical and physiological aspects of the local circuitry may map onto elements of the graphical model implementation we discuss in particular the roles of three major types of inhibitory neurons small fast spiking basket cells large layer 2 3 basket cells and double bouquet neurons subpopulations of strongly interconnected neurons with their unique connectivity patterns in different cortical layers and the possible role of minicolumns in the realization of the population based maximum operation

attractor dynamics in feedforward neural networks we study the probabilistic generative models parameterized by feedforward neural networks an attractor dynamics for probabilistic inference in these models is derived from a mean field approximation for large layered sigmoidal networks fixed points of the dynamics correspond to solutions of the mean field equations which relate the statistics of each unit to those of its markov blanket we establish global convergence of the dynamics by providing a lyapunov function and show that the dynamics generate the signals required for unsupervised learning our results for feedforward networks provide a counterpart to those of cohen grossberg and hopfield for symmetric networks

approximating probabilistic inference in bayesian belief networks is np hard abstract it is known that exact computation of conditional probabilities in belief networks is np hard many investigators in the ai community have tacitly assumed that algorithms for performing approximate inference with belief networks are of polynomial complexity indeed special cases of approximate inference can be performed in time polynomial in the input size however we have discovered that the general problem of approximating conditional probabilities with belief networks like exact inference resides in the np hard complexity class we develop a complexity analysis to elucidate the difficulty of approximate probabilistic inference more specifically we show that the existence of a polynomial time relative approximation algorithm for major classes of problem instances implies that np p we present our proof and explore the implications of the result

the computational complexity of probabilistic inference using bayesian belief networks research note abstract bayesian belief networks provide a natural efficient method for representing probabilistic dependencies among a set of variables for these reasons numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks more general classes of belief networks however have eluded efforts to develop efficient inference algorithms we show that probabilistic inference using belief networks is np hard therefore it seems unlikely that an exact algorithm can be developed to perform probabilistic inference efficiently over all classes of belief networks this result suggests that research should be directed away from the search for a general efficient probabilistic inference algorithm and toward the design of efficient special case average case and approximation algorithms

nondeterministic moore automata and brzozowski s minimization algorithm moore automata represent a model that has many applications in this paper we define a notion of coherent nondeterministic moore automaton nma and show that such a model has the same computational power of the classical deterministic moore automaton we consider also the problem of constructing the minimal deterministic moore automaton equivalent to a given nma we propose an algorithm that is a variant of brzozowski s minimization algorithm in the sense that it is essentially structured as reverse operation and subset construction performed twice moreover we explore more general classes of nma and analyze the applicability of the algorithm for some of such classes the algorithm does not return the minimal equivalent deterministic automaton

an optimal approximation algorithm for bayesian inference abstract approximating the inference probability pr x x e e in any sense even for a single evidence node e is np hard this result holds for belief networks that are allowed to contain extreme conditional probabilities that is conditional probabilities arbitrarily close to 0 nevertheless all previous approximation algorithms have failed to approximate efficiently many inferences even for belief networks without extreme conditional probabilities we prove that we can approximate efficiently probabilistic inference in belief networks without extreme conditional probabilities we construct a randomized approximation algorithm the bounded variance algorithm that is a variant of the known likelihood weighting algorithm the bounded variance algorithm is the first algorithm with provably fast inference approximation on all belief networks without extreme conditional probabilities from the bounded variance algorithm we construct a deterministic approximation algorithm using current advances in the theory of pseudorandom generators in contrast to the exponential worst case behavior of all previous deterministic approximations the deterministic bounded variance algorithm approximates inference probabilities in worst case time that is subexponential 2 log n d for some integer d that is a linear function of the depth of the belief network

understanding the scalability of bayesian network inference using clique tree growth curves one of the main approaches to performing computation in bayesian networks bns is clique tree clustering and propagation the clique tree approach consists of propagation in a clique tree compiled from a bn and while it was introduced in the 1980s there is still a lack of understanding of how clique tree computation time depends on variations in bn size and structure in this article we improve this understanding by developing an approach to characterizing clique tree growth as a function of parameters that can be computed in polynomial time from bns specifically i the ratio of the number of a bn s non root nodes to the number of root nodes and ii the expected number of moral edges in their moral graphs analytically we partition the set of cliques in a clique tree into different sets and introduce a growth curve for the total size of each set for the special case of bipartite bns there are two sets and two growth curves a mixed clique growth curve and a root clique growth curve in experiments where random bipartite bns generated using the bpart algorithm are studied we systematically increase the out degree of the root nodes in bipartite bayesian networks by increasing the number of leaf nodes surprisingly root clique growth is well approximated by gompertz growth curves an s shaped family of curves that has previously been used to describe growth processes in biology medicine and neuroscience we believe that this research improves the understanding of the scaling behavior of clique tree clustering for a certain class of bayesian networks presents an aid for trade off studies of clique tree clustering using growth curves and ultimately provides a foundation for benchmarking and developing improved bn inference and machine learning algorithms

standard sturmian words and automata minimization algorithms the study of some close connections between the combinatorial properties of words and the performance of the automata minimization process constitutes the main focus of this paper these relationships have been in fact the basis of the study of the tightness and the extremal cases of hopcroft s algorithm that is up to now the most efficient minimization method for deterministic finite state automata recently increasing attention has been paid to another minimization method that unlike the approach proposed by hopcroft is not based on refinement of the set of states of the automaton but on automata operations such as determinization and reverse and is also applicable to non deterministic finite automata however even for deterministic automata it was proved that the method incurs almost surely in an explosion of the number of the states very recently some polynomial variants of brzozowski s method have been introduced in this paper by using some combinatorial properties of words we analyze the performance of one of such algorithms when applied to a particular infinite family of automata called standard word automata constructed by using standard sturmian words in particular n log n is the worst case time complexity when the algorithm is applied to the infinite family of word automata associated to fibonacci words representing also the worst case of hopcroft s minimization algorithm

on extremal cases of hopcroft s algorithm in this paper we consider the problem of minimization of deterministic finite automata dfa with reference to hopcroft s algorithm hopcroft s algorithm has several degrees of freedom so there can exist different executions that can lead to different sequences of refinements of the set of the states up to the final partition we find an infinite family of binary automata for which such a process is unique whatever strategy is chosen some recent papers cf berstel and carton 2004 3 castiglione et al 2008 6 and berstel et al 2009 1 have been devoted to find families of automata for which hopcroft s algorithm has its worst execution time they are unary automata associated with circular words however automata minimization can be achieved also in linear time when the alphabet has only one letter cf paige et al 1985 14 but such a method does not seem to extend to larger alphabet so in this paper we face the tightness of hopcroft s algorithm when the alphabet contains more than one letter in particular we define an infinite family of binary automata representing the worst case of hopcroft s algorithm for each execution they are automata associated with particular trees and we deepen the connection between the refinement process of hopcroft s algorithm and the combinatorial properties of such trees

improving generalization of fuzzy if then rules by maximizing fuzzy entropy when fuzzy if then rules initially extracted from data have not a satisfying performance we consider that the rules require refinement distinct from most existing rule refinement approaches that are based on the further reduction of training error this paper proposes a new rule refinement scheme that is based on the maximization of fuzzy entropy on the training set the new scheme which is realized by solving a quadratic programming problem is expected to have the advantages of improving the generalization capability of initial fuzzy if then rules and simultaneously overcoming the overfitting of refinement experimental results on a number of selected databases demonstrate the expected improvement of generalization capability and the prevention of overfitting by a comparison of both training and testing accuracy before and after the refinement

multiresolution image classification by hierarchical modeling with two dimensional hidden markov models this paper treats a multiresolution hidden markov model for classifying images each image is represented by feature vectors at several resolutions which are statistically dependent as modeled by the underlying state process a multiscale markov mesh unknowns in the model are estimated by maximum likelihood in particular by employing the expectation maximization algorithm an image is classified by finding the optimal set of states with maximum a posteriori probability states are then mapped into classes the multiresolution model enables multiscale information about context to be incorporated into classification suboptimal algorithms based on the model provide progressive classification that is much faster than the algorithm based on single resolution hidden markov models

on the hardness of approximate reasoning abstract many ai problems when formalized reduce to evaluating the probability that a propositional expression is true in this paper we show that this problem is computationally intractable even in surprisingly restricted cases and even if we settle for an approximation to this probability we consider various methods used in approximate reasoning such as computing degree of belief and bayesian belief networks as well as reasoning techniques such as constraint satisfaction and knowledge compilation that use approximation to avoid computational difficulties and reduce them to model counting problems over a propositional domain we prove that counting satisfying assignments of propositional languages is intractable even for horn and monotone formulae and even when the size of clauses and number of occurrences of the variables are extremely limited this should be contrasted with the case of deductive reasoning where horn theories and theories with binary clauses are distinguished by the existence of linear time satisfiability algorithms what is even more surprising is that as we show even approximating the number of satisfying assignments i e approximating approximate reasoning is intractable for most of these restricted theories we also identify some restricted classes of propositional formulae for which efficient algorithms for counting satisfying assignments can be given

finding non overlapping clusters for generalized inference over graphical models graphical models use graphs to compactly capture stochastic dependencies amongst a collection of random variables inference over graphical models corresponds to finding marginal probability distributions given joint probability distributions in general this is computationally intractable which has led to a quest for finding efficient approximate inference algorithms we propose a framework for generalized inference over graphical models that can be used as a wrapper for improving the estimates of approximate inference algorithms instead of applying an inference algorithm to the original graph we apply the inference algorithm to a block graph defined as a graph in which the nodes are non overlapping clusters of nodes from the original graph this results in marginal estimates of a cluster of nodes which we further marginalize to get the marginal estimates of each node our proposed block graph construction algorithm is simple efficient and motivated by the observation that approximate inference is more accurate on graphs with longer cycles we present extensive numerical simulations that illustrate our block graph framework with a variety of inference algorithms e g those in the libdai software package these simulations show the improvements provided by our framework

model reductions for inference generality of pairwise binary and planar factor graphs we offer a solution to the problem of efficiently translating algorithms between different types of discrete statistical model we investigate the expressive power of three classes of model those with binary variables with pairwise factors and with planar topology as well as their four intersections we formalize a notion of simple reduction for the problem of inferring marginal probabilities and consider whether it is possible to simply reduce marginal inference from general discrete factor graphs to factor graphs in each of these seven subclasses we characterize the reducibility of each class showing in particular that the class of binary pairwise factor graphs is able to simply reduce only positive models we also exhibit a continuous spectral reduction based on polynomial interpolation which overcomes this limitation experiments assess the performance of standard approximate inference algorithms on the outputs of our reductions

diagnosis under compound effects and multiple causes by means of the conditional causal possibility approach abstract the paper addresses uncertain reasoning on a causal model given by two layered networks where nodes in one layer express possible causes and those in the other are possible effects uncertainty of causalities is expressed in a novel manner i e by conditional causal possibilities the expression has two advantages over the conventional way with conditional possibilities it expresses the exact degrees of possibility of causalities and the number of necessary conditional causal possibilities is far smaller than that of conditional possibilities however it has a weakness that it cannot handle causalities with compound effects such as synergistic and canceling effects by multiple causes the paper discusses the weakness and proposes a solution first it discusses how to deal with the compound effects and proposes a new causal model with conditional causal possibilities by multiple causes then it defines a causality consistency problem that calculates possibility of a hypothesis given some observed events and shows a way to solve the problem

parameter uncertainties in models of equivariant dynamical systems abstract we examine the advantages of using equivariant versus nonequivariant phenomenological models for chaotic systems that possess an inversion symmetry numerical experiments used the minimum description length mdl criteria for model selection they indicated that the selection between equivariant versus nonequivariant models should be made on an a priori basis we also examine the relationship between the optimal truncation accuracy of the parameters of the model as predicted by the mdl criteria and the true uncertainty in the parameters of the model numerical experiments indicate that the true uncertainties are poorly approximated by the optimal truncation accuracy they also indicate that the true uncertainties may be useful for determining the presence of an inversion symmetry in reconstructed attractors

inner product spaces for bayesian networks bayesian networks have become one of the major models used for statistical inference we study the question whether the decisions computed by a bayesian network can be represented within a low dimensional inner product space we focus on two label classification tasks over the boolean domain as main results we establish upper and lower bounds on the dimension of the inner product space for bayesian networks with an explicitly given full or reduced parameter collection in particular these bounds are tight up to a factor of 2 for some nontrivial cases of bayesian networks we even determine the exact values of this dimension we further consider logistic autoregressive bayesian networks and show that every sufficiently expressive inner product space must have dimension at least n2 where n is the number of network nodes we also derive the bound 2 n for an artificial variant of this network thereby demonstrating the limits of our approach and raising an interesting open question as a major technical contribution this work reveals combinatorial and algebraic structures within bayesian networks such that known methods for the derivation of lower bounds on the dimension of inner product spaces can be brought into play

a simple graphical approach for understanding probabilistic inference in bayesian networks we present a simple graphical method for understanding exact probabilistic inference in discrete bayesian networks bns a conditional probability table conditional is depicted as a directed acyclic graph involving one or more black vertices and zero or more white vertices the probability information propagated in a network can then be graphically illustrated by introducing the black variable elimination bve algorithm we prove the correctness of bve and establish its polynomial time complexity our method possesses two salient characteristics this purely graphical approach can be used as a pedagogical tool to introduce bn inference to beginners this is important as it is commonly stated that newcomers have difficulty learning bn inference due to intricate mathematical equations and notation secondly bve provides a more precise description of bn inference than the state of the art discrete bn inference technique called lazy ar lazy ar propagates potentials which are not well defined probability distributions our approach only involves conditionals a special case of potential

importance sampling based estimation over and or search spaces for graphical models it is well known that the accuracy of importance sampling can be improved by reducing the variance of its sample mean and therefore variance reduction schemes have been the subject of much research in this paper we introduce a family of variance reduction schemes that generalize the sample mean from the conventional or search space to the and or search space for graphical models the new and or sample means allow trading time and space with variance at one end is the and or sample tree mean which has the same time and space complexity as the conventional or sample tree mean but has smaller variance at other end is the and or sample graph mean which requires more time and space to compute but has the smallest variance theoretically we show that the variance is smaller in the and or space because the and or sample mean is defined over a larger virtual sample size compared with the or sample mean empirically we demonstrate that the and or sample mean is far closer to the true mean than the or sample mean

a note on kirkwood s algebraic method for decision problems the main objectives of this note are to point out some strengths and some limitations of kirkwood s algebraic method for decision problems and to propose a modification to address the limitations the modification is based on the valuation network technique and it enables us to better relate kirkwood s method to the graphical techniques of valuation networks and influence diagrams

a comparison of graphical techniques for decision analysis recently we proposed a new method for representing and solving decision problems based on the framework of valuation based systems the new representation is called a valuation network and the new solution method is called a fusion algorithm in this paper we compare valuation networks to decision trees and influence diagrams we also compare the fusion algorithm to the backward recursion method of decision trees and to the arc reversal method of influence diagrams recently we proposed a new method for representing and solving decision problems based on the axiomatic framework of valuation based systems shenoy 1993 1992a the new representation is called a valuation network and the new solution method is called a fusion algorithm in this paper we compare valuation networks to decision trees and influence diagrams we also compare the fusion algorithm to the backward recursion method of decision trees and the arc reversal method of influence diagrams a popular method for representing and solving decision problems is decision trees decision trees have their genesis in the pioneering work of von neumann and morgenstern 1944 on extensive form games decision trees graphically depict all possible scenarios the decision tree representation allows computation of an optimal strategy by the backward recursion method of dynamic programming raiffa and schlaifer 1961 raiffa and schlaifer 1961 call the dynamic

dynamic uncertain causality graph for knowledge representation and probabilistic reasoning statistics base matrix and application graphical models for probabilistic reasoning are now in widespread use many approaches have been developed such as bayesian network a newly developed approach named as dynamic uncertain causality graph ducg is initially presented in a previous paper in which only the inference algorithm in terms of individual events and probabilities is addressed in this paper we first explain the statistic basis of ducg then we extend the algorithm to the form of matrices of events and probabilities it is revealed that the representation of ducg can be incomplete and the exact probabilistic inference may still be made a real application of ducg for fault diagnoses of a generator system of a nuclear power plant is demonstrated which involves variables most inferences take with a laptop computer the causal logic between inference result and observations is graphically displayed to users so that they know not only the result but also why the result obtained

reinforcement learning with factored states and actions a novel approximation method is presented for approximating the value function and selecting good actions for markov decision processes with large state and action spaces the method approximates state action values as negative free energies in an undirected graphical model called a product of experts the model parameters can be learned efficiently because values and derivatives can be efficiently computed for a product of experts actions can be found even in large factored action spaces by the use of markov chain monte carlo sampling simulation results show that the product of experts approximation can be used to solve large problems in one simulation it is used to find actions in action spaces of size 240

finding maps for belief networks is np hard given a probabilistic world model an important problem is to find the maximum a posteriori probability map instantiation of all the random variables given the evidence numerous researchers using such models employ some graph representation for the distributions such as a bayesian belief network this representation simplifies the complexity of specifying the distributions from exponential in n the number of variables in the model to linear in n in many interesting cases we show however that finding the map is np hard in the general case when these representations are used even if the size of the representation happens to be linear in n furthermore minor modifications to the proof show that the problem remains np hard for various restrictions of the topology of the graphs the same technique can be applied to the results of a related paper by cooper to further restrict belief network topology in the proof that probabilistic inference is np hard

circular sturmian words and hopcroft s algorithm in order to analyze some extremal cases of hopcroft s algorithm we investigate the relationships between the combinatorial properties of a circular sturmian word x and the run of the algorithm on the cyclic automaton a x associated to x the combinatorial properties of words taken into account make use of sturmian morphisms and give rise to the notion of reduction tree of a circular sturmian word we prove that the shape of this tree uniquely characterizes the word itself the properties of the run of hopcroft s algorithm are expressed in terms of the derivation tree of the automaton which is a tree that represents the refinement process that in the execution of hopcroft s algorithm leads to the coarsest congruence of the automaton we prove that the shape of the reduction tree of a circular sturmian word x coincides with that of the derivation tree t a x of the automaton a x from this we derive a recursive formula to compute the running time of hopcroft s algorithm on the automaton a x expressed in terms of parameters of the reduction tree of x as a special application we obtain the time complexity q nlogn of the algorithm in the case of automata associated to fibonacci words

a new look at survey propagation and its generalizations this article provides a new conceptual perspective on survey propagation which is an iterative algorithm recently introduced by the statistical physics community that is very effective in solving random k sat problems even with densities close to the satisfiability threshold we first describe how any sat formula can be associated with a novel family of markov random fields mrfs parameterized by a real number 0 1 we then show that applying belief propagation a well known message passing technique for estimating marginal probabilities to this family of mrfs recovers a known family of algorithms ranging from pure survey propagation at one extreme e 1 to standard belief propagation on the uniform distribution over sat assignments at the other extreme e 0 configurations in these mrfs have a natural interpretation as partial satisfiability assignments on which a partial order can be defined we isolate cores as minimal elements in this partial ordering which are also fixed points of survey propagation and the only assignments with positive probability in the mrf for e 1 our experimental results for k e 3 suggest that solutions of random formulas typically do not possess non trivial cores this makes it necessary to study the structure of the space of partial assignments for 0 can be viewed as a smoothed version of the uniform distribution over satisfying assignments e 0 finally we isolate properties of gibbs sampling and message passing algorithms that are typical for an ensemble of k sat problems

variational approximations for categorical causal modeling with latent variables latent class models in the social and behavioral sciences have remained structurally simple one reason for this is that inference in statistical models can be computationally difficult methods for approximate inference known as variational approximations which have been developed in the machine learning graphical modeling and statistical physics literatures can be used to alleviate the computational difficulties of inference for latent variable models the aim of the present article is to set these methods alongside some social and behavioral science literature to which they are relevant and in particular to consider their potential for categorical causal modeling using latent class analysis we have collated a number of popular categorical data models with latent variables and causal structure typically incorporating a markovian structure the efficacy of the approximation methods has been demonstrated through simulations related to an important behavioral science model

on the properties of concept classes induced by multivalued bayesian networks the concept class c n induced by a bayesian network n can be embedded into some euclidean inner product space the vapnik chervonenkis vc dimension of the concept class and the minimum dimension of the inner product space are very important indicators for evaluating the classification capability of the bayesian network in this paper we investigate the properties of the concept class c n k induced by a multivalued bayesian network n k where each node x i of n k is a k valued variable we focus on the values of two dimensions i the vc dimension of the concept class c n k denoted as vcdim n k and ii the minimum dimension of the inner product space into which c n k can be embedded we show that the values of these two dimensions are k n 1 for fully connected k valued bayesian networks n f k with n variables for non fully connected k valued bayesian networks n k without v structure we prove that the two dimensional values are k 1 i 1 nk m i 1 where m i denotes the number of parents for the i t h variable we also derive the upper and lower bounds on the minimum dimension of the inner product space induced by non fully connected bayesian networks

the computational complexity of abduction abstract the problem of abduction can be characterized as finding the best explanation of a set of data in this paper we focus on one type of abduction in which the best explanation is the most plausible combination of hypotheses that explains all the data we then present several computational complexity results demonstrating that this type of abduction is intractable np hard in general in particular choosing between incompatible hypotheses reasoning about cancellation effects among hypotheses and satisfying the maximum plausibility requirement are major factors leading to intractability we also identify a tractable but restricted class of abduction problems

variational mixture of bayesian independent component analyzers there has been growing interest in subspace data modeling over the past few years methods such as principal component analysis factor analysis and independent component analysis have gained in popularity and have found many applications in image modeling signal processing and data compression to name just a few as applications and computing power grow more and more sophisticated analyses and meaningful representations are sought mixture modeling methods have been proposed for principal and factor analyzers that exploit local gaussian features in the subspace manifolds meaningful representations may be lost however if these local features are nongaussian or discontinuous in this article we propose extending the gaussian analyzers mixture model to an independent component analyzers mixture model we employ recent developments in variational bayesian inference and structure determination to construct a novel approach for modeling nongaussian discontinuous manifolds we automatically determine the local dimensionality of each manifold and use variational inference to calculate the optimum number of ica components needed in our mixture model we demonstrate our framework on complex synthetic data and illustrate its application to real data by decomposing functional magnetic resonance images into meaningful and medically useful features

approximating probabilistic inference in bayesian belief networks a belief network comprises a graphical representation of dependencies between variables of a domain and a set of conditional probabilities associated with each dependency unless rho np an efficient exact algorithm does not exist to compute probabilistic inference in belief networks stochastic simulation methods which often improve run times provide an alternative to exact inference algorithms such a stochastic simulation algorithm d bnras which is a randomized approximation scheme is presented to analyze the run time belief networks are parameterized by the dependence value d sub xi which is a measure of the cumulative strengths of the belief network dependencies given background evidence xi this parameterization defines the class of f dependence networks the run time of d bnras is polynomial when f is a polynomial function thus the results prove the existence of a class of belief networks for which inference approximation is polynomial and hence provably faster than any exact algorithm

on the complexity of inference about probabilistic relational models abstract we investigate the complexity of probabilistic inference from knowledge bases that encode probability distributions on finite domain relational structures our interest here lies in the complexity in terms of the domain under consideration in a specific application instance we obtain the result that assuming netime etime this problem is not polynomial for reasonably expressive representation systems the main consequence of this result is that it is unlikely to find inference techniques with a better worst case behavior than the commonly employed strategy of constructing standard bayesian networks over ground atoms knowledge based model construction

possibilistic causality consistency problem based on asymmetrically valued causal model the paper addresses uncertain reasoning based on causal knowledge given by two layered networks where nodes in one layer express possible causes and those in the other are possible effects uncertainties of the causalities are given by conditional causal possibilities which were proposed to express the exact degrees of possibility of causalities the expression of the uncertainty also has an advantage over the conventional conditional possibilities in the number of possibilistic values that should be given as a priori knowledge the number of conditional causal possibilities given as knowledge is far smaller than that of conventional conditional possibilities the paper starts with the definition of a causal model called asymmetrically valued causal model the conditional causal possibilities are defined on the causal model and their mathematical properties are discussed then the paper defines the possibilistic causality consistency problem based on the proposed model and shows how to solve the problem the discussed problem is one to calculate the conditional possibility of a hypothesis about presence and absence of unknown events when the states of some other events are known

induction of multiple fuzzy decision trees based on rough set technique the integration of fuzzy sets and rough sets can lead to a hybrid soft computing technique which has been applied successfully to many fields such as machine learning pattern recognition and image processing the key to this soft computing technique is how to set up and make use of the fuzzy attribute reduct in fuzzy rough set theory given a fuzzy information system we may find many fuzzy attribute reducts and each of them can have different contributions to decision making if only one of the fuzzy attribute reducts which may be the most important one is selected to induce decision rules some useful information hidden in the other reducts for the decision making will be losing unavoidably to sufficiently make use of the information provided by every individual fuzzy attribute reduct in a fuzzy information system this paper presents a novel induction of multiple fuzzy decision trees based on rough set technique the induction consists of three stages first several fuzzy attribute reducts are found by a similarity based approach and then a fuzzy decision tree for each fuzzy attribute reduct is generated according to the fuzzy id3 algorithm the fuzzy integral is finally considered as a fusion tool to integrate the generated decision trees which combines together all outputs of the multiple fuzzy decision trees and forms the final decision result an illustration is given to show the proposed fusion scheme a numerical experiment on real data indicates that the proposed multiple tree induction is superior to the single tree induction based on the individual reduct or on the entire feature set for learning problems with many attributes

fusion and propagation with multiple observations in belief networks abstract the polytree algorithm of kim and pearl 16 19 is a fundamental method for evidential reasoning in belief networks not only does it provide exact solutions to singly connected networks using efficient local computations but variations of it can be applied to more general networks as well when a belief network is singly connected the polytree algorithm can compute a posterior marginal distribution for each variable by visiting each node at most once for each piece of evidence by contrast the related algorithms based on undirected graphs 1 14 16 28 only need to visit each node at most twice no matter how much evidence is observed in this paper a revised polytree algorithm is developed with the same complexity as the undirected methods but within the directed framework of the polytree algorithm when this new algorithm is applied via cutset conditioning to general networks it obtains not just the corresponding significant improvement in speed but also a much simpler form for combination furthermore the revised algorithm requires only minor modifications to existing implementations of the polytree algorithm

generalizations of suffix arrays to multi dimensional matrices we propose multi dimensional index data structures that generalize suffix arrays to square matrices and cubic matrices giancarlo proposed a two dimensional index data structure the lsuffix tree that generalizes suffix trees to square matrices however the construction algorithm for lsuffix trees maintains complicated data structures and uses a large amount of space we present simple construction algorithms for multi dimensional suffix arrays by applying a new partitioning technique to lexicographic sorting our contributions are the first efficient algorithms for constructing two dimensional and three dimensional suffix arrays directly

compression based averaging of selective naive bayes classifiers the naive bayes classifier has proved to be very effective on many real data applications its performance usually benefits from an accurate estimation of univariate conditional probabilities and from variable selection however although variable selection is a desirable feature it is prone to overfitting in this paper we introduce a bayesian regularization technique to select the most probable subset of variables compliant with the naive bayes assumption we also study the limits of bayesian model averaging in the case of the naive bayes assumption and introduce a new weighting scheme based on the ability of the models to conditionally compress the class labels the weighting scheme on the models reduces to a weighting scheme on the variables and finally results in a naive bayes classifier with soft variable selection extensive experiments show that the compression based averaged classifier outperforms the bayesian model averaging scheme

convergence analysis of reweighted sum product algorithms markov random fields are designed to represent structured dependencies among large collections of random variables and are well suited to capture the structure of real world signals many fundamental tasks in signal processing e g smoothing denoising segmentation etc require efficient methods for computing approximate marginal probabilities over subsets of nodes in the graph the marginalization problem though solvable in linear time for graphs without cycles is computationally intractable for general graphs with cycles this intractability motivates the use of approximate ldquomessage passingrdquo algorithms this paper studies the convergence and stability properties of the family of reweighted sum product algorithms a generalization of the widely used sum product or belief propagation algorithm in which messages are adjusted with graph dependent weights for pairwise markov random fields we derive various conditions that are sufficient to ensure convergence and also provide bounds on the geometric convergence rates when specialized to the ordinary sum product algorithm these results provide strengthening of previous analyses we prove that some of our conditions are necessary and sufficient for subclasses of homogeneous models but not for general models the experimental simulations on various classes of graphs validate our theoretical results

bilinear generalized approximate message passing part i derivation in this paper we extend the generalized approximate message passing g amp approach originally proposed for high dimensional generalized linear regression in the context of compressive sensing to the generalized bilinear case which enables its application to matrix completion robust pca dictionary learning and related matrix factorization problems here in part i of a two part paper we derive our bilinear g amp big amp algorithm as an approximation of the sum product belief propagation algorithm in the high dimensional limit where central limit theorem arguments and taylor series approximations apply and under the assumption of statistically independent matrix entries with known priors in addition we propose an adaptive damping mechanism that aids convergence under finite problem sizes an expectation maximization em based method to automatically tune the parameters of the assumed priors and two rank selection strategies in part ii of the paper we will discuss the specializations of em big amp to the problems of matrix completion robust pca and dictionary learning and we will present the results of an extensive empirical study comparing em big amp to state of the art algorithms on each problem

tree based reparameterization framework for analysis of sum product and related algorithms we present a tree based reparameterization trp framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles this class includes the belief propagation bp or sum product algorithm as well as variations and extensions of bp algorithms in this class can be formulated as a sequence of reparameterization updates each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph i e a tree or more generally a hypertree the ultimate goal is to obtain an alternative but equivalent factorization using functions that represent exact or approximate marginal distributions on cliques of the graph our framework highlights an important property of the sum product algorithm and the larger class of reparameterization algorithms the original distribution on the graph with cycles is not changed the perspective of tree based updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency we develop interpretations of these results in terms of information geometry the invariance of the distribution in conjunction with the fixed point characterization enables us to derive an exact expression for the difference between the true marginals on an arbitrary graph with cycles and the approximations provided by belief propagation more broadly our analysis applies to any algorithm that minimizes the bethe free energy we also develop bounds on the approximation error which illuminate the conditions that govern their accuracy finally we show how the reparameterization perspective extends naturally to generalizations of bp e g kikuchi 1951 approximations and variants via the notion of hypertree reparameterization

graphical models for structured classification with an application to interpreting images of protein subcellular location patterns in structured classification problems there is a direct conflict between expressive models and efficient inference while graphical models such as markov random fields or factor graphs can represent arbitrary dependences among instance labels the cost of inference via belief propagation in these models grows rapidly as the graph structure becomes more complicated one important source of complexity in belief propagation is the need to marginalize large factors to compute messages this operation takes time exponential in the number of variables in the factor and can limit the expressiveness of the models we can use in this paper we study a new class of potential functions which we call decomposable k way potentials and provide efficient algorithms for computing messages from these potentials during belief propagation we believe these new potentials provide a good balance between expressive power and efficient inference in practical structured classification problems we discuss three instances of decomposable potentials the associative markov network potential the nested junction tree and a new type of potential which we call the voting potential we use these potentials to classify images of protein subcellular location patterns in groups of cells classifying subcellular location patterns can help us answer many important questions in computational biology including questions about how various treatments affect the synthesis and behavior of proteins and networks of proteins within a cell our new representation and algorithm lead to substantial improvements in both inference speed and classification accuracy

discriminative learning of max sum classifiers the max sum classifier predicts n tuple of labels from n tuple of observable variables by maximizing a sum of quality functions defined over neighbouring pairs of labels and observable variables predicting labels as map assignments of a random markov field is a particular example of the max sum classifier learning parameters of the max sum classifier is a challenging problem because even computing the response of such classifier is np complete in general estimating parameters using the maximum likelihood approach is feasible only for a subclass of max sum classifiers with an acyclic structure of neighbouring pairs recently the discriminative methods represented by the perceptron and the support vector machines originally designed for binary linear classifiers have been extended for learning some subclasses of the max sum classifier besides the max sum classifiers with the acyclic neighbouring structure it has been shown that the discriminative learning is possible even with arbitrary neighbouring structure provided the quality functions fulfill some additional constraints in this article we extend the discriminative approach to other three classes of max sum classifiers with an arbitrary neighbourhood structure we derive learning algorithms for two subclasses of max sum classifiers whose response can be computed in polynomial time i the max sum classifiers with supermodular quality functions and ii the max sum classifiers whose response can be computed exactly by a linear programming relaxation moreover we show that the learning problem can be approximately solved even for a general max sum classifier

approximate belief updating in max 2 connected bayes networks is np hard a max 2 connected bayes network is one where there are at most 2 distinct directed paths between any two nodes we show that even for this restricted topology null evidence belief updating is hard to approximate

mini buckets a general scheme for bounded inference this article presents a class of approximation algorithms that extend the idea of bounded complexity inference inspired by successful constraint propagation algorithms to probabilistic inference and combinatorial optimization the idea is to bound the dimensionality of dependencies created by inference algorithms this yields a parameterized scheme called mini buckets that offers adjustable trade off between accuracy and efficiency the mini bucket approach to optimization problems such as finding the most probable explanation mpe in bayesian networks generates both an approximate solution and bounds on the solution quality we present empirical results demonstrating successful performance of the proposed approximation scheme for the mpe task both on randomly generated problems and on realistic domains such as medical diagnosis and probabilistic decoding

knowledge representation and inference in similarity networks and bayesian multinets we examine two representation schemes for uncertain knowledge the similarity network heckerman 1991 and the bayesian multinet these schemes are extensions of the bayesian network model in that they represent asymmetric independence assertions we explicate the notion of relevance upon which similarity networks are based and present an efficient inference algorithm that works under the assumption that every event has a nonzero probability another inference algorithm is developed that works under no restriction albeit less efficiently we show that similarity networks are not inferentially complete namely not every query can be answered nonetheless we show that a similarity network can always answer any query of the form what is the posterior probability of an hypothesis given evidence we call this property diagnostic completeizess finally we describe a generalization of similarity networks that can encode more types of asymmetric conditional independence assertions than can ordinary similarity networks

a linear time solution to the single function coarsest partition problem abstract the problem of finding the coarsest partition of a set s with respect to another partition of s one or more functions on s has several applications one of which is the state minimization of finite state automata in 1971 hopcroft presented an algorithm to solve the many function coarsest partition problem for sets of n elements in o n log n time and o n space in 1974 aho hopcroft and ullman presented an o n log n algorithm that solves the special case of this problem for only one function both these algorithms use a negative strategy that repeatedly refines the original partition until a solution is found we present a new algorithm to solve the single function coarsest partition problem in o n time and space using a different constructive approach our algorithm can be applied to the automated manufacturing of woven fabric

optimal dynamic treatment regimes summary a dynamic treatment regime is a list of decision rules one per time interval for how the level of treatment will be tailored through time to an individual s changing status the goal of this paper is to use experimental or observational data to estimate decision regimes that result in a maximal mean response to explicate our objective and to state the assumptions we use the potential outcomes model the method proposed makes smooth parametric assumptions only on quantities that are directly relevant to the goal of estimating the optimal rules we illustrate the methodology proposed via a small simulation