Title: Approximate inference of marginals using the IBIA framework
Venue: NeurIPS 2023
Accepted: yes
Keywords: Bayesian inference

posterior marginals

probabilistic graphical models
Abstract: Exact inference of marginals in probabilistic graphical models (PGM) is known to be intractable, necessitating the use of approximate methods. Most of the existing variational techniques perform iterative message passing in loopy graphs which is slow to converge for many benchmarks. In this paper, we propose a new algorithm for marginal inference that is based on the incremental build-infer-approximate (IBIA) paradigm. Our algorithm converts the PGM into a sequence of linked clique tree forests (SLCTF) with bounded clique sizes, and then uses a heuristic belief update algorithm to infer the marginals. For the special case of Bayesian networks, we show that if the incremental build step in IBIA uses the topological order of variables then (a) the prior marginals are consistent in all CTFs in the SLCTF  and (b) the posterior marginals are consistent once all evidence variables are added to the SLCTF. In our approach, the belief propagation step is non-iterative and the accuracy-complexity trade-off is controlled using user-defined clique size bounds. Results for several benchmark sets from recent UAI competitions show that our method gives either better or comparable accuracy than existing variational and sampling based methods, with smaller runtimes.

Approximate inference of marginals using the IBIA framework

Abstract

Exact inference of marginals in probabilistic graphical models (PGM) is known to be intractable, necessitating the use of approximate methods. Most of the existing variational techniques perform iterative message passing in loopy graphs which is slow to converge for many benchmarks. In this paper, we propose a new algorithm for marginal inference that is based on the incremental build-infer-approximate (IBIA) paradigm. Our algorithm converts the PGM into a sequence of linked clique tree forests (SLCTF) with bounded clique sizes, and then uses a heuristic belief update algorithm to infer the marginals. For the special case of Bayesian networks, we show that if the incremental build step in IBIA uses the topological order of variables then (a) the prior marginals are consistent in all CTFs in the SLCTF and (b) the posterior marginals are consistent once all evidence variables are added to the SLCTF. In our approach, the belief propagation step is non-iterative and the accuracy-complexity trade-off is controlled using user-defined clique size bounds. Results for several benchmark sets from recent UAI competitions show that our method gives either better or comparable accuracy than existing variational and sampling based methods, with smaller runtimes.

1 Introduction

Discrete probabilistic graphical models (PGM) including Bayesian networks (BN) and Markov networks (MN) are used for probabilistic inference in a wide variety of applications. An important task in probabilistic reasoning is the computation of posterior marginals of all the variables in the network. Exact inference is known to be #P-complete [Roth, 1996], thus necessitating approximations. Approximate techniques can be broadly classified as sampling based and variational methods.

Sampling based methods include Markov chain Monte Carlo based techniques like Gibbs sampling [Gelfand, 2000, Kelly et al., 2019] and importance sampling based methods [Gogate and Dechter, 2011, Friedman and Van den Broeck, 2018, Kask et al., 2020, Broka, 2018, Lou et al., 2019, 2017a,b, Marinescu et al., 2019, 2018]. An advantage of these methods is that accuracy can be improved with time without increasing the required memory. However, in many benchmarks the improvement becomes slow with time. Moreover, many of the recent sampling/search based techniques Kask et al. [2020], Broka [2018], Lou et al. [2019, 2017a,b], Marinescu et al. [2019, 2018] have been evaluated either for approximate inference of partition function (PR) or for finding the marginal maximum a posteriori assignment (MMAP). Currently, there are no published results for posterior marginals (MAR) using these methods, and the publicly available implementations do not support the MAR task. Alternatively, variational techniques can be used. These include loopy belief propagation (LBP) [Frey and MacKay, 1998] region-graph based techniques like generalized belief propagation (GBP) [Yedidia et al., 2000] and its variants [Heskes et al., 2003, Mooij and Kappen, 2007, Lin et al., 2020], mini-bucket based schemes like iterative join graph propagation

(IJGP) [Mateescu et al., 2010] and weighted mini-bucket elimination (WMB) [Liu and Ihler, 2011] and methods that simplify the graph structure like edge deletion belief propagation (EDBP) and the related relax-compensate-recover (RCR) techniques [Choi et al., 2005, Choi and Darwiche, 2006, 2010]. While the accuracy-complexity trade-off can be achieved using a single user-defined clique size bound in mini-bucket based methods, it is non-trivial in many of the other region graph based methods. Most of these techniques use iterative message passing to solve an optimization problem, for which convergence is not guaranteed and even if possible, can be slow to achieve. Non-iterative methods like Deep Bucket Elimination (DBE) [Razeghi et al., 2021] and NeuroBE [Agarwal et al., 2022] are extensions of bucket elimination that approximate messages using neural networks. However, training these networks takes several hours. Moreover, the publicly available implementations of these methods do not support the MAR task.

The recently proposed incremental build-infer-approximate (IBIA) framework [Bathla and Vasudevan, 2023] uses a different approach. It converts the PGM into a sequence of calibrated clique tree forests (SCTF) with clique sizes bounded to a user-defined value. Bathla and Vasudevan [2023] show that the normalization constant (NC) of clique beliefs in the last CTF in the sequence is a good approximation of the partition function of the overall distribution. This framework has two main advantages. Firstly, since it is based on clique trees and not loopy graphs, the belief propagation step is non-iterative. Therefore, it is fast and has no issues related to convergence. Secondly, it provides an easy control of the accuracy complexity trade-off using two user-defined parameters and hence can be used in anytime manner. However, the framework in Bathla and Vasudevan [2023] cannot be used to infer marginals. This is because only the clique beliefs in the last CTF account for all factors in the PGM. Beliefs in all other CTFs account for a subset of factors and thus, cannot be used for inference of marginals.

Contributions of this work: In this paper, we propose a method for marginal inference that uses the IBIA framework. We show that the approximation algorithm used in this framework preserves the within-clique beliefs. Based on this property, we modify the data structure generated by IBIA to add links between adjacent CTFs. We refer to the modified data structure as a sequence of linked clique tree forests (SLCTF). We propose a heuristic belief update algorithm that back-propagates beliefs from the last CTF to the previous CTFs via the links and re-calibrates each CTF so that the updated beliefs account for all factors in the PGM. We also propose a greedy heuristic for the choice of links used for belief update. Results for several UAI benchmark sets show that our method gives an accuracy that is better than or comparable to the existing variational and sampling based methods, with competitive runtimes.

For the special case of BNs, we show that if the incremental build step in IBIA is performed in the topological order of variables then (a) the estimated partition function is guaranteed to be one if no evidence variables are present (b) the prior marginals of all variables are consistent across all CTFs in the sequence and (c) once all the evidence variables have been added to the SLCTF, the posterior marginals of variables in subsequent CTFs are consistent. Our results show that using the topological ordering for BNs leads to better estimates of partition function, prior marginals and posterior marginals in most benchmarks.

2 Background

2.1 Discrete Probabilistic Graphical Models

Let X = { X 1 , X 2 , · · · X n } be a set of random variables with associated domains D = { D X 1 , D X 2 , · · · D X n } . The probabilistic graphical model (PGM) over X consists of a set of factors, Φ , where each factor ϕ α ( X α ) ∈ Φ is defined over a subset of variables, Scope ( ϕ α ) = X α . If D α denotes the Cartesian product of the domains of variables in X α , then ϕ α : D α → R ≥ 0 . The joint probability distribution captured by the PGM is P ( X ) = 1 Z ∏ α ϕ α , where Z = ∑ X ∏ α ϕ α is the partition function. PGMs can be broadly classified as Markov networks (MN) which are the undirected models and Bayesian networks (BN) which are the directed models.

One method to perform exact inference involves converting the PGM into a clique tree (CT), which is a hypertree where each node C i is a clique that contains a subset of variables. We use the term C i as a label for the clique as well as to denote the set of variables in the clique. An edge between C i and C j is associated with a set of sepset variables denoted S i,j = C i ∩ C j . Exact inference in a CT is done using the belief propagation (BP) algorithm [Lauritzen and Spiegelhalter, 1988] that is

Incremental

Build

Add factors

while max-

clique-size

mcs

p

CTF

k

Φ

k

= 1

CTF

0

Infer

Calibrate

using Belief

Propagation

Φ

.empty

()

Approximate

Reduce

clique sizes

to

mcs

im

CTF

k

,

a

k

=

k

+

1

No

CTF

k

Stop

Yes

(a)

Φ

CTF

0

CTF

1

CTF

1

,

a

CTF

2

Initialize

Incremental Build

+ Infer

Approximate

Incremental Build

+ Infer

dmo

jh

il

dmo

fdhm

hmj

dhk

hmj

hi

il

fmn

hm

i

h

dm

dh

fm

mo

fhm

hl

dhk

h

m

h

fmoh

ohkl

dhk

ho

hk

(b)

equivalent to two rounds of message passing along the edges of the CT. The algorithm returns a CT with calibrated clique beliefs β ( C i ) and sepset beliefs µ ( S i,j ) . In a calibrated CT, all clique beliefs have the same normalization constant ( Z ) and beliefs of all adjacent cliques agree over the marginals of the sepset variables. The joint probability distribution, P ( X ) , can be rewritten as follows.

where V T and E T are the set of nodes and edges in the CT. The marginal probability distribution of a variable X i can be obtained by marginalizing the belief of any clique C that contains X i as follows.

We use the following definitions in this paper.

Definition 1. Clique Tree Forest (CTF): Set of disjoint clique trees.

Definition 2. Clique size: The clique size cs of a clique C is the effective number of binary variables contained in C . It is computed as follows.

where | D X i | is the cardinality or the number of states in the domain of the variable X i .

Definition 3. Prior marginals ( P ( X i ) ): It is the marginal probability of a variable X i when the PGM has no evidence variables.

Definition 4. Posterior marginals ( P ( X i | E = e ) ): It is the conditional probability distribution of a variable X i , given a fixed evidence state e .

2.2 Overview of IBIA framework

Methodology: The IBIA framework proposed in Bathla and Vasudevan [2023] converts the PGM into a sequence of calibrated CTFs (SCTF) with bounded clique sizes. IBIA starts with an initial

CTF, CTF 0 , that contains cliques corresponding to factors with disjoint scope. Figure 1a illustrates the overall methodology used in the framework. It uses three main steps as described below.

Incremental Build: In this step, a CTF, CTF k , is constructed by incrementally adding factors in the PGM to an initial CTF ( CTF 0 or CTF k -1 ,a ) until the maximum clique size reaches a user-specified bound mcs p . Methods for incremental construction of CTs have been proposed in Bathla and Vasudevan [2023] and Flores et al. [2002]. Either of the two methods can be used to incrementally add new factors.

Infer: In this step, all CTs in CTF k are calibrated using the standard belief propagation algorithm.

Approximate: In this step, CTF k is approximated to give an approximate CTF, CTF k,a , that has clique sizes reduced to another user-specified bound mcs im .

As shown in Figure 1a, IBIA constructs the SCTF = { CTF 1 , . . . , CT F n } by repeatedly using the incremental build, infer and approximate steps. This process continues until all factors in the PGM are added to some CTF in the SCTF. Figure 1b shows the SCTF generated by IBIA for an example PGM, Φ (specified in the caption to the figure), with clique size bounds mcs p and mcs im set to 4 and 3 respectively. For the purpose of illustrating all steps, the disjoint cliques corresponding to factors ϕ ( d, m, o ) , ϕ ( i, l ) and ϕ ( j, h ) are chosen as the initial CTF, CTF 0 . CTF 1 is constructed by incrementally adding factors to CTF 0 . All factors except ϕ ( k, l, o ) and ϕ ( f, o ) are added to CTF 1 . These two factors are deferred since their addition results in clique sizes greater than mcs p = 4 . CTF 1 is calibrated using BP and then approximated to give CTF 1 ,a with clique sizes bounded to mcs im = 3 . CTF 2 is constructed by adding the remaining factors to CTF 1 ,a . We will use this example to explain the steps in our method for inference of marginals.

Approximate step: Since our inference algorithm is based on the properties of the approximate CTF, we explain this step in more detail using the running example shown in Figure 1b. Variables f, k, l and o in CTF 1 are also present in the deferred factors ϕ ( k, l, o ) and ϕ ( f, o ) . These variables are needed for the construction of subsequent CTFs and are called interface variables (IV). All other variables in CTF 1 are called non-interface variables (NIV). CTF 1 ,a is initialized as the minimal subgraph that connects the IVs. This subgraph contains all cliques in CTF 1 except clique fmn . Approximation involves two main steps to reduce the number of cliques and clique sizes.

1. Exact marginalization: The goal of this step is to remove as many NIVs as possible while ensuring that the overall joint distribution is preserved. NIV j is present in a single clique hmj and is marginalized out from it by summing over the states of j . The resulting clique hm is a non-maximal clique that is contained in clique fdhm , and is thus removed. NIV i is removed after collapsing the two containing cliques hi and il . Exact marginalization of the other NIVs results in collapsed cliques with size greater than mcs im = 3 , and is not performed.

2. Local marginalization: In this step, clique sizes are reduced by marginalizing variables from individual cliques with size greater than mcs im while ensuring that (a) CTF 1 ,a is a valid CTF that satisfies the running intersection property (RIP) (b) a connected CT in CTF 1 remains connected in CTF 1 ,a and (c) CTF 1 ,a contains all IVs. To reduce the size of the large-sized clique fdhm , NIV d is locally marginalized from this clique. In order to satisfy RIP, it needs to be marginalized from either clique dmo or dhk as well as the corresponding sepsets. Here, d is locally marginalized from clique dmo and the corresponding sepsets to give CTF 1 ,a as shown in the figure. Since all cliques containing d are not collapsed before marginalization, this results in an approximate joint distribution. Propositions 5 and 6 in Bathla and Vasudevan [2023] show that all CTs in the approximate CTF obtained after exact and local marginalization are valid and calibrated.

3 Inference of marginals

In this section, we first discuss some of the properties satisfied by the SCTF generated by IBIA. Based on these properties, we then describe the proposed methodology for inference of marginals.

3.1 Properties of SCTF

We show that each CTF k in SCTF and the corresponding approximate CTF, CTF k,a , satisfy the following properties. The detailed proofs for these properties are included in the supplementary material and the main ideas used in the proofs are discussed here.

Proposition 1. The joint belief of variables contained within any clique in the approximate CTF, CTF k,a , is the same as that in CTF k .

Proof Sketch. Both exact and local marginalization involve summing clique beliefs over the states of a variable, which does not alter the joint belief of the remaining variables in the clique.

Proposition 2. The clique beliefs in CTF k account for all factors added to { CTF 1 , . . . , CT F k }.

Proof Sketch. CTF 1 is exact, with clique beliefs corresponding to the partial set of factors used to form CTF 1 . CTF 1 ,a is a calibrated CTF that is obtained after approximating CTF 1 . Thus, the joint belief of variables in CTF 1 ,a is an approximation of the beliefs encoded by factors added to CTF 1 . CTF 2 is obtained after adding new factors to CTF 1 ,a . Therefore, after calibration, clique beliefs in CTF 2 account for factors added to CTF 1 and CTF 2 . The proposition follows, since the same argument holds true for all CTFs in the sequence.

BNs can be handled in a similar manner as MNs by using the undirected moralized graph corresponding to the BN. Another possibility is to use the directed acyclic graph (DAG) corresponding to the BN to guide the incremental build step. In contrast to MNs where all factors are un-normalized, each factor in a BN is the conditional probability distribution (CPD) of a variable y given the state of its parents in the DAG ( Pa y ). Factors corresponding to the evidence variables are simplified based on the given state and hence become un-normalized.

For BNs, the following properties hold true if each CTF in the SCTF is built by adding factors in the topological order. By this, we mean that the factor corresponding to the variable y is added only after the factors corresponding to all its parent variables, Pa y , have been added to some CTF in the sequence. Let Y k denote the set of variables whose CPDs are added during construction of CTF k , e k denote the evidence states of all evidence variables in Y k and Pa Y k denote the parents of variables in the set Y k .

Proposition 3. The product of factors added in CTFs, { CTF 1 , . . . , CT F k } is a valid joint probability distribution whose normalization constant is the probability of evidence states e 1 , . . . , e k .

Proof. For each variable y ∈ { Y 1 , . . . , Y k } , the corresponding CPD, P ( y | Pa y ) , is added to some CTF in { CTF 1 , . . . , CT F k } . The proposition follows since the CPDs corresponding to parents of y are always added to a CTF before the CPD of y is added.

Proposition 4. The normalization constant of the distribution encoded by the calibrated beliefs in CTF k is the estimate of probability of evidence states e 1 , . . . , e k .

Proof Sketch. Using Proposition 3, the normalization constant (NC) of the distribution encoded by CTF 1 is P ( e 1 ) . Using Proposition 1, the approximation algorithm preserves the within-clique beliefs and hence the NC. Thus, the NC of CTF 1 ,a is also P ( e 1 ) . Although the NC is the same, the overall distribution corresponding to CTF 1 ,a is approximate due to local marginalization. CTF 2 is constructed by adding CPDs of variables in Y 2 to CTF 1 ,a . CPDs of parent variables in Pa Y 2 are added either in CTF 1 or CTF 2 . Hence, after calibration, we get a valid probability distribution with NC as the estimate of probability of evidence states e 1 , e 2 . A similar procedure can be used to show that the property holds for all CTFs.

Corollary 1. For a BN with no evidence variables, the normalization constant of any CT in CTF k is guaranteed to be one.

Theorem 1. Let I E denote the index of the last CTF in the sequence where the factor corresponding to an evidence variable is added. The posterior marginals of variables present in CTFs { CTF k , k ≥ I E } are preserved and can be computed from any of these CTFs.

Proof Sketch. Once all evidence variables are added, additional CPDs added in each new CTF in { CTF k , k > I E } correspond to the successors in the BN. Since none of the successors are evidence variables, the corresponding CPDs are normalized and hence cannot change the beliefs of the previous variables.

Corollary 2. For a BN with no evidence variables, the estimate of prior marginals obtained from any CTF in the sequence is the same.

3.2 Proposed algorithm for inference of marginals

We first explain our approach for estimation of marginals with the help of the example shown in Figure 1b. Following this, we formally describe the main steps in our algorithm.

The SCTF generated by IBIA for the example in Figure 1b contains two CTFs, CTF 1 and CTF 2 . Using Proposition 2, we know that calibrated clique beliefs in CTF 2 account for all factors in the PGM, Φ . Therefore, the marginals of all variables present in it can be inferred using Equation 2. However, clique beliefs in CTF 1 do not account for factors ϕ ( k, l, o ) and ϕ ( f, o ) which were added during the construction of CTF 2 . Therefore, in order to infer the marginals of variables n, j, i that are present only in CTF 1 , we need to update the beliefs to account for these two factors.

̸

Figure 2 shows CTF 1 , CTF 1 ,a and CTF 2 for the example. Using Proposition 1, we know that the joint belief of variables present within any clique in CTF 1 ,a is the same as that in CTF 1 . However, this belief changes when new factors are added during the construction of CTF 2 . For instance, β ( C 2 ′ ) = ∑ d β ( C 2 ) = ∑ o β ( ˜ C 2 ) . To reflect the effect of new factors added in CTF 2 , the joint belief of variables in clique C 2 can be updated as follows.

To make sure that CTF 1 remains calibrated, this must be followed by a single round of message passing in CTF 1 with C 2 as the root node. It is clear that a similar belief update is needed for all the variables in CTF 1 ,a . However, every update and subsequent round of message passing will override the previous updates.

Hence, the beliefs in CTF 1 will only approximately reflect the effect of additional factors in CTF 2 . To improve accuracy, we propose a heuristic procedure for belief update sequence.

Formally, the steps in our algorithm are as follows. Variables present in CTF k,a are present in both CTF k and CTF k +1 . We refer to these variables as the link variables . We first find links between corresponding cliques C ∈ CTF k , C ′ ∈ CTF k,a and ˜ C ∈ CTF k +1 . Each link ( C, C ′ , ˜ C ) is associated with a set of link variables V l = C ∩ C ′ . For the example, links between CTF 1 , CTF 1 ,a and CTF 2 , and the corresponding link variables are shown in magenta in Figure 2. The first part of a link contains cliques C ′ and C . It is obtained as follows.

(a) If C ′ is obtained after collapsing a set of cliques { C 1 , · · · C m } in CTF k ,

CTF

1

:

˜

C

3

:

dhk

C

3

′

:

dhk

C

3

:

dhk

C

2

:

fdhm

C

4

:

hi

C

5

:

il

C

1

:

dmo

C

0

:

fmn

C

6

:

hmj

CTF

1

,

a

:

C

1

′

:

mo

C

2

′

:

fhm

C

4

′

:

hl

CTF

2

:

˜

C

2

:

fmoh

˜

C

4

:

ohkl

fhm

h

dhk

mo

l

C ′ is linked to each of { C 1 , · · · C m } . For example, C 4 ′ is linked to C 4 and C 5 , which were collapsed during exact marginalization of variable i .

(b) If C ′ is obtained from C in CTF k after local marginalization, C ′ is linked to C . In the example, cliques C 1 ′ and C 2 ′ are obtained after local marginalization of variable d from cliques C 1 and C 2 respectively. Hence, the corresponding tuples are ( C 1 , C 1 ′ ) and ( C 2 , C 2 ′ ) .

(c) If C ′ is same as clique C in CTF k , C ′ is linked to C . For example, C 3 is linked to C 3 ′ .

The second part links C ′ to ˜ C in CTF k +1 such that C ′ ⊆ ˜ C . This is always possible since CTF k +1 is obtained after incrementally modifying CTF k,a to add new factors. Thus, each clique in CTF k,a is contained in some clique in CTF k +1 . For example, C 1 ′ ⊂ ˜ C 2 and the link is ( C 1 , C 1 ′ , ˜ C 2 ) .

Let L k denote the set of links between CTF k and CTF k +1 . We refer to the modified data structure consisting of the sequence of calibrated CTFs, SCTF = { CTF k } and a list of links between all adjacent CTFs, SL = { L k } , as the sequence of linked CTFs (SLCTF) . Once the SLCTF is created, starting from the final CTF in the SLCTF, we successively back-propagate beliefs to the preceding CTFs via links between adjacent CTFs. To back-propagate beliefs from CTF k +1 to CTF k , we choose a subset of links in L k based on heuristics, which will be discussed later. Then, for each selected link ( C, C ′ , ˜ C ) , we update the belief associated with clique C as follows.

where, V l = C ∩ C ′ . This is followed by one round of message passing from C to all other cliques in the CT containing C . Once all CTFs are re-calibrated, we infer the marginal distribution of a variable using Equation 2 from the last CTF in which it is present.

For BNs, if incremental build is performed by adding variables in the topological order, then as shown in Theorem 1, the singleton marginals are consistent in CTFs { CTF k ≥ I E } , where I E is the index of the last CTF to which the factor corresponding to an evidence variable is added. Therefore, in this case, back-propagation of beliefs can be performed starting from CTF I E instead of starting from the last CTF. This reduces the effort required for belief update.

Heuristics for choice of links : To ensure that beliefs of all CTs in CTF k are updated, at least one link must be chosen for each CT. It is also clear that for any CT in CTF k more than one link may be required since variables that have low correlations in CTF k could become tightly correlated when new factors are added in CTF k +1 . However, belief update via all links is expensive since a round of message passing is required for each link. Based on results over many benchmarks, we propose the following greedy heuristic to choose and schedule links for backward belief update.

(a) To minimize the number of selected links, we first choose a subset of link variables for which the difference in posterior marginals in adjacent CTFs, CTF k and CTF k +1 , is greater than a threshold. Next, for belief update, we select a minimal set of links that cover these variables.

(b) The updated beliefs depend on the order in which the links are used for update. Based on the difference in marginals, we form a priority queue with the cliques containing link variables that have the lowest change in marginals having the highest priority. This is to make sure that large belief updates do not get over-written by smaller ones. This could happen for example, if two variables, v 1 and v 2 , that are highly correlated in CTF k become relatively uncorrelated in CTF k +1 due to the approximation. Assume that new factors added to CTF k +1 affect v 1 but not v 2 . A belief update via the link containing v 1 will make sure that its belief is consistent in CTF k and CTF k +1 . Later, if we perform a belief update using a link containing v 2 , the previous larger belief update of v 1 will be overwritten by something smaller since the belief of v 2 is not very different in the two CTFs.

Complexity : Let N CTF be the number of CTFs in the sequence, and N l be the maximum number of selected links between any two adjacent CTFs. IBIA requires 2 N CTF rounds of message passing for calibrating CTFs and ( N CTF -1) N l rounds for belief update. Therefore, the runtime complexity of IBIA is O ( N CTF N l 2 mcs p ) . To perform backpropagation of beliefs, all CTFs need to be stored and the memory complexity is O ( N CTF 2 mcs p ) .

4 Results

All experiments were carried out on an Intel i9-12900 Linux system running Ubuntu 22.04.

Error metrics: For each non-evidence variable X i , we measure error in terms of Hellinger distance between the exact marginal distribution P ( X i ) and the approximate marginal distribution Q ( X i ) . It is computed as follows.

We use two metrics namely, the average Hellinger distance (denoted as HD avg ) and the maximum Hellinger distance (denoted as HD max ) over all non-evidence variables in the network.

Benchmarks: Weused the benchmark sets included in UAI repository [Ihler, 2006] and the Bayesian network repository [Scutari, 2007]. We classify instances for which exact solutions are present in the repository as 'small' and others as 'large' .

Methods used for comparison: In the UAI 2022 inference competition [UAI, 2022], the uai 14 _ mar solver had the highest score for the MAR task. It is an amalgam of solvers that dumps solutions with different methods based on the given time and memory constraints. It uses loopy BP (LBP), generalized BP on loopy graphs where outer regions are selected using mini-bucket heuristics, and cutset conditioning of GBP approximations. The implementation of this solver is not publicly available. Therefore, we have compared our results individually with methods that belong to categories of methods used in uai 14 _ mar . This includes LBP [Murphy et al., 1999], IJGP [Mateescu et al., 2010] and sample search [Gogate and Dechter, 2011] which is an importance sampling based technique that uses an IJGP based proposal and cutset sampling (referred to as 'ISSwc' in this paper). We also compare our results with weighted mini-bucket elimination (WMB) [Liu and Ihler, 2011]. Additional results showing a comparison with results published in Kelly et al. [2019] are included in the supplementary material.

Evaluation setup: The implementation of LBP and WMB were taken from LibDAI [Mooij, 2010, 2012] and Merlin [Marinescu, 2016] respectively. For IJGP and ISSwc, we used implementations [Gogate, 2010, 2020] provided by the authors of these methods. LBP, IJGP, ISSwc and WMB are implemented in C++. IBIA on the other hand has been implemented in Python3 and is thus, at a disadvantage in terms of runtime. We report results with runtime limits of 2 min and 20 min for small instances. In all cases, the memory limit was set to 8GB, which is the same as that used in UAI 2022 competition [UAI, 2022]. For IBIA, we set the maximum clique size bound mcs p to 20 (referred to as 'IBIA20' ) when the time limit is 2 min and we set it to 23 (referred to as 'IBIA23' ) when the time limit is 20 min. mcs im is empirically chosen as 5 less than mcs p . The evaluation setup used for other methods is included in the supplementary material.

Results: Table 1 has the results for the small benchmark sets. It reports the average of HD avg and HD max over all instances in each set. We compare results obtained using LBP, WMB, IJGP, IBIA20, IBIA23 and ISSwc for both time constraints. The minimum error obtained for each time limit is marked in bold. IBIA20 and IBIA23 solve all small instances within 2 min and 20 min respectively. In 2 min, the accuracy obtained with IBIA20 is better than all other solvers for most benchmarks. For ObjDetect, Segment and Protein, it is comparable to WMB, IJGP/ISSwc and LBP respectively, which give the least errors for these testcases. In 20 min, IBIA23 gives lower or comparable errors in all testcases except DBN and ObjDetect. For DBNs, ISSwc reduces to exact inference in most instances and hence error obtained is close to zero. For ObjDetect, IJGP gives the least error closely followed by IBIA23. Note that for many benchmarks the accuracy obtained with IBIA20 in 2 min is either better than or comparable to the accuracy obtained with other solvers in 20 min.

A comparison of IBIA with results published in Kelly et al. [2019] for Gibbs sampling with Raoblackwellisation (ARB) and IJGP is included in the supplementary material. It is seen that error obtained with IBIA is lower than both methods in majority of the testcases.

For BN instances, Table 2 compares the results obtained using IBIA20 when CTFs are constructed by adding factors in the topological order (columns marked as 'TP') with that obtained using a non-topological order (columns marked as 'NTP'). We compare the maximum error in partition function (PR) and the average HD max over all instances in each benchmark set.

We observe that the topological ordering gives better accuracy for both PR and marginals in all testcases except Pedigree. The advantage of this ordering is that once all the evidence variables are added, no belief-update is needed is needed for the subsequent CTFs (using Theorem 1). So, the number of belief update steps is lower, resulting in lower errors. However, a drawback of this ordering is that it is rigid and it sometimes results in a larger number of CTFs in the sequence which could lead to larger errors if all the evidence variables are added in later CTFs. When no evidence variables are present (e.g.

GridBN, Bnlearn), both runtime and memory complexity is lower with topological ordering since marginals are consistent in all CTFs (using Corollary 2) and belief update is not needed. The average runtime with and without topological ordering was 1s and 146s respectively for GridBN instances and 0.3s and 1.3s for Bnlearn testcases.

To evaluate the scalability of the proposed algorithm, we ran it for large networks where the exact solutions are not known. Table 3 tabulates the percentage of large instances in each benchmark set that could be solved using IBIA within 2 min, 20 min and 60 min. For this experiment, we start with mcs p =20 and allow it to increase if incremental build results in a CTF with larger clique sizes. IBIA could solve all large instances in benchmark sets BN, Promedas, ObjDetect and Segmentation and most instances in Protein within 20 min. For other benchmarks, additional

instances could be solved when the runtime was increased to 60 min. The memory required for the remaining Grids, DBN, CSP and Protein instances is more than 8 GB. The increased memory usage is due to the following reasons. Firstly, all calibrated CTFs in the SLCTF need to be stored in order to allow for back-propagation of beliefs and the memory required increases with the number of CTFs. The average number of CTFs in the remaining Grid, DBN and CSP benchmarks is 22, 58 and 80 respectively. Secondly, for benchmarks with large variable domain sizes, the number of variables present in each clique in a CTF is small. Therefore, approximation using exact and local marginalization becomes infeasible and the subsequent CTFs have clique sizes greater than mcs p , which results in increased memory usage. This is seen in 9 out of 395 Protein instances and 12 out of 54 CSP instances. In addition to memory, the remaining Type4b instances also require additional runtime. This is because during belief update of each CTF, we perform one round of message passing for each selected link and the number of links is large in these instances.

5 Discussion

Limitations: While the belief update algorithm performs well for most benchmarks, it has some limitations. It is sequential and is performed link by link for each CTF that needs to be updated. The time and space complexity depends on the number of CTFs in the sequence and the number of selected links, which is large in some testcases. Also, after belief-update of all CTFs is completed, beliefs of variables present in multiple CTFs need not be consistent. However, good accuracies are obtained when beliefs are inferred from the last CTF containing the variable. For BNs, we found that building CTFs in the topological order gives larger errors in some cases. A possible extension would be to have an efficient build strategy where the ordering is decided dynamically based on the properties of the graph structure.

Comparison with related work : IBIA is similar to mini-bucket schemes in the sense that the accuracy-complexity tradeoff is controlled using a user-defined maximum clique size bound. While mini-bucket based schemes like IJGP [Dechter et al., 2002] and join graph linear programming [Ihler et al., 2012] use iterative message passing in loopy graphs, others like mini-bucket elimination (MBE), WMB[Liu and Ihler, 2011] and mini-clustering [Mateescu et al., 2010] are non-iterative approaches that approximate the messages by migrating the sum operator inside the product term. In contrast, IBIA constructs a sequence of clique trees. It performs belief propagation on approximate clique trees so that messages are exact and there are no issues of convergence.

Unlike sampling based techniques, there is no inherent randomness in IBIA that is each run gives the same results. There could be a variation if the order in which factors are added is changed. However, this variation is minimal since subsets of factors are added together in the incremental build step. In that sense, it is like mini-bucket based methods where results are the same if the variable elimination order and the partitioning technique used to generate the initial choice of mini-buckets are the same.

In order to construct the sequence, IBIA requires a fast and accurate method to approximate clique trees by reducing clique sizes. The aim is to preserve as much as possible, the joint distribution of the interface variables. This is achieved by marginalizing out variables from large-sized cliques and a minimal set of neighbors without disconnecting clique trees. IBIA gives good accuracy since variables that are removed are chosen based on a mutual information (MI) based metric and a sufficient number of non-interface variables are retained so that a CT is never disconnected. In contrast, the Boyen Koller (BK) [Boyen and Koller, 1998, Murphy, 2002] and factored frontier (FF) approximations, although fast, retain only the interface variables which can disconnect the CT resulting in larger errors due to the underlying independence approximation. The thin junction tree approximations proposed in Kjærulff [1994] and Hutter et al. [2004] split cliques that contain a pair of variables that is not present in any of its sepsets. However, if there are large cliques that have only sepset variables (which is typical in most benchmarks), then the split has to be done iteratively starting from leaf nodes of multiple branches, until the large-sized clique can be split. When such cliques are centrally located in the CT, this process is both time-consuming and would result in approximation of a much larger set of cliques. Similar to BK and FF, this method can also disconnect the CT.

The other thin junction tree methods [Bach and Jordan, 2001, Elidan and Gould, 2008, Dafna and Guestrin, 2009] choose an optimum set of features based on either KL distance or a MI based metric. They cannot be directly used since IBIA requires a good approximation of the joint distribution of only the interface variables. Also, these methods are typically iterative and not very fast.

References

Sakshi Agarwal, Kalev Kask, Alex Ihler, and Rina Dechter. NeuroBE: Escalating neural network approximations of bucket elimination. In Uncertainty in Artificial Intelligence , pages 11-21, 2022.

Francis Bach and Michael Jordan. Thin junction trees. Advances in neural information processing systems , 14, 2001.

Shivani Bathla and Vinita Vasudevan. IBIA: An incremental build-infer-approximate framework for approximate inference of partition function. Transactions on Machine Learning Research , 2023. ISSN 2835-8856.

Xavier Boyen and Daphne Koller. Tractable inference for complex stochastic processes. In Uncertainty in Artificial Intelligence , pages 33-42, 1998.

Filjor Broka. Abstraction sampling in graphical models. In AAAI Conference on Artificial Intelligence , pages 8010-8011, 2018.

Choi, H. Chan, and A. Darwiche. On Bayesian network approximation by edge deletion. In Uncertainty in Artificial Intelligence , pages 128-135, 2005.

Arthur Choi and Adnan Darwiche. An edge deletion semantics for belief propagation and its practical impact on approximation quality. In National Conference on Artificial Intelligence and Innovative Applications of Artificial Intelligence Conference , pages 1107-1114, 2006.

Arthur Choi and Adnan Darwiche. Relax, compensate and then recover. In JSAI International Symposium on Artificial Intelligence , pages 167-180, 2010.

Shahaf Dafna and Carlos Guestrin. Learning thin junction trees via graph cuts. In Artificial Intelligence and Statistics , pages 113-120, 2009.

Rina Dechter, Kalev Kask, and Robert Mateescu. Iterative join-graph propagation. In Uncertainty in Artificial Intelligence , pages 128-136, 2002.

Gal Elidan and Stephen Gould. Learning bounded treewidth Bayesian networks. Journal of Machine Learning Research , 9(12), 2008.

MJulia Flores, José A Gámez, and Kristian G Olesen. Incremental compilation of Bayesian networks. In Uncertainty in Artificial Intelligence , pages 233-240, 2002.

Brendan J Frey and David MacKay. A revolution: Belief propagation in graphs with cycles. In Advances in Neural Information Processing Systems , volume 10, 1998.

Tal Friedman and Guy Van den Broeck. Approximate knowledge compilation by online collapsed importance sampling. Advances in neural information processing systems , 31, 2018.

Alan E Gelfand. Gibbs sampling. Journal of the American statistical Association , 95(452):1300-1304, 2000.

Vibhav Gogate. Iterative join graph propagation. https://personal.utdallas.edu/~vibhav. gogate/ijgp.html , 2010. Accessed: 2023-04-15.

Vibhav Gogate. IJGP-sampling and samplesearch (PR and MAR tasks). https://github.com/ dechterlab/ijgp-samplesearch , 2020. Accessed: 2023-01-15.

Vibhav Gogate and Rina Dechter. Samplesearch: Importance sampling in presence of determinism. Artificial Intelligence , 175(2):694-729, 2011.

Tom Heskes, Kees Albers, and Bert Kappen. Approximate inference and constrained optimization. In Uncertainty in Artificial Intelligence , pages 313-320, 2003.

Frank Hutter, Brenda Ng, and Richard Dearden. Incremental thin junction trees for dynamic Bayesian networks. Darmstadt University of Technology, Tech. Rep , 2004.

Alexander Ihler. UAI model files and solutions. http://sli.ics.uci.edu/~ihler/uai-data/ , 2006. Accessed: 2021-10-15.

Alexander T Ihler, Natalia Flerova, Rina Dechter, and Lars Otten. Join-graph based cost-shifting schemes. In Uncertainty in Artificial Intelligence , 2012.

Kalev Kask, Bobak Pezeshki, Filjor Broka, Alexander Ihler, and Rina Dechter. Scaling up and/or abstraction sampling. In International Joint Conference on Artificial Intelligence , pages 4266-4274, 2020.

Craig Kelly, Somdeb Sarkhel, and Deepak Venugopal. Adaptive Rao-Blackwellisation in Gibbs sampling for probabilistic graphical models. In Artificial Intelligence and Statistics , pages 29072915. PMLR, 2019.

Uffe Kjærulff. Reduction of computational complexity in Bayesian networks through removal of weak dependences. In Uncertainty in Artificial Intelligence , page 374-382, San Francisco, CA, USA, 1994.

Steffen L Lauritzen and David J Spiegelhalter. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society: Series B (Methodological) , 50(2):157-194, 1988.

Peng Lin, Martin Neil, and Norman Fenton. Improved high dimensional discrete Bayesian network inference using triplet region construction. Journal of Artificial Intelligence Research , 69:231-295, 2020.

Qiang Liu and Alexander Ihler. Bounding the partition function using holder's inequality. In International Conference on Machine Learning , pages 849-856, 2011.

Qi Lou, Rina Dechter, and Alexander Ihler. Anytime anyspace and/or search for bounding the partition function. In AAAI Conference on Artificial Intelligence , volume 31, 2017a.

Qi Lou, Rina Dechter, and Alexander T Ihler. Dynamic importance sampling for anytime bounds of the partition function. Advances in Neural Information Processing Systems , 30, 2017b.

Qi Lou, Rina Dechter, and Alexander Ihler. Interleave variational optimization with Monte Carlo sampling: A tale of two approximate inference paradigms. In AAAI Conference on Artificial Intelligence , volume 33, pages 7900-7907, 2019.

Radu Marinescu. Merlin. https://github.com/radum2275/merlin/ , 2016. Accessed: 202110-15.

Radu Marinescu, Junkyu Lee, Rina Dechter, and Alexander Ihler. AND/OR search for marginal map. Journal of Artificial Intelligence Research , 63:875-921, 2018.

Radu Marinescu, Akihiro Kishimoto, Adi Botea, Rina Dechter, and Alexander Ihler. Anytime recursive best-first search for bounding marginal map. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 7924-7932, 2019.

Robert Mateescu, Kalev Kask, Vibhav Gogate, and Rina Dechter. Join-graph propagation algorithms. Journal of Artificial Intelligence Research , 37:279-328, 2010.

Joris M. Mooij. libDAI: A free and open source C++ library for discrete approximate inference in graphical models. Journal of Machine Learning Research , 11:2169-2173, August 2010.

Joris M. Mooij. libDAI - A free/open source C++ library for discrete approximate inference. https://github.com/dbtsai/libDAI/ , 2012. Accessed: 2021-10-15.

Joris M. Mooij and Hilbert J. Kappen. Loop corrections for approximate inference on factor graphs. Journal of Machine Learning Research , 8(40):1113-1143, 2007.

Kevin P. Murphy, Yair Weiss, and Michael I. Jordan. Loopy belief propagation for approximate inference: An empirical study. In Uncertainty in Artificial Intelligence , pages 467-475, 1999.

Kevin Patrick Murphy. Dynamic Bayesian networks: Representation, inference and learning, dissertation. PhD thesis, UC Berkley, Dept. Comp. Sci , 1, 2002.

Yasaman Razeghi, Kalev Kask, Yadong Lu, Pierre Baldi, Sakshi Agarwal, and Rina Dechter. Deep bucket elimination. In International Joint Conference on Artificial Intelligence , pages 4235-4242, 2021.

Dan Roth. On the hardness of approximate reasoning. Artificial Intelligence , 82(1-2):273-302, 1996.

Marco Scutari. Bayesian network repository. https://www.bnlearn.com/bnrepository/ , 2007. Accessed: 2021-10-15.

UAI. UAI 2022 - Competition. https://www.auai.org/uai2022/uai2022_competition , 2022.

Jonathan S Yedidia, William T Freeman, Yair Weiss, et al. Generalized belief propagation. In Advances in Neural Information Processing Systems , volume 13, pages 689-695, 2000.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new approximation scheme for computing posterior marginals in graphical models such as Bayesian networks and Markov networks. The proposed approach is based on a previously introduced scheme called IBIA (Incremental Build Infer Approximate) which was initially developed for approximating the partition function. The idea is to construct a sequence of bounded size clique trees (called a clique tree forest) that eventually spans all the input factors and subsequently compute the posterior marginals by carefully selecting the appropriate cliques from the forest. The experimental evaluation is carried out on standard benchmarks for graphical models and the proposed scheme is compared with several state-of-the-art approximation scheme based on message passing. The results demonstrate that the proposed approximation scheme is competitive and some times outperforms existing approaches.\n\nSTRENGTHS:\nThe paper is relatively well written and organised. It is also relatively self contained and most of the concepts are discussed in sufficient detail. The empirical evaluation is sound and the results are presented in a relatively clear manner. Some theoretical properties of the proposed method are also discussed.\n\nWEAKNESSES:\nFirstly, the contribution presented in this paper appears to be quite incremental over the previous work the IBIA framework. Namely it consists of the message passing scheme over the additional links between adjacent clique trees. The proposed heuristic for choosing links appears to be quite arbitrary. \n\nSecondly, I think the proposed method is highly sensitive to the order in which factors are added to the clique tree forest, as well as the initial collection of factors chosen for initialisation. Therefore, the paper should discuss this aspect in more detail.\n\nThirdly, the paper does not mention anything about the convergence of the scheme nor its time/space complexity.\n\nQUESTIONS:\n1. As presented, the performance measure considered assumes that the exact marginals are available. However, many of the problem instances from the experimental section have large induced widths (e.g., 37 for GridBNs) and therefore are difficult to solve exactly, unless a lot of evidence is asserted in the network. Therefore, how many evidence variables are considered for each of the benchmark classes? And also, how do you compute the score when exact inference results are not available?\n\n2. What's the reason for defining the clique size in terms of log instead of just the number of variables in the clique? I found the log based definition confusing.\n\n3. Why do we need to consider two bounds on the clique size? I think the scheme is well defined with only one.\n\nLIMITATIONS:\nThe proposed scheme is clearly related to several existing scheme that aim to bound the size of the join-tree. Prominent examples are the mini-bucket scheme for variable elimination, the mini-clustering approach for cluster tree (join tree) elimination, as well as the iterative join-graph propagation scheme (all from Rina Dechter's group). Similarly, the Incremental Thin Junction Tree scheme by Frank Hutter is also looking at bounding the clique size of a join tree by splitting a larger clique into smaller cliques. Therefore, I think a deeper discussion on the connection between these previous schemes and the proposed IBIA based one is clearly called for (it is currently missing from the paper).\n\nThe empirical evaluation only considers a subset of the existing message passing schemes for approximating marginals e.g, LBP, IJGP, and SampleSearch. These methods are more than one decade old (if not more), do not really fall into the more recent category of variational inference methods. I think it would be interesting to compare with more recent methods based on weighted mini-buckets and cost-shifting such as those developed in Alex Ihler's group (including the importance sampling based ones). If you look at the recent UAI-2022 competition results, it looks like these kinds of methods performed best in the MAP (and to some extent in the PR category as well).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new approximate inference algorithm for probabilistic graphical models (PGMs). The approach uses a framework called IBIA to perform marginal inference in discrete PGMs. The main idea is to perform Belief Propagation on a sequence of clique trees. In each step the clique tree is approximated such that the number of variables in any factor is less than a bound and BP is performed on this approximate tree. Factors are then added to this tree and the messages are updated using heuristics to get the next clique tree in the sequence.\n\nSTRENGTHS:\nThe approach proposed in the paper seems to work well over a large set of UAI benchmarks in comparison with well-known inference methods. \nThe idea of computing marginals based on a sequence of linked Clique Tree Forests seems novel.\nThe idea of running BP for a single iteration to calibrate and obtaining good experimental results seems interesting since it can reduce computational complexity.\n\nWEAKNESSES:\nThe presentation could be improved since it seems a bit hard to understand all the steps. Perhaps a pseudo-code like algorithm might have helped. Also, since there have been a lot of approximate inference algorithms in PGMs, the main motivation for this approach was a bit unclear. For example, if I understand correctly the mini-bucket methods (Dechter 2003) also have a similar idea of approximating factors through mini-buckets. I think in the related work section the significance of the method can be perhaps better motivated in relation to these methods.\n\nOne aspect that was not very clear was when BP converges on the approximate clique tree, if we change the cliques (by adding factors) we cannot just update the beliefs without performing inference again I assumed. In that case, if we use heuristics to update the beliefs (as is mentioned) won’t the errors propagate through the sequence of clique trees? I think there are convergence guarantees on he marginals (e.g. as we add more clique trees in the sequence), that seems like a significant drawback since the scheme will not be any-time (as compared to several other techniques which have this property).\n\nQUESTIONS:\nWhat is the effect of longer vs shorter sequences, how will this affect the experiment results?\nHow is the idea conceptually related to other methods such as mini-buckets, etc.?\n\nLIMITATIONS:\nLimitations are mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new approximate inference method for marginals of probabilistic graphical models (PGMs) based on the incremental build-infer-approximate (IBIA) framework. IBIA was introduced by Bathla & Vasudevan in an arXiv preprint in April 2023 to estimate the partition function (normalizing constant). It achieves this by converting the PGM into a sequence of clique tree forests (SCTFs) with a user-defined clique size bound. The sequence is constructed iteratively: the first step (Build) adds factors of the PGM to the current CTF until the maximum clique size is reached; the second step (Infer) calibrates all clique trees using standard belief propagation; and the final step (Approximate) reduces the clique size to another user-defined bound.\n\nThis paper builds upon the previous IBIA work in the following ways: it observes and proves several properties of the SCTF generated by the IBIA framework, which it uses to design an extension of IBIA to approximate marginals (in addition to the partition function). This extension works by introducing links between cliques in subsequent steps of the SCTF based on shared variables, called sequence of linked CTFs (SLCTF). It backpropagates beliefs from the last CTF to previous CTFs via the links and re-calibrates each CTF with one round of message passing. Finally the marginal distribution can be inferred from the CTF containing the desired variable. The paper also describes heuristics for the choice of links and notes that in the directed case (i.e. Bayesian networks), it is advantageous to add variables in topological order during the incremental build step.\n\nThe experimental evaluation compares with two existing methods, loopy belief propagation (LBP) and iterative join graph propagation (IJGP), on benchmarks from recent UAI competitions with time limits of 2 min, 20 min, and, for some benchmarks, 1h. They report the number of instances solved within the time limit and the average and maximum Hellinger distance over all non-evidence variables. The new method performs performs better than LBP and IJGP on most examples and is similar on the remaining ones, even though IBIA is implemented in Python whereas the baselines are written in C++.\n\nSTRENGTHS:\nThe proposed approximate inference method is interesting and improves the performance compared to existing methods (LBP and IJGP). The extension of the IBIA framework with links is new and useful. The explanation is adequate given the page constraints and the use of a running example is very helpful to understand the algorithm.\n\nWEAKNESSES:\nThe contribution of this paper (adding links) seems to me to be relatively minor compared to the introduction of IBIA itself in previous work. Extending it in some way to relate successive CTFs (here done via links) seems quite obvious (although that might be the benefit of hindsight).\n\nRegarding the experimental evaluation, I had a look at the recent UAI 2022 competition, since the paper mentions that the same benchmarks were used for recent UAI competitions, if I understand line 14 correctly. It looks like IBIA was a competitor there and while performing well, it did not come in first, as the experimental evaluation of this paper might suggest. For this reason, I'm not convinced that the baselines in the experimental section are representative of the field of PGM inference and whether comparisons with more methods are needed to get an accurate picture of IBIA's performance (see my question to the authors below).\n\nAs a minor point: while the presentation is generally fine, I found the notation confusing at times, see below.\n\nQUESTIONS:\n(I will update my rating if the author's answers address my concerns.)\n\nMy main question is about the UAI 2022 competition: it seems to me that other submissions there performed better than the baselines in this paper (LBP, IJGP). Is this understanding correct? If so, what is the reason for choosing what seem to be inferior baselines?\n\nI was also confused by line 375, which seems to say that topological ordering gives large errors sometimes. Previously, it sounded like topological ordering was essential for good performance and for the theoretical results in Section 3.1 (line 163). Can you clarify the drawbacks and benefits of topological ordering? Do you know why it causes large errors sometimes?\n\nI also noticed some confusing notation, which I mention here for lack of a better place in the review form:\n- equation (1): (i - j) looks like subtraction. In graph theory, an edge is often written as a pair (i, j) instead.\n- equation (3): I don't understand the point of the quantifier ($\\forall$) in the product notation and have never seen this before.\n- line 100: $CTF_{k-1,a}$ first looked like a double-index to me, but I assume the \"a\" just stands for \"approximation\".\n\nTypos:\n- line 69: I think it should be $Scope(\\phi_\\alpha)$\n- Figure 1, incremental build: I think it should be \"while\" instead of \"until\"\n\nLIMITATIONS:\nThe authors have adequately discussed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}