Learning Tractable Probabilistic Models from Inconsistent Local Estimates
Authors: Shasha Jin
Vasundhara Komaragiri
Tahrima Rahman
Vibhav Giridhar Gogate
Venue: NeurIPS 2022 Accept (2022)
Keywords: Tractable Probabilistic Models
Parameter Learning
Inconsistent Data
Local Estimates

Abstract: Tractable probabilistic models such as cutset networks which admit exact linear time posterior marginal inference are often preferred in practice over intractable models such as Bayesian and Markov networks. This is because although tractable models, when learned from data, are slightly inferior to the intractable ones in terms of goodness-of-fit measures such as log-likelihood, they do not use approximate inference at prediction time and as a result exhibit superior predictive performance. In this paper, we consider the problem of improving a tractable model using a large number of local probability estimates, each defined over a small subset of variables that are either available from experts or via an external process. Given a model learned from fully-observed, but small amount of possibly noisy data, the key idea in our approach is to update the parameters of the model via a gradient descent procedure that seeks to minimize a convex combination of two quantities: one that enforces closeness via KL divergence to the local estimates and another that enforces closeness to the given model. We show that although the gradients are NP-hard to compute on arbitrary graphical models, they can be efficiently computed over tractable models. We show via experiments that our approach yields tractable models that are significantly superior to the ones learned from small amount of possibly noisy data, even when the local estimates are inconsistent.

Learning Tractable Probabilistic Models from Inconsistent Local Estimates
Shasha Jin, Vasundhara Komaragiri, Tahrima Rahman, and Vibhav Gogate The University of Texas at Dallas
Abstract
Tractable probabilistic models such as cutsel networks which admit exact linear time posterior marginal inference are often preferred in practice over intractable models such as Bayesian and Markov networks. This because although tractable models, when learned from data, slightly inferior to the intractable ones in terms of goodness-of inference at prediction time and result exhibit superior predictive performance. In this paper, we consider the problem of improving a tractable model using a large number of local probability estimates, each defined over small subset of variables that are either available from experts or via an external process. Given a model learned from fully-observed, but small amount of possibly noisy data, the key idea in our approach is to update the parameters of the model via gradient descent procedure that seeks minimize convex combination of two quantities: that enforces closeness via KI. divergence to the local estimates and another that enforces closeness the given model. We show that although the gradients are NP- hard to compute on arbitrary graphical models, they can be efficiently computed over tractable models. We show experiments that our approach yields tractable models that are significantly superior to the ones learned from small amount of possibly noisy data, even when the local estimates are inconsistent.
measures such as log-likelihook they do not use approximate
Introduction
A major issue with probabilistic graphical models (PGMs) 3 g such as Bayesian and Markov networks is that in most real-world domains, exact inference is computationally infeasible. result, approximate inference algorithms often used in practice, but these algorithms can be inaccurate and may exhibit high variability. Thus, even if PGM performs well on goodness-offfr measures such as test set log- lieelood,oood, its predictive performance, because of the use of approximate inference methods, can be quite poor.
To combat this problem, the field of tractable probabilistic models (TPMs) probabilistic circuits (PCs) IT 20 seeks to learn probabilistic models that admit polynomial time exact inference algorithms. TPMs are more biased or less expressive than general probabilistic models (because they enforce tractability) and result may yield slightly inferior (model) fit. But, the hope that because no approximation are used at prediction time the slight drop in goodness-of-ft measures is offset by superior predictive performance. This hope is often realized practice, and demonstrated in numerous empirical studies in literature (for example, see [18 20
To date, algorithms for learning the structure and parameters of tractable probabilistic models such as cutset networks [17], sum-product networks [14] and probabilistic sentential decision diagrams 9 assume access either full data which has no missing values or almost full data in which only a few variables have missing values. Often in the real world, however, the following scenario is quite
common. The learning algorithm has access to a small amount of full, possibly noisy data and large
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

number of local marginal probability estimates that are derived from some combination of local data, imperfect domain knowledge and local, independent predictive models. For instance:
Due to privacy concerns in application domains such as social networks 22 only limited global data is available. But local statistics, such as information about a person's 's contacts/connections can be retrieved easily.
In lazy learning of generative models
we derive sufficient statistics needed for inducing a probabilistic model at test time (when query is made) from various sources such as local classifiers for each statistic or via query made to a large database.
18
26
In active learning [23], the learning algorithm interactively solicits a user for labels or in general local marginal probability distributions for certain variables that the user is expert at given observations.
hallmark of all the scenarios just outlined is that the local marginal estimates are often inconsistent, namely, there may not exist joint probability distribution that is consistent with the local estimates. For example, in the active learning case, consider two pairwise marginal estimates PA (1 X2) and (22 Zz) over three random variables X1,, X2, X3 derived by querying two users A and B respectively. A joint probability distribution P(z Ø2, æ3) over the three variables that is consistent
with the two estimates exists iff (x = P'B (22) where (a(2)) Ex, (2 X2) and P' (12)
Σz; PB(#2 rz). Unfortunately, because of factors such precision errors and user bias, PA will not be equal to (122)))
In this paper, we focus on the following learning problem over TPMs. We assume that the learner has access to: (1) a TPM learned from small dataset having few or no missing variables, and (2) set containing local estimates where each estimate is a marginal probability distribution over a small subset of variables given observations.
Contributions. We propose to express the above learning task as the following minimization problem: update the parameters of the given tractable model such that linear combination of the following two KL distances is minimized: (1) the distance between the distribution represented by the original parameters and the one represented by the updated parameters; and (2) the sum of the distances between the given local, possibly inconsistent estimates and the ones computed using the updated distribution. We derive gradient-basse method for solving this optimization task. Since the gradients require computation of marginal distributions over subsets of variables, they are NP-hard in general on arbitrary probabilistic models but can be computed efficiently on tractable models [24]. This shows the virtue of using tractable models in our learning settings.
We performed a controlled empirical evaluation of our proposed method using 20 popular datasets that have been used in numerous studies on tractable models [[O] Our results show that our approach that leverages local estimates yields sigmificant improvements in both generative and predictive performance over the original model learned from small amount of training data, even when the local estimates are inconsistent. Moreover, since the optimization problem is smooth, our procedure is guaranteed to reach a local optima under mild conditions.
2 Related Work
Vomlel 225 studied the problem of integrating probabilistic knowledge bases where a joint probability distribution is constructed from low-dimensional probability distributions (local estimates). Vomlel used a classic optimization method called iterative proportional fitting procedure (IPFP) (6] and pro- posed variant called the generalized expectation maximization algorithm (GEMA) for solving this problem. Vomlel provided convergence proofs for these methods; showing that IPFP converges when the local estimates are consistent and GEMA converges even if the local estimates are inconsistent. Unfortunately, Vomlel's 's approach has high computational complexity (is exponential in the treewidth of the graph defined over the local estimates) and is not practical. The method proposed in this paper does not have this limitation. Peng and Ding [3] proposed two polynomial time approximations for IPFP and applied them to Bayesian networks. However, their preliminary experimental study demon- strates that the error due to their approximations is quite high and convergence is not guaranteed if the local estimates are inconsistent. In contrast, our proposed method makes very few approximations and leverages tractable inference to yield practical scheme.

---

3 Notation and Background
We use the following notation. Bold upper-case letters (e.g., X, Y, U, etc.) are used to denote sets of discrete random variables while bold lower-case letters (e.g., a, y, etc.) denote an assignment of values all variables in the corresponding set denoted by bold upper case letters (thus æ and y denote an assignment to all variables in the set X and Y respectively). We use upper-case letters (e.g.. X, U, V. etc.) to denote the variables. For simplicity of exposition, we assume that all variables are binary taking values from the set {0, 1}. A lower case letter (e.g., 2, y, etc.) denotes an assignment of a value the corresponding variable denoted by the upper case letter.
For simplicity of exposition, we present our algorithm and experimental results on a specific type of tractable model called cutset networks [17] which combines Bayesian networks and AND/OR graphs 5.. noting that the algorithm in this paper can be easily extended to other tractable models such as sum-product networks (14), thin junction trees [I] and arithmetic eircuits [2].
3.1 Bayesian Networks
use directed acyelic graphs (DAGs) and conditional probability tables (CPTs) to compactly represent joint probability distributions over a set of random variables. Each node in the DAG denotes a random variable and is associated with a conditional probability distribution of the corresponding variable given its parents in the DAG. Formally, Bayesian network is a triple (X, G, F) where X = (XI...., } J is a set of random variables; F = [Fi..... set of conditional probability tables (CPTs); and G(V, E) is directed acyelic graph such that each vertex Vi E V is associated with the variable X; E X and E is the set of directed edges. Each node V, is associated with the CPT F; (X( that represents the conditional probability distribution of the variable X; given its parents U, X X;
Bayesian networks (BNs) 3 12
......... Fn is
Let æ: (Z)) be an assignment of values to all variables in the set X and let u, denote the value assignment to all variables in the set U, that compatible with æ. namely u, is the projection of x on U, We will parameterize the Bayesian network using set of parameters Q where each 22 u. E θ is equal to the conditional probability P(X, = 1|U, = u.). Note that since P(X, 1|U, u:) + P(X, 0JU;- u.) 1, we do not need to have separate parameter for P(X, = 0jU; = u.). Under this parameterization, the probability distribution represented by the Bayesian network can be written as:
P(x) II (0.,,,)*:11 Oz 1--2, 1=1
(I)
The two main reasoning tasks over Bayesian networks are most-probable explanation (MPB) and posterior marginal inference (MAR). The former seeks to find an assignment of values to all variables given evidence such that the probability is maximized. Formally, let E C X be a set of evidence variables, let Z X E, and e be an assignment to all variables in E, then the MPE task is given by argmax P(z, e). The MAR task seeks find the marginal probability distribution over each
non-evidence variable given namely compute P(zzle le) for each variable Zj Z. Both tasks are known to be NP-hard in general but can be solved efficiently in practice on Bayesian networks having small treewidth using bucket (or variable) elimination and propagation algorithms 4.. Specifically, given a Bayesian network having treewidth w, the time and space complexity of these exact inference algorithms is bounded by ((n2*+1) and O(n2") respectively.
3.2 Cutset Networks
Cutset networks (CNs) [5 17 are tractable probabilistic models that combine two well-known classes of tractable models: AND/OR graphs S and tree Bayesian networks. Formally, CN defined over set of variables X (X may include latent variables) is defined recursively using the following three conditions: (1) A tree Bayesian network over X is CN; (2) An OR node labeled by variable
X, E X such that X > with two child CNs, each defined over the set X \ (X.) a CN. We follow the convention that the left child of the OR node labeled by X; represents conditioning over
X, = and the right child represents conditioning over X, 1. The edges from the OR node to its child nodes are labeled with probability values in R such that they sum to 1; and (3) Let (X,, X²)
3

---

be a partition of X such that [X] > 1. Then, an AND node with two child CNs, one defined over X1 and the second defined over X₂ a CN.
Fig. D shows a CN defined over eight variables. In general, full assign- ment æ yields rooted sub-graph in a CN.
PIA is Up-i
MCA 1)
Following our notation for Bayesian networks, let θ denote the set of pa- rameters of the cutset network, such that 0z E ④ is equal to the con- ditional probability P(X; = 1 u.). Given assignment æ: when X; an OR node, u; denotes the assign- ment along the path from the root of the CN to X,. Alternativly, when X; belongs to tree Bayesian network in the CN, U4 denotes the assignment formed by composing the assignment along the path from the root of the CN the tree Bayesian network with the assignment to the parent of X; (if X; has a parent in the network). Given this parameterization, the probability distribution associated with the cutset network is given by
PiNç
A cutset network defined over eight binary vari- ables (X1...., Xs). OR nodes are labeled with variables and denoted by circles, AND nodes by cross mark in cir- cles and leaf nodes (tree Bayesian networks ......, Tr) by shadowed dotted rectangles. Left and right edges emanating from OR node correspond to assignment of 0 and respectively to the corresponding variable and are labeled with conditional probabilities. For instance, the nodes and edges highlighted in red show the sub-tree consistent with the assignment {X1 02 1}.
Figure 1:
P(x) II (0n,u)) ) Oz, 0z ) 1=1
(2)
A distinguishing feature of CNs that when they have no latent variables, both MAR and MPE inference be performed over them in time that scales linearly with the size of the network. This can be accomplished by converting the CN in linear time to an AND/OR graph (5) or an arithmetic circuit [2] and performing wwo-passes over these circuits. When latent variables are present, CNs admit linear time MAR inference only while MPE inference is intractable in general.
4 Our Approach
In this section, define the optimization problem for learning the parameters of cutset network from local estimates and present a gradient- algorithm for solving it.
4.1 The Learning Problem
The high-level goal of this paper is to improve the prediction quality and model fit of an existing model that is induced using global information or full data alone by leveraging local information. The latter is often available in practice from various sources such as prior knowledge, expert domain knowledge, external sensors or processes and (local) data over small subset features. distinguishing property of local information is that it is typically available in large quantities and thus more robust as compared to global information/full data. However, because of various issues such as user bias (c.g., when information is obtained from different experts), locality and privacy constraints, local information is often noisy and inconsistent. Therefore, our specific goal is to learn cutset networks such that they optimally combine both local and global information, regularizing the two appropriately based on background knowledge.
We begin by describing required notation and simplifying assumptions. We assume that we have access to local information that can be summarized via pairwise distributions. Note that our algorithm can be easily extended to arbitrary for clarity of presentation only. Let D denote the KI. divergence (distance) between two probability distributions defined over the same set of variables. Namely, D(A, B) Ez A(x) log A(x)
local marginal distributions and we make the assumption

---

Ex A(x) (x)) where A and B are two distributions defined over X. The first term equals the negative entropy the distribution A and the second term equals the negative cross moss-entropy between A and B.
Let & denote subset of pairs of random variables, namely & E ((x, X X X and k). XR) E E) is consistent iff there exists a probability distribution P such that each estimate Pj X1. marginal probability
Definition 1. We say that set of local estimates (P (P Xx))
distribution Pf Otherwise, we say that the set is inconsistent.
Given set of local estimates and data, the learning problem is given by:
Given: A cutset network structure with parameters θ representing a probability distribution R. set of pairwise local distributions (X,, where Xx) e & and a fully observed dataset X' (xx(1) .,...,,, (m)))
To do: Find assignment of values all parameters in @ such that the negative log-likelihood &' w.r.t. Rg is minimized and the KL distance between Pau (X, J, Xx) and Rø(X,, Xs) equals zero for all (X,, Xx) E
Mathematically, we can express it as:
argmin
Σ
log (e(x!d) s.d. A (Xj, XR) EE, D(P))(X,,XA) Xx), Rx Xx)) =
Unfortunately, if the set (P)x (x), X E) is inconsistent (see Def. 1. then the above I
constrained optimization problem is infeasible To address this issue, propose combination the objective and constraints (which akin to Lagrange relaxation). Mathematically,
linear
argmin A
Σ (XP Xx), (X Xx)) Aa (XG,
Σ xx(d)
log Re (ax
where d 2 and X2 2 are hyperparameters (technically, we only need one hyperparameter; two for convenience) that model the relative importance of the local and global statis- ties respectively. Here, (X,,,(X,, Xx), Re (X,, Xx)) Eny Pj #x) log Pyu (#j, Exgre Pyu (Z),, log Ro (o( xx)
4.2 Simplifying the Learning Problem
We simplify the optimization problem given in Eq. (4) by making the following observation. Since Eny (I, xx) log (., 2x) is constant, we can remove it from the objective function and replace each term D(P)*(I), mx), Re(I, zx.) in Eq. by - (T 2x) log Re Ix).
Negating the objective and making the substitutions described above yields the following maximiza- tion problem:
argmax A
Σ Σ Pie 26) log Re (R + A2
Σ log Ro (x((1))
(5)
(X,,X))EE Zy.ZA
(n(d) E
The optimization problem given in Eq. (5) is not concave in the parameters θ but it is smooth. Therefore, it can be solved using an iterative, gradient ascent algorithm. However, a drawback of this algorithm is that the gradient of the second term in Eq. (5) requires data at each iteration yielding a slow algorithm. Although, stochastic gradient descent or mini-hatch approaches can be used to address this issue, their convergence quite slow in practice. Therefore, in order to reduce the computational complexity, we propose the following moment-matching approach.
go
the whole
Let Q denote the distribution associated with the cutset network having the same structure as R. Thus, there a one-to-one correspondence between the parameters Q and R. Let II denote the set
Note that this condition is sound but not complete. For example, the problem is inconsistent if the estimates are consistent but the cutset network does not have enough representation power to faithfully represent the constraints given in Eq. (3).
5

---

of parameters of Q. Thus, given parameter T., E II, there is a corresponding parameter ox,,u. in Ô. Let the set of parameters II be learned from data by maximizing the log- log-likelihood. Since the parameters of cutset networks are conditional probability distributions, given Q learned from data, we can use negative cross-entropy between parameters of Q and R lieu of the second term (log-ikeliiood) of Eq. (5). Mathematically,
argmax A
Σ
E
Po 24 oxg (Z,, +
(X), By
(6)
A2
Tayu log Ox m + (1 Tz ) log(1 - 0as,u)))
8x 2
EB
4.3 Solving the Learning Problem via Gradient Ascent
We propose to solve the optimization task given in Eq. (6) using gradient ascent methods. To this end, we derive the gradients w.r.t. each parameter 0z.u. The partial derivative of the second term w.r.t. On is straight-forward and given by
Tz
1 Tz.
X2
1 Oz
The partial derivative of the first term of Eq. (6) is more involved and we summarize it in the following proposition. (Proofs are given in the appendix.) Proposition 2. The partial derivative of
A1
Σ Puu x&) log Re (2)5 xk)
(K)) Zyink
w.r.l. O Ozijus is given by
Re RR(u,; 1/2
Ro uu X, = 0fz,,, 2kk)
A
Σ Σ (sn(
(8)
0 22
O 0.r,
(X Zy.ze
Formal Algorithm
Next, we formally present an algorithm that leverages the gradient equations given in Eqs. 48) (see Algorithm [ for solving the learning problem given in Eq. (6). The algorithm, which we call learning cutset networks with local inconsistent statistics (LCN-LIS). takes as input training dataset X, local statistics 3 (X,: Xx), two hyperparameters A and X2 (real numbers) and an integer bound T on the number of iterations. It begins by learning cutset network Q from the dataset X' (step 2) and initializes to have the same structure as Q (step 3). In steps 4-19, the algorithm runs the gradient ascent steps. The gradient ascent begins by setting all parameters
random number between 0 and 1. Then, at each iteration t, for each pair Xx) E E and its possible assignments (æy, 2x), it sets X; Iÿ and X 2g evidence and runs two-pass inference algorithm over the cutset network [1 to compute the required conditional probabilities Ryy (u; X; = 1 و. 24)) and Re (u, Xx 0|2 xx) (step 9). The algorithm then updates the gradient for each parameter 0..,, (steps 10-13; also see Eqs. ( given the assignment X, = and X₃ Tx. In steps 15-17, the algorithm updates the parameters using the gradient estimates fz..u and learning rate a. The algorithm terminates the gradient ascent on convergence or when the bound T on the number of iterations is reached. At termination, the algorithm returns with parameters O'.
(8)
The main virtue of our algorithm is that has polynomial computational complexity. The time (and space) complexity of step 9 is O(@@)) using two-pass algorithm over the CN that calculates the conditional probabilities of the parameters given evidence (see 2 The time complexity of updating the gradients (steps 10- (3-13) is also O())). Thus, the time complexity of steps 9-13 is O((e)). Since these steps can be executed a maximum of ((E) x T) times (step 8 and step 6 respectively), where T is the bound the number of iterations, the overall complexity is O( T (0))-
Remarks. Note that feasible solution to the optimization problem given in Eq. G). and thus the one returned by Algorithm I filters inconsistency the local estimates {P (XX) Xx) E}

---

Algorithm 1: LCN-LIS (X, (Pz,(X, Xx), E 8), X1, X2 7)
Input
(1) Training examples defined over a set variables X; (2) a set inconsistent pairwise marginal statistics (,, Xx) where Xc) E
and & S [[X3, Xx),,xxx]225 y,,, Xx E X and j k]; (3) Two hyper parameters A 2 0 and A 2 0, and
(4) a bound 7' the number of iterations
O Cutset Network R
begin
Learn a Cutset Network Q from Data X' using learning algorithms from litcrature (cf.
13
Initialize cutset network R to have the same structure as Q. Let @ (O...u.) and II [Targ:
denote the set of parameters of R and Q respectively. Initialize: all parameters 0x of R to random number between and 2..u, denote initial value 2,,.u:
Initialize: 1 =
repeat
Initialize: Izq.u: O for each Uz E θ
for each possible value assignment X; = x, and X = my where (X,, Xx) E Set X, 2y and Xg = as evidence in R. and compute the conditional prohabilities Rg: (u.,2i/25, 2x) for each Oz,m, θ
for each parameter Ozcin E do Let:
Ro (ur X1 1|15, rk) and 5-= Ro (egr(u., X₁ = 0f Ik)
8
01
(1 0z.
Update:
IEc.u: Ieccus + Pa -8 +
Tay
Ta,
1 0%
13
end
end
15
for each parameter Ouging do
0 0r s, + gr /l oc: learning rate end
--t+1 until convergence or > T:
19
29
return R with parameters @*
end
because it yields globally consistent model Rg. Unlike previously proposed techniques for solving the optimization task Eq. (3) such as the iterative proportional fitting procedure (IPFP) [6 25 which will not converge when the local estimates are inconsistent Algorithm I will converge to local optimum (because the objective is smooth).
To summarize, we derived and presented gradient- algorithm for learning the parameters of cutset networks presence of local estimates (see Algorithm(1] [T] and showed that the algorithm requires time and space that scales linearly with the number of given local statistics. Note that Theorem 2 can be easily extended to Bayesian and Markov networks. However, the problem is that computing the terms in the numerator in Eq. R will be NP-hard in general on Bayesian and Markov networks. This highlights another virtue of tractable models, local information, even if is inconsistent can be efficiently integrated into tractable model.
Experiments
We performed a detailed, controlled experimental study to evaluate the impact of using inconsistent local statistics the quality of the learned model. Specifically, we used the following controls: (1) the
1IPFP extension called GEMA proposed by Vomlel 25 are computationally infeasible unless treewidth of the primal graph obtained by combining the edges corresponding to E and the cutset network is bounded by small constant and the cutset network is an I-map (cf. (8) of Markov network whose potentials given by E. The second requirement particularly difficult to satisfy in practice.

---

accuracy of the cutset network Q learned from data X' in Algorithm 11 (2) the strength/level of incon- (3) the cutset network architecture; and (4) the number of evidence variables or observations available at test time (to test discriminative performance).
sistency in the local statistics (X Xx):
We used 20 benchmark datasets that have been widely used in previous studies 20 21 evaluate
our new approach. The number of variables in these datasets vary from 16 to 1556, and all variables
binary. We ran Algorithm P maximum of 48 hours or 1000 iterations (namely T = 1000) or convergence,whicheeee whichever was earlier. We used 5-fold cross-validation to select the values of the hyperparameters X1 and A2-
Local estimates: For each dataset, we learned mixture of cutset net- works [66 17] and used it as the true model P. We generated local statistics from P as follows. Since P tractable model, we can ef- ficiently (in linear time) compute (X, Xx) for all Xx E X. To make them inconsistent, we added randomly sam- pled from a normal distribution with mean and standard deviation o. We experimented with five values of (0.001, 0.01, 0.05, 0.1, 0.2). Note that after adding e, we have to normalize the distributions to ensure that they are valid.
Table 1: Generative (0% evidence) performance measured
using the negative cross entropy between P and Q and between P and R on three models: CLTs, CNs. MCNs.
Negative enossyy with 0% Evidence
Datasets #var
CLTs
CNs
MCNs
Q
R
Q
R
Q
R
nltes msnbe
16
-7.62
-6.85
-6.86
-6.17
-7.32
-6.03
value e that
17
-7.01
-6.58
-6.99
-6.36
-7.08
-6.08
kdd
-5.20
-2.80
-5.86
-2.77
-5.52
-2.69
plants
69
-18.64
-17.23
-17.70
-15.55
-17.04
-15.05
audio
100
-47.16
-45.52
-47.15
-44.01
-44.20
-42.57
jester
100
-60.95
-59.62
-61.08
-57.45
-57.92
-56.70
netflix
100
-62.98
-61.46
-63.5
-60.93
-58.31
-57.73
accidents
111
-38.00
-34.21
-37.06
-33.14
-35.72
-32.43
retail
135
-15.54
-11.59
-17.22
-11.61
-16.68
-11.61
We used 10% of the ran- domly chosen examples in the train- ing set learn Q. Thus, the dataset used by Algorithm | has 90% fewer examples than the one used for learn- ing P. We did this to ensure that Q, learned from a much smaller amount of data, differs significantly from the true model P. which in turn will help us evaluate how local information im- proves the quality from an inferior starting point. We further controlled the quality of Q using a parameter which we call the perturb rate. h lies
Data model:
pumsb*
163
-37.23
-32.86
-33.25
-27.61
-38.55
-30.46
dna 180
-103.18
-95.89
-224.06
-99.41
-98.53
.90.58
kosarek 190
-21.04
-12.98
-19.46
-12.45
-18.23
-10.68
msweb
294
-24.57
-12.28
-24.37
-12.33
-22.48
-11.05
book
500
-58.07
-1,80
-56.28
-41.46
-51.81
-34.49
movie 500
-97.27
-77.21
-94.39
-70.00
-64.99
-34.14
webkb reuters
839
-126.06
889 1152.53-113342-44660
-111.99
20newsg bbc
-194.55-157.70 -187777--88177---11
-55.48
-177.83 -152.65
1058 335.551 -270.74-265
210..5
223.82277777 -223.60
ad
1556-112.01
-47.27
-93.96
-41.34
-96.79
-53.70
Total AVG
-79.81 -64.03
-75.14 -59.01
-69.67
-55.43
between 0 and 100, and given a value for h, we replaced h% of the parameters in Q with a random number. We normalized to ensure that it is valid probability distribution.
We used three types of cutset network architectures: (1) cutset networks with depth which are equivalent to Chow-Liu trees (CLTs); (2) cutset networks with no latent variables (CNs); and (3) mixtures of cutset networks (MCNs). The latter state-of-the-ar model [9]]. We learned both discriminative and generative cutset networks. In discriminative networks, we set L% of random
variables as evidence E and learn a probability distribution over the variables X E given an assignment e to the evidence variables. We used values for L {0, 20, 50, 80}. When L = 0. get a generative model while the remaining models are discriminative.
As our evaluation criteria, we used negative cross-entropy between the true P and the learned model. The higher the negative cross-entropy, the better the model.
6.1 Results
Improved Model Quality. For lack of space, we present plots showing the impact of the level of noise Q, controlled by h;; and the level of inconsistency in the pairwise estimates (Xx(X,, Xx). controlled by on one randomly chosen dataset. We present the plots for the remaining datasets in
8

---

-30
-20
124
-300
-22
2.20%
1
(a) movie; CLT
(b) movie; CN
(c) movie: MCN
Figure 2: Negative Cross Entropy between P and Q, and between P and R. with evidence of 0%, 20%, 50% and 80% on three different models: CLT, CN, and MCN, as a function of perturb rate for a randomly chosen dataset: movie.
Model
bataset pundit
Modet MCN
-20-
0041
6.00
3641
620
0001
CDl
1
A1
K00
K00
(a) pumsb"; CLT
(b) pumsb"; CN
(c) pumsb"; MCN
Figure 3: Negative Cross Entropy between P and Q, and between P and R, with evidence of 0%, 20%, 50%, and 80% on three different models: CLT, CN, and MCN, as function of standard deviation a (of Gaussian noise that is applied to the local statistics (Xx(X,, Xx)) on one randomly chosen dataset: pumsb*.
the supplementary material. 2 presents the negative cross-entropy scores achieved by CLTs, CNs, and MCNs as function of the perturb rate (h) for randomly chosen dataset 'movie', and for different values of L (the percentage of evidence variables). These plots help us evaluate the impact that the quality of Q has (namely the impact of the starting point) the model learned by Algorithm|]
T We observe that as the perturb rate increases, there a substantial drop in the negative cross-entropy between P and Q. However, the negative cross entropy between P and R remains relatively flat. This shows that the use of local statistics significantly improves the model quality, especially when the model based on global data alone is inaccurate.
Figure shows the negative cross-entropy scores achieved by CLTs, CNs and MCNs as a function of the standard deviation a E {0.001, 0.01, 0.05, 0.1, of the Gaussian noise that is applied to the local statistics P) (X,. Xx)) for the dataset "pumsb*', and for different values of L (the percentage of evidence variables). The figure clearly shows that when the local statistics contain more noise, the performance degrades as expected. When the a is large (a > 0.2), there a non-zero chance that the final model R will be worse than Q. Thus, based on our experimental results, our proposed algorithm is more likely to yield a better model than the one based on global information alone when the Gaussian noise has standard deviation < 0.1.
Improved Generative and Discriminative Performances. Tables I and 2 show the generative and discriminative (L- = 50 and L 80) performances of Q and R respectively. We present the results for L 20 in the supplement. We used a 0.1 and 0 to generate the experimental data given in the tables. Each value in the tables is an average over 5 runs. To avoid clutter, we do not report the standard deviation because it was fairly small over all the runs. These results help us analyze the impact of the cutset network architecture and the number of evidence variables on our evaluation criteria. We observe that, on average, using local inconsistent statisties improves the negative cross entropy of each architecture by 17-23%. MCNs are the best performing model overall

---

Table 2: Discriminative (50% and 80% evidence) performance of Q and R as measured using negative cross-entropy on three models: CLT', CNs, MCNs.
50% Evidence
80 % Evidence
Dataset
CLTs
CNs
MCNs
CLTs
CNs
MCNs
Q
Q
R
Q
Q
Q
R
nites
-4.14
-3,53
-4.03
-3.66
-3.53
-3.18
-0,45
-0.27
-0,42
-0.33
-0.44
-0.29
msnbe
-5.14
-4.68
-5.34
-5.30
-5.03
-2.14
-400
-2.77
-3.24
-2.84
-3.36
-2.70
kdd
-4.28
-2.97
-5.12
-2.89
-3.82
-2.19
-3.51
-1.94
-3.02
-1.67
-2.99
-1.68
plants
-16.07
-8.75
-16.32
-9.72
-15.15
-8.28
-10.27
-7.20
-10.23
-7.70
-9.35
-6.64
audio
-32.14
-26.68
-31.59
-30.76
-28.83
-25.93
-27.23
-25.88
-21.55
-17.76
-21.02
-18.20
jester
-49.55
-45.73
-49.1
-88.08
-44.55
-42.27
-4.68
-29.78
-35.07
-34.24
-35.51
-27.13
netflix
-56.43
-56.14
-51.97
-48.84
-48.28
-46.68
-55.29
-53.86
-47.55
-46.98
-46.38
-45.28
accidents
-34.27
-31.73
-33.03
-29.3
-30.1
-28.34
-34.19
-28.96
-32.36
-27.84
-29.53
-25.56
retail
-11.75
-9.02
-12.44
-11.27
-11.07
-9.32
-8.86
-6.08
-8.09
-5.95
-7.38
-4.98
pumsb*
-29.69
-25.78
-25.57
-21.92
-25.18
-22.10
-16.75
-15.62
-13.88
-11.22
-12.31
-11.32
dna
-90.48
-82.46
107.81
-88.33
-85.73
-74.24
-79.33
-65.40
.99.28
-82.15
-75.45
-60.68
kosarek
-12.62
-9.00
-11.95
-5.73
-9.23
-4.17
-10388
-8.63
-8.82
-4.40
-7.87
-5.22
msweb -12.28
-8.18
-11.97
-9.89
-10.27
-9.35
-11.66
-5.66
-7.72
-6.87
-7.72
-6.31
book -23.42
-15.05
200.22
-18.05
-00.99
-1399
-17.36
-15.17
-15.83
-13.05
14.54
-11.72
movic
-77.15
-59.46
-62.22
-54.47
-55.2
-36.89
-59,2
-46.65
-52.7
-39.36
-39.65
-21.40
webkb
-15.721 -15500
139.86
134.67
123.38
105.83
-72.08
-91.48
-61.68
-90.19
-55.65
reuters
-113.17
-91.79 105.21
-85.56
110.26
-89.12
-83.56
-6.866
-74.57
-62.59
-77.75
-65.24
20ncwsg
-128.9
-143.25
139989
144.61
122.772
-93.59
-85.91
-87.6
-74.44
-79.68
-48.44
bhc
-247.79
-209.2093300 223.775
-174.1
-18.991884.8
-171.31
116.00--6444
-49.022-67..2-155..
ad
-88.33
-34.38
-65.11
-34.5
-72.13
-34.78
-80.7
-26.19
-57.95
-3.099
-55.6
-27.13
Avg
-61.67 -50,28
-8.07
48.12
-55
-44.55
45.91
-3..5
41.78
-33.56
-39.19
-30.19
and Chow-Liu trees (CLTs) are significantly worse. There was no significant difference the amount of improvement increased the number of evidence nodes. This suggests that our approach is equally useful for both discriminative and generative models.
7 Conclusion
In this paper, we presented new method for learning the parameters of cutset networks in presence of inconsistent local estimates. Unlike conventional algorithms which use full i.i.d. data during the learning process, proposed a novel approach that uses noisy local information to learn a more accurate and robust model. The key advantage using local estimates that they are often readily available compared to full i.idd. data. We also showed via experiments benchmark datasets that our new algorithm greatly improves the quality of the initial model learned from i.i.d. data, even when the local estimates are inconsistent and noisy.
Future work includes: (1) developing new structure learning algorithms that use local inconsistent statistics; (2) applying our approach to other tractable models such as sum-product networks and probabilistic sentential decision diagrams; (3) improving human feedback improve explanation quality; (4) developing lazy learning algorithms; etc.
Acknowledgments and Disclosure of Funding
This work was supported in part by the DARPA Explainable Artificial Intelligence (XAI) Program under contract number 166001-17-2-4033 the DARPA Perceptually-Enabble Task Guidance (PTG) Program under contract number HR001 12220005 and by the National Science Foundation grants 11S-1652835 and 11S-1528037.
References
[1] Francis Bach and Michael Jordan. Thin Junction Trees. Advances Neural Information Processing Systems, pages 569-576. MIT Press, 2001.
10

---

[2] Adnan Darwiche. A differential approach to inference in bayesian networks. Journal of the ACM (JACM), 50(3):280-305, 2003.
[3] Adnan Darwiche. Modeling and reasoning with Bayesian networks. Cambridge university press, 2009.
[4] Rina Dechter. Bucket elimination: unifying framework for reasoning. Artificial Intelligence, 113(1-2):41-85 1999.
[5] Rina Dechter and Robert Mateescu. And/or search spaces for graphical models. Artificial intelligence, 171(2-3):73- 106, 2007.
[6] W. Edwards Deming and Frederick F. Stephan. On least squares adjustment of sampled frequency table when the expected marginal totals are known. The Annals of Mathematical Statistics, 1144:42774444 1940.
[7] Michel Forsé and Alain Degenne. Introducing social networks. Sage, 1999.
[8] Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
[9] Yitao Liang, Jessa Bekker, and Guy Van den Broeck. Learning the structure of probabilistic sentential decision diagrams. In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence ( (UAI), 2017.
[10] Daniel Lowd and Jesse Davis. Learning markov network structure with decision trees. In ICDM, pages 334-343. IEFE, 2010.
[11] Daniel Lowd and Pedro M. Domingos. Learning arithmetic circuits. In Uncertainty in Artificial Intelligence, pages 383-392, 2008.
[12] Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Nenworks of Plausible Inference. Morgan Kaufmann, 1988.
[13] Yun Peng and Zhongli Ding. Modifying bayesian networks by probability constraints. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, pages 459 2005.
[14] Hoifung Poon and Pedro Domingos. Sum hum-product networks: new deep architecture. In Proceedings of the Twenty-eventt Conference on Uncertainty in Artificial Intelligence, pages 337-346, 2011.
[15] Tahrima Rahman and Vibhav Gogate. Merging strategies for sum-product networks: From trees graphs. In Proceedings of the Thirly-Second Conference Conference Uncertainty in Artificial Intelligence, pages 617-626, 2016.
[16] Tahrima Rahman and Vibhav Gogate. Learning Ensembles of Cutset Networks. In AAAI conference on Artificial Intelligence, pages 3301-3307, 2016.
[17] Tahrima Rahman, Prasanna Kothalkar, and Vibhav Gogate. Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees. In Machine Learning and Knowledge Discovery in Databases European Conference, ECML PKDD 2014, Nancy, France, September 15-19, 2014. Proceedings, Part II, pages 630-645, 2014.
[18] Tahrima Rahman, Shasha Jin, and Vibhav Gogate. Cutset Bayesian Networks: New Represen- tation for Learning Rao- alo-Blackwellised Graphical Models. In Sarit Kraus, editor, Proceedings of the Twenty-Eights International Joint Conference on Artificial Intelligence, pages 5751-5757, 2019.
[19] Tahrima Rahman, Shasha Jin, and Vibhav Gogate. Look Ma. Latent Variables: Accurate Cutset Networks via Compilation. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the Thirty-Siahh International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5311-5320, PMLR, 2019.

---

[20] Amirmohammad Rooshenas and Daniel Lowd. Learning tractable graphical models using mixture of arithmetic circuits. Proceedings of the 17th AAAI Conference on Late-Breakinn Breekinng Developments in the Field of Artificial Intelligence, pages 104-106, 2013.
[21]
Amirmohammad Rooshenas and Daniel Lowd. Learning sum-product networks with direct and indirect variable interactions. In Proceedings of the Thirty-First International Conference on Machine Learning, pages 710-718, 2014.
[22] John Scott. Social network analysis. Sociology, 22(1):109-127, 1988.
[23] Burr Settles. Active Learning, volume 18. Morgan Claypool Publishers, 2012.
[24] Antonio Vergari, YooJung Choi, Anji Liu, Stefano Teso and Guy Van den Broeck. A com- positional atlas of tractable circuit operations for probabilistic inference. Advances in Neural Information Processing Systems, 34: 13189-13201, 2021.
[25] Jiff Vomlel. Integrating inconsistent data in a probabilistic model. Journal of Applied Non- Classical Logics, 14(3):367-386, 2004.
[26] Min-Ling Zhang and Zhi-Hus Zhou. MI-knn: lazy learning approach to multi-labe learning. Pattern recognition, (1T))2038-2044 2007.
[27] Zijian Zheng and Geoffrey I Webb. Lazy learning of bayesian rules, Machine Learning, 41(1): 53-84, 2000.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Ye]] In the supplement.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] In the supplementary material
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed reproduce the main perimental results (either in the supplemental material or as URL)? [Yes] In the Supplement
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)? [Yes] In the section on Experiments and the Supplement
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A] Standard deviation was very small and so error bars were not provided.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] In the supplement
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
12

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "tSYx22SO8Xf", "reviewer": "Reviewer_L5Sp", "summary": "The paper considers the problem of learning tractable probabilistic models (TPMs) and presents a gradient-based method for learning a TPM from local (possibly inconsistent) estimates. Specifically, a TPM is first learnt from the input dataset and is subsequently refined by updating its parameters using a set of local probability estimates defined over small subsets of variables (i.e., local marginal distributions). The parameters are updated using a gradient descent based approach with an objective function that ensures closeness to the local estimates as well as the learnt model. The empirical evaluation demonstrates that the proposed method produces tractable models that are superior to those learned solely from data.\n", "questions": "1) How realistic is the assumption that the marginal probabilistic estimates are available? Could you describe a real-world application where this information is readily available? The experimental setup in Section 5, especially the generation of the local estimates, looks a bit artificial.\n\n2) I was wondering how previous methods like IPFP perform at least for the case when the local estimates are consistent.\n\n3) How sensitive is the method to the learning rate? Did you try using an adaptive learning rate?\n\n[Post Rebuttal] Thanks for your answers. They addressed my concerns.", "limitations": "see above", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is fairly well written and organised. The quality of the presentation is overall very good and therefore the paper is relatively easy to follow. Most of the concepts are introduced and discussed if a fairly clear manner. The empirical evaluation is sound and demonstrates clearly the strength of the proposed method.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "Yh0nrEzBw9", "reviewer": "Reviewer_gNZo", "summary": "This paper proposes to incorporate so-called local estimates into the optimization of probabilistic models. Since such estimates are typically intractable, the paper focuses on using tractable probabilistic models in order to enforce them into the loss function. The main discussion centers on learning cutset networks with (possible inconsistent) local estimates. To do so the paper formulates this as a constrained optimization problem (constraints being the local estimates) and takes a Lagrangian relaxation approach which basically ends up looking like a weighted joint optimization problem. Finally, due to the properties of cutset networks where parameters map to conditional probabilities, the loss function variance can be reduced through moment-matching instead of SGD, leading to faster convergence. Experiments are run on the 20-Datasets benchmark, showing that incorporating local estimates leads to better learned models when evaluated on KL divergence against an oracle model (chosen to be a mixture of cutset networks).", "questions": ".", "limitations": ".", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "My overall opinion of this paper is poor. The paper has three significant and pervasive shortcomings.\n\nFrom start to finish, the assumption of local estimates felt impractical and not well-justified. Are there really many scenarios where we would have large access to local estimates? If so, why are there no experiments on real datasets with real local estimates? The current experiments are completely synthetic, in the sense that the paper learns an oracle model using the full dataset to synthetically generate the local estimates, and then synthetically hides partial dataset from the learner models. \n\nWhy are the experiments only testing Q vs R, with no reasonable baselines? It is completely expected that R will do better than Q, since it is given strictly more information (essentially local statistics from the oracle model)? Where are the ablations using SGD, using other TPMs, or using any other method to incorporate local estimates? Altogether, I find it the experimental evaluation sorely lacking.\n\nWhy is there such emphasis on cutset networks? The paper tries to start off general, making statements that try to encompass all TPMs, but the entirety of the experiments end up being focused on cutset networks. Even the moment-matching technique is specific to cutset networks. General TPMs do not have the conditional distribution interpretation in their parameters -- would probably need to rely on SGD for optimization. Given this, there should absolutely have been studies ran on other families of TPMs. As it stands, the paper (and its title) should advertise only cutset networks.\n\n- on this, why don't we use general probabilistic circuits? What are the drawbacks of PCs compared to cutset networks?\n\n\n\n**Strengths**\n\n- the formulation is reasonable, and the derivations look correct\n- the writing is reasonably clear\n\n**Weaknesses**\n\n- poor justification of access to local estimates. At the very least there should be experiments on datasets with real local estimates, not synthetically generated.\n- experiments section is weak and far from 'detailed'. Missing ablations (SGD instead of moment matching), other TPMs, choices of oracle model...\n- unjustified choice of the heavy emphasis on cutset networks.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "YNbkBhpyzWx", "reviewer": "Reviewer_WY8E", "summary": "The paper proposes an algorithm for learning a tractable probabilistic model (TPM) from a dataset consisting of (1) a (possibly small) complete dataset and (2) (possibly many) local marginal estimates.  The tractable probabilistic model is a cutset network (or mixtures of).  They propose a tractable gradient algorithm for learning such a model, and show empirically how their algorithm can significantly improve the quality of a learned model when local marginal estimates can be taken into account.\n\nCutset networks are tractable; given complete data their parameters can be learned in closed form, and otherwise their gradients can also be computed in polynomial time.  The learning algorithm is based on gradient descent, where the local marginal estimates are incorporated as a soft constraint in the objective function.\n\nAs the authors have mentioned, the learning problem (or variations of it) have been considered before in the probabilistic graphical models literature, but there are some fundamental difficulties with this learning task on Bayesian networks and Markov networks.  Namely, checking whether local marginal constraints are satisfied requires inference in the network, which is computationall hard for BNs/MNs but can be done in polytime on tractable models such as cutset networks.\n\nTo my knowledge, the exploitation of a tractable probablistic model (TPM) for this learning task is novel.  There are certainly practitioners who are interested in this learning task for their applications, and I believe this type of paper would help spread the practical use of TPMs.\n", "questions": "I am familiar with this setting, and the developments to me were clear enough to me with no big surprises.\n", "limitations": "no negative societal impact, and otherwise limitations and relations to related work were satisfactory.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The proposed problem (learning a probabilistic model from data + marginals) is relevant in practice, and the authors apply a more modern class of (tractable) statistical models that is able to overcome limitations of classical models (like Bayesian networks).  The experiments demonstrate the utility on the learning problem.  Because of the (computational) limitations inherent in Bayesian networks, one did not see this learning scenario as often in practice, but given the use of tractable models, it may see increased visibility.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Ne5__m48apa", "reviewer": "Reviewer_QR7n", "summary": "This paper investigates a learning setting where training data is paired with noisy local estimates, focusing on Cutset Networks, a popular class of tractable probabilistic models (although the results seem easily extendable to other TPM classes).\n\nTractable models enable the efficient evaluation of the gradients required to solve this learning problem. The paper then proposes an iterative, gradient ascent approach. To further speed up convergence, instead of maximizing the original objective involving both the local estimates and training data, the paper proposes a moment-matching variant where the local estimates are considered as a post-processing step after training the CN on data using standard techniques. \n", "questions": "1- Can you further substantiate that local estimates can be easily obtained in many real-world applications?\n\n2- Why not including a comparison of the proposed moment-matching approach with a variant where stochastic gradient ascent is used to optimize the objective in Eq. 5?\n\n---\n\nMinor comment:\n\n\"To avoid clutter, we do not report the standard deviation because it was fairly small over all runs.\"\n\n- I would still report how (relatively) small it is or add the full table in the supplementary material.\n\n", "limitations": "I did not find any unaddressed limitations in the paper. I am looking forward to reading an answer to Q1 in order to better assess the impact of this work, possibly raising my 'Contribution' and overall rating.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nI found the paper generally well-written. I think it is quite accessible even for non-experts in tractable models while being sufficiently detailed in the description of the contribution. The empirical evaluation is also convincing.\n\n---\n\nI am not completely convinced that having access to these local estimates is \"quite common in real-world applications\". The work cited\nat lines 38-39 is not specifically supporting the claim. Considering that prior work in this setting is quite limited (according to the Related Work section), it would be nice if the paper substantiated more this claim with real-world examples.\n\n[Updated after rebuttal] The authors addressed my concerns in their response and included the changes in an updated version of the paper.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept