Title: Linear Uncertainty Quantification of Graphical Model Inference
Venue: NeurIPS 2024
Accepted: yes
Keywords: graphical models

belief propagation

uncertainty quantification
Abstract: Uncertainty Quantification (UQ) is vital for decision makers as it offers insights into the potential reliability of data and model, enabling more informed and risk-aware decision-making. 
Graphical models, capable of representing data with complex dependencies, are widely used across domains.
Existing sampling-based UQ methods are unbiased but cannot guarantee convergence and are time-consuming on large-scale graphs. 
There are fast UQ methods for graphical models with closed-form solutions and convergence guarantee but with uncertainty underestimation.
We propose *LinUProp*, a UQ method that utilizes a novel linear propagation of uncertainty to model uncertainty among related nodes additively instead of multiplicatively, to offer linear scalability, guaranteed convergence, and closed-form solutions without underestimating uncertainty.
Theoretically, we decompose the expected prediction error of the graphical model and prove that the uncertainty computed by *LinUProp* is the *generalized variance component* of the decomposition.
Experimentally, we demonstrate that *LinUProp* is consistent with the sampling-based method but with linear scalability and fast convergence.
Moreover, *LinUProp* outperforms competitors in uncertainty-based active learning on four real-world graph datasets, achieving higher accuracy with a lower labeling budget.

Linear Uncertainty Quantification of Graphical Model Inference

Abstract

Uncertainty Quantification (UQ) is vital for decision makers as it offers insights into the potential reliability of data and model, enabling more informed and risk-aware decision-making. Graphical models, capable of representing data with complex dependencies, are widely used across domains. Existing sampling-based UQ methods are unbiased but cannot guarantee convergence and are time-consuming on largescale graphs. There are fast UQ methods for graphical models with closed-form solutions and convergence guarantee but with uncertainty underestimation. We propose LinUProp , a UQ method that utilizes a novel linear propagation of uncertainty to model uncertainty among related nodes additively instead of multiplicatively, to offer linear scalability, guaranteed convergence, and closed-form solutions without underestimating uncertainty. Theoretically, we decompose the expected prediction error of the graphical model and prove that the uncertainty computed by LinUProp is the generalized variance component of the decomposition. Experimentally, we demonstrate that LinUProp is consistent with the sampling-based method but with linear scalability and fast convergence. Moreover, LinUProp outperforms competitors in uncertainty-based active learning on four real-world graph datasets, achieving higher accuracy with a lower labeling budget.

1 Introduction

Graphical models are known for their capability to represent data with complex dependencies. These models have been extensively applied to various fields, ranging from social networks [9, 39], fraud detection [26, 8], recommendation [14, 37] and crowdsourcing [25, 16], to more recent applications in enhancing graph neural networks (GNNs) [10, 28] and large language models (LLMs) [15, 34]. Among the many inference techniques in graphical models [19, 13, 12, 17], Belief Propagation (BP) [27] stands out as a powerful iterative message-passing algorithm. BP takes an initial guess (or 'prior belief') of each node and refines it using information propagation, resulting in an updated, more accurate estimate called 'posterior belief'. However, a crucial limitation of BP is that it only provides

point estimates for these posterior beliefs [7] and does not capture their potential uncertainty, leading to decisions unaware of the underlying data and model unreliability. For example, in a social network, a higher inferred probability of Alice's interest in sports over Bob's may lead to erroneous decisions if, in fact, Alice's inferred probability has high uncertainty and only a slightly higher interest.

To handle the uncertainty present in the beliefs, some well-known sampling-based uncertainty quantification (UQ) tools, such as Monte Carlo (MC) simulations, can provide unbiased UQ. However, these techniques are time-consuming for large-scale graphs and cannot ensure convergence within a reasonable time frame in practice [21]. Existing works [7, 35], based on Bayesian theory, have derived UQ methods with provable convergence and scalability by modeling beliefs as Dirichlet distributions and treating neighboring nodes as observations. However, this means that any neighbor of a node will necessarily reduce the uncertainty, even neighbors with noise or missing information. As illustrated in Figure 1, consider a user A with no preference information about being a music enthusiast, and A has two friends who are music enthusiasts (represented by a strong preference with Beta distribution parameters like B (3 , 1) , where the parameters represent positive and negative preference counts). If A gains an additional friend D who has no preference information (represented by a uniform distribution B (1 , 1) ), the existing methods still reduce A's uncertainty, incorrectly suggesting increased confidence that A is a music enthusiast. This results in an underestimation of posterior uncertainty. Furthermore, none of existing works provided a theoretical relationship between the computed uncertainty and the expected model prediction error, making it difficult for decision-makers to understand and trust the UQ results.

To address the challenges mentioned above, we introduce Linear Uncertainty Propagation ( LinUProp ), a method to quantify the uncertainty in posterior beliefs resulting from multiple iterations of propagation of uncertainty in node priors, which offers the following advantages:

Methodologically, we propose LinUProp (Sec. 3), a novel linear method that spreads the uncertainty from each node to the entire graph and additively aggregates this uncertainty to avoid underestimating posterior uncertainty. LinUProp offers interpretability due to the additive of neighbor uncertainty (Fig. 1), enabling tracking the contributions of other nodes to the computed uncertainty and allowing users to understand its sources.

Theoretically, LinUProp possesses a closedform solution, provable convergence related to the spectral radius of a matrix representing the dependencies between nodes (Sec. 4.1), and proven linear scalability (Appendix A.3). Moreover, by employing the bias-variance decomposition, we demonstrate that the posterior uncertainty is a generalized variance component of the expected model prediction error (Sec. 4.2).

Experimentally, we demonstrate the following with LinUProp : (1) The uncertainty quantified by LinUProp is accurate due to its strong positive correlation with MC simu-

lations (Fig. 7 in Appendix B.1); (2) LinUProp exhibits both fast convergence and linear scalability (Fig. 4); (3) LinUProp is interpretable (Fig. 3); (4) When applied to graph active learning guided by uncertainty, LinUProp achieves superior accuracy with a smaller budget compared to the baseline method that underestimates uncertainty (Figs. 5-6)

A

C

B

A

C

B

D

A

LinUProp

(Wider interval indicates

higher uncertainty)

A

's

Distribution

by

NETCONF/SocNL

(Larger

/u1D736

+

/u1D737

indicates

lower uncertainty)

G

1

G

1

G

2

G

2

1

1

3

1

1 1

1 1

3

7

8

4

αA

|

G

1

=

αA

+

αB

+

αC

αA

|

G

1

=

αA

+

αB

+

αC

β

A

|

G

1

=

β

A

+

β

B

+

β

C

β

A

|

G

1

=

β

A

+

β

B

+

β

C

αA

|

G

2

=

αA

+

αB

+

αC

+

αD

αA

|

G

2

=

αA

+

αB

+

αC

+

αD

β

A

|

G

2

=

β

A

+

β

B

+

β

C

+

β

D

β

A

|

G

2

=

β

A

+

β

B

+

β

C

+

β

D

Graphs and Initial

B

C

B C

A

B

C

D

B

C A

D

After One Step of Propagation

α β

Beta(

/u1D736

,

/u1D737

)

Treating Neighbors as

Observations Leads to

Underestimation of

A

's Uncertainty

A

Properly Incorporates

Neighbors' Uncertainty

1

1

1

3

3

3

2 Preliminaries

Belief Propagation (BP) [27] iteratively passes messages on a graph to infer posterior distributions of variables on a graphical model. We focus on the uncertainty in the inferred posteriors used for node classification on Markov Random Field (MRF). Bayesian networks are another important type of graphical models that can be converted into MRF [27].

Consider a graph G = ( V , E ) with n random variable nodes, each with k possible classes. The prior e s for node s is a k -dimension probability distribution and e s ( i ) specifies the prior probability for node s belonging to class i . Each edge ( s, t ) from G represents the dependencies between the two random variables s and t . In particular, the dependency is represented by a pairwise potential function, which is a k × k compatibility matrix H and H ( i, j ) denotes the degree of association between class i of node s and class j of node t . We assume that H is a symmetric matrix, as in [11, 7]. Specifically, for binary classification, the compatibility matrix is assumed to be

Similarly, an example form of H for multi-class problems is setting the diagonal elements to 1 k +( k -1) ϵ and the other elements to 1 k -ϵ . | ϵ | is typically close to 0, and a positive/negative ϵ specifies a homophily/heterophily relationship between any two connected nodes. Unlike existing methods that require all edges to have the same compatibility matrix, our proposed LinUProp can handle situations where each edge has a different compatibility matrix.

BP updates the k -dimensional message m ts sent from node t to node s by:

where N ( t ) is the set of neighboring nodes of t . Eq. (2) is applied iteratively until convergence or a maximum number of iterations is reached. Then the posterior b s for node s is

where Z s is for normalization such that ∑ k i =1 b s ( i ) = 1 .

Centered BP [11] is a linearized version of Eqs. (2-3):

where ˆ e s ( i ) = e s ( i ) -1 k , ˆ b s ( i ) = b s ( i ) -1 k , ˆ m ts ( i ) = m ts ( i ) -1 , ˆ H ( i, j ) = H ( i, j ) -1 k , are the centralized version of the prior, belief, message, and compatibility matrix.

NETCONF [7] models both the belief and uncertainty of each node using a Dirichlet distribution. The certainty of a node is represented by the sum of its Dirichlet parameters, where a higher sum indicates greater certainty. Each node u is initialized with a prior Dirichlet belief vector ˇ e u . The posterior Dirichlet belief ˇ b u is updated using multinomial messages from its neighbors:

where ˇ m vu is the message from node v to u , and M is a modulation matrix derived from H .

Problem 1 (Quantifying Uncertainty in Posteriors).

Given : (1) An undirected graph G = ( V , E ) consisting of n nodes and an adjacency matrix A , (2) |E| compatibility matrices, where each matrix illustrates the dependency relationship between a pair of connected nodes, and (3) n k -dimensional prior uncertainty vectors ❡ i ( i = 1 , . . . , n ) , with each vector representing the uncertainty in prior beliefs for a node as interval widths across the k classes. Find n k -dimensional posterior uncertainty vectors ❜ i ( i = 1 , . . . , n ) , where each vector represents the uncertainty of posterior beliefs for k classes of a node based on interval width, with wider interval indicating higher uncertainty.

̂

̂

̂

̂

̂

̂

1

H

′

12

H

′

12

H

′

21

H

′

21

H

′

23

H

′

23

H

′

32

H

′

32

H

′

2

12

H

′

2

12

∑

t

H

′

2

2

t

∑

t

H

′

2

2

t

H

′

2

32

H

′

2

32

=

+

+

(

)

Inputs

Outputs

2

1

3

H

12

=

H

12

=

H

23

=

H

23

=

/u1D556

1 =

-

/u1D556

2 =

-

/u1D556

3 =

-

e

+

3

-

e

-

3

e

+

3

-

e

-

3

e

+

1

-

e

-

1

e

+

1

-

e

-

1

e

+

2

-

e

-

2

e

+

2

-

e

-

2

{

{

{

1

2

3

(/u1D539)

=

(/u1D53C)

+

(

Ψ

′

+

Diag

(

Ψ

′

Q

)

)

(/u1D539)

(/u1D539)

=

(/u1D53C)

+

(

Ψ

′

+

Diag

(

Ψ

′

Q

)

)

(/u1D539)

̂

̂

2

1

2

3 Linear Bound Propagation

Recall that our objective is to quickly quantify the uncertainty in posterior beliefs and not assume that neighbors necessarily reduce uncertainty, in order to avoid uncertainty underestimation. Interval arithmetic [6] is a non-probabilistic method for quantifying uncertainty without assumptions regarding neighbors. However, it is unclear how to directly apply interval arithmetic rules to Eqs. (2-3) of BP while meeting the above objectives.

A solution to these challenges lies in linearization. The linearized BP is more amenable to interval arithmetic due to its exclusive reliance on interval addition, sidestepping the interval multiplication that may underestimate uncertainty. Consequently, this facilitates the derivation of closed-form solution, endowing the method with linear scalability, interpretability, and guaranteed convergence.

We will next introduce LinUProp and its iterative variant, which is computationally efficient in practical applications. Prior to that, we need some additional notation pertinent to LinUProp . Let vec denotes the operation that vertically concatenates the rows of a given matrix into a single column vector, and let Diag denote the transformation of an nk × k block matrix into a block diagonal matrix. ❊ = [ ❡ 1 , ❡ 2 , · · · , ❡ n ] T and ❇ = [ ❜ 1 , ❜ 2 , · · · , ❜ n ] T respectively represent the uncertainties of prior and posterior beliefs for all n nodes, each of dimension n × k . Q is an nk × k matrix formed by vertically stacking n identity matrices, each of size k × k . Ψ ′ 1 and Ψ ′ 2 are nk × nk block matrices formed from ˆ H ′ st and ˆ H ′ 2 st , respectively. For any given edge ( s, t ) , ˆ H ′ st denotes the centralized compatibility matrix corresponding to that edge, with entries ˆ H ′ st defined as ˆ H ′ st ( i, j ) = | H st ( i, j ) -1 k | . In cases where the edge ( s, t ) does not exist, the matrix defaults to zero.

Theorem 3.1 ( LinUProp ) . For a multi-class node classification task on an MRF, given the matrix ❊ which represents the prior belief uncertainty of all nodes, matrices Ψ ′ 1 and Ψ ′ 2 which denote the dependencies among these nodes. The posterior belief uncertainty for all nodes, represented by ❇ and in terms of interval widths, is determined by the linear equation system:

Figure (2) illustrates Eq. (6) using a 3-node chain as an example. From this, it can be observed that LinUProp can be implemented using matrix multiplication without computing the uncertainty propagated between each pair of nodes. In practice, a more effective strategy for computing the

̂

̂

̂

̂

̂

̂

posterior belief interval widths vec ( ) is to use an iterative update version of LinUProp :

❇ ❊ ❇ where l denotes the iteration round and vec ( ❇ ) (0) can be set to vec ( ❊ ) . Although Eq. (6) is similar in form to another method, LinBP (Eq. (13), [11]), which is a point estimation inference approach rather than a UQ method, simply replacing prior and posterior beliefs in LinBP with interval widths does not yield LinUProp . LinUProp is specifically designed to quantify uncertainty through derivations involving the upper and lower bounds of messages and beliefs. For detailed derivations, please refer to Appendix A.1. Its guaranteed convergence is proven in Sec. 4.1. The proof of its linear scalability is provided in Appendix A.3, showing that the time complexity per iteration is O ( |V| ) when |V| > |E| , otherwise O ( |E| ) .

4 Analysis

We present theoretical analyses of LinUProp , including the derivation of a closed-form solution, the proof of its convergence, and the provision of an interpretable approach to the uncertainty quantified by LinUProp . Furthermore, by employing the bias-variance decomposition, we addressed the question of what the uncertainty computed by LinUProp represents and established the connection between this uncertainty and the expected prediction error.

4.1 Theoretical Analysis of LinUProp

Closed-form solution. By simplifying Eq. (6), we can derive a closed-form solution for LinUProp :

where I represents the identity matrix. This closed-form solution is primarily used for subsequent theoretical analysis; in practice, the iterative version Eq. (7) is employed, which does not require the computation of the matrix inverse.

Convergence. Let Ψ ′ 1 + Diag ( Ψ ′ 2 Q ) be denoted as T . The sufficient and necessary criteria for the convergence of LinUProp is that the spectral radius of T is less than 1:

Proof. The closed-form solution of LinUProp (Eq. (8)) conforms to a general linear equation system, y = ( I -P ) -1 x , where y , P , and x are generic terms. Such linear equation systems can be solved by the Jacobi method [29], which converges if and only if the spectral radius of P is less than 1.

From the above convergence condition, LinUProp has a limitation: in large graphs with strong global (most edges) homophily/heterophily, ρ ( T ) may be large, leading to non-convergence. However, LinUProp can still converge if such strong homophily/heterophily is only local (a few edges).

Interpretability. When LinUProp converges, with ρ ( T ) < 1 , we can expand the closed-form solution Eq. (8) using the Neumann series, yielding:

The uncertainty of the posterior belief for a certain class of a node, vec ( ) v can be expanded as

❇ ❊ ❊ ❊ where T v vec ( ❊ ) , ( T 2 ) v vec ( ❊ ) , etc. can be expanded to ∑ w T v,w vec ( ❊ ) w , ∑ w ( T 2 ) v,w vec ( ❊ ) w and so on, the subscript v represents the v -th row of matrices T , T 2 , · · · , while the subscript w represents the w -th column of matrices T , T 2 , · · · and the w -th component of vec ( ❊ ) . Finally, we derive the contribution of the prior uncertainty of a certain class of any other variable node vec ( ) w towards the posterior belief uncertainty vec ( ) v as

This linear equation facilitates simple explanations of the uncertainty computed by LinUProp . Since ρ ( T ) < 1 ensures convergence, the influence of higher-order terms, such as T 3 and T 4 , decays rapidly. Therefore, in practice, considering only the first few terms often provides an accurate and interpretable estimate of uncertainty contributions.

4.2 Theoretical Connection with Bias-Variance Decomposition

To delve into the theoretical understanding of the uncertainty in the node's posterior belief, we employed the bias-variance decomposition [2], an effective tool for analyzing prediction errors and decomposing model uncertainty [41]. We demonstrated that the posterior uncertainty computed by LinUProp is a generalized variance component of the expected model prediction error, giving LinUProp a solid theoretical basis. This relationship between the uncertainty computed by UQ methods and the expected model prediction error has seldom been explored by existing work [7, 36, 35]. To begin with, we first derive a linearized BP suitable for edges with distinct potential functions to facilitate bias-variance decomposition.

From the update equation of message ˆ m ts as shown in Eq. (4), we can derive the update rule for the reverse message ˆ m st and substitute it back. This leads us to the stable message ˆ m ts when the algorithm converges. By inserting the stable message into Eq. (5) and simplifying, we can derive the closed-form solution for posterior beliefs where each edge has a distinct potential function (detailed proof in Appendix A):

where vec and Diag are the same as those in Eq. (6). ˆ B = [ ˆ b 1 , ˆ b 2 , · · · , ˆ b n ] T and ˆ E = [ˆ e 1 , ˆ e 2 , · · · , ˆ e n ] T are n × k matrices composed of the centralized posterior and prior beliefs of all nodes. Q is an nk × k matrix formed by vertically stacking n identity matrices, each of size k × k . Ψ 1 and Ψ 2 are nk × nk block matrices formed from ˆ H st and ˆ H 2 st , respectively. For any given edge ( s, t ) , ˆ H st denotes the centralized compatibility matrix corresponding to that edge, with entries ˆ H st defined as ˆ H st ( i, j ) = H st ( i, j ) -1 k . If the edge ( s, t ) does not exist, the matrix defaults to zero.

This closed-form solution shows that the posterior belief can be regarded as a linear model of the prior belief. Specifically, consider the element vec ( ˆ B ) v . This element represents the centralized posterior belief of a specific class corresponding to a particular node. It is formulated as a linear combination of all centralized prior beliefs, represented by F v vec ( ˆ E ) . Here, F v denotes the v -th row of F . This linear equation enables us to derive a bias-variance decomposition of the uncertainty in the posterior belief. We define h ( ˆ E ) as the centralized posterior belief corresponding to a specific class for a particular node in an unknown true linear model, and represent the centralized belief about ˆ E computed using Eq. (13) as vec ( ˆ B ) v . Then, the expected model prediction error can be decomposed into bias and variance as

Expanding the variance term yields

where Σ vec ( ˆ E ) is the covariance matrix of vec ( ˆ E ) (proof in Appendix A.4). If we assume that the priors of different nodes and class probabilities within each node's prior are nearly independent, Σ vec ( ˆ E ) can be approximated as a diagonal matrix. Consequently, the sign of the elements in F v

will not affect the result of Eq. (15). Ignoring the small term Diag ( · ) , the variance component can be approximated as F ′ v Σ vec ( ˆ E ) F ′ T v . The posterior interval width for v computed by LinUProp is vec ( ❇ ) v = F ′ v vec ( ❊ ) , leading to vec ( ❇ ) v ( vec ( ❇ ) T ) v = F ′ v ( vec ( ❊ ) vec ( ❊ ) T ) F ′ T v . This implies that the variance component is a special case of LinUProp where the outer product of the prior interval width vec ( ❊ ) vec ( ❊ ) T is the covariance matrix Σ vec ( ˆ E ) . In other words , the uncertainty computed by LinUProp is a generalized variance component of the expected prediction error.

5 Experiments

We begin with a simple case study to illustrate the correctness and interpretability of quantified uncertainty. We then present quantitative evidence of our method's ability to accurately quantify uncertainty, and demonstrate the convergence and scalability of our methods on real-world datasets. Finally, we compare the efficacy of LinUProp to other competitors in graph active learning tasks.

5.1 Case Study on a Simple Graph

Correctness of quantified uncertainty. To verify the correctness of the uncertainty quantified by LinUProp for posterior beliefs, we first qualitatively analyze the belief bounds on a simple graph. Specifically, we conduct the experiment on a 4 × 4 grid, as shown in Figure 3(a), with priors set to

(a)

0.00

0.05

0.10

0.15

0.20

0.25

0.30

(b)

0

2

4

6

8

Iterations

0.2

0.3

0.4

0.5

0.6

Average Belief Bound Width

(a)

0

10000

20000

30000

40000

Number of Edges

0.00

0.02

0.04

Time in s

(b)

Citeseer

Pubmed

PolBlogs

B (9 , 1) (red nodes) or B (1 , 9) (green node) for labeled nodes, and B (1 , 1) for the unlabeled nodes. We use Beta distributions as priors in binary classification because they are a natural choice for binary random variables [2]. For each variable node, the prior interval width for each class is set to twice the standard deviation of the corresponding prior Beta distribution. For each edge, the compatibility matrix is set as in Eq. (1), with ϵ = 0 . 1 .

Figure 3(b) shows that the more labeled nodes around each node s , the smaller the uncertainty in the belief; the more unlabeled nodes around s , the larger the uncertainty in the belief, which is intuitive. In addition to the qualitative analysis mentioned above, we also compared the uncertainty quantified by LinUProp with that quantified by MC simulations. MC simulations are adopted as the ground-truth due to its ability to provide accurate approximations through a sufficient amount of sampling [30], which are feasible for small-scale graphs. The experimental results show a strong positive correlation between the uncertainties quantified by the two methods (PCC=0.9084), which can be found in Appendix B.1.

Interpretability. By Eq. (12), we can compute the contribution of any node t to the uncertainty in the belief of node s computed by LinUProp . Figure 3(b) shows that the main source of uncertainty for the bold node is itself because it is an unlabeled node. The unlabeled nodes in the neighborhood are the secondary sources, and the unlabeled nodes within 2 hops are the tertiary sources. It demonstrates that the uncertainty in the beliefs quantified by LinUProp has good interpretability, which can enhance users' trust in UQ results computed by LinUProp .

5.2 Experiments on Real Data

We validate the properties of LinUProp on three popular citation networks (Cora, Citeseer, and PubMed) [18] and a political blog hyperlink network (PolBlogs) [1]. For further details regarding these datasets and experimental configurations, please refer to Appendix B.2. Our experimental findings can be summarized into the following three aspects.

Convergence. We set node priors based on classification type: Beta distributions for binary and Dirichlet distributions for multi-class ( k classes), both using parameter vector α of length k . In datasets, 30% of nodes are randomly labeled; if labeled as class i , α i = 10 , otherwise, entries are 1. Unlabeled nodes have α = 1 . Prior interval widths for each class are twice the standard deviation of the distribution, capturing uncertainty by representing the interval as mean ± std.

To simulate the diversity of dependencies between nodes, each edge's ϵ is randomly selected from {1e-4, 5e-4, 1e-3, 5e-3, 1e-2} and linked to a k × k compatibility matrix. Diagonal elements are 1 k +( k -1) ϵ , and other elements are 1 k -ϵ . We monitor LinUProp 's convergence (iterative version in Eq. (7)) by measuring the average belief bound width, ∑ n p =1 ∑ k q =1 ❇ ( p, q ) / ( n ∗ k ) .

Figure 4(a) shows that LinUProp converges within 10 iterations across all four datasets, demonstrating its rapid convergence. Using an iterative update version of LinUProp , the posterior belief bound width for each node increases over iterations due to uncertainty propagation from other nodes.

Scalability. We initialize the prior interval width for nodes and the compatibility matrix for edges following the same procedure as in the convergence experiment. Then we uniformly sampled different numbers of edges from four datasets and recorded the running time of LinUProp for 10 iterations. Figure 4(b) shows that the running time scales linearly in the number of edges. For more details on the runtime comparison between NETCONF and LinUProp , please refer to Appendix B.4.

Effectiveness. For large-scale graphs, MC sampling is computationally impractical due to the significantly increased time required for each sample. Therefore, we evaluate LinUProp 's effectiveness using active learning as a downstream task. Specifically, we use uncertainty-based sampling to select the next node for label acquisition, as detailed in [23]. In this context, users seek a labeled dataset with minimal uncertainty, maximum accuracy, and minimal labeling budget.

We simulate a realistic scenario as in [40], using training set V train , validation set V val , test set V test , and unlabeled pool V ulp (node numbers in Table 1, Appendix B.2), with query batch size b . Initially, training set nodes are labeled and others are unlabeled. Node priors follow the convergence experiment procedure. Edge compatibility matrices match the correctness experiment, enabling comparisons with existing methods that cannot handle diverse potential functions. Due to noisy labeling, the unlabeled pool remains unchanged to allow re-labeling [31].

For each selected node s , we update its Dirichlet prior by incrementing the parameter for the given label class by 1. After each labeling iteration, all nodes undergo inference. Noisy labels may not guarantee improved inference accuracy, so we use the iteration yielding the highest validation accuracy to evaluate the test set, which determines the test accuracy for the current labeling budget. MC-based methods are unsuitable for active learning as it requires time-consuming sampling for each iteration. We use the following strategies in each iteration:

Random: Select b nodes randomly.

Least Confidence (LC): Calculate uncertainty as U LC ( s ) = 1 -argmax i b s ( i ) .

Entropy: Calculate uncertainty as U Entropy ( s ) = -∑ i b s ( i ) log b s ( i ) .

Certainty Score (CS): Let ˘ b s be the parameter vector of the posterior Dirichlet distribution of node s inferred by NETCONF [7]. Uncertainty is U CS ( s ) = -∑ i ˘ b s ( i ) . Due to assumptions on distribution forms of priors, messages, and posterior beliefs, only applicable with NETCONF.

50

100

Size of labeling budget

40

45

50

55

60

65

70

Test accuracy(%)

[100%] [Cora]

25

50

75

100

125

Size of labeling budget

20

25

30

35

40

45

[100%] [Citeseer]

20

40

60

Size of labeling budget

30

40

50

60

70

[100%] [PubMed]

10

20

30

40

Size of labeling budget

50

60

70

80

90

[100%] [PolBlogs]

Random

LC

Entropy

BB

LC+BB

50

100

Size of labeling budget

40

45

50

55

60

65

70

Test accuracy(%)

[100%] [Cora]

25

50

75

100

125

Size of labeling budget

25

30

35

40

45

[100%] [Citeseer]

20

40

60

Size of labeling budget

40

45

50

55

60

65

70

[100%] [PubMed]

10

20

30

40

Size of labeling budget

60

70

80

90

[100%] [PolBlogs]

Random

LC

Entropy

BB

LC+BB

CS

LC+CS

Belief Bound (BB): Based on LinUProp with the same prior bound width setting as in the convergence experiment, uncertainty is U BB ( s ) = s .

❜ · LC+CS: Perform Min-Max normalization on U LC ( s ) and U CS ( s ) to obtain U norm LC ( s ) and U norm CS ( s ) . Uncertainty is U LC + CS ( s ) = U norm LC ( s ) + U norm CS ( s ) .

LC+BB: Combine LC and BB. Compute uncertainty as: U LC + BB ( s ) = U norm LC ( s ) + U norm BB ( s ) .

We set query batch size b = 2 k ( k is the number of classes) and maximum labeling budget to 20 b . We evaluate node selection strategy performance with annotator labeling accuracy at 70%, 80%, 90%, and 100%. To reduce randomness in results, we repeat each method ten times (re-partitioning datasets and changing random seeds) and record the mean of test set inference accuracies. To demonstrate labeling budget impact, we conduct experiments evaluating various node selection strategies under different labeling budgets on four datasets, varying the budget from 2 b to 20 b .

As shown in Figures 5 and 6, whether using BP or NETCONF for inference, the test accuracy of the LinUProp -based node selection strategy (BB) and its variations (LC+BB) generally grow faster than other baselines as the budget increases. This is especially the case when compared to the native UQ method in NETCONF (CS) and its variations (LC+CS). As shown in Figure 6, strategies based on NETCONF (CS/LC+CS) prioritize labeling low-degree nodes due to their inherent assumption that neighbors necessarily reduce the uncertainty. As a result, nodes with many neighbors are often mistakenly viewed as having low uncertainty and are left unlabeled, which further leads to high uncertainty in the majority of nodes. This phenomenon is more pronounced in graphs with a large number of low-degree nodes, like PolBlogs, leading to a very slow increase in accuracy. Experiments with a lower labeling accuracy yielded similar conclusions, as shown in Appendix B.3.1.

We also evaluate all strategies under a fixed labeling budget of 20 b for a fair comparison, with results displayed in Appendix B.3.2. The conclusion is that whether using BP or NETCONF for inference,

BB and LC+BB outperform baselines across different labeling accuracies and datasets in most cases. From the results in active learning tasks, we see node selection strategies guided by LinUProp 's UQ results achieves higher labeling accuracy with lower labeling budget. This indicates that uncertainty quantified by LinUProp is accurate, effective, and insensitive to the inference method.

6 Related Work

Uncertainty in posterior belief. In addition to traditional UQ methods like MC simulations, which require extensive sampling, some research focuses on modeling uncertainty in posterior belief by deriving closed-form solutions to incorporate uncertainty into inference results. Existing methods [7, 36, 35] exhibit scalability but limited as they assume uniform potential functions across all edges. Furthermore, these methods with closed-form solutions assume that any neighbor of a node will necessarily reduce the uncertainty even neighbors affected by noise or lacking information, potentially leading to uncertainty underestimation. Moreover, few existing studies have theoretically linked calculated uncertainty to the expected model prediction error, making it difficult for decision-makers to understand and trust the UQ results.

Existing works [22, 20] utilize bound propagation without the aforementioned assumptions regarding neighbors. However, they focus on quantifying the error between the posterior beliefs and the true marginal probabilities of the variable nodes, rather than quantifying uncertainty. This fundamentally differs from LinUProp , which quantifies the posterior uncertainty as the generalized variance component of the expected prediction error.

Human understanding of PGM inference. Humans are unlikely to adopt inference outcomes without reasonable interpretation [33]. In [24, 32], the authors studied explainable Bayesian networks. Recently, explanations of inference on Bayesian network and MRF were found by differentiation [3, 5], so that a set of important network parameters (potentials) can explain the changes in the inferred posterior distribution of a target variable. Interpretable graphical models are also studied under the hood of topic models [4] or GNN [38]. None of the aforementioned studies can provide an explanation for the uncertainty in the inference results.

7 Conclusion

In this paper, we proposed LinUProp , a UQ method for graphical model inference that utilizes a novel linear propagation of uncertainty to model uncertainty among related nodes additively. LinUProp provides linear scalability, guaranteed convergence, and is interpretable. Unlike its competitors, LinUProp does not assume neighbors necessarily reduce uncertainty and thus avoids uncertainty underestimation. To gain deeper insights, we decompose the expected prediction error of the graphical model and prove that the uncertainty computed by LinUProp is the generalized variance component of the decomposition. Experimental analysis shows LinUProp possesses aforementioned properties and outperforms competitors in downstream tasks. However, the study has not yet explored the interpretability of uncertainty with human involvement in real-world decision-making processes, an area we aim to address in future research. We would also like to apply LinUProp to more applications of graphical models to demonstrate its utility in the future.

Acknowledgments and Disclosure of Funding

Chenghua Guo and Xi Zhang were supported by the Natural Science Foundation of China (No. 62372057). This material is based upon work supported by the National Science Foundation under Grant Number 2008155. Sihong Xie was supported in part by the National Key R&D Program of China (Grant No. 2023YFF0725001), the Guangzhou-HKUST(GZ) Joint Funding Program (Grant No. 2023A03J0008), and Education Bureau of Guangzhou Municipality. Qi Li was supported in part by the National Science Foundation under NSF Grants IIS 2007941. Chao Chen was supported by the National Key Research and Development Program of China (No. 2023YFB3106504), and Pengcheng-China Mobile Jointly Funded Project (No. 2024ZY2B0050).

References

L. A. Adamic and N. Glance. The political blogosphere and the 2004 us election: divided they blog. In Proceedings of the 3rd international workshop on Link discovery , pages 36-43, 2005.

C. M. Bishop and N. M. Nasrabadi. Pattern recognition and machine learning . Springer, 2006.

H. Chan and A. Darwiche. Sensitivity analysis in markov networks. In IJCAI , 2005.

J. Chang, J. Boyd-Graber, C. Wang, S. Gerrish, and D. M. Blei. Reading tea leaves: How humans interpret topic models. In NeurIPS , 2009.

A. Darwiche. A differential approach to inference in bayesian networks. Journal of the ACM (JACM) , 2003.

H. Dawood. Theories of interval arithmetic: mathematical foundations and applications . LAP Lambert Academic Publishing, 2011.

D. Eswaran, S. Günnemann, and C. Faloutsos. The power of certainty: A dirichlet-multinomial model for belief propagation. In SDM . SIAM, 2017.

D. Eswaran, S. Günnemann, C. Faloutsos, D. Makhija, and M. Kumar. Zoobp: Belief propagation for heterogeneous networks. Proceedings of the VLDB Endowment , 10(5):625-636, 2017.

A. Farasat, A. Nikolaev, S. N. Srihari, and R. H. Blair. Probabilistic graphical models in modern social network analysis. Social Network Analysis and Mining , 5:1-18, 2015.

H. Gao, J. Pei, and H. Huang. Conditional random field enhanced graph convolutional neural networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 276-284, 2019.

W. Gatterbauer, S. Günnemann, D. Koutra, and C. Faloutsos. Linearized and single-pass belief propagation. Proceedings of the VLDB Endowment , 2015.

S. Geman and D. Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on pattern analysis and machine intelligence , (6):721-741, 1984.

W. R. Gilks, S. Richardson, and D. Spiegelhalter. Markov chain Monte Carlo in practice . CRC press, 1995.

J. Ha, S.-H. Kwon, S.-W. Kim, C. Faloutsos, and S. Park. Top-n recommendation through belief propagation. In Proceedings of the 21st ACM international conference on Information and knowledge management , pages 2343-2346, 2012.

Z. Hu, G. Wu, S. Mitra, R. Zhang, T. Sun, H. Huang, and V. Swaminathan. Token-level adversarial prompt detection based on perplexity measures and contextual information. arXiv preprint arXiv:2311.11509 , 2023.

Y. Jin, M. Carman, Y. Zhu, and Y. Xiang. A technical survey on statistical modelling and design methods for crowdsourcing quality control. Artificial Intelligence , 2020.

M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. Machine learning , 37:183-233, 1999.

T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICML , 2017.

S. L. Lauritzen and D. J. Spiegelhalter. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society: Series B (Methodological) , 50(2):157-194, 1988.

M. Leisink and B. Kappen. Bound propagation. JAIR , 2003.

F. Lindsten, J. Helske, and M. Vihola. Graphical model inference: Sequential monte carlo meets deterministic approximations. Advances in Neural Information Processing Systems , 31, 2018.

J. M. Mooij and H. Kappen. Bounds on marginal probability distributions. NeurIPS , 2008.

V.-A. Nguyen, P. Shi, J. Ramakrishnan, N. Torabi, N. S. Arora, U. Weinsberg, and M. Tingley. Crowdsourcing with contextual uncertainty. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, page 3645-3655, New York, NY, USA, 2022. Association for Computing Machinery.

S. W. Norton. An explanation mechanism for bayesian inferencing systems. In UAI , 1986.

J. Ok, S. Oh, J. Shin, and Y . Yi. Optimality of belief propagation for crowdsourced classification. In ICML , 2016.

S. Pandit, D. H. Chau, S. Wang, and C. Faloutsos. Netprobe: a fast and scalable system for fraud detection in online auction networks. In WWW , 2007.

J. Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference . Morgan kaufmann, 1988.

M. Qu, Y. Bengio, and J. Tang. Gmnn: Graph markov neural networks. In International conference on machine learning , pages 5241-5250. PMLR, 2019.

Y. Saad. Iterative methods for sparse linear systems . SIAM, 2003.

S. S. Sawilowsky. You think you've got trivials? Journal of Modern Applied Statistical Methods , 2:218-225, 2003.

V. S. Sheng, F. Provost, and P. G. Ipeirotis. Get another label? improving data quality and data mining using multiple, noisy labelers. In SIGKDD , 2008.

H. J. Suermondt. Explanation in Bayesian belief networks . Stanford University, 1992.

R. L. Teach and E. H. Shortliffe. An analysis of physician attitudes regarding computer-based clinical consultation systems. Computers and Biomedical Research , 1981.

R. Tutunov, A. Grosnit, J. Ziomek, J. Wang, and H. Bou-Ammar. Why can large language models generate correct chain-of-thoughts? arXiv preprint arXiv:2310.13571 , 2023.

Y. Yamaguchi, C. Faloutsos, and H. Kitagawa. Socnl: Bayesian label propagation with confidence. In Advances in Knowledge Discovery and Data Mining: 19th Pacific-Asia Conference, PAKDD 2015, Ho Chi Minh City, Vietnam, May 19-22, 2015, Proceedings, Part I 19 , pages 633-645. Springer, 2015.

Y. Yamaguchi, C. Faloutsos, and H. Kitagawa. Camlp: Confidence-aware modulated label propagation. In Proceedings of the 2016 SIAM International Conference on Data Mining , pages 513-521. SIAM, 2016.

X. Yang, Y. Guo, and Y. Liu. Bayesian-inference-based recommendation in online social networks. IEEE Transactions on Parallel and Distributed Systems , 24(4):642-651, 2012.

Z. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS . 2019.

J. Yoo, S. Jo, and U. Kang. Supervised belief propagation: Scalable supervised inference on attributed networks. In 2017 IEEE International Conference on Data Mining (ICDM) , pages 595-604. IEEE, 2017.

W. Zhang, Y. Wang, Z. You, M. Cao, P. Huang, J. Shan, Z. Yang, and B. CUI. Information gain propagation: a new way to graph active learning with soft labels. In ICLR , 2022.

X. Zhou, H. Liu, F. Pourpanah, T. Zeng, and X. Wang. A survey on epistemic (model) uncertainty in supervised learning: Recent advances and applications. Neurocomputing , 489:449-465, 2022.

A Proofs

A.1 Proof of Theorem 3.1

Let vec denotes the operation that vertically concatenates the rows of a given matrix into a single column vector, and let Diag denote the transformation of an nk × k block matrix into a block diagonal matrix. ❊ = [ ❡ 1 , ❡ 2 , · · · , ❡ n ] T and ❇ = [ ❜ 1 , ❜ 2 , · · · , ❜ n ] T respectively represent the uncertainties of prior and posterior beliefs for all n nodes, each of dimension n × k . Q is an nk × k matrix formed by vertically stacking n identity matrices, each of size k × k . Ψ 1 and Ψ 2 are nk × nk block matrices formed from ˆ H ′ st and ˆ H ′ 2 st , respectively. For any given edge ( s, t ) , ˆ H ′ st denotes the centralized compatibility matrix corresponding to that edge, with entries ˆ H ′ st defined as ˆ H ′ st ( i, j ) = | H st ( i, j ) -1 k | . In cases where the edge ( s, t ) does not exist, the matrix defaults to zero.

Theorem 3.1 ( LinUProp ) . For a multi-class node classification task on an MRF, given the matrix ❊ which represents the prior belief uncertainty of all nodes, matrices Ψ ′ 1 and Ψ ′ 2 which denote the dependencies among these nodes. The posterior belief uncertainty for all nodes, represented by ❇ and in terms of interval widths, is determined by the linear equation system:

Proof. Based on the linear approximation of BP messages defined in Eq. (4), we can approximate the lower and upper bounds of BP messages using interval arithmetic rules [6]:

The idea behind Eqs. (16-19) is that the determination of the lower and upper bounds of ˆ m ts ( i ) is contingent upon the sign of ˆ H st ( i, j ) . Specifically, when determining the lower bound of ˆ m ts ( i ) , if ˆ H st ( i, j ) < 0 , then ˆ b t \ s ( j ) should be maximized, i.e., taking its upper bound ˆ b + t ( j ) -1 k ˆ m -st ( j ) ; if ˆ H st ( i, j ) > 0 , then ˆ b t \ s ( j ) should be minimized, i.e., taking its lower bound ˆ b -t ( j ) -1 k ˆ m + st ( j ) . A similar idea applies to the computation of the upper bound of ˆ m ts ( i ) . Then by subtracting Eq. (16) from Eq. (17), we obtain the interval width ♠ ts ( i ) to reflect the uncertainty of ˆ m ts ( i ) :

where ❜ t is the interval width vector reflecting the posterior belief uncertainty at node t , as defined in the 'Problem Definitions' of Sec. 2. Eq. (20) leaves only the interval width, making the specific location irrelevant for subsequent steps. Thus, LinUProp is also unaffected by the exact location of the interval. Similarly, the uncertainty of the message in the opposite direction can be quantified by interval width as:

Substituting Eq. (21) into Eq. (20) leads to

When the algorithm converges, all messages are at a stable state, so we can treat ♠ ts on both sides of Eq. (22) as equal and replace the update formula with the equation:

Let us denote | H st ( i, j ) -1 k | as ˆ H ′ st ( i, j ) , then the stable state message bound can be simplified as:

♠ ❜ ❜ where the message bound width, denoted as ♠ ts , and the posterior belief bound width of nodes s and t , denoted as ❜ s and ❜ t respectively, are all k -dimensional vectors. Then the interval width ❜ s reflecting the uncertainty of posterior belief ˆ b s can be calculated by subtracting its lower bound ˆ b -s = ˆ e -s + 1 k ∑ t ∈N ( s ) ˆ m -ts from its upper bound ˆ b + s = ˆ e + s + 1 k ∑ t ∈N ( s ) ˆ m + ts :

where ❡ s is the prior bound width vector with k -dimensional. Then plugging Eq. (24) into Eq. (25):

Due to ˆ H ′ st being the centralized matrix, ( I -ˆ H ′ 2 st ) ≈ I , then ( I -ˆ H ′ 2 st ) -1 ˆ H ′ st ≈ ˆ H ′ st and we get

By using the matrices introduced at the outset of this section, Eq. (26) can be rewritten as Eq. (6).

A.2 Proof of Eq. (13)

Proof. Based on the linear approximation of BP messages (Eq. (4)), we can similarly get ˆ m st :

Substituting Eq. (27) into Eq. (4) leads to

When the algorithm converges, all messages are at a stable state, so we can treat ˆ m ts on both sides of Eq. (28) as equal and replace the update formula with the equation:

This stable state message can be simplified as:

Then plugging Eq. (30) into Eq. (5):

Due to ˆ H st being the centralized matrix, ( I -ˆ H 2 st ) ≈ I , then ( I -ˆ H 2 st ) -1 ˆ H st ≈ ˆ H st and we get

By using Ψ 1 and Ψ 2 introduced at the outset of Sec. 4.2, Eq. (31) can be rewritten as Eq. (13).

A.3 Proof of the Linear Scalability of LinUProp

Proof. To demonstrate the linear scalability of LinUProp , we will start with the time complexity analysis of the iterative version of LinUProp (Eq. (7)):

❇ ❊ ❇ Before the iteration starts, vec ( ❊ ) and Ψ ′ 1 + Diag ( Ψ ′ 2 Q ) (denoted as T ) are known and fixed as input. In the l -th iteration, we need to compute the matrix-vector multiplication T · vec ( ❇ ) ( l ) and then add the result to vec ( ❊ ) . Since T is a block sparse matrix, we can use sparse matrix operations to simplify the computation. The number of non-zero elements in T is k 2 ∗ ( |E| + |V| ) , where |E| is the number of edges, |V| is the number of nodes and k is the number of node classes. Then the time complexity of T · vec ( ❇ ) ( l ) is O ( k 2 ∗ ( |E| + |V| )) . Adding this result to vec ( ❊ ) has a complexity of O ( k |V| ) . Therefore, the time complexity of each iteration of LinUProp is O ( k 2 ( |E| + |V| ) + k |V| ) . From this, we can conclude that the time complexity is O ( |V| ) when |V| > |E| , otherwise O ( |E| ) , which reflects the linear scalability of LinUProp .

A.4 Proof of Eq. 15

In order to expand the variance term, we first apply the property of variance, which states that the variance of a random variable is equal to the difference between the expected value of its square and the square of its expected value. Next, we substitute the linear function vec ( ˆ B ) v = F v vec ( ˆ E ) into the equation and appropriately expand the squared terms. Following this, we manipulate and simplify the terms using the properties of expectation and matrix operations. At this point, we factor out the linear function's coefficient vector F v . The resulting expression includes the covariance matrix of the random vector, denoted as Σ vec ( ˆ E ) . Finally, we present the concise form of the expression, which demonstrates the linear relationship between the variance term of each node and the prior covariances of all nodes. The detailed proof steps are provided below:

B Additional Details and Results

B.1 Quantitative Validation of Correctness on a 4 × 4 Grid

We also quantitatively verify the correctness of LinUProp by comparing it with the uncertainty computed by MC simulations on a 4 × 4 grid. Specifically, we first sample prior beliefs based on the prior distributions of each node, and then run BP to infer the posterior beliefs of each node. After repeating the above sampling process for 100,000 times, we use the empirical standard deviation of the posterior belief of each node sampled as an estimation of the ground truth uncertainty.

Figure 7 shows that the uncertainties quantified by MC simulation and LinUProp have a strong positive correlation, as evidenced by a Pearson Correlation Coefficient (PCC) of 0.9084 between the uncertainties obtained for all nodes using both approaches. This confirms the consistency of the uncertainty in the beliefs quantified by our method with that of MC simulation.

0.10

0.15

0.20

0.25

0.30

Std. of belief by MC

1.0

1.1

1.2

1.3

1.4

1.5

Belief bound width by LinUProp

PCC=0.9084 (p=1.14e-06)

B.2 More Experimental Details

Running environment. We conducted convergence and scalability experiments on the Apple M2 chip, and convergence experiments on a 2.2 GHz Intel Xeon CPU.

Datasets. Table 1 shows the statistical information and partitioning of three citation network datasets.

B.3 More Effectiveness Results for LinUProp

B.3.1 Results under Varying Labeling Budgets with More Labeling Accuracy Settings

125

50

100

40

50

60

Test accuracy(%)

[70%] [Cora]

50

100

40

50

60

[80%] [Cora]

50

100

40

50

60

[90%] [Cora]

50

100

40

50

60

70

[100%] [Cora]

25

50

75

100

125

20

30

40

Test accuracy(%)

[70%] [Citeseer]

25

50

75

100

125

20

30

40

[80%] [Citeseer]

25

50

75

100

125

20

30

40

[90%] [Citeseer]

25

50

75

100

20

30

40

[100%] [Citeseer]

20

40

60

40

60

Test accuracy(%)

[70%] [PubMed]

20

40

60

40

60

[80%] [PubMed]

20

40

60

40

60

[90%] [PubMed]

20

40

60

40

60

[100%] [PubMed]

10

20

30

40

Size of labeling budget

60

80

Test accuracy(%)

[70%] [PolBlogs]

10

20

30

40

Size of labeling budget

60

80

[80%] [PolBlogs]

10

20

30

40

Size of labeling budget

60

80

[90%] [PolBlogs]

10

20

30

40

Size of labeling budget

60

80

[100%] [PolBlogs]

Random

LC

Entropy

BB

LC+BB

50

100

40

50

60

Test accuracy(%)

[70%] [Cora]

50

100

40

50

60

[80%] [Cora]

50

100

40

50

60

70

[90%] [Cora]

50

100

40

50

60

70

[100%] [Cora]

25

50

75

100

125

30

35

40

Test accuracy(%)

[70%] [Citeseer]

25

50

75

100

125

30

40

[80%] [Citeseer]

25

50

75

100

125

30

40

[90%] [Citeseer]

25

50

75

100

125

30

40

[100%] [Citeseer]

20

40

60

40

50

60

Test accuracy(%)

[70%] [PubMed]

20

40

60

40

50

60

[80%] [PubMed]

20

40

60

40

50

60

[90%] [PubMed]

20

40

60

40

50

60

70

[100%] [PubMed]

10

20

30

40

Size of labeling budget

40

60

80

Test accuracy(%)

[70%] [PolBlogs]

10

20

30

40

Size of labeling budget

50

60

70

80

[80%] [PolBlogs]

10

20

30

40

Size of labeling budget

60

80

[90%] [PolBlogs]

10

20

30

40

Size of labeling budget

60

80

[100%] [PolBlogs]

Random

LC

Entropy

BB

LC+BB

CS

LC+CS

B.3.2 Results under a fixed Labeling Budgets

Tables 2 and 3 show the mean test accuracies and standard deviations for a fixed labeling budget (20 b ) with BP/NETCONF inference on noisy labeled nodes. LinUProp achieves the highest mean accuracy in 28 of 32 cases, consistently outperforming its competitors across all four datasets, two inference methods, and four labeling accuracies. While the performance improvements are not always statistically significant, LinUProp maintains lower standard deviations (below 7%) and avoids poor accuracy. In contrast, LC achieves only 55% accuracy on Polblogs in Table 3, while LinUProp consistently exceeds 80%. The Entropy method shows high standard deviations (up to 14%) on Polblogs, whereas LinUProp generally has more stable performance, as underlined in the results.

B.4 Additional Runtime Results

Under the same experimental conditions as the linear scalability experiments in Figure 4(b), we tested NETCONF on four datasets, comparing its computation time (including all edges) with that of LinUProp . As shown in Table 4, the computation times of NETCONF and LinUProp are quite similar. However, LinUProp demonstrates a significant performance advantage over the NETCONF-based UQ method, as evident from Table 3 (BB/LC+BB vs. CS/LC+CS).

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction present the main contributions, including theoretical, methodological, and experimental contributions.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have outlined the limitations in the conclusion and discussed convergence limitations in Sec. 4.1.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The complete and correct proofs for the theoretical analyses and results are provided in Sec. 4 and Appendix A.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All the information needed to reproduce all the experimental results are disclosed in Sec. 5 and Appendix B.2.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code is public at

https://github.com/chenghuaguo/LinUProp .

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The sufficient information concerning the experimental settings is comprehensively disclosed in Sec. 5 and Appendix B.2.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All figures and tables related to test accuracy for varying/fixed labeling budgets have provided error bars.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The sufficient information concerning the computation resources is comprehensively disclosed in Appendix B.2.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conforms with the NeurIPS Code of Ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The original papers for the datasets are cited properly.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
comments: SUMMARY:
This paper proposes an uncertainty quantification method for graphical models that uses linear propagation to model uncertainty. Experiments show that it can achieve competitive results and fast convergence in downstream tasks.

STRENGTHS:
1: The paper is well-written. The motivation for the paper and the proposed solutions are clear, making it easy for readers to follow.

2: Theoretical analysis is solid. The authors clearly explain the necessary preliminaries and provide detailed proofs.

WEAKNESSES:
1: Most of the datasets used for validating the properties of the proposed method are datasets with high homophily, which makes the validation less comprehensive. It would be better to conduct experiments on several datasets with high heterophily.

2: The illustrations in some of the figures are vague. For example, Figure 1 is quite inclusive but lacks some necessary explanations. The meaning of distribution Beta and the values of alpha and beta are hard to understand (though I clearly understand the calculation part).

3: The formatting of the paper is less professional, especially in the appendix. In B.3.2, there are large blanks between the title and Table 2. Also, Table 3 is oversized.

QUESTIONS:
Can you please explain the upper part of Figure 2? I'm a bit confused about its meaning.

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n[Edit: After discussion, the Authors have addressed my concerns on the presentation and discussion of their results. I have accordingly increased my score (previously 5).]\n\nThe paper proposes an alternative algorithm for message passing for uncertainty quantification in graphical models. The method is linear, with (presumed) computational performance gains over existing methods, less susceptibility to certain kinds of bias, and supporting theoretical justification. The method is empirically explored in simulation studies.\n\nSTRENGTHS:\nTheoretical justification of the method is helpful.\n\nWEAKNESSES:\nSome areas were hard to understand. E.g. the motivating discussion of why some (but not all) existing methods could only reduce uncertainty.\n\nLimitations of the method have not really been addressed. It seems the proposed method is perfect!\n\nQUESTIONS:\nI was a little confused on the comparative studies.\n\n1) It isn't clear to me whether the proposed method produces unbiased results (though my guess is that it does not, and I note the content in section 4.2 to this end, and the claim it avoids bias in Section 7). Other methods, as discussed, do provide unbiased results, but perhaps they are very computational or have other downsides. So its ok for the proposed method to be biased if there is a substantial speed gain, for example. (*Is* there a speed gain? Actual comparisons with other methods don't seem to have been made.)  Or, put another way, the method presumably gains lots by designing for linear speed. What did it lose?\n\n2) I couldn't find anywhere which described what \"BP\" or \"NETCONF\" were (Figures 5-6). This somewhat reduces the readers ability to understand the information being presented. \n\n3) Tables 2 and 3 (Appendix) have generously bolded the results corresponding to the papers methodology, but the +/- standard errors of the means presented strongly suggest that in many cases there is no statistically significant difference between many of the methods (bolded, or otherwise). This would seem to undermine the accuracy performance claims.\n\nLIMITATIONS:\nThe paper checklist states that the limitations are outlined in the Conclusion (section 7). However, no limitations of the proposed methods seem to be discussed here. (The statement \"However, ...\" is not a limitation.) I would expect that the authors really ought to have some things to say here. (Presumed) strong computational gains of methods usually have a tradeoff.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of calculating uncertainty in the infererence results on probabilistic graphical models. Their method is based on a previously published linearization of the belief propagation method, to provide scalability, interpretability, and unbiasedness. The benefits of the new algorithm is demonstrated through experiments comparing with existing methods including Monte Carlo sampling, and uncertainty-based active learning on graphs.\n\nSTRENGTHS:\nOriginality: the studied problem is unique in that a linearly scalable, interpretable, provably convergence uncertainty quantification methods on graph is still missing. The proposed method is novel with a new definition of uncertainty quantity that is based on interval width that is decomposed into bias and variance. Such a definition is not seen in prior work.\nQuality: Technically, the formula and algorithm are developed rigorously and I don’t find errors therein. Their experiments protocols are designed carefully to evaluate their methods and prove the claimed advantages.\nClarity: the process of conducting the experiments, including datasets, graph constructions, baseline setup, are clearly described and reproducing the results is feasible. Figure 2 helps in understanding the dimension and operations of the linear operators on graphs.\nSignificance: their problem definition targets at gaps in uncertainty quantification on graphical model. In particular, previous methods, such as Monte Carlo and other uncertainty calculation methods on graphs, are either not scalable, biased, without convergence guarantee, or non-interpretable. In this aspect, the work contributes to the need of a scalable, interpretable, provably convergent, and unbiased uncertainty quantification algorithm for graphical models. In terms of theoretical significance, the decomposition of the computed interval width further makes sense of their proposed definition of uncertainty. Unlikely previous work that use variance of a distribution as a notion of uncertainty, the computed uncertainty is in fact a bias and variance term, thus shedding lights on the seemingly unmotivated definition of interval width.\n\nWEAKNESSES:\n-The proposed method is based on a previously developed linearization method, making their innovation limited.\n-In the section of “Correctness of quantified uncertainty”, the correctness is only validated through a small simple graph, and no experiments are conducted on larger-scale graphs. Maybe the authors can explain why it is so designed?\n\nQUESTIONS:\n-In Figure 6, the legend shows 7 methods, while in some of the subfigures, there are only 5-6 curves. Can you explain why?\n-In the same figure, LC seems to have performance between that of LC+BB and LC+CS. Is there any explanation about this observation?\n\nLIMITATIONS:\nPlease see the weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA method for quantifying the uncertainty in the graphical model is proposed. The authors claimed that the method is superior over prior methods including NETCONF and Monte Carlo in the aspects of scalability, interpretability, and unbiasedness. Active learning that uses node uncertainty estimation for unlabeled node selection is conducted to show the benefits of the computed uncertainty.\n\nSTRENGTHS:\n- I find the decomposition of the prediction error into bias and generalized variance novel, and the assignment of the interval width to the variance term is creative.\n- The submission is well-prepared and contain necessary components to make it a completed piece of work.\n- The presentation of preliminaries clearly show the background to facilitate the understanding of the more complicated part in Section 3. Overall, the paper is well-organized with clear logics.\n- The proof in Section 4 provide significant insight into the belief propagation algorithm and also the uncertainty quantification. Experiments on various setting proved the usefulness of the studied problem and the LinUProp method.\n\nWEAKNESSES:\n-\tI have doubt about the interpretability of LinUProp, as it involves high-order terms such as $T^2$, $T^3$, etc. that are not easy to be understood.\n-\tEq. (8) seems to be a simple extension of the original LinBP method.\n-\tIt is not clear what are the $\\mathbb{x}$, $\\mathbb{y}$, and $\\mathbb{P}$ terms in the proof in Eq. (9). What’s their relationship to the LinUProp method?\n\nQUESTIONS:\n-\tThe uncertainty is defined using interval width. Does the location of the interval matter and why?\n-\tDue to the proof of Eqs. (14) and (15), is it possible to compute the generalized variance component directly without LinUProp?\n\nLIMITATIONS:\nThe clarity of Figure 2 can be improved: what is H_12, H_23, e_1, etc.? The authors should point to the definitions in the main texts or to label these symbols in the figure.\n\nThe GNN models are more popular, while its uncertainty quantification has been studied. Though GNN and probabilistic graphical models are not comparable, but I expect the authors to have certain discussion of GNN and make potential connections between methods for these two sorts of models.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}