Title: Diffusion Model for Graph Inverse Problems: Towards Effective Source Localization on Complex Networks
Venue: NeurIPS 2023
Accepted: yes
Keywords: Diffusion Model

Graph Inverse Problems

Source Localization

Information Diffusion
Abstract: Information diffusion problems, such as the spread of epidemics or rumors, are widespread in society. The inverse problems of graph diffusion, which involve locating the sources and identifying the paths of diffusion based on currently observed diffusion graphs, are crucial to controlling the spread of information. The problem of localizing the source of diffusion is highly ill-posed, presenting a major obstacle in accurately assessing the uncertainty involved. Besides, while comprehending how information diffuses through a graph is crucial, there is a scarcity of research on reconstructing the paths of information propagation. To tackle these challenges, we propose a probabilistic model called DDMSL (Discrete Diffusion Model for Source Localization). Our approach is based on the natural diffusion process of information propagation over complex networks, which can be formulated using a message-passing function. First, we model the forward diffusion of information using Markov chains. Then, we design a reversible residual network to construct a denoising-diffusion model in discrete space for both source localization and reconstruction of information diffusion paths. We provide rigorous theoretical guarantees for DDMSL and demonstrate its effectiveness through extensive experiments on five real-world datasets.

Diffusion Model for Graph Inverse Problems: Towards Effective Source Localization on Complex Networks

Abstract

Information diffusion problems, such as the spread of epidemics or rumors, are widespread in society. The inverse problems of graph diffusion, which involve locating the sources and identifying the paths of diffusion based on currently observed diffusion graphs, are crucial to controlling the spread of information. The problem of localizing the source of diffusion is highly ill-posed, presenting a major obstacle in accurately assessing the uncertainty involved. Besides, while comprehending how information diffuses through a graph is crucial, there is a scarcity of research on reconstructing the paths of information propagation. To tackle these challenges, we propose a probabilistic model called DDMSL (Discrete Diffusion Model for Source Localization). Our approach is based on the natural diffusion process of information propagation over complex networks, which can be formulated using a message-passing function. First, we model the forward diffusion of information using Markov chains. Then, we design a reversible residual network to construct a denoising-diffusion model in discrete space for both source localization and reconstruction of information diffusion paths. We provide rigorous theoretical guarantees for DDMSL and demonstrate its effectiveness through extensive experiments on five real-world datasets.

1 Introduction

Information diffusion is a pervasive phenomenon in our daily lives. Data scientists have conducted extensive research on information diffusion along the direction of entropy increase, such as maximizing the influence of nodes in social networks [15, 3], and developing control policies to curb the scale of epidemics in disease transmission networks [53]. However, merely comprehending the mechanism of forward diffusion is insufficient. When destructive information spreads across a network, it can cause huge damage on the entire system. For example, the rapid spread of rumors in social networks can cause harm to society [54], and the spread of computer viruses on the Internet can paralyze a system consisting of millions of users [1]. Additionally, pandemics such as SARS and COVID-19 in human interaction networks pose serious challenges to human health and social functioning [31]. Therefore, accurately identifying the sources of transmission and cutting off their possible transmission paths in time is crucial. This can help limit the spread of negative information and maintain the stability of the network.

Common information dissemination models like SIR (Susceptible-Infected-Recovered) and SI (Susceptible-Infected) [19] are subject to uncertainties during the process of information diffusion. As illustrated in Figure 1, the diffusion graph of information should follow a non-explicit distribution, implying that a single diffusion source may correspond to multiple different diffusion

graphs, and a diffusion graph may have multiple different diffusion sources. Therefore, the inverse problem of information diffusion is underdetermined.

P -1

…

…

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

)

|

(

2

1

x

p x

)

|

(

2

1

x

p x

)

|

(

2

1

x

p x

)

|

(

2

1

x

p x

)

|

(

2

1

x

p x

)

|

(

3

2

x

p x

)

|

(

3

2

x

p x

)

|

(

1

T

T

x

p x

-

)

|

(

1

T

T

x

p x

-

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

t=1

t=2

t=T

P

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

A

B

C

D

E

F

G

H

I

J

K

t=T+1

Susceptibles

Infectives

Recovered

1

Q

2

Q

T

Q

1

+

T

Q

1

Q

2

Q

T

Q

1

+

T

Q

Previous source localization research has primarily relied on manually formulating rules to filter source nodes, such as using Jordan centrality [24, 26] to locate multiple source nodes in SI model, or unbiased intermediary centrality to identify source nodes in SIR model [49]. Besides, IVGD [43] exploited fixed point theorem for source localization of IC model [18]. However, these algorithms impose relatively strict requirements on information diffusion patterns , which could hardly be satisfied in real-world applications. To address this issue, some algorithms [38, 36] have been proposed using graph neural networks (GNN) [35] to learn source nodes under different diffusion models. Unfortunately, most deep learning-based algorithms ignore the underdeterminacy of information propagation and attempt to establish a direct mapping between observed data and source nodes. As a result, inference results are often deterministic and fail to quantify the uncertainty of source localization . In addition to source localization, it is equally important to recover the possible propagation paths of information . For example, when an infectious disease breaks out, it is necessary to promptly trace the virus transmission trajectory and corresponding close contacts. However, in reality, this can only be achieved by analyzing the genetic evolution tree of the virus strain [40], which is both time-consuming and labor-intensive. In summary, there is a current dearth of methods that can simultaneously locate information sources and restore information propagation paths .

Inspired by diffusion phenomena, the diffusion denoising model [16] has garnered significant achievements in the realm of image generation. It effectively restores the original image within a noise distribution through gradual refinement. This process bears resemblance to the challenges encountered in information reconstruction and diffusion, demanding our attention. Throughout the propagation process, the source information continually undergoes blurring, presenting an exceedingly arduous task of restoring the original source node. In this scenario, employing a diffusion model for the reverse recovery of the entire information dissemination process proves highly appropriate. However, using diffusion models to solve inverse problems in graphs presents several challenges: (1) Information diffusion occurs on non-Euclidean space graphs, making learning discrete non-Euclidean data difficult; (2) Diffusion over graphs is governed by information propagation rules that operate in non-Euclidean spaces, making it challenging to establish both the forward process and reverse inference process of information diffusion within the network; and (3) Most existing algorithms only work for specific propagation patterns, suffering from the problem of model generalization.

To address the aforementioned challenges, we propose a new framework, DDMSL 2 (Discrete Diffusion Model for Source Localization). DDMSL excels at source localization and reconstructing the evolution of diffusion, showing promising results across different propagation patterns. Our contributions can be summarized as follows:

We model the information diffusion process using Markov chains and demonstrate that at each moment, the state transition matrix converges, enabling us to reconstruct the information diffusion paths from any time point.

We design a residual network block based on graph convolutional networks to approximate the information diffusion process and provide a theoretical proof for the model's reversibility.

We propose an end-to-end framework being capable of both reconstructing the evolution of information diffusion and source localization. To the best of our knowledge, this is the first study to employ denoising diffusion models for solving graph inverse problems. Extensive experiments on five real-world datasets demonstrate its effectiveness.

2 Related work

Diffusion models for inverse problems . The diffusion denoising model [16, 29] is currently one of the best available deep generative models, which has been extensively applied in image-related inverse problems [33, 23, 34]. Previous studies have extended DP3Ms [2] to discrete spaces and used this work as a foundation for developing multiple generative diffusion models for graph data [14, 42]. Additionally, Stevens et al. [39] and Chung et al. [7] demonstrated that the diffusion model can solve nonlinear inverse problems. These works collectively demonstrate the effectiveness of diffusion models across different domains in addressing inverse problems.

Localization of information sources on the graph . To facilitate the localization of infection sources, various algorithms have been proposed [25, 52, 44, 9]. Early algorithms focused on screening source nodes by employing feature engineering. For instance, Luo et al. [25] identified the diffusion source by the centrality of the diffusion subgraphs, while Prakash et al. [30] and Zhu et al. [52] respectively developed NETSLEUTH and OJC algorithms based on the minimum description length of nodes and the shortest path. Wang et al. [44] designed a series of label propagation algorithms based on the aggregation process of information diffusion, on which Dong et al. [9] further improved using GCN [20]. Recently, the proposed reversible perception algorithm IVGD based on graph diffusion [43] has been applied to the IC model. However, these algorithms generally have strict limitations on diffusion patterns or network structures. DDMIX [8] was the first algorithm to use a generative model for reconstructing the dissemination paths of information. It leverages a V AE (variational autoencoder) to learn the state of nodes across all time steps; however, as the number of time steps increases, the solution space of the dissemination path becomes significantly more complex, leading to low accuracy in source localization. Ling et al. later introduced SLV AE [22], which is also based on VAE and can efficiently localize the source node but cannot directly reconstruct the propagation paths. Current generative model-based approaches for source localization cannot simultaneously handle the problems of source localization and reconstructing diffusion paths .

3 Methodology

3.1 Problem definition

We consider an undirected graph G = ( V , E ) where V = { x 1 , x 2 , . . . , x N } is the node set and E is the edge set. Let S = { x s 1 0 , x s 2 0 , . . . x s m 0 } denote the set of initial diffusion source nodes at t = 0 . The source nodes undergo diffusion on the graph G for T time steps, governed by the diffusion pattern g ( · ) , and the set of node states at time step t is denoted by X t = { x 1 t , . . . , x N t } with x i t ∈ { 0 , 1 } M (e.g., if g ( · ) is the SIR model, then M ∈ { S, I, R } , where S, I, R denote susceptible, infected and recovered state, respectively). If the node i ∈ S , then x i 0 = 1 , otherwise x i 0 = 0 . We define the research problem as finding the intermediate state X = { X 0 , X 1 , . . . , X T -1 } that maximizes the likelihood function X ∗ = argmax P { X T | X , G} X , given the observed X T .

3.2 Information diffusion in discrete spaces

Susceptible-Infected-Recovered (SIR) and Susceptible-Infected (SI) models [19] are commonly used to model diffusion phenomena in nature, such as the spread of epidemics [51] and rumors [50]. In this paper, we will demonstrate our approach using the SIR model. The SIR model categorizes node states into susceptible (S), infected (I), and recovered (R) states. The S state represents individuals who are

susceptible to infection, while the I state represents those who have been infected, and the R state indicates recovery from the infected state. The transition between these three states is irreversible. We assume that all nodes on the graph are homogeneous and follow the same transition process for the different states of the SIR model 3 :

And, the SIR diffusion process can be represented using a state transfer matrix:

where Q i t denotes the state transfer matrix of node i at moment t , and [ Q i t ] uv denotes the probability of transferring from state u to state v 4 . β i I ( t ) and γ i R ( t ) denote the infection rate and recovery rate at moment t , respectively. Let [ P i S ( t ) , P i I ( t ) , P i R ( t )] to be the probabilities of node i being in three states S , I , and R at time t , then β i I ( t ) and γ i R ( t ) can be calculated by the following equation:

Theorem 3.1 Given graph G and the infected seed set S , it can be determined that the state transfer matrix Q i t for the SIR diffusion process on G converges at all times.

Sketch of Proof . Given G and S , the initial state of information diffusion can be determined. Based on the SIR propagation rule, an iterative equation can be constructed for the state distribution of the nodes at any given time. Finally, Q i t can be calculated according to Equations 2 and 3.

Referring to Equations 2 and 3, we can readily demonstrate the convergence of the state transition matrix Q i t at every moment, rendering the discrete diffusion model a practicable solution [2]. The proof is shown in Theorem3.1.

3.3 Discrete diffusion model for source localization

3.3.1 Forward process

We represent the state of node i by an one-hot vector x i t ∈ R 1 × M , and the state distribution of node i at time t is written as:

Information diffusion is a Markov process that allows for inference of the node state at any given moment based on the initial state:

3.3.2 Reverse process

Reconstructing the information diffusion process requires backward inference of the forward process, which can be achieved through Bayesian formulation 5 :

In the absence of knowledge about x i 0 , the posterior distribution q ( x i t -1 | x i t , x i 0 ) becomes intractable to compute directly. As a result, we must approximate this distribution using other methods. Continuing with Bayes' theorem, we can express the posterior distribution as follows:

We utilize a neural network model ( nn θ ) to learn about p θ ( x i t -1 | x i t ) and estimate q ( x i 0 | x i t ) . The detailed design of nn θ will be elaborated later in Section 3.4.

Similar to [2], we predict x i 0 and then use this prediction to learn the distribution of q ( x i 0 | x i t ) . This approach offers several benefits. First, it leverages prior knowledge by exploiting the fact that at t = 0 there are only two states: susceptible and infected. This makes nn θ easier to train. Second, locating x i 0 is a key target of our task. Finally, predicting x i 0 can help constrain errors in the information reconstruction process (Refer to Appendix C).

Equation 9 presents the variational lower bound loss function of the denoising diffusion model [16], where D KL represents relative entropy. Furthermore, as highlighted in our earlier discussion, it is crucial for us to supervise x i 0 directly. As a result, we arrive at the simplified variational lower bound loss ( L simple ), which is denoted as:

3.3.3 Supervision of information propagation rules

Our reconstructed information diffusion must adhere to the propagation rules of model g ( · ) . During early training, the predictions of nn θ for x i t -1 may not be accurate enough, which could lead to violations of the propagation rules. For example, using the SIR model, if the state of node i at time t is classified as I , then the state of i at time t -1 can only be either S or I . However, the prediction result of nn θ may erroneously output state R , resulting in the failure to calculate q ( x i t -1 | x i t , x i 0 ) .

To prevent violations of the propagation rules, we can take two measures: (1) In cases where the inference result of nn θ violates the propagation rule, q ( x i t -1 | x i t , x i 0 ) should be set to [1 , 0 , 0] . and (2) To supervise the training of nn θ , the propagation rule loss function L constrain = L constrain 1 + L constrain 2 is introduced. The detailed derivations of (1) and (2) are shown in Appendix C.

In order to evaluate Equation 11, we must first sample X t -1 from the discrete probability distribution computed in Equation 8. To ensure gradient preservation during the sampling process, we employ the Gumbel -Softmax technique [17]: X t -1 ∼ Gumbel -Softmax ( q ( x i t -1 | x i t , x i 0 )) . Equation 11 demonstrates that each node's contribution to the information diffusion process is non-negative. In summary, the constrained loss function is L constrain = = L constrain 1 + L constrain 2 , and the loss function of DDMSL is:

3.4 Design of the inference model

In this section, we discuss the design of the nn θ in detail . Let X 0 = { x 1 0 , x 2 0 . . . x N 0 } and Y T = { x 1 T , x 2 T . . . , x N T } . The diffusion of information on complex networks can be modeled using the following processes [46, 43]:

Within this framework, f w and F serve as the feature vector construction and feature propagation func-

x1

x2

…

S

I

I

…

I

R

I

I

I

…

R R

…

…

YT

Dense

…

Residual Block ×  n

GCN

BN

s

GCN

BN

s

GCN

BN

s

Residual Block

GCN

BN

s

Residual Block

DP

DP

FC

Sigmoid

…

…

…

S

xN

S

I

I

I

S

xN

S

I

I

I

…

Residual Block

F

tions, respectively. The relationship between Y T and X 0 be expressed as: Y T = P ( X 0 ) = F ( f w ( X 0 )) . Our research objective focuses on the inverse problem of the SIR or SI diffusion model, which entails a large number of stochastic processes. Thus, when F = RD , solving P -1 can be extremely challenging. In the following subsections, we will explore how to construct suitable functions f w and F to facilitate the calculation of P -1 .

3.4.1 Relationship between GCN and SIR diffusion models

According to [37], diffusion models such as SIR are analogous to message passing neural networks (MPNNs [11]), where each node's state is only updated based on the states of its neighboring nodes. However, the operational architecture must be designed to enable each node to aggregate both its own features and those of its neighbors, and update its state via nonlinear activation as shown in Equation 26. Therefore, we propose using MPNNs to remodel the SIR diffusion process. Denoting P i Ω ( t ) ≡ P ( x i t = Ω) as the probability that node i is in state Ω ∈ { S, I, R } at time t in the SIR diffusion process. It can be observed that this process is structurally equivalent to:

where M t is the aggregate function and U t represents the node status update function. σ ( · ) is a nonlinear activation function. e ij represents the edge between nodes i and j. Additionally, since h ( P i Ω ( t )) already contains nonlinear transformations, more complex forms of transformation are not necessary and U t can be defined as: U t ( a, b ) = a + b . To sum up, Equation 14 can be simplified as:

We can achieve this using a residual block composed of graph convolutional networks, which allows us to easily fit the SIR model:

Among them, U is a linear transformation, h ( l ) i,t denotes the representation of the l -th layer of the network, D is the degree matrix, SN ( · ) and BN ( · ) represent spectral normalization and batch normalization, respectively. σ ( · ) and σ g ( · ) are nonlinear activation functions 6 . In fact, f w and F 7 in Equation 13 are the SN ( U ( · )) and residual blocks in Equation 16.

3.4.2 Reversibility of residual blocks

We now discuss the invertibility of Equation 13 as reconstructed from Equation 16. Equation 13 can be written as:

Lemma 3.1 Denoting the Lipschitz constants of f w and f θ to be L w and L θ , then L w < 1 and L θ < 1 .

Sketch of Proof . Since f w and BN ( g ( ξ )) are spectrally normalized, their Lipschitz coefficients are less than 1 [28]. Additionally, since the σ ( · ) is set to Mish ( · ) , it can be determined that the Lipschitz constant of their composite function (i.e., f θ ) is less than 1.

Theorem 3.2 If L w < 1 and L θ < 1 , then Y T = F ( f w ( X 0 )) is reversible.

Sketch of Proof . Y T = F ( f w ( X 0 )) can be written as { X 0 = ξ + X 0 -f w ( X 0 ) ξ = Y T -f θ ( ξ ) . Since f w < 1 and f θ < 1 , constructing the iteration according to the Banach fixed point theorem [4], Y T = F ( f w ( X 0 )) is invertible.

In practical applications, we typically use multiple residual blocks to form a residual network. This approach is employed to improve the receptive field of the GCN and alleviate the problem of node smoothing caused by multi-layer graph convolution. The multi-layer reversible residual network that we designed is depicted in Figure 2, where DP represents dropout. If the dropout rate is r and the i -th layer residual block is F i , then we have F i ( ξ ) = DP ( ξ + f w ( ξ )) .

Theorem 3.3 If dropout -rate = 0 . 5 , L w < 1 , and L θ < 1 , the residual network Y T = ( F 1 ◦ F 2 ◦ . . . ◦ F n )( f w ( X 0 )) is reversible.

Sketch of Proof . Denoting this multilayer residual network as F , the upper bound of L F is the product of the Lipschitz constants of each function [13]. Additionally, with a dropout rate of r , the Lipschitz constants of the functions will be limited to (1 -r ) times their original values. Using these two conclusions, we can calculate the upper bound of the Lipschitz constant of a multilayer reversible network with L F ≤ 1 when r = 0 . 5 .

4 Experiments

4.1 Datasets and evaluation metrics

Datasets . The diffusion of information occurs in a broad range of network types, and DDMSL was evaluated on five distinct realistic datasets: Karate [48], Jazz [12], Cora-ML [27], Power Grid [45], and PGP [5]. The detailed parameters of these networks are provided in the Appendix F.2. Following previous research [43, 9, 22], we conducted SIR and SI diffusion simulations on each dataset by randomly selecting 10% of the nodes as source nodes at the initial moment, and stopping the simulation when approximately 50% of the nodes were infected 8 (For PGP dataset, simulation stopped at 30% infection rate). We randomly divided each generated dataset into a training set, a validation set and a test set in the ratio of 8 : 1 : 1 .

Evaluation Metrics . Our objective consists of two components: source localization and information diffusion paths recovery. The source localization task is a binary classification task, and thus evaluated using four metrics: Precision ( PR ), which denotes the proportion of nodes predicted as sources that are true sources; Recall ( RE ), which represents the proportion of actual source nodes that are correctly predicted; F1 score, the harmonic mean of PR and RE; and ROC-AUC ( AUC ), quantifying the model's ability to classify accurately. To evaluate the performance of the recovering information diffusion paths, we adopted the Mean Squared Error (MSE) error in [8]: MSE = 1 NT ∑ T -1 t =0 ∥ ∥ ∥ X t -X ˆ t ∥ ∥ ∥ 2 ,

where ˆ X t is the ground truth. This metric is solely computed for infected nodes, with nodes in the recovered and susceptible states being marked as 0 .

4.2 Baseline algorithms and experimental settings

Six algorithms for source location were selected to compare with DDMSL: DDMIX [8] employs a Variational Autoencoder (VAE) to reconstruct the outbreak propagation process; LPSI [44] aggregates information to learn source detection through label propagation; GCNSI [9] uses graph convolutional network to learn manually formulated node features, enabling the discrimination of multiple source nodes; OJC [52] locates the source node by minimising the eccentricity of the infection; NETSLEUTH [30] filters multi-source nodes based on minimum description distance, but works solely on the SI diffusion model; and SLVAE [22] deploys generative models to learn the distribution of source nodes. Appendix F.1 provides the experimental settings and tuned hyperparameters for these algorithms, while implementation details on DDMSL are shown in Appendix E.

4.3 Experimental results

while comparing DDMSL with other algorithms on different diffusion models, we have focused on the widely used SIR and SI models, which are effective in capturing the dynamics of various information diffusion processes found in nature. It is noteworthy that DDMSL's flexibility and robustness enable it to handle real-world scenarios where the diffusion process may undergo significant variations by simply designing the corresponding state transfer matrices 9 .

Source Location . Table 1 shows the performance of DDMSL in SIR diffusion mode, where it outperforms all baseline algorithms regarding almost all metrics. Notably, DDMSL leads the best of the baseline algorithms by approximately 25% and 14% on F1 score and AUC, respectively. Meanwhile, Table 2 shows similar results in the SI diffusion mode. While SLVAE has a higher precision than DDMSL, DDMSL surpasses SLVAE by approximately 21% and 17% in terms of F1 score and AUC, indicating that DDMSL is more accurate in identifying source nodes. Moreover, the performance of DDMSL varies across datasets and is mainly influenced by the network's typologies (Refer to Table 6) and the scale of information diffusion. The performance of DDMSL in solving the inverse problem generally has a negative correlation with the average clustering coefficient and density of the network. The larger these two parameters are, the more complex the scenario of information diffusion is, and the accuracy of DDMSL in detecting source points decreases, which aligns with our intuition.

0.057

0.116

0.117

0.123

0.102

0.111

0.175

0.137

0.151

0.253

Karate

Jazz

Power Grid

PGP

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

MSE

DDMSL

(a) MSE reconstruction error in SIR mode.

0.074

0.116

0.115

0.116

0.112

0.098

0.122

0.173

0.201

0.299

Karate

Jazz

Power Grid

PGP

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

MSE

DDMSL

(b) MSE reconstruction error in SI mode.

Reconstructing information diffusion paths . Both DDMSL and DDMIX can reconstruct the evolution of information diffusion, but DDMIX can only recover the states of susceptible nodes ( S ) and infected nodes ( I ), whereas DDMSL can reconstruct nodes of all states. To facilitate fair comparison, we calculate MSE loss for node reconstruction in states S and I , as shown in Figure 3. DDMSL consistently outperforms DDMIX on all datasets, especially in SI diffusion mode where DDMSL's average reconstruction loss is 69% lower than DDMIX's. Unlike DDMIX, which recovers the entire graph state, DDMSL allows for fine-grained node-level reconstruction.

Additional experiments . DDMSL represents a source detection algorithm relying on diffusion models, necessitating an assessment of its aptitude for generalization. Concurrently, we assessed its computational complexity while also incorporating experiments utilizing two real diffusion datasets. We have documented the results of these experiments in Appendix G.

Visualization . To further showcase the efficacy of DDMSL in solving the inverse problem of information diffusion on complex networks, we designed a visualization that demonstrates source localization and the propagation evolution process. Due to space constraints, we have presented the visualization in Appendix H.

4.4 Ablation study

To evaluate the effectiveness of each component, we conducted ablation experiments on DDMSL in SIR diffusion mode. One essential component of DDMSL is the reversible residual network. We removed this component and replaced each reversible residual block with a GCN network, denoted as DDMSL(a). Additionally, DDMSL is also supervised by L constarin on the propagation rules. The model without the propagation rule supervision module is denoted as DDMSL(b). Finally, to assess the impact of GCN (Graph Convolutional Network) in reversible residual blocks, DDMSL(c) represents the model after removing the GCN modules from residual blocks. We ran the three variants on five datasets and compared the results,

Karate

Jazz

Power Grid

PGP

/s48

/s46

/s48

/s48

/s48

/s46

/s48

/s53

/s48

/s46

/s49

/s48

/s48

/s46

/s49

/s53

/s48

/s46

/s50

/s48

/s48

/s46

/s50

/s53

/s48

/s46

/s51

/s48

/s48

/s46

/s51

/s53

MSE

DDMSL

DDMSL(a)

DDMSL(c)

Figure 4: MSE error in ablations.

which are shown in Table 3 and Figure 4. As can be seen in table 3, we have these observations: (1) removing the residual module (DDMSL(a)) leads to a significant degradation in model performance, with the F1 score and AUC decreasing by 49% and 17%, respectively. Although the GCN network is also a form of MPNNs, we noticed that the output of the inference network composed of GCNs were very similar, indicating node feature oversmoothing, and further highlighting the effectiveness of the reversible residual network module; (2) L constraint is also effective, which contributes to 5% and 2% performance improvement in terms of F1 score and AUC, respectively. Similar results can be obtained in Figure 4 regarding the task of reconstructing the information diffusion processes; and (3) upon the drop of the GCNs from the reversible residual network (DDMSL(c)), a noteworthy deterioration model performance was observed, manifesting as a substantial decrease in F1 scores and AUC by 59% and 32% respectively. This compellingly signifies the indispensable role of GCN in acquiring the diffusion mode of the SIR model through the process of learning. Similar results can be obtained in Figure 4 regarding the task of reconstructing the information diffusion processes.

5 Conclusions

In this paper, we introduced a reversible residual network block based on the relationship between diffusion phenomena and message passing neural networks, while ensuring the reversibility of the network by limiting its Lipshitz coefficient. Using this, we constructed a discrete denoising diffusion model (DDMSL) which can locate the source of graph diffusion and restore the diffusion paths. Extensive experiments on five real datasets have demonstrated the effectiveness of DDMSL and its constituent modules. Our work offers insights into how to calculate the distribution of solutions to graph diffusion inverse problems based on the information propagation laws on complex networks.

Solving the inverse problem of graph diffusion plays a crucial role in many social operations, including controlling the spread of infectious diseases, rumors, and computer viruses. It provides valuable insights on enhancing source detection performance and fills the gap in methods for recovering diffusion evolution. However, our work has some limitations. For instance, DDMSL requires the prior knowledge about propagation models, such as infection rate and recovery rate. Although we can infer these parameters from existing observation data [32, 21], it limits the application of DDMSL in situations with insufficient observations of propagation conditions. Our future research will focus on reducing the dependence of DDMSL on prior conditions.

References

Olalekan Adeyinka. Internet attack methods and internet security technology. In 2008 Second Asia International Conference on Modelling & Simulation (AMS) , pages 77-82, 2008.

Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems , 34:17981-17993, 2021.

Suman Banerjee, Mamata Jenamani, and Dilip Kumar Pratihar. A survey on influence maximization in a social network. Knowledge and Information Systems , 62:3417-3455, 2020.

J. Behrmann, W. Grathwohl, Rtq Chen, D. Duvenaud, and J. H. Jacobsen. Invertible residual networks. In International Conference on Machine Learning , 2019.

Marián Boguná, Romualdo Pastor-Satorras, Albert Díaz-Guilera, and Alex Arenas. Models of social networks based on social distance attachment. Physical review E , 70(5):056122, 2004.

Zongmai Cao, Kai Han, and Jianfu Zhu. Information diffusion prediction via dynamic graph neural networks. In 2021 IEEE 24th international conference on computer supported cooperative work in design (CSCWD) , pages 1099-1104. IEEE, 2021.

Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687 , 2022.

Gojko ˇ Cutura, Boning Li, Ananthram Swami, and Santiago Segarra. Deep demixing: Reconstructing the evolution of epidemics using graph neural networks. In 2021 29th European Signal Processing Conference (EUSIPCO) , pages 2204-2208. IEEE, 2021.

Ming Dong, Bolong Zheng, Nguyen Quoc Viet Hung, Han Su, and Guohui Li. Multiple rumor source detection with graph convolutional networks. In Proceedings of the 28th ACM international conference on information and knowledge management , pages 569-578, 2019.

Fei Gao, Jiang Zhang, and Yan Zhang. Neural enhanced dynamic message passing. In International Conference on Artificial Intelligence and Statistics , pages 10471-10482. PMLR, 2022.

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning , pages 1263-1272. PMLR, 2017.

Pablo M Gleiser and Leon Danon. Community structure in jazz. Advances in complex systems , 6(04):565-573, 2003.

H. Gouk, E. Frank, B. Pfahringer, and M. J. Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning , 110(2):393-416, 2021.

Kilian Konstantin Haefeli, Karolis Martinkus, Nathanaël Perraudin, and Roger Wattenhofer. Diffusion models for graphs benefit from discrete state spaces. arXiv preprint arXiv:2210.01549 , 2022.

Qiang He, Hui Fang, Jie Zhang, and Xingwei Wang. Dynamic opinion maximization in social networks. IEEE Transactions on Knowledge and Data Engineering , 35(1):350-361, 2021.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems , 33:6840-6851, 2020.

Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 , 2016.

David Kempe, Jon Kleinberg, and Éva Tardos. Maximizing the spread of influence through a social network. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining , pages 137-146, 2003.

William Ogilvy Kermack and Anderson G McKendrick. A contribution to the mathematical theory of epidemics. Proceedings of the royal society of london. Series A, Containing papers of a mathematical and physical character , 115(772):700-721, 1927.

Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ArXiv , abs/1609.02907, 2017.

Daniel B Larremore, Bailey K Fosdick, Kate M Bubar, Sam Zhang, and Yonatan H Grad. Estimating sars-cov-2 seroprevalence and epidemiological parameters with uncertainty from serological surveys. eLife Sciences , 10, 2021.

Chen Ling, Junji Jiang, Junxiang Wang, and Zhao Liang. Source localization of graph diffusion via variational autoencoders for graph inverse problems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 1010-1020, 2022.

Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11461-11471, 2022.

Wuqiong Luo and Wee Peng Tay. Estimating infection sources in a network with incomplete observations. In 2013 IEEE Global Conference on Signal and Information Processing , pages 301-304. IEEE, 2013.

Wuqiong Luo and Wee Peng Tay. Estimating infection sources in a network with incomplete observations. In 2013 IEEE Global Conference on Signal and Information Processing , pages 301-304. IEEE, 2013.

Wuqiong Luo, Wee Peng Tay, and Mei Leng. How to identify an infection source with limited observations. IEEE Journal of Selected Topics in Signal Processing , 8(4):586-597, 2014.

Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval , 3:127-163, 2000.

Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 , 2018.

Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning , pages 8162-8171. PMLR, 2021.

B Aditya Prakash, Jilles Vreeken, and Christos Faloutsos. Spotting culprits in epidemics: How many and which ones? In 2012 IEEE 12th International Conference on Data Mining , pages 11-20. IEEE, 2012.

Maciel M Queiroz, Dmitry Ivanov, Alexandre Dolgui, and Samuel Fosso Wamba. Impacts of epidemic outbreaks on supply chains: mapping a research agenda amid the covid-19 pandemic through a structured literature review. Annals of operations research , pages 1-38, 2020.

Jonathan M. Read, Jessica R. E. Bridgen, Derek A. T. Cummings, Antonia Ho, and Chris P. Jewell. Novel coronavirus 2019-ncov (covid-19): early estimation of epidemiological parameters and epidemic size estimates. Philosophical Transactions of the Royal Society B: Biological Sciences , 376(1829):20200265-, 2021.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10684-10695, 2022.

Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks , 20(1):61-80, 2008.

Hao Sha, Mohammad Al Hasan, and George Mohler. Source detection on networks using spatial temporal graph convolutional networks. In 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA) , pages 1-11. IEEE, 2021.

Chintan Shah, Nima Dehmamy, Nicola Perra, Matteo Chinazzi, Albert-László Barabási, Alessandro Vespignani, and Rose Yu. Finding patient zero: Learning contagion source with graph neural networks. CoRR , abs/2006.11913, 2020.

Xincheng Shu, Bin Yu, Zhongyuan Ruan, Qingpeng Zhang, and Qi Xuan. Information source estimation with multi-channel graph neural network. In Graph Data Mining , pages 1-27. Springer, 2021.

Tristan SW Stevens, Jean-Luc Robert, Faik C Yu, Jun Seob Shin, and Ruud JG van Sloun. Removing structured noise with diffusion models. arXiv preprint arXiv:2302.05290 , 2023.

Xiaolu Tang, Changcheng Wu, Xiang Li, Yuhe Song, Xinmin Yao, Xinkai Wu, Yuange Duan, Hong Zhang, Yirong Wang, Zhaohui Qian, et al. On the origin and continuing evolution of sars-cov-2. National science review , 7(6):1012-1023, 2020.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209.14734 , 2022.

Junxiang Wang, Junji Jiang, and Liang Zhao. An invertible graph diffusion neural network for source localization. In Proceedings of the ACM Web Conference 2022 , pages 1058-1069, 2022.

Zheng Wang, Chaokun Wang, Jisheng Pei, and Xiaojun Ye. Multiple source detection without knowing the underlying propagation model. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence , AAAI'17, page 217-223. AAAI Press, 2017.

Duncan J Watts and Steven H Strogatz. Collective dynamics of 'small-world'networks. nature , 393(6684):440-442, 1998.

Wenwen Xia, Yuchen Li, Jun Wu, and Shenghong Li. Deepis: Susceptibility estimation on social networks. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining , pages 761-769, 2021.

KiJung Yoon, Renjie Liao, Yuwen Xiong, Lisa Zhang, Ethan Fetaya, Raquel Urtasun, Richard Zemel, and Xaq Pitkow. Inference in probabilistic graphical models by graph neural networks. In 2019 53rd Asilomar Conference on Signals, Systems, and Computers , pages 868-875. IEEE, 2019.

Wayne W Zachary. An information flow model for conflict and fission in small groups. Journal of anthropological research , 33(4):452-473, 1977.

Wenyu Zang, Peng Zhang, Chuan Zhou, and Li Guo. Locating multiple sources in social networks under the sir model: A divide-and-conquer approach. Journal of Computational Science , 10:278-287, 2015.

Laijun Zhao, Hongxin Cui, Xiaoyan Qiu, Xiaoli Wang, and Jiajia Wang. Sir rumor spreading model in the new media age. Physica A: Statistical Mechanics and its Applications , 392(4):9951003, 2013.

Linhua Zhou and Meng Fan. Dynamics of an sir epidemic model with limited medical resources revisited. Nonlinear Analysis: Real World Applications , 13(1):312-324, 2012.

Kai Zhu, Zhen Chen, and Lei Ying. Catch'em all: Locating multiple diffusion sources in networks with partial observations. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence , AAAI'17, page 1676-1682. AAAI Press, 2017.

Linhe Zhu, Gui Guan, and Yimin Li. Nonlinear dynamical analysis and control strategies of a network-based sis epidemic model with time delay. Applied Mathematical Modelling , 70:512-531, 2019.

Arkaitz Zubiaga, Ahmet Aker, Kalina Bontcheva, Maria Liakata, and Rob Procter. Detection and resolution of rumours in social media: A survey. ACM Computing Surveys (CSUR) , 51(2):1-36, 2018.

A Broader impacts

Overall, our work offers valuable insights into how to limit the spread of malicious information. For example, by tracing the spread of infectious diseases, we can identify potential contacts of infected individuals, effectively containing the outbreak. However, it is important to consider the potential negative implications of this approach for society, such as compromising the privacy of individuals living with infectious diseases like HIV. Addressing these concerns should be important.

B Derivation of forward and backward processes

Derivation of forward processes . Extending Equation 4 by means of the Markov property:

Where I is the identity matrix.

Derivation of backward processes . The detailed derivation of Equation 6 is as follows:

From the Bayesian formula, it follows that:

Fitting this distribution using a neural network:

C Propagation rule constraint of information diffusion reconstruction

To investigate the circumstances under which the propagation rules may be violated, let's revisit Equation 6. Note that the denominator serves as the normalization term, while the numerator is composed of two key terms -( x i t Q i t T ) and ( x i 0 ¯ Q i t -1 ) - that are crucial in preserving the propagation rule.

The three rows of Q i t T correspond to the distribution of q ( x i t | x i t -1 , x i 0 ) when x i t is in one of three states: S , I , and R . Similarly, the three rows of ¯ Q i t represent the distribution of q ( x i t -1 | x i 0 ) when x i 0 is in one of three states: S , I , and R . Let [ ¯ Q i t ] 12 and [ ¯ Q i t ] 13 be denoted by q a t and q b t respectively, then q a 1 = γ i 1 , q b 1 = 0 .

When x i t and x i 0 are in different states, the results of q ( x i t -1 | x i t , x i 0 ) = ( x i t Q i t T ) ⊙ ( x i 0 ¯ Q i t -1 ) are shown in Table 4. The table reveals that q ( x i t -1 | x i t , x i 0 ) = [0 , 0 , 0] when the propagation rule is violated. Since we are predicting x i 0 , there are only two possible states for x i 0 : S (Susceptible) and I (Infected), with x i 0 = R (Recovered) being excluded. In such instances, q ( x i t -1 | x i t , x i 0 ) can be set to [1 , 0 , 0] to resolve the issue. Furthermore, if x i t = R and x i 0 = S , then x i t -1 = S would violate the propagation rules. However, as shown in the 9 th row of Table 4, the probability of x i t -1 = S is much smaller than that of x i t -1 = I and x i t -1 = R , and such situations will not lead to training failure. Hence, there is no need for any specific handling of this scenario.

To further minimize propagation rule violations during the training process, we incorporate supervision of the propagation rule. Specifically, when using this supervision function, nodes that have a state of R are set to I to enforce the propagation rule.

where ( A + I ) X 0 represents the total number of infected nodes in a given node's first-order neighborhood. Equation 23 penalizes X 0 if the node is infected and there are no other infected nodes within its first-order neighborhood. To maintain the stability of inferred X 0 generating X t -1 , we apply the same constraint to the process. Specifically, we utilize the monotonicity regularization of information diffusion from [22]. If the source set X ( i ) 0 is a superset of X ( j ) 0 , then the generated X t -1 resulting from their diffusion needs to satisfy the following equation.

D Proofs of lemmas and theorems

D.1 The proof of theorem 3.1

Proof Set the initial infection seed set as: S = { x s 1 0 , x s 2 0 , . . . x s m 0 } . At the initial moment, the infection status distribution of node i is:

̸

At time t , the infection status distribution of node i is:

where A is the adjacency matrix and j is the neighbor of node i . When dealing with a static graph G , A is fixed, allowing for determination of the state distribution at any time based on the initial node state. Specifically, if both graph G and the seed node set S are known, it becomes possible to calculate the state distribution of each node at any given time using Equation 26. Utilizing Equations 2 and 3, Q i t can also be determined.

D.2 Proof of lemma 3.1

Proof The graph convolution layer with batch normalization BN ( g ( ξ )) can be abbreviated as GConv . In our approach, we apply spectral normalization to both the linear transformation U and convolutional layers GConv . As a result, the weight parameters of both the networks f w and GConv possess 1-Lipschitz continuity after spectral normalization, as described in [28].

̸

Let GConv : R n → R m , x 1 , x 2 ∈ R n . Note that the nonlinear activation function σ ( · ) is set to Mish ( · ) , which obviously possesses 1-Lipschitz continuity.

where ∥·∥ p represents the p-norm ( p = 2 or p = ∞ ). Therefore, the Lipschitz constant of f θ is less than 1.

D.3 Proof of theorem 3.2

Proof To prove that Y T = F ( f w ( X 0 )) is reversible, it is necessary to ensure that F and f w are reversible [43].

We construct the following iterative formula:

By Lemma 2, we can ensure that L w < 1 and L θ < 1 . Moreover, the Lipschitz constant of ( X k 0 -f w ( X k 0 ) is 1 -L w , which is less than 1. Thus, when the number of iterations k is sufficiently large, it follows that the transformation Y T = F ( f w ( X 0 )) is reversible according to the Banach fixed point theorem [4].

D.4 Proof of theorem 3.3

Proof Specifically, Theorem 3.2 proves that ( F 1 ◦ F 2 ◦ . . . ◦ F n )( ξ ) is invertible when n = 1 . Therefore, we are currently examining whether ( F 1 ◦ F 2 ◦ . . . ◦ F n )( ξ ) retains its reversibility for n > 1 . To make the notation simpler, we denote ( F 1 ◦ F 2 ◦ . . . ◦ F i )( ξ ) as ˆ F i .

The application of the dropout function DP will limit the Lipschitz constant of any function f to 1 -r times its original value: L DP ( f ) = (1 -r ) L f . Additionally, we have L ˆ F i ≤ ∏ i j =1 ( L F j ) ≤ (1 -r ) i (1 + L f w ) i [13]. Therefore, the Lipschitz constant of ˆ F n is expressed as:

Note that when L f w < 1 and n > 1 , we only need to set r = 1 / 2 to ensure that L ˆ F n < 1 , hereby guaranteeing that ˆ F n is reversible.

E DDMSL implementation details

The DDMSL approach has been previously explained, and we will now provide further details on the implementation of DDMSL. The linear transform U refers to a fully connected layer, and in Figure 2, the dense layer is composed of two fully connected layers that undergo spectral normalization. The

final output of the nn θ is represented by an N × 1 matrix, indicating the probability that each node is in an infected state at t = 0 . We designate nodes with an infection probability higher than a certain threshold as infected nodes.

Given a complete information diffusion instance X = { X 0 , . . . , X T } where X t = { x 1 t , . . . , x N t } , we sample t ∈ { 1 , . . . , T } to be included in the neural network nn θ based on the probability distribution p ( t ) ∼ t ∑ T t =1 t , and use the sine-cosine position encoding [41] to embed t . The training and inference processes are shown in Algorithm 1 and Algorithm 2, respectively. Additionally, the variable T remains consistent with the information diffusion step size.

Algorithm 1: Training

Input:

X 0 , G , X t , threshold α

1 repeat

2 Q t ← Equation 2 and Equation 3

3 q ( x t -1 | x t ) ← Calculate Equation 6 using X t and Q t

4 t ∼ P( { 1 , . . . , T } ) , P ( t ) = t ∑ T t =1 t

// Using data from t , nn θ infers the source node X t -1 0 , which is then used to reconstruct the diffusion graph at t -1 .

5 X t -1 0 = nn θ ( G , X t , t )

6 X t -1 0 [ X t -1 0 > α ] = 1

8 Q ′ t ← Equation 2 and Equation 3

7 X t -1 0 [ X t -1 0 ! = 1] = 0

// Q ′ t is generated by X t -1 0 .

9 P θ ( x t -1 | x t ) ← Calculate Equation 8 using X t , X t -1 0 and Q ′ t

11 Take gradient descent step on:

12 ∇ θ ( L simple + L constrain )

Algorithm 2: Infering

Input: X t , G , Empty set X , threshold α Output: X 1 for t starts from T to 1 do 2 X t - 1 0 = nn θ ( G , X t , t ) 3 X t - 1 0 [ X t - 1 0 > α ] = 1 4 X t - 1 0 [ X t - 1 0 ! = 1] = 0 5 Q ′ t ← Equation 2 and Equation 3 6 P θ ( x t - 1 | x t ) ← Calculate Equation 8 using X t , X t - 1 0 and Q ′ t 7 X t - 1 ← Gumbel - Softmax ( P θ ( x t - 1 | x t )) 8 X [ t ] ← X t - 1 9 end

In Algorithm 1, we have Q t = { Q 1 t , . . . , Q N t } , where Q i t can be calculated using Equations 2 and 3. Additionally, the P i S ( t ) , P i I ( t ) , and P i R ( t ) in Equation 3 can be computed using various methods. Monte Carlo simulations provide the most accurate results, but require at least 10 5 simulations to be sufficiently precise, leading to a high time complexity. An alternative approach is to utilize a neural network model [10, 47] to learn [ P i S ( t ) , P i I ( t ) , P i R ( t ) ] , which significantly reduces the training time of the model. When applied to the SI model, DDMSL utilizes the state transfer matrix by:

where X t -1 0 represents the predicted source node using X t , while Q ′ t is generated using the same method as above. Ultimately, P θ ( x t -1 | x t ) is calculated using X t -1 0 , X t , and X t -1 0 . We obtain X t -1 by sampling from P θ ( x t -1 | x t ) , and label nodes as 0( S ), 1( I ) , or 2( R ) based on their state. Algorithm 2 proceeds in a similar manner to the process described above.

F Additional algorithms and dataset parameters

F.1 Hyperparameters of the algorithms

The hyperparameters for each algorithm have been set according to the values shown in Table 5. Any parameter that is not stated as default is common to both SI and SIR models. In the updated version of SLVAE, the original three-layer MLP network encoder was replaced with a three-layer GCN network, resulting in improved performance. For hyperparameters and implementation details of other algorithms, please refer to the corresponding original papers. DDMSL and deep learning comparison

algorithms are both running on Windows 10 systems and trained using a 4090 graphics card. The code for DDMSL is already open source, please refer to: https://github.com/Ashenone2/DDMSL .

F.2 Additional details of the datasets

The description of the data sets used for the experiments and their statistics are shown as below:

Karate [48]: It includes a network of interrelationships between 34 members of the Karate club, comprising 34 nodes and 78 edges. The Karate dataset is a real dataset, widely employed in complex network community discovery research.

Jazz [12]: The Jazz dataset is a network dataset that captures the collaborative relationships between jazz musicians. It comprises 198 nodes and 2,742 edges, and has been extensively used in research on complex network community discovery, node importance metrics, and other related studies.

Cora-ML [27]: Cora-ML is a citation network dataset containing papers from the field of machine learning. Nodes represent papers and edges represent citation relationships between papers.

Power Grid [45]: The Power Grid dataset is a network dataset describing the topology of the Northeastern US power grid, containing 4,941 nodes and 6,594 edges.

PGP [5]: It is a User network of the Pretty-Good-Privacy algorithm for secure information exchange, consisting of 10,680 nodes and 24,316 edges.

G Additional experiments

Experiments on Real Diffusion Datasets . In order to gauge the efficacy of DDMSL on realworld propagation datasets, we opted for the Twitter [6] and Douban [6] datasets, encompassing 12,627 nodes with 309,631 edges, and 23,123 nodes with 348,280 edges, respectively. The detailed performance metrics can be found in Table 7.

Generalization Performance . We conducted extensive tests on datasets of varying scales to assess the generalization performance of both DDMSL and SLVAE algorithms. The comparative results are presented in Table 8, revealing that DDMSL demonstrates commendable generalization performance across a majority of scenarios.

Table 8: Additional generalization experiments: Test results on different network topologies after one training on a real network, where the original performance represents the test performance of DDMSL on real networks.

Time Complexity . Lastly, we conducted a comparative evaluation of the time complexity between DDMSL and baseline algorithms across diverse datasets, revealing the outcomes illustrated in Table 9. Owing to the gradual inference of diffusion state for each time step, the time complexity of DDMSL tends to be substantial. However, optimization through parallel computing can effectively mitigate this disparity.

H Visualization

H.1 Visualization of reconstructing diffusion paths

To conserve space, we displayed the actual node states and corresponding prediction results every 20% of the time. Figures 5 to 9 showcase the results, where the blue nodes denote susceptible ones, the red nodes denote infected nodes, and the green nodes denote recovered nodes. The findings indicate that DDMSL accurately restored the node states at different times. In contrast, DDMIX could only restore infected nodes, revealing that DDMSL far surpasses DDMIX in its expression capability.















H.2 Visual comparison of source localization

Due to spatial limitations, we only presented the source localization results of DDMSL and benchmark algorithms on the Karate and Jazz datasets. The results are depicted in Figures 10 and 11. The baseline algorithm's performance is unsatisfactory, as evidenced by significant positioning errors in the source nodes. On the contrary, DDMSL outperforms other benchmark algorithms in accurately identifying source nodes. Moreover, even when misidentifying source nodes, DDMSL locates them near the actual nodes.





















































































Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work discusses the challenges associated with tracing the origin and path of information diffusion in complex networks, such as those involved in epidemics or rumors. To address these, the authors propose a probabilistic model, DDMSL (Discrete Diffusion Model for Source Localization), which utilizes Markov chains to model information diffusion and a reversible residual network to localize the source and reconstruct the paths of information diffusion, with its efficacy supported by extensive tests on five real-world datasets.\n\nSTRENGTHS:\nThe general topic is interesting, and the paper studies the problem of source localization under two information propagation patterns: SI and SIR. The proposed methods are sound with theoretical guarantee. The overall writing is easy to follow, but the use of notations are sometimes less formal. The idea of simultaneously handling the problems of source localization and reconstructing diffusion paths seems to be interesting, but the motivation is less promising. The experiments are strong, and the proposed method is better than most of existing baselines.\n\nWEAKNESSES:\nThere are three main questions of this work. Please refer to the questions section.\n\nQUESTIONS:\n1. The motivation for recovering the diffusion path is weak. I feel like the source localization only needs to predict the correct set of seed nodes in order to identify the culprit, why would it be necessary to recover the diffusion paths with the already given diffusion status/observation? I am still having a hard to convince myself with the provided motivation.\n2. The authors claim \"existing inference results are often deterministic and fail to quantify the uncertainty of source localization\", but it seems like the proposed method's results are also deterministic from Figure 2. Does the proposed method output a range of possible source nodes with [S, I, R] labels?\n3. In terms of the experiment section, the biggest problem is the efficiency issue of the proposed framework. With the step-wise diffusion model to recover the overall diffusion path, the runtime of the proposed method and the complexity analysis should be demonstrated.\n\nLIMITATIONS:\nI am not seeing any potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a discrete denoising diffusion model for source localization called DDMSL. DDMSL can simultaneously locate information sources and restore information propagation paths. Experiments results on real-world datasets demonstrate DDMSL’s effectiveness.\n\nSTRENGTHS:\n1. This is the first study to solve information spread problems based on denoising diffusion model.\n2. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed method.\n\nWEAKNESSES:\n1. The motivation of employing denoising diffusion model is not clear. Why is this generative model used for source localization problem? Compared to other source localization models, what are the advantages of the denoising diffusion model?\n2. I do not agree that the proposed method is a denoising diffusion model. The final distribution of the forward process and the initial distribution of the reverse process are both random because they are simulated by the SIR model. It is more like a sequence model trained by the SIR data.\n3. The motivation behind the model design is not clear. For example, why are SN and BN employed in Eq. 16?\n\nQUESTIONS:\n1. The phrase “graph inverse problem” or “inverse problem” sounds peculiar, perhaps the diffusion inverse problem or the spread inverse problem would be more comprehensible.\n2. The formulation of the proposed denoising diffusion model is not clear. In Eqs. (4) and (5), why is $x^i_t$ included in $q(x^i_t | x^i_{t-1})$ and $q(x^i_t | x^i_0)$? If we already know $x^i_t$, is it necessary to have $x^i_{t-1} or x^i_0$ to calculate $x^i_t$?\n3. How to obtain the distribution $Cat(x^i_t; p)$?\n4. What are reaction diffusion models? Maybe they should be included in the Section Related Work?\n\nLIMITATIONS:\n1. The proposed model requires simulation of SIR from t_0 to t_T to obtain the training data, whereas diffusion models can be trained at any step by sampling Gaussian noise at an indicated time step. It seems that the model proposed in this paper needs more training time and is difficult to apply to large-scale graphs, such as OGB datasets. It is also necessary to compare time complexity with baseline methods.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nInformation diffusion is common in various domains, such as social networks, the internet, and disease propagation. Obtaining the diffusion paths and localizing the source based on the final diffusion node states is beneficial for researchers to identify key transmission pathways during information dissemination and facilitate control over the propagation. This article proposes a framework for information diffusion called Discrete Diffusion Model for Source Localization (DDMSL) using the Susceptible-Infected-Recovered (SIR) model. The framework utilizes a discrete Markov Chain to characterize information diffusion and recover diffusion paths based on observed results while achieving source localization. For the neural network module in the framework, the authors design an inference model based on a residual network and rigorously prove its reversibility to justify the rationality of the model design. The experiments are performed on five real-world networks, where propagation data is generated using SIR and SI models. The experiment results demonstrate that, compared to the baseline, DDMSL performs better in source localization and diffusion path recovery across all datasets.\n\nSTRENGTHS:\n- Interesting idea. This paper proposes a method to establish a correlation between the information propagation process and the denoising diffusion model, which is worth further research.\n- Rigorous theoretical guarantees. This paper presents reliable theoretical proofs to explain the designs of the model, which are also proven effective in the experiment stage.\n- Well performance on different networks(datasets). The experiment results demonstrate the consistent performance across different networks and the improvement compared to the baselines.\n\nWEAKNESSES:\n- This paper needs to clarify why the diffusion model is being used for this task. Only the achievements of the diffusion model in the image inverse problems are mentioned. Personally, it seems that there is a significant difference between the inverse problems of images and the task addressed in this paper.\n- In my view, the method proposed is not highly correlated to the so-called denoising diffusion model, as the initial input is not a particular noise, and the diffusion steps $T$ are set equal to the steps of information propagation.\n- The paper does not show the model's generalization ability on real-world propagation datasets (only SIR or SI model as presented in the paper, no real-world datasets like *Digg* and *Memetracker*).\n- Compared to previous works, the proposed method requires fine-grained diffusion history information, which may largely limit its application scenarios.\n- Compared to previous works that do not explicitly simulate every diffusion step, such as SL-VAE, this method seems much more inefficient. And no efficiency study is provided.\n\n## After Rebuttal\nThe authors' response has addressed most of my concerns. I suggest authors discuss the following points in the revised versions.\n1. How to decide the number of diffusion/denoising steps in the diffusion model, the same as the real-world forward diffusion process on networks? What if this number is extremely large?\n\n2. Please provide generalization performance on other real-world large networks, and comparison with baseline methods.\n\n3. Requiring fine-grained diffusion history information is still a large limitation. Can this forward diffusion history be inferred with parameter estimation?\n\nQUESTIONS:\nMy most concerning questions are mostly mentioned in the *Weaknesses* part, which are:\n\n1. The reason why the proposed method utilizes the denoising diffusion model for this task.\n2. The performance of the proposed method on real-world propagation datasets is not provided.\n3. In IVGD(Wang et al.), the invertibility of the network is proven so that fixed-point iterations can be performed. I am concerned about how the reversibility of residual blocks, strictly proven, contributes to the prediction of $X_0$.(In Figure 2, the input is $Y_T$, while in the text of sec.3.4, the input is $X_0$.)\n4. Moreover, I would like to know whether the model has been tested on different graph topologies under a single training. Because in most application scenarios, the trained model is asked to generalize on the changing graphs. I wonder whether the results in the paper are from the model trained and tested on the same static network topology(dataset).\n\nLIMITATIONS:\nThe authors have illustrated the main limitation that the model requires prior knowledge about propagation models. \n\nThe provided source code cannot be run directly.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of source identification in a stochastic\nnetwork diffusion process, having observed the set of nodes that are\ninfected at time T. It considers SIR and SI models. The authors propose a\nneural network based solution called DDMSL. The authors approach is based\non a reaction-diffusion process and formulate it using a message-passing\nfunction and use a reversible residual network for source identification\nand reconstruction of the cascade. DDMSL is compared with several\nbaselines and an ablation study is conducted to evaluate the significance of\nthe reversible residual network and propagation rule supervision module.\n\nSTRENGTHS:\nThe authors use a novel reversible residual network construction for the\nsource identification problem.\n\nThe paper has a strong experiments component considering multiple networks\nand several baselines.\n\nDDMSL shows much superior performance compared to the baselines.\n\nWEAKNESSES:\nThe presentation could be greatly improved. Many notation and crucial\nconcepts are missing. The derivations are very dense and lack intuitive\nexplanations. So, it is hard to understand the results. Details follow.\n\nDiffusion models are not clearly defined: There are several types of SIR\nand SI models in the literature. The authors fail to describe their model\nclearly. Is it a discrete-time SIR process, i.e., an infected node infects\nits neighbors with probability equal to the edge weight at each time step\nuntil it recovers, which again is determined by some probability? A second\ntype of process is a Gillespie process which is a discrete-event process\nwhich involves rates as the authors have used. Also, the authors seem to\nuse a single transmission rate per node. This is important to clarify as\nthe baselines considered in the paper use their own definitions of SIR\nmodels. Is it consistent with the definition of the SIR model in this\npaper? If not, then the comparison wouldn't be fair.\n\nReaction-diffusion process is not defined. In section 3.4.1, the authors\nseem to relate GCN with reaction-diffusion models. This is not clear at all\nas the concept is not introduced.\n\nTheorem 3.1: What is the purpose of this theorem? Further, its proof is not\npresented, even in the appendix. Also, it is not referred to anywhere.\nEquation (3) is not used anywhere as well. It is not clear why the rates\ncan be expressed in that manner. Also, even though these are mentioned as\nrates, beta and gamma are used as probabilities.\n\nSeveral symbols are not defined: nn_\\theta, KL, D_KL, Cat(), Q_k^i. It is\nvery difficult to follow the derivations. They are dense and lack\nexplanations.\n\nAblation: In DDMSL(a), the reverse residual block is removed, but replaced\nby a GCN network. Typically, in an ablation study, the component is removed\nand the performance evaluated. It is possible that the GCN network\ndeteriorates the performance.\n\nThe Figures 6 to 11 in the Appendix are very difficult to read and infer.\nBesides, these are just few instances where DDMSL does well. Table 2\nalready quantifies the performance.\n\nQUESTIONS:\nSeveral definitions of notation are missing. Diffusion model is not\ndefined. These are mentioned in the Weakness section.\n\nIn Algorithm 1 line 4, t is sampled from a distribution where the\nprobability of t increases as its value increases. This is not explained\nanywhere. Why is this sampling necessary?\n\nLIMITATIONS:\nNo negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-09"}