Paper: Denoising Graph Dissipation Model Improves Graph Representation Learning (ICLR-2024)

Reviewer comment (Experiments):
In this work, the authors did not mention the time complexity of GDM and its runtime in experiments. As GDM requires eigendecomposition of the graph Laplacian matrix, I wonder if the authors could further discuss GDM's time complexity and also provide some results of the GDM's runtime compared to other baselines in the link prediction and node classification experiments.

Author response:
Since we leverage Laplacian smoothing as a noise source in the forward process of GDM, it does not require the eigendecomposition of graph Laplacian matrix. We mentioned eigendecomposition for providing insight on graph spectral domain. We are sorry for the misleading typo. We will immediately revise it. 
As there is no algorithm in the training procedure, GDM's runtime is 55seconds on OGB-Collab which is comparable with conventional GNNs which take 20~30 seconds on the dataset. It is notable that GDM achieves competitive results in link prediction tasks compared to SEAL or Neo-GNNs where runtime is fairly longer ($>3 \text{min}$) due to extracting enclosing subgraphs or generalizing algorithms.

Reviewer comment (Reproducibility):
(Minor) I did not find any supplementary materials discussing the details of the implementation of GDM and the experiments conducted in the paper. There is also no code implementation of GDM to reproduce the experimental results presented in the paper.

Author response:
*Minor*
(3) We will upload the code implementation of GDM soon.

Reviewer comment (Writing):
(Minor) Typo: In the Implementation Details of Section 5.1, \
"Also we set iffusion state to 3 for OGB-Citation2" $\rightarrow$ "Also we set diffusion state to 3 for OGB-Citation2"

Author response:
*Minor*
(4) Thanks to the reviewer, we will fix typos in the revised paper.

Reviewer comment (Experiments):
As the over-smoothing issue appears after only several Laplacian smoothing operations (i.e., node representations converge to identical after only several steps), it seems the value of time step $t$ can be small if we set the over-smoothing as the convergence state. Therefore, I wonder how to choose a proper $t$ to ensure sufficient diffusion and if the authors have done some experiments on the selection of $t$.

Author response:
This is a great point! Right. Unlike diffusion models proposed for image or graph generation, GDM does not require extremely large numbers as time step values, such as 1000 or 10000. The oversmoothing issue in GNNs is known to occur when there are at least five layers. Based on this, we determined that setting the time step ($T$) to a minimum of 5 is sufficient. Through experiments, we found that setting $T$ to 6 for Collab, PPA, and DDI is sufficient. Adaptive learning of $T$ value is something we consider for future work.
Additionally, we will include the sensitivity analysis for the selection of $T$ in the appendix.

Reviewer comment (Novelty):
The novelty of the structure diffusion process with randomly removing edges is limited, which also appears in [3]. Further, this reverse process of the used structure diffusion cannot correspond to the forward process.

Author response:
We acknowledge that the term 'edge removal' can cause misleading and it does not express the goal of the process. 

Our edge removal is completely different from random edge removal from [3].
Because we aim to reflect the decay of signal/information on the graph, we designed the subgraph sampling that can lift feature decay to the graph structure. 
Specifically, edges are sampled to be dropped from the given graph structure stochastically based on the amount of feature smoothed as drop parameter $p$ during the structure diffusion process. 
This is significantly different from [3] as this sampling is coupled with the feature diffusion process and incorporates removing edges in a way that is influenced by the degree of feature smoothing. This sampling procedure is proposed to lift dissipation of features to the structure.

On the contrary, [3] employs edge removal for generic graph generation by randomly selecting nodes and then probabilistically removing edges connected to those selected nodes solely relying on randomness. Edge removal of [3] may be suitable for graph generation since it requires diverse structures to generate unique structures but with similar graph statistics.

To clarify the novelty of GDM and to prevent misleading, we will use the term "dissipative structure sampling" instead of the term "edge removal" in the revised paper.

Reviewer comment (Theory):
In Eq(9), the Feature prediction loss and structure dissipation loss are both confusing. How to calculate the $q(X_{t-1}|X_{t},X_{0})$ and $q(A_{t-1}|A_{t},A_{0})$? The relationship between ELOB (Eq. (8)) and final loss (Eq 9) should be rigorously proved.

Author response:
This is a good point. First, we will derive the loss (Eq. 9) from the negative of evidence lower bound (ELBO), then explain q( X\_{t-1} | X_{t}, X\_{0} ) $ and $ q( A\_{t-1} | A_{t}, A\_{0} ) $.

Let $G\_0$ be a given observed graph data consisting of $(X, A), denoting node features and adjacency matrix, respectively. Taking negative log-likelihood with evidence lower bound (ELBO), we get

$$
 -\log p\_{\theta}(G_{0}) \leq  -\int q(G_{1:T} | G_{0}) \log p_{\theta}(G_{0}|G_{1:T}) \frac{p_{\theta}(G_{1:T})}{q(G_{1:T}|G_{G\_{0}})} \\,dG
$$
$$
= \mathbb{E}\_{q(G_{1:T} | G_{0})} \left[ -\log \frac{p_{\theta}(G_{0:T})}{q(G_{1:T}|G\_{0})} \right] .
$$

Note that the forward feature process is defined as $ q(X\_{1:T} | X_{0}) = \prod_{t=1}^{T} q(X_{t} | X\_{0}) $. As we lift feature dissipation to the forward structural process, under mild assumption, approximated $q(A\_{t}|A_{t-1}) \approx q(A_{t}|A\_{0}) $ can be used as follows:
$$
= \mathbb{E}\_{q(G_{1:T} | G_{0})} \left[ -\log \frac{p_{\theta}(G_{T}) \prod_{t=1}^{T} p_{\theta}(G_{t-1}|G_{t})}{\prod_{t=1}^{T} q(G_{t}|G\_{0})} \right]
$$
$$
= \mathbb{E}\_{q(G_{1:T} | G_{0})} \left[ -\log \frac{p_{\theta}(G_{T})}{q(G_{T}|G_{0})} -\log \prod_{t=2}^{T} \frac{p_{\theta}(G_{t-1}|G_{t})}{q(G_{t-1}|G_{0}) } -\log p_{\theta}(G_{0}|G\_{1}) \right]
$$
$$
= \mathbb{E}\_{q(G_{1:T} | G_{0})} \left[ -\log \cancel{\frac{p(G_{T})}{q(G_{T}|G_{0})}} -\sum_{t=2}^{T} \log \frac{p_{\theta}( G_{t-1}| G_{t} ) }{q(G_{t-1}|G_{0})} -\log p_{\theta}(G_{0} | G\_{1}) \right].
$$
The second term can be interpreted as KL divergence $D\_{KL} [ q(G_{t-1}|G_{0}) \Vert P_{\theta}(G_{t-1}|G\_{t}) ]$. Posterior $q( G\_{t-1} | G\_{0})$ cannot be expressed in a closed-form solution because $q(G\_{t-1}|G\_{0})$ is unknown as several well-known distributions are inadequate to define a network graph due to its structural characteristics. 
Especially, it becomes more difficult because GDM targets to learn feature-structure integrated latent representation. Therefore, posterior $q( G\_{t-1} | G\_{0})$ is intaractable.

However, we define $\mathbb{E}\_{q(G_{1:T} | G_{0})} \left[ -\sum_{t=2}^{T} \log \frac{p_{\theta}( G_{t-1}| G_{t} ) }{q(G_{t-1}|G\_{0})} \right]$ by decomposing $G$ to feature $X$ and structure $A$.
Posterior on feature $q(X\_{t-1}|X\_{0})$ is straight forward according to Eq.(3): $q(X\_{t-1}|X_{0}) = (I - L)^{t-1}X\_{0}$. 
Thus, we define $\mathbb{E}\_{q(X_{1:T} | X_{0})} \left[ -\sum_{t=2}^{T} \log \frac{p_{\theta}( X_{t-1}| X_{t} ) }{q(X_{t-1}|X\_{0})} \right]$ as predicting $(I-L)^{t-1}X_{0}$ from $X\_{t}$, i.e., $\lVert f_{\theta}(X_{t}, A_{t})-X\_{t-1} \rVert\_{2}^{2}$.

Posterior on structure $q(A\_{t-1}|A\_{0})$ is approximately obtained as follows:
$$
q(A\_{ij}^{(t-1)}|A_{ij}^{(0)}) = 
\mathcal{B}(A_{ij}^{(t-1)};p \propto  L^{t-1}X), \quad  \text{if } A\_{ij}^{(0)}=1 
$$
$$
q(A\_{ij}^{(t-1)}|A_{ij}^{(0)}) = 
\mathcal{B}(A_{ij}^{(t-1)};p=0),  \quad  \text{if }A\_{ij}^{(0)}=0 
$$
Only in the first case, edge existence probability $p$ is uncertain.
Note that edge probability $p$ is correlated to Laplacian matrix which feature dissipation relied on. 
To learn comprehensive and integrated latent representation, we defined the forward structural process with stochastic structure sampling based on connected node pairs' similarity. Consequently, the forward structural process makes the graph structure sparser as the node features converge to oversmoothing. However, due to its uncertainty, the edge probability $p$ cannot be estimated by just restoring feature similarity. 
The intuition behind the forward structural process is to lift signal dissipation to graph structure. Leveraging this intuition, the edge probability $p$ is estimated by discrepancy of structural information which implies dissipation upon graph structure. Therefore, we define $ \mathbb{E}\_{q(A_{1:T} | A_{0})} \left[ -\sum_{t=2}^{T} \log \frac{p_{\theta}( A_{t-1}| A_{t} ) }{q(A_{t-1}|A\_{0})} \right]$ with predicting the discrepancy between graph Laplacian where dissipation is dependent, i.e., $\lVert f_{\theta}(X_{t}, A_{t})-(L_{0}-L_{t-1})  \rVert\_{2}^{2}$.

Reviewer comment (Theory):
Eq (6) is confusing since the $A_t$ is sampled from Eq 5, which is unrelated to $A_{t-1}$. So, how to calculate the elements of $A_t$?

Author response:
Sorry for the confusion.
Eq.(5) expresses each edge is sampled to be dropped by the parameter $p$, lifting dissipation of feature to the graph structure. We meant to show Bernoulli sampling on each edge.
Eq.(6) expresses the sampling for whole adjacency $A_t$.

We will clarify the Eq.(5) as follows:

$ A\_{t}[ij] \sim \text{Bern}(A_{t} | A_{t-1}[ij]=1, p=s(  \hat{\mathbf{x}}_{i}^{(t-1)} , \hat{\mathbf{x}}\_{j}^{(t-1)} )). $

Reviewer comment (Experiments):
The experimental results show the proposed method doesn’t achieve competitive performance in Link prediction (https://ogb.stanford.edu/docs/leader_linkprop/). Some important baselines are missing, such as GIN, on the node classification task.

Author response:
The top-ranked performances on the OGB leaderboard are mostly a result of combining various auxiliary techniques such as augmentations or additional plug-in methods. 

It is noteworthy that the goal of GDM is to learn latent graph representations universally applicable to network graph learning tasks but capturing underlying latent factors that are intrinsically capable of given network graph learning tasks.
We compared our model against widely-known effective GNN-based models without such plug-ins, including SEAL which exhibits powerful performance on link prediction tasks among other models in OGB Leaderboard. 

Besides, the significance of our work arises from the motivation behind the GDM that existing graph representation learning models are investigated for improving particular graph representation learning tasks, yet they are often vulnerable to other tasks. For instance, in the link prediction task, some existing models assume generalizing some graph heuristics to find missing links. However, those existing models are not capable of learning node embeddings for node classification tasks.

We demonstrate that GDM mitigates its motivation as GDM results in competitive and meaningful performance on both major graph representation learning tasks: link prediction and semi-supervised node classification. Especially, in link prediction tasks, each dataset entails some specifics regarding whether graph structure property is more essential than node embeddings or vice versa. Conventional GNNs show poor performance often since they heavily rely on locality of node embeddings, whereas some competitive models designed for link prediction often fall on a dataset that weights more attention to node embeddings than graph structure properties.
However, GDM consistently show comparable outputs across all realistic large networks graphs in OGB.
 
> Some important baselines are missing, such as GIN, on the node classification task.

We indeed know GIN is outstanding in graph representation learning, particularly in graph classification. Since its motivation is to  capture graph isomorphism, ultimately learning small graph representations that are invariant to the same isomorphism, GIN sometimes shows trivial result in network graph representation learning tasks.

As reviewer suggested, we conducted semi-supervised node classification task on OGB-Arxiv. The results are as follows:

|        | **K=1** | **K=5** | **K=10** |
|----------------|---------|---------|----------|
| **GIN**        | $5.32\pm{5.92}$  | $29.22\pm{2.28}$ | $34.98\pm{2.43}$  |
| **GCN**        | $31.69\pm{2.74}$ | $52.97\pm{0.94}$ | $58.39\pm{0.50}$  |
| **GDM (Ours)** | $38.40\pm{1.64}$ | $57.22\pm{0.85}$ | $60.97\pm{0.40}$  |

Note that our semi-supervised node classification setting is extremely restrained, hence, some GNNs may show low performances if mechanism does not propagate information. We verified that GIN is not suitable for our task. 
Indeed, our experiments did not omit strong baselines.

Reviewer comment (Theory):
Eq (9) is out of bounds.

Author response:
To address the reviewer's concern, we provide more precise expressions for the loss function and its derivation. Besides, we acknowledge there are some typos that may cause misleading and confusion. We will correct typos and provide the derivation for loss function in the revised paper including appendix.

Reviewer comment (Novelty):
The claim “there has been no work on diffusion models for graph representation learning in both feature and structural aspects” is inappropriate because there exist related works such as MoleculeSDE [1], [2].

Author response:
Our statement "there has been no work on diffusion models for graph representation learning in both feature and structural aspects" holds true without any exaggeration. 

To address potential misunderstanding about the meaning of our work, we will provide a clearer explanation of the significance of GDM. 
The motivation behind GDM is that existing graph representation learning models have focused on defining task-dependent / task-specific assumptions for a particular graph learning task. For instance, existing GNNs that mostly rely on message-passing, usually show good performance in node classification tasks while showing limited results in some link prediction tasks. Consequently, there are no powerful models that are capable of solving both link prediction and node classification tasks. 
It is noteworthy that we initially raised the need for this motivation since such motivation has not been raised before.

To mitigate this motivation, we introduce GDM, a diffusion model-based graph representation that is universally applicable to network graph representation learning tasks. Regarding both features and structure of a network graph, the goal of GDM is to learn comprehensive latent representations universally applicable to network graph learning tasks, capturing underlying latent factors from a graph that are inherent in a given task. Leveraging the intuition of diffusion models to capture arbitrary data distributions, GDM learns comprehensive and integrated latent representations crucial for given network graph representation learning tasks (e.g., link prediction) without task-oriented assumptions. Remarkably, it achieves competitive results in both major graph representation learning tasks, link prediction and node classification, without applying task-specific assumptions or biases into a model. As far as our knowledge extends, such research has not been introduced before, highlighting the novelty of our study.

Unlike our GDM, MoleculeSDE[1] and [2] are diffusion models for molecular graph learning which requires readout/pooling to obtain final outputs and diffusion model for graph generation tasks.
In other words, GDM and [1], [2] target different types of graphs. Molecular graphs are distinctive from network graphs. Molecular graphs have node and edge categorical information, and there are multiple graphs in the dataset, making it possible to define the distribution needed for the diffusion model. This is why diffusion models for graph has been numerously studied. On the contrary, network graphs consist of a single given graph, and sometimes the node's categorical information is incomplete or absent, making it insufficient to define the distribution of the graph. Thus, the diffusion models for graph representation learning has not been investigated thoroughly as molecular graphs.

Indeed, GDM and the reference [1], [2] that the reviewer mentioned are completely different research areas.
To address the misunderstanding, we will revise the statement it is for *network* graph representation learning.

Reviewer comment (Writing):
The formula at the bottom of page 3 lacks the explanation of $x$.

Author response:
$x$ located at the bottom of page 3 denotes an arbitrary data point to explain Laplacian smoothing. Since this equation is trivial, we will shorten this part in the revised paper.

Reviewer comment (Writing):
Is there $\zeta$ in Eq(5)?

Author response:
$\hat{x}_{i}^{(t-1)}$ denotes a row vector corresponding to node $i$ in $\hat{X}_{t-1}$ in Eq.(4). $zeta$ is a variance parameter of normal distribution to avoid $p$ converges to $1.0$ by giving very small random noise to oversmoothed features.

Reviewer comment (Experiments):
From the Leaderboards of OGB (https://ogb.stanford.edu/docs/leader_linkprop/), the experimental results of this paper are not very competitive. Why does the GDM not use a powerful GNN as the denoising network? In my understanding, the Loss $L_{diff}$ can be used in any GNN for graph representation learning.

Author response:
The reason for defining and using our own Denoising network in GDM is as follows: At the convergence state T, node features are strongly oversmoothed and graph structure is highly scarce. 
Since existing GNNs rely on a message-passing mechanism with an adjacency matrix, most of them suffered from oversmoothing problem, indicating limitations in finding the meaningful latent factors from graph data from the convergence state. Therefore, we define the Denoising network with a learnable parameter, the latent Laplacian, to ensure GDM can extract latent information as much as possible.

> Q(1). From the Leaderboards of OGB(https://ogb.stanford.edu/docs/leader_linkprop/), the experimental results of this paper are not very competitive. 
/ W(4). The experimental results show the proposed method doesn’t achieve competitive performance in Link prediction (https://ogb.stanford.edu/docs/leader_linkprop/). 

The top-ranked performances on the OGB leaderboard are mostly a result of combining various auxiliary techniques such as augmentations or additional plug-in methods. 

It is noteworthy that the goal of GDM is to learn latent graph representations universally applicable to network graph learning tasks but capturing underlying latent factors that are intrinsically capable of given network graph learning tasks.
We compared our model against widely-known effective GNN-based models without such plug-ins, including SEAL which exhibits powerful performance on link prediction tasks among other models in OGB Leaderboard. 

Besides, the significance of our work arises from the motivation behind the GDM that existing graph representation learning models are investigated for improving particular graph representation learning tasks, yet they are often vulnerable to other tasks. For instance, in the link prediction task, some existing models assume generalizing some graph heuristics to find missing links. However, those existing models are not capable of learning node embeddings for node classification tasks.

We demonstrate that GDM mitigates its motivation as GDM results in competitive and meaningful performance on both major graph representation learning tasks: link prediction and semi-supervised node classification. Especially, in link prediction tasks, each dataset entails some specifics regarding whether graph structure property is more essential than node embeddings or vice versa. Conventional GNNs show poor performance often since they heavily rely on locality of node embeddings, whereas some competitive models designed for link prediction often fall on a dataset that weights more attention to node embeddings than graph structure properties.
However, GDM consistently show comparable outputs across all realistic large networks graphs in OGB.

Reviewer comment (Novelty):
What is the relationship between GDM and Digress [1]? The GDM seems to be a specific case of Digress.

Author response:
GDM is completely different from DiGress (Vignac et al., 2022). It differs in data domain, motivation, and noise source.

Difference from DiGress (Vignac et al., 2022): While GDM is a model for graph representation learning in network-shaped graphs, DiGress is a model for molecular graph generation. In other words, GDM and DiGress target different types of graphs. Molecular graphs have node and edge categorical information, and there are multiple graphs in the dataset, making it easier to define the distribution needed for the diffusion model. However, network graphs consist of a single given graph, and sometimes the node's categorical information is incomplete or absent, making it insufficient to define the distribution of the graph.

Reviewer comment (Novelty):
The extension of Rissanen et al., 22' work, using Laplacian smoothing for graphs, was natural and even mentioned in the original paper's discussion section. The claim of *no work on diffusion models for graph representation learning in both feature and structural aspects* feels like an exaggeration, as Vignac et al. 22' (also cited in the manuscript) uses both node features and structural information.

Author response:
Our statement holds true without any exaggeration. Also, the studies the reviewer mentioned are significantly different from GDM.
To address potential misleading about the meaning of our work, we will provide a clearer explanation of the significance of GDM. 

The motivation behind GDM is to learn latent graph representations universally applicable to network graph learning tasks, capturing underlying latent factors inherent in a given network graph representation learning tasks. Existing work has focused on defining task-dependent / task-specific assumptions for a particular graph learning task. For instance, in the link prediction task, some existing models assume generalizing some graph heuristics to find missing links. However, those existing models are not capable of learning node embeddings for node classification tasks. On the other hand, existing GNNs usually show good performance in node classification tasks while showing limitation in link prediction tasks. Hence, there are no powerful models that can solve both link prediction and node classification tasks.

To address this issue, we introduce GDM, a diffusion model-based graph representation learning approach that considers both features and structure of a network graph. Leveraging the capabilities of diffusion models to capture arbitrary data distributions, GDM learns comprehensive and integrated latent representations crucial for given network graph representation learning tasks (e.g., link prediction). Remarkably, it achieves competitive results in both major graph representation learning tasks, link prediction and node classification, without applying task-specific assumptions or biases into a model. As far as our knowledge extends, such research has not been introduced before, highlighting the novelty of our study.

Besides, GDM is completely different from IHDM (Rissanen et al., 2022) and DiGress (Vignac et al., 2022). 
It differs in data domain, motivation, and noise source. 
- Difference from IHDM (Rissanen et al., 2022):
Unlike GDM, IHDM incorporates the multi-resolution nature of the image domain as an inductive bias to enhance the quality of image generation. Not only is IHDM not in the graph domain, but its motivation is also different as it focuses on image generation. While both GDM and IHDM use smoothing/blurring in the forward process, their purposes and definitions differ. GDM defines the noise source as Laplacian smoothing since nodes, the data instances in the graph domain, have dependencies, necessitating the incorporation of this characteristic as a noise source. Therefore, we define GDM's noise source as Laplacian smoothing, resulting in an oversmoothed final state graph. In contrast, IHDM incorporates the multi-resolution nature of images and uses approximated Gaussian blur in continuous space.
- Difference from DiGress (Vignac et al., 2022):
While GDM is a model for graph representation learning in network-shaped graphs, DiGress is a model for molecular graph generation. In other words, GDM and DiGress target different types of graphs. Molecular graphs have node and edge categorical information, and there are multiple graphs in the dataset, making it easier to define the distribution needed for the diffusion model. However, network graphs consist of a single given graph, and sometimes the node's categorical information is incomplete or absent, making it insufficient to define the distribution of the graph.

Reviewer comment (Experiments):
The experiments are not convincing to support the authors' claim on the new GDM. It is unclear whether it learns both feature and structural level information: table 1 only showed it outperforms SEAL on DDI and underperforms on the other three tasks.

Author response:
The motivation behind GDM is to capture latent factors inherent in network graph representation learning tasks through the intuition of diffusion models, thereby learning integrated graph representations that encompass the feature- and structure-related latent information necessary for a given task. To validate this, we conducted experiments focusing on the Link Prediction task. This is because Link Prediction requires learning not only node embeddings but also structural information [1]. 
However, The dependencies on node feature information and graph structural information vary with the dataset. Referring to Table 1 in our work, it implies that OGB-PPA and OGB-DDI place higher importance on structural information and feature information, respectively. OGB-Collab and Citation2 imply the necessity of both types of information for optimal prediction. 

Our model, GDM, consistently shows competitive (best or second-best) performance across all four datasets. While SEAL, a model specialized for link prediction, also demonstrates good performance, it shows poor performance in OGB-DDI, compared to GDM. Given that OGB-DDI appears to heavily rely on node embeddings, this result demonstrates that GDM is capable of capturing both latent factors that are informative for solving a given task, unlike other models specialized to specific factors. In other words, GDM’s competitive performance across the four datasets indicates its ability to effectively extract and learn integrated representation about the necessary structure and features for link prediction within a given graph. It is important to note that we did not incorporate task-specific biases into the model. Therefore, we empirically validated GDM has addressed its motivation, demonstrating its ability to capture essential underlying latent factors in a given graph required to handle the link prediction task.

Reviewer comment (Experiments):
The authors need more experiments/analysis to support the claim that their model can learn both features/structural information well.

Author response:
To demonstrate that our model effectively learns both feature and structural information in network graphs, we conducted link prediction experiments, which are known to require not only node embeddings but also graph structural information. In these experiments, our model shows highly competitive performance, verifying its ability to capture both aspects effectively. Additionally, to demonstrate its ability to learn node embedding, we conducted semi-supervised node classification tasks. GDM outperforms on all datasets. Furthermore,  through the ablation study, we analyzed the validity of whether GDM effectively captures feature and structure latent factors by feature process and structural process.

Reviewer comment (Theory):
It would be more helpful if the authors can explore a bit more on the spectral meanings of Laplacian smoothing aside from information dissipation. The authors did mention it decays the high-frequency components in the spectral domain; can this be expanded further? Do we gain additional insights from using Laplacian smoothing?

Author response:
Yes, we would like to provide the explanation of the insight into the decay of high frequency.

 As high frequency gradually decays, the difference between signals will also gradually diminish. In the spatial domain of a graph, it is interpreted as a loss of information regarding the distinct features among nodes. This implies that the amount of lost signal or information varies for each node at each time step, suggesting that our model GDM can learn the latent factors of a given graph by recovering this dissipated signal or information. This aligns with the philosophy and characteristics of diffusion models.
Additionally, in the real world, there can be noise or missing information (e.g., missing links) in the features or adjacency of a network graph. In other words, observations may not constitute a perfect ground truth. In real-world scenarios, graph representation learning involves learning from a noisy observed graph to approach a more optimal graph representation. From the resolution perspective, our insight is analogous to utilizing a coarse-to-fine strategy to enhance image resolution quality.

Reviewer comment (Writing):
In the abstract, *...model leverages Laplacian smoothing and subgraph sampling as a noise source.* What does subgraph sampling mean here? Edge removal?

Author response:
Sorry for the confusion. As the same point has been mentioned in Question (3), we respond with the same explanation in Question (3).
We will clarify the term in the revised manuscript.

Reviewer comment (Writing):
In the abstract, *...Graph dissipation model that captures latent factors for any given downstream task.* This statement needs to be toned down.

Author response:
Since this point aligns with Weakness (1), we can tell our statement is without exaggeration. We raise the motivation of universally applicable model that learns network graph representation while naturally considering specifics of a given task without injecting task-oriented biases. To address the motivation, we proposed GDM and demonstrated effectiveness in major graph representation learning tasks: link prediction task and semi-supervised node classification task. 

However, we will rephrase "for any given downstream task" into "for any given graph representation learning task." to distinguish it from graph generation tasks which is a very different research area.

Reviewer comment (Writing):
Some parts of the paper are overly verbose; for example, is Corollary 3.1 truly needed?

Author response:
Since understanding diffusion models and over-smoothing requires background knowledge, we decided to provide detailed explanations for a more straightforward understanding of intuition behind GDM, which is inseparably related to diffusion models and relation between Laplacian smoothing and over-smoothing.

Reviewer comment (Writing):
There are typos, for example on pg5 *graph-strudtured* and on pg8 *iffusion*.

Author response:
Thanks to the reviewer, we will fix typos in the revised manuscript.