Title: Improving Generalization of Dynamic Graph Learning via Environment Prompt
Venue: NeurIPS 2024
Accepted: yes
Keywords: Dynamic graph

spatio-temporal graph learning

out-of-distribution generalization

causal theory

prompt learning

subgraph learning.
Abstract: Out-of-distribution (OOD) generalization issue is a well-known challenge within deep learning tasks. In dynamic graphs, the change of temporal environments is regarded as the main cause of data distribution shift. While numerous OOD studies focusing on environment factors have achieved remarkable performance, they still fail to systematically solve the two issue of environment inference and utilization. In this work, we propose a novel dynamic graph learning model named EpoD based on prompt learning and structural causal model to comprehensively enhance both environment inference and utilization. Inspired by the superior performance of prompt learning in understanding underlying semantic and causal associations, we first design a self-prompted  learning  mechanism to infer unseen environment factors. We then rethink the role of environment variable within spatio-temporal causal structure model, and introduce a novel causal pathway where dynamic subgraphs serve as mediating variables. The extracted dynamic subgraph can effectively capture the data distribution shift  by incorporating the inferred environment variables into the node-wise dependencies. Theoretical discussions and intuitive analysis support the generalizability and interpretability of EpoD. Extensive experiments on seven real-world datasets across domains showcase the superiority of EpoD against baselines, and toy example experiments further verify the powerful interpretability and rationality of our EpoD.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work investigates the issue of spatio-temporal data distribution shift, which is a long-standing challenge in dynamic graph learning. First, the authors systematically analyze the limitations of existing works over OoD  challenge, and then propose a comprehensive solution to address their limitations.  Specifically, a self-prompted learning mechanism is proposed to extract underlying environment variables that potentially influence data distribution, and a novel causal pathway that leverages dynamic subgraphs as mediating variables is further introduced to effectively utilize the inferred environment embedding.  Comprehensive experiments on seven real-world datasets demonstrate the superiority of proposed EpoD.\n\nSTRENGTHS:\n- The motivation of this work is clearly described and convincing. This work proposes a novel framework to systematically tackle the  environment inference and utilization problem via profound understanding of the environment perception and limitations in existing OoD literatures.\n\n- This work proposes some interesting and pioneering learning components for dynamic graphs, including self-prompted learning and dynamic subgraph learning. \n\n- Sufficient experimental studies support the insights and framework of the paper. A toy dataset  verifies that the model can capture interpretable causal associations.\n\nWEAKNESSES:\n- It is widely agreed that unobservable environmental factors are the primary cause of shifts in data distribution. How does the self-prompted design ensure that prompt answers can capture unobserved environmental factors, without overlapping with the observable information?\n\n- There are some  minor typos, e.g.,  in Line105,  more consistent with -> is more consistent with.\n\nQUESTIONS:\n-  How can we understand the sentence \"Historical observations $X$ can be divided into the accessible environment features $X_X$ and observed labels with evolution patterns $Y_X$\" in Line 90?\n\n- This work presents a winding causal path to guide the utilization of environment variables. Is this causal path specific to dynamic graphs, can it be extended to other types of data? \n\n- It is intuitively meaningful to use the asymmetric quantization principle to extract the node-centered subgraph. Is there more illustrative exampls to depict such asymmetry?\n\nLIMITATIONS:\nThis manuscript contains sections \"Broader impacts\" and \"Limitations and Future Works\". The authors have described the impact of this work, and discuss its limitations in those sections.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a novel dynamic graph learning framework EpoD to tackle the temporal distribution shift issue by exploiting prompt learning. EpoD addresses two limitations of existing works regarding inference and exploitation of unseen environments. The EpoD includes two modules, i.e., self-prompted environment inference component and dynamic subgraph learning component, where the former extracts underlying environment variables that potentially influence data distribution, while the latter effectively and interpretably exploits the inferred environment embedding for dynamic subgraph generation. The experiments design several distribution scenarios for evaluation and empirically demonstrate impressive performance under distribution shift scenarios.\n\nSTRENGTHS:\n**S1.**  Well-written and clearly organized. This work decouples the OOD issue into environment inference and utilization challenges, addressing each separately. Moreover, empirical examples and theoretical analysis enhance the accessibility of the paper for readers.\n\n**S2.** Qualified technical contributions. EpoD addresses two limitations of existing works regarding inference and exploitation of unseen environments with two well-designed modules, environment prompting and dynamic subgraph learning, which innovatively leverage successful experiences from LLMs to address the OOD generalization issue in dynamic graph learning. \n\n**S3.** Good experiment designs and comprehensive evaluations. The authors especially design several distribution scenarios for OOD task evaluation with consideration of the COVID-19 period.  Seven cross-domain real-world dynamic graph datasets are selected to evaluate the performance of EpoD.\n\nWEAKNESSES:\n**W1.** More detailed explanations and analysis will be more welcome and help readers better understand this manuscript. What is the unique superiority of the self-prompted learning mechanism in inferring unseen environments in spatio-temporal graph data? \n\n**W2.** It seems that self-prompt is borrowed from LLMs, thus what is the intuitive designing idea of self-prompt learning here, and how it works for environment inference?\n\n**W3.** Any other considerations besides efficiency when designing node-specific and time-shared prompt tokens? \n\n**W4.** There are many methods to extract subgraphs, the most popular one is to drop edges to realize the partition of graph data. What do you think are the advantages of node-centered dynamic subgraphs?\n\n**W5.** Some typos: \n\n- In line 254, \"dynamics graph\" should be modified to \"dynamic graph\".\n\n- In line 385, \"we\" should be modified to \"our\".\n\nQUESTIONS:\nPlease answer the questions in Weaknesses.\n\nLIMITATIONS:\nThe authors have discussed the broader social impacts and limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

comments: SUMMARY:
The paper is about dynamic graph learning, which is an interesting topic. The authors propose a novel dynamic graph learning model named EpoD based on prompt learning and structural causal model to comprehensively enhance both environment inference and utilization. The paper is well written and well organized. However, there are several concerns in the current version of the paper that addressing them will increase the quality of this paper.

STRENGTHS:
1 New perspectives on theory.

2 Good writing.

3 Fully experimented.

WEAKNESSES:
1 Regarding the OOD research of dynamic graphs, I have a question: when new nodes appear in the data (rather than the known number of nodes), how should the model construction be handled?

2 Would like to see a discussion of computational complexity, both time complexity and space complexity.

3 The details of the datasets should be included in the text (rather than in the appendix), such as the size of the datasets. In addition, it seems that the size of these datasets is limited.

QUESTIONS:
As above.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces EpoD, a novel dynamic graph learning model that leverages prompt learning and structural causal models to address out-of-distribution (OOD) generalization challenges. EpoD features a self-prompted learning mechanism for inferring environment variables and a node-centered subgraph extractor to capture data distribution shifts. Extensive experiments across various real-world datasets demonstrate EpoD's superior performance and interpretability, highlighting its potential for practical applications in fields like traffic forecasting and social network analysis.\n\nSTRENGTHS:\n1. The paper introduces a novel dynamic graph learning model named EpoD, leveraging prompt learning and structural causal models to address environment inference and utilization. The self-prompted learning mechanism and node-centered subgraph extraction represent significant advancements in the field.\n\n2. The authors provide a solid theoretical framework supporting the generalizability and interpretability of EpoD. The incorporation of structural causal models and dynamic subgraphs as mediating variables showcases a deep understanding of the underlying principles of dynamic graph learning.\n\n3. The paper presents extensive experiments across seven real-world datasets from diverse domains, demonstrating the superiority of EpoD over several baseline models. The inclusion of a toy example experiment further validates the interpretability and rationality of the proposed model.\n\n4. The results indicate that EpoD consistently outperforms existing methods in terms of both mean absolute error (MAE) and root mean square error (RMSE) for traffic flow prediction, as well as AUC scores for social link prediction tasks. The significant improvements on large-scale datasets highlight the model's robustness and scalability.\n\n5. The dynamic subgraph extraction process and the use of node-centered subgraphs enhance the interpretability of the model. The paper effectively demonstrates how dynamic subgraphs can capture the impact of environment variable shifts on data distribution.\n\n6. The model's ability to generalize across different domains and datasets suggests strong potential for practical applications in various fields, such as traffic forecasting, social network analysis, and air quality prediction.\n\nWEAKNESSES:\nTo be honest, I did not find obvious drawbacks of this work. Some minor revision might be helpful: \n\n- use the mentioned all-in-one (citation [39] in their manuscript), a more flexible graph prompt as your environment vector. Or see the impact of different graph prompt formats. \n- The discussion on future work and the limitations of the current approach could be expanded. For example, I think this work might be helpful to be applied in some sociological analysis. The environment concept might be also inspiring to the similar concept in this paper: Self-supervised Hypergraph Representation Learning for Sociological Analysis. TKDE. 2023.\n\n\nOverall, this paper presents a significant contribution to the field of dynamic graph learning, with innovative methods and strong experimental results.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nSee in the weakness", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}