Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes a new approach for representation learning on temporal graphs (or CTDGs). The proposal can be viewed as an extension of the walk sampling scheme from CAW [1], modified to also count on node degree and time traversal information. The paper also employs a NeuralODE-based approach aiming to better leverage fine-grained temporal information. Results on three benchmark datasets show the effectiveness of the proposal.

strengths_and_weaknesses: Overall, the paper is easy to follow despite some typos and minor technical errors. I like the simplicity of the proposed method and the fact it leads to promising results. Also, the paper discusses a relevant topic in AI/ML. However, I would rank novelty as limited: modification of existing approaches (sampling strategy of CAW) combined with previous strategies (ODE-based solvers, contrastive loss) and evaluated using a standard setup. Also, no theoretical insights or claims are established in the paper. The proposal seems to incur a computation burden on a model that is already slow (CAW, [1]). Last, the paper only considers three benchmarks as opposed to six in [1]. 

Strengths:
- Promising results
- Simplicity

Weaknesses: 
- Lack of a more principled motivation behind the proposed design choices
- Limited evaluation setup
- Limited technical novelty


questions: - In the Introduction, the paper says “[..] studies in dynamic graph modelling extract expressive patterns manually […]” and cites [1]. Although [1] uses triadic closure as a motivation, the design in [1] can capture a large collection of structures/motifs. Why is this regarded as manual motif extraction?
- “[..] time encoding hurts model performance because temporal dependencies are modelled implicitly”. What is the basis for this strong claim?
- In Definition 1: $i \in \mathbb{N}^+$ means that $i$ lies in the natural numbers without zero, which conflicts with $0 \leq i$. Also, by definition of $G_{\cdot, t}$, no pair $(\cdot, t_i)$ can belong to ${G}_{\cdot, t_i}$. This definition deserves some polishing. 
- What can continuous evolution bring in terms of expressive power that sequence-based models (e.g., RNNs) cannot?
- The chosen datasets are small. It would be helpful to report the performance of NeurTW on the Reddit and MOOC datasets.
- Is the proposed model able to deal with (node-level and edge-level) deletion events? How would so?
- Can the model be applied to node-level prediction tasks? How could we obtain node embeddings using NeurTW?
- In the ablation study Ab.4, what is done to replace the continuous evolution procedure? Is the temporal information completely disregarded? 
- It would be helpful to report a time comparison between NeurTW vs. TGN and CAW. 

**Typos:** 
- Line 165: "a nodes u"
- Line 162: "to allows"
- Line 126: "Specifically, Given"
- Line 187: "with the length $l$"
- Line 234: "We details"

[1] CAW: https://arxiv.org/pdf/2101.05974.pdf 
limitations: The authors do not discuss the limitations of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a method based on the concepts of temporal walks and neural ordinary differential equations to learn effective node representations on continuous-time dynamic graphs. The main contributions of this work are in three aspects. Firstly, it proposes a more effective way to sample temporal walks from dynamic graphs. Secondly, it learns continuous latent dynamics of temporal walks, showing significant performance gains (Tab. 3). Thirdly, it borrows the idea of contrastive learning to optimize the dynamic graph learning problems. Compared with the existing methods, especially for its predecessor CAWs, the proposed method shows significantly better performances on downstream tasks.
strengths_and_weaknesses: Pros:

+	In general, this paper is well-presented. The learning problem on dynamic graphs is clearly formulated, and the technical writing in Sec. 4 is easy to follow. The overall architecture design makes sense to me, where the contributions and improvements over CAWs can be easily distinguished.

+	The experimental design is straightforward and comprehensive. Firstly, the effectiveness of the proposed method can be proved compared with state-of-the-art baselines. Secondly, necessary ablation and parameter studies are also clearly presented in the paper's main text. Thirdly, I can see supplementary experiments in the appendices, including the analysis of important designs and the visualization results, which help confirm the effectiveness of the proposed method.

Cons:

+	The readability can be further improved. It requires the readers to have some background knowledge on anonymous and temporal walks to fully understand the proposed method.

+	Some description is unclear. For instance, how is the spatial information handled in the proposed neural temporal walk encoding method?
questions: 1.	How is the spatial information handled in the proposed neural temporal walk encoding method?

2.	Why have the authors not experimented on the widely adopted Wikipedia and Reddit datasets used in TGAT, TGN, and CAWs?

3.	What are the connections and comparisons between the random walk-based methods (e.g., NeurTWs) and message passing-based methods (e.g., TGAT and TGN)? In other words, what is the significance between NeurTWs and those local aggregation methods?

limitations: Yes. The authors have addressed the limitations of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a method based on random walks and neural ordinary differential equations to capture and encode irregularly-sampled walks consisting of temporal nodes. The authors also provide in-depth analysis to reveal the connections between temporal graph motifs and temporal walks and why it is essential to capture them to learn temporal graph representations. Compared with previous work, this paper demonstrates good performance in both attributive and non-attributive scenarios.
strengths_and_weaknesses: Pros: 

The paper is overall well-written, and the motivations are justified clearly. In general, the improvements over existing work on dynamic graph learning are significant based on the experimental results.

- Representation learning on continuous-time dynamic graphs is an important but less explored task. This paper presented an effective method for solving this problem.

- Overall, the novelty of the proposed method is somewhat significant compared with existing random walk-based methods. Notably, capturing the information of time intervals with NODE demonstrates significant improvements over the widely adopted time encoding approach, which may open a new possibility in modeling continuous-time dynamic graphs.

Cons:

- Regarding the walk anonymization to establish the structural connections between walks, there is no significant breakthrough compared with CAW.

- The complexity analysis is prone to error, e.g., the time complexity of walk anonymization has not been counted when calculating the total time complexity of the proposed method.

- Optimizing Eq. 8 may be problematic. In real-world dynamic graphs, the time intervals may be extremely large (in seconds), which leads to large integral intervals when solving the ODE.

questions: - How the proposed method deals with large time intervals compared with the time encoding approach? 

- How to deal with the case where the number of the spanned neighborhood is large? Based on the complexity analysis in the appendices, the time complexity of the proposed sampling algorithm is proportion to $k_s$, which makes the proposed method hard to scale on dynamic graphs with higher average temporal node degree.

limitations: - The discussions of how the proposed method can learn on dynamic graphs with large average time intervals and higher temporal node degrees are necessary.

- The details of paper writing can be further improved, e.g., u->b in Fig. 3 can also be a temporal-biased walk since the timestamp of node b is the same as node f.
review_title: null

Meta-review: metareview: The paper proposes a method based on the concepts of temporal walks and neural ordinary differential equations to learn effective node representations on continuous-time dynamic graphs. All the reviewers are positive about the paper and the rebuttal/discussion helped clearing out the concerns they had.
confidence: Certain
Decision: Accept