Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, they carefully design and propose the GraphMixer network for temporal link prediction in dynamic graphs. With a carefully designed and ablate architecture separated into three components, the link-encoder, node-encoder, and MLP-Mixer based link classifier, they are able to achieve high performance on this task that has only previously been attained with RNN and Self-Attention based architectures. They include extensive experiments to study the different architecture choices they made and their effect on performance and generalization.
strength_and_weaknesses: Strengths:
- Proposes a novel architecture that achieves performance competitive with other networks without memory or attention mechanisms.
- Strong analysis and ablation of the different design choices made.

Weaknesses:
- The model takes significantly longer to train and there is not significant analysis of the inference cost. This lack of discussion in the main paper also makes some conclusions about faster convergence somewhat misleading.

Questions:
Why are the train average precisions for the competing methods in Figure 3 so much lower than the final results in Table 1. For example, DySAT training AP is around 60-70% while the final eval is 96.71. Are all the results from reimplementations or from taken from other papers.

It would likely be helpful to bring appendix C.3 into the paper and analyze and discuss more about the model efficiency since the model does take significantly longer to train than some baselines like TGN. If some of this is due to preprocessing, it would be helpful to separate the time required for each and/or analyze the inference FLOPs for your model compared to the others.

clarity,_quality,_novelty_and_reproducibility: The work is mostly clear and understandable. Some of the claims around model convergence is a bit misleading and the authors should consider bringing appendix on training speed C.3 into the main paper.
summary_of_the_review: This work proposes a novel architecture for link prediction in temporal graph networks that achieves very good performance without RNN or self-attention mechanisms. They achieve this using a carefully constructed network with MLP-mixer. They do detailed experiments to analyze and ablate the different design choices they made. A large strength of this paper is the extensive experiments ablating the details of the design choices they made how it affects performance, generalization, and training stability. This greatly helps the community understand how these architecture choices affected their model and may extend to other models. The paper may have somewhat limited novelty since the different parts of the design have all been explored elsewhere and in terms of setting a new SOTA, the model still has some under-analyzed weaknesses such as the slow training speed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors propose GraphMixer, a conceptually and technically simple architecture that consists of three components: 1 a link-encoder that is only based on multi-layer perceptions (MLP) to summarize the information from temporal links, 2 a node-encoder that
is only based on neighbor mean-pooling to summarize node information, and 3 an MLP-based link classifier that performs link prediction based on the outputs of the encoders. The authors show that GraphMixer attains an outstanding performance on temporal link prediction benchmarks with faster convergence and better generalization performance.
strength_and_weaknesses: Pros: Focused on the temporal graph learning, the authors propose a conceptually and technically simple architecture GraphMixer for temporal link prediction. GraphMixer not only outperforms all baselines but also enjoys a faster convergence speed and better generalization ability. The author conduct empirical study to identify three key factors that contribute to the success of GraphMixer and highlights the importance of simpler neural architecture and input data structure.

Cons: There is not obvious weakness in the draft.
clarity,_quality,_novelty_and_reproducibility: The paper is clear, novel, and easy to reproduce.
summary_of_the_review: Please refer to the above points.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a simple architecture named GraphMixer for temporal graph learning. The GraphMixer is based on simple MLP structure and achieves state-of-the-art performances on temporal link prediction benchmarks.
strength_and_weaknesses: First of all, I'm not the expert in the graph learning area and my research focus is computer vision. From my side, I would to say: A simple method along with state-of-the-art performances satisfies me. 

When I read this paper, I'm satisfied by the simple method, the detailed experiments. Thus I would like to give an accept rating.

However, there is still a little bit concerns:

1. I think the major novelty lies in the time-encoding part, also the proposed GraphMixer performs extremely better on LastFM dataset (c.f., Table 2). Could the author give explanations of why GraphMixer work well on LastFM? Since I only observe marginal improvements on other datasets.
clarity,_quality,_novelty_and_reproducibility: Though I am not the expert in graph learning area, I enjoyed reading this paper. I would like to give this paper an borderline accept rating. I think a simple but effective method satisfied me.
summary_of_the_review: I think this paper is a simple and effective paper, thus I want to give it an accept rating.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: I Summary:

- I.1 Investigated Problem:
The paper investigates if memory and self-attention mechanisms are indispensable for temporal graph learning. Motivated by the fact that it is non-trivial to understand which parts of the self-attention and memory-based models contribute to its success and whether these components are indispensable, the paper identifies 3 key factors to a simpler and elegant solution named GraphMixer.

- I.2 Proposed Solution: GraphMixer is conceptually and technically a simpler architecture for temporal link prediction that achieves:
   - Better generalization compared to all tested baselines;
   - Faster convergence speed.

- I.3 Validity Proof of the Proposed Solution:
Illustrative and intuitive visualization as well as an empirical piece of evidence is provided to support the validity of the proposed solution.
Moreover, an extensive study identifying the following 3 key factors contributing to the success of GraphMixer is conducted: 
    - The use of a fixed time-encoding function: 
        - The authors demonstrated that a trainable time-encoding function could cause instability during training which leads to a non-smooth landscape objective;
    - The use of MLP-Mixer in GraphMixer’s Link Encoder is a good alternative to self-attention and its simplicity leads to a better generalization performance;
    - Using a simple input structure that is better aligned with their labels allows a simple neural network model to capture the underlying mapping between the input and labels which leads to a better generalization.

II Strengths:

- II.1 From a structural point of view:
    - The authors formulated the problem investigated in terms of questions and presented the content of the paper as answers to these questions;
    - Used circled numbers and bullet points in the presentation of the proposed solution as well as question and answer type of analysis for the experiments. This is an example of a very well-written and structured paper showcasing the simplicity and effectiveness of a novel method.

II.2 From an analytical point of view: 
 - The clarity of the motivation: Presented in terms of questions;
 - The presentation of the designed method: Simple and elegant presentation for a simple and elegant solution;
 - The visual and empirical evidence provided and the analysis performed to identify the key factors to the performance of the method provide good intuition about the effectiveness of the proposed solution more intuitive to understand;
 - Comparisons are conducted with several features of the existing methods. In fact, it has been shown that the presented solution is comparable to or better than most baselines;
 - The transparency aspect of the submission as open-source code is provided for reproducibility purposes and the authors are planning to clean up and release the code for all baseline implementations to gather important information for model analysis and plotting.

- II.3 From a perspective of soundness (development, unity, and coherence) and completeness (correctness): 
  - The strength points mentioned above are sufficient evidence of the soundness and completeness of the paper.  

III Addressing what can be thought of as weaknesses:

As has been mentioned by most reviewers, there isn’t an obvious weakness to mention. Most of what could be thought of as weaknesses have been addressed and clarified by the authors during the rebuttal period. Unanimously, the reviewers agree on the acceptance of the submission.

IV. Potential of the paper:

- IV.1 From a Potential perspective (Potential of the paper to the community): The proposed solution has great potential to be of benefit to the whole community, As mentioned by the authors in the conclusion, not limited to temporal graph learning and to name a few, an interesting direction would be to design algorithms that could automatically select the best input data and data pre-processing strategies for different downstream tasks.
justification_for_why_not_lower_score: The authors present a simple and elegant solution that achieves state-of-the-art performance on temporal graph learning-related tasks. The proposed solution has the potential to open the door to research for simpler, more interpretable, yet effective solutions.
Decision: Accept: notable-top-5%