Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The submission proposed a digraph data augmentation method called Laplacian perturbation to conduct digraph contrastive learning. Moreover, the authors gave theoretically analyze how it provides contrastive information without changing the digraph structure. In addition, the authors train it using multi-task curriculum learning to progressively learn from multiple easy-to-hard contrastive views. Basically, the authors blend many different existing techniques to address the digraph CL. Although the work is novel to some extent, the most novel part actually does not report the major performance gain. The major gain comes from curriculum learning which however lacks novelty. I thus think the work to be a borderline one.
main_review: Pros:
1). The work is well presented and easy to follow.

2). Many theoretical analyses guarantee correctness.

3). Experiments are thorough.

Cons:

1). Basically, the novel part is to introduce Laplacian perturbation to generate augmentation for contrastive learning with a theoretical guarantee. However, from Table 1 ablation results, it is clear Laplacian perturbation parts actually are not comparable to many baseline approaches. The major performance gain comes from curriculum learning which however is not a very novel technique. 

2). The authors blend many techniques in the work. This makes the contribution not very clear. I would suggest the authors isolate the contributions.


limitations_and_societal_impact: There is no limitations on societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper mainly focus on handling some fundamental problems arising in digraph contrastive learning. First, it introduces a concept named Laplacian perturbation and analyze how it provides contrastive information without changing the digraph structure. Then, it presents the multi-view digraph contrastive learning framework, which learns from all possible contrastive views generated by Laplacian perturbation. And finally, it trains the model using multi-task curriculum learning to progressively learn from multiple easy-to-hard contrastive views. 
main_review: First, Laplacian perturbation can also be achieved by adjusting the teleport probability alpha, which in the random walk strategy means probability return to the beginning node from current position. However, it means that we manually add an edge from current node to the starting node when alpha doesn’t equal to 0 so that we have a chance to return back. And in this perspective, Laplacian perturbation also equals to change the digraph structure. 

Second, since Laplacian perturbation also changes the digraph structure like other perturbation strategies do, the reviewer does not agree with the opinion that DiGCL provides more contrastive views than other existing models. Theoretically, by using different perturbation strategy and scale, one can get different views as many as one wants. So DiGCL doesn’t provides more views than other models although it does offer many contrastive views. 

Third, pacing function takes effect by automatically adjusting the difficulty scores along with training proceeding, while pacing function relies heavily on the pre-defined parameters such as starting and ending difficulty scores, step size and even the type of pacing functions. So, it is hard to agree with the opinion that DiGCL doesn’t rely on the pre-defined parameters. In my opinion, DiGCL depends heavily on the pre-defined parameters like other models do.

Finally, I did not find much relationship between DiGCL with directed graphs. That is, it is not specifically designed for directed networks.

There are some typos such as the link to Table 2, page 6th and some prove mistakes such as equation 5.

limitations_and_societal_impact: no
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper designs a contrastive learning framework for directed graphs (Digraph). It tackles two problems in the domain: 
- previous method to generate contrastive views (e.g., dropping edge/node) can mislead the graph topology. 
- there is a limitation to generate thorough contrastive views. 

To address the above two issues, this paper improves previous work [1] that uses digraph Laplacian to create the contrastive learning objective with 3 extensions:
- Different from [1], using a single pre-defined teleport probability \alpha to generate contrastive views, the paper propose a multi-view objective, trying to optimize over every pair of possible teleport probabilities (\alpha_1, \alpha_2,  \alpha_1 \neq \alpha2). 
- To speed up the learning, the paper fixed \alpha_1 to be 0 and consider a sequence of \alpha_2 to generate contrastive views. This reduce the complex problem in a to a simple multi-task learning problem
-. Propose an importance function for each task

Superior performance for node classification tasks in digraphs. 

[1] Z. Tong etc, "Digraph Inception Convolutional Networks", NeurIPS 2020

main_review: Pros

1. Clean idea and easy. Neat problem definition and step-by-step description of the intuition and model presentation. The key innovation is constructing all pairs of construct views is superior than a single view with pre-defined parameter. 
2. The multi-task curriculum learning formula is well motivated. It is a descent model design reducing the complex intractable expectation problem to a simple multi-task learning problem. 
3. Good experimental results and solid result analysis. 
4. Good experiments description and open-sourced data and code. Readers should be able to reproduce the results. 

Cons:

1. Novelty and significance is kind of limited. First of all, I appreciate the idea and the work presented in the paper, but the contribution for the large machine learning community can be incremental. Digraph Laplacian has been proposed in [1]  and reduce multi-view contrastive learning as a multi-task problem has been explored as well. A combination of the two is kind of natural and straight-forward. 
2. It is not clear how does the approach work for edge level problem, e.g, link prediction.  The paper presents results on node classification only. It would be more promising if performance on link prediction results are provided. 

limitations_and_societal_impact: None
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a novel directed graph contrastive learning algorithms with a novel data augmentation method guaranteed by theoretical analysis and a multi-view contrastive objective without limitation on the number of graph views.
main_review: [Summary]
This paper presents a very interesting digraph contrastive learning algorithm with a novel Laplacian-based graph augmentation scheme and a novel multi-view contrastive objective that is adaptive trained using curriculum learning. The whole framework is easy to follow and technically sound. Extensive experiments on public benchmark datasets demonstrate its effectiveness.

[Strengths]
* Considering that there has less attention for bigraph contrastive learning, the proposed method is technically novel.
* A Laplacian-based data augmentation scheme is introduced for graph contrastive learning with solid theoretical analysis.
* A easy-to-hard curriculum learning scheme is proposed for multiview contrastive objectives that can be used to contrast unlimited number of graph views.

[Weaknesses]
* As the authors have noted in Section 2.2, computing the Laplacian perturbation is relatively complex. I wonder how the proposed method scales to large-scale datasets. Experiments on real-world large-scale datasets are needed.
* Since I know little about curriculum learning, I wonder how the proposed easy to hard scheme affects the model performance. Also, the authors may consider comparing it with the AugMin scheme (What Makes for Good Views for Contrastive Learning? By Tian et al.) that selects to contrasts between two most distinct views for a more convincing evaluation.
* A further inspection of the Laplacian perturbation may further strengthen the argument. A qualitative analysis of certain positional characteristics of the perturbed bigraphs may be interesting, as the Laplacian matrix connects to the spectrum of the graph.
limitations_and_societal_impact: I have no further concerns of this work.
review_title: null

Meta-review: This paper proposes for digraphs a contrastive learning pipeline, which consists of Laplacian permutation and a multi-view digraph contrastive learning framework that leverages multi-task curriculum learning. The proposed Laplacian permutation is a data augmentation method that is proved theoretically to provide contrastive information without changing the digraph structure and comes with a fast algorithm for computation. While there are initial concerns on lack of novelty and the importance of maintaining the digraph structure, the authors have provided detailed responses, which have included additional experiments on link predictions and appropriate ablation studies,  to satisfactorily address these concerns. After reading the authors' rebuttal, there is a general consensus among the reviewers to accept this paper. The authors are encouraged to carefully incorporate into the revision the new information provided in the rebuttal, which will help enhance the clarity and strengthen the contributions. 
Decision: Accept (Poster)