Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a simple subgraph modelling framework by adding a subgraph-level position encoding to node (whether a node belongs to subgraph) to node feature. In this way, the GNN is able to differentiate nodes from different subgraphs, and outperform previous model SubGNN. 
main_review: This paper argues that a subgraph-level positional encoding is enough for subgraph representation learning. It gives some theoretical analysis that such trick could fulfill the six properties that subGNN proposed. It also introduces three self-supervised learning task to guide GNN learning.

I think the approach is reasonable and the analysis is sound and convincing. The main concern is:

1) This paper adds the self-supervised learning into the framework, and compare with subGNN without such SSL learning. I think this comparison is not fair. On the one hand, I think SSL learning should be agnostic to GNN model. I think you should compare your model with other GNN model in the same setting, i.e., with SSL training or without.
2) The motivation and insights of the SSL learning task is less than the subgraph encoding. I think the authors could consider either remove this part, or at least compare with other SSL learning methods, such as graph contrastive learning or generative learning.
3) The idea of adding structural encoding is also explored in graph transformer (Do Transformers Really Perform Bad for Graph Representation?)
I recommend the authors also compare this method into subgraph-level modelling.


summary_of_the_review: The method is simple and effective, but I doubt the current evaluation setup as it uses additional self-supervision to train the model against baselines.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on predicting the properties of subgraphs in the whole graph. The authors propose a labeling trick to help GNN distinguish nodes inside and outside the subgraph. The authors rigorously analyze the effectiveness of the proposed method. Experiments demonstrate that the proposed method achieves state-of-the-art performance on several benchmarks.
main_review: Strengths:
1) This work is well-motivated. The authors claim that distinguishing inside and outside nodes of subgraphs is crucial for subgraph tasks. Based on the insight, the authors propose a novel method for node-level message passing GNNs to perform subgraph representation learning effectively and efficiently.
2) The authors theoretically analyze the effectiveness of the max-zero-one labeling trick and the expressiveness power of the proposed method.
3) Experiments demonstrate that the proposed model with labeling tricks outperforms state-of-the-art subgraph representation learning methods on several synthetic and real-world datasets. Experiments on running time also show that the proposed model is more computationally efficient than the subgraph-level learning method SubGNN.

Weakness:
The authors may want to improve the presentation of this paper. For example, the introduction and analysis of SubGNN take up too much space that the illustration of the proposed method and experimental settings are not sufficiently detailed in the main text. 

summary_of_the_review: The authors propose a labeling trick to enhance plain GNNs for subgraph tasks, and the theoretical and empirical results demonstrate its effectiveness and efficiency. However, the authors may want to detail the model structure and experimental settings in the main text.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose GLASS, a GNN designed for subgraph tasks using labelling tricks. A simple zero-one trick can be used to differentiate nodes in and out of a target subgraphs. However, this would cause efficiency problem as different target subgraphs would require different labels. To fit the classic batch training, a batch of target subgraphs are used to label the nodes together. To resolve conflicts among subgraphs, a zero-max-one trick is proposed. GLASS is easier / intuitive to implement and more scalable. The zero-one trick version is also claimed to have more expressive power than plain GNNs and can capture the six properties in the state-of-the-art method SubGNN. Experiments have been conducted on both real-world and synthetic datasets with generally promising results.
main_review: Strength:

1. The paper addresses the overlooked problem of subgraph-level tasks, as most previous GNN models focus on node or graph-level tasks.

2. The proposed approach is simple and intuitive with little overhead on top of plain GNNs. It can be easily implemented and can be trained efficiently.

3. There is a good balance of theoretical and empirical results.

Weakness:

1. The novelty is somewhat incremental. Node labeling trick has been used on GNNs, although not specifically on the subgraph level. There does not seem to have significant challenges (or rather, it is quite straightforward) in extending plain GNNs to the zero-one / max-zero-one strategies.

2. The main theoretical results only apply to the zero-one strategy. For the max-zero-one strategy, "it can be achieved if the subgraphs are sparsely located in the graph"--- which is dataset dependent. Furthermore, many real-world graphs have a small diameter, meaning that two random subgraph can easily overlapped k-hop neighborhood even when k is a small number.

3. Proposition 2 states that GLASS can cover the six properties of SubGNN. In this case, what is the advantage of GLASS over SubGNN? Any additional property that SubGNN does not have? In the experiments, it is quite clear that GLASS is more efficient, but GLASS also holds a significant accuracy edge. I wonder if this accuracy advantage actually comes from the self-supervised loss, rather than the labeling trick? In particular, in the ablation study, even after removing the labeling trick, the results are still better than SubGNN on the real datasets. In other words, if SubGNN is also trained with the same self-supervised loss, would its performance become much better?

Minor issues:

4. Figure 2 is a bit hard to read, especially for S and S', since the green shapes intersect with many edges. Maybe using two different colors to highlight the nodes/edges, instead of using another shape to enclose it.

5. It is mentioned that to solve label inconsistency in a batch, concatenation cannot be used as it leads to variable lengths of
node labels. I don't quite follow this point. If each batch contain the same number of subgraphs, wouldn't all label vectors have the same length?

--- update after rebuttal ---
Some of the response is valid, especially regarding the self-supervised loss. Hence I'm upgrading my rating.
summary_of_the_review: My main reason is incremental novelty (W1), and insufficient support for certain claims (W2/W3).
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a simple GNN approach to predict labels of subgraphs. In experiments, it works better than an existing approach. Complementary they show their approach can predict characteristics of a graph such as cut ratio. Additionally, they augment the loss with self-supervised losses of predicting node, edge, and subgraph level information. 

main_review: The authors study the problem of subgraph classification and propose a simple adaptation of existing GNNs for this task. In particular, their approach augments the initial representation of the node features with label information if a node belongs to a subgraph in a batch.

The paper is clearly written, provides supportive illustrations, and closes theoretical gaps of preceding works. 

Authors can improve the paper in the following ways. 

1. Provide an ablation study on the self-supervised losses. How much does it affect performance? What if you employ the same losses for other approaches? 

2. From the last figure in the appendix it's clear that increasing the batch generally degrades performance, hence the batch size of 1 is the most promising one. What batch size you used in experiments? How much is it a problem from training time perspective? Your approach can't clearly have a very large batch size as it would include all nodes of the graph in the subgraph labels and hence there would be no difference between your approach and plain GNNs. So what's the largest batch size you can afford? What's the trade-off between batch size, time, and performance?

3. Can you please add graph-agnostic MLP and GBDT approaches as baselines for the experiments? They often show superior performance than GNNs (see [here](https://openreview.net/forum?id=ebS5NUfoMKL) for example). 

4. One of the biggest concerns, however, is not with the paper, but with a studied problem. While authors give an example where predicting subgraph labels can be useful, I don't believe GNNs should be explored there (it's unlikely that the performance of the department is evaluated based on the graph structure rather on their deliverables). The same is true for the employed datasets, even though they are inherited from the previous work. In particular, ppi-bp, hpo-metab, hpo-neuro ask to predict a label for a subset of proteins, which are obtained from the clinical trials, which are expensive to run and utilize information beyond of what's available in the graphs. Besides the subgraph number of nodes in these datasets is about 10, which are very small graphs to evaluate. em-user is even worse: predicting a gender based on workout history graph is a very contrived dataset and thus it's not very insightful. I understand that it's not the authors who introduced the problem and datasets; however, making it clear why this problem is important and why it cannot be solved by other methods better would significantly strengthen the paper. Maybe you can find some relevant applications in the subgraph regression problem, where your approach could be seamlessly adopted. 

I am likely to update my score based on the authors' responses.
summary_of_the_review: This is a well-written paper, that could be improved by providing more motivation and exploring the performance in more depth by using other baselines and providing ablation studies.
review_title: null

Meta-review: This paper proposes a labeling trick for subgraph representation learning with GNNs. The proposed method, GLASS, improves on subgraph-level tasks. The topic of subgraph representation learning is relatively new, and this paper makes progress in that community which would be appreciated by other researchers interested in the same problem. 

The paper in the original submission state raised some concerns from the reviewers about unclear writing of the motivation and potential applications, technical novelty, and comparisons with existing approaches (even one that are not specifically designed for subgraph representation learning). It is good that the authors conducted additional experiments to show the effect of SSL (that the approach makes improvements without SSL). This and other clarifications from the authors convinced the reviewers to recommend acceptance.
Decision: Accept (Poster)