Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors deal with outlier detection on graphs.
They start with the paper on InfoGraph "INFOGRAPH: UNSUPERVISED AND SEMI-SUPERVISED
GRAPH-LEVEL REPRESENTATION LEARNING VIA MUTUAL INFORMATION MAXIMIZATION" by Sun et al , combined with SVDD, and propose two modifications.

1. DOHSC adds on top of infograph+SVDD a singular vector standardization and a projection on the first dimensions corresponding to the largest k' = 8 directions on the right side of the SVD (equation 9) . (see notes in Strength and weaknesses, that the code does actually something different)

2. DO2HSC replaces the SVDD objective by a two-sided objective which aims to identify outliers by samples which have a too large or too small distances from the center.

This comes with a 2 sided loss (penalizing too large and too small distances from the center), and calculates a distance measure which is positive if the sample distance from the center is either too large or too small.


strength_and_weaknesses: Strengths:
the paper does several experiments on graph datasets.

Weaknesses:

1. the down projection does something else than the text states,

when applying W = V_{k'} \Lambda^{-1}_{k'}  
to H= U \Lambda V^\top 
then the result is:
U_{k'} (in math) aka U[:,:d]  (in python/pytorch)

It is NOT U[:,:d] I_d V^\top[:d,:] as what would be the orthogonal projection onto the first d dimensions of a SVD decomposition.

What did the authors want to compute  ? 
 If it is U[:,:d], then the approach needs to be renamed, as that is not an orthogonal projection onto the first dimensions of U[:,:d] I_d V^\top[:d,:] .

2. It is not clear whether the singular vector standardization (by \Lambda^{-1}_{k'}) or the downprojection onto the first k' helps to improve DOHSC over infograph+SVDD. 
To understand this, an experiment would be useful where one only projects onto the top k' dimensions in the Orthogonal_Projector module, without the .matmul(torch.linalg.inv(S)[:d,:d]) in the weightConstraint class  (or just .matmul(torch.eye(d))  )

3. the outlier detection is based on class-labels only. Depending on the dataset this might cluster too easily compared to outliers in the wild. 

4. an analysis is missing why prohibiting too small distances from the center is a good idea.

clarity,_quality,_novelty_and_reproducibility: The paper readability is okay. 

As for the quality, there is a certain issue with weakness 1. 
As for experiments see weakness 3. 
There is no analysis why the two proposed changes are helpful. 
These factors reduce the quality somewhat.

Regarding quality and reproducibility: several AUCs in the baseline experiments are substantially below 0.5. This indicates that one would obtain a very good AUC if one inverses the prediction rule for outliers. This can be due to the instability of the baselines or due to suboptimal parameters. One can choose not to hold it against the submission.

The two propositions over infograph+SVDD are novel. 
summary_of_the_review: The paper proposes two interesting changes. The orthogonal projection could be a mistaken implementation, see weakness 1, which is a point of lack of clarity and quality which has to be addressed.

*edit* after reading the rebuttal, the reviewer is agreeable to increase his rating to marginally above the accept threshold.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The work presents a new one-class classification method for graph-level anomaly detection. The new components added to the original one-class objective include a local-global graph representation learning, a SVD-based representation projection, and a bi-hypersphere learning. The first two newly added components are directly taken from existing work. The bi-hypersphere learning is new, as far as I know. The method is evaluated on six graph datasets and compared with graph kernel and GNN-based methods.
strength_and_weaknesses: Strengths of the paper are:
- The work tackles an important yet under-explored problem -- graph-level anomaly detection. Unlike node anomaly detection, graph-level anomaly detection methods, especially deep neural network-based methods, are relatively limited.
- The bi-hypersphere learning objective is an improved version of popular deep one-class classifiers, which is new, to the best of my knowledge.
- The effectiveness of the SVD-based representation projection and the bi-hypersphere learning is justified via the ablation study.

The negative aspects of the paper include:
- Although the paper is focused on graph-level anomaly detection, the key design or the newly proposed component (i.e., the bi-hypersphere learning) is generic and does not take into account of the graph-level graph mining tasks. The only component relevant to graph-level detection is the mutual information maximization between local and global representations, which is taken directly from existing work, such as Infograph. The main concern here is that the presented method is not designed specifically for graph-level anomaly detection. 
- The bi-hypersphere learning seems to be generalizable to different types of data. Results on image/tabular data would be important  to justify whether the main argument of the bi-hypersphere learning is effective. 
- There have been many variants of deep one-class classifier for learning more meaningful one-class models, such as [1-4]. They can be easily combined with loss functions like Infograph to adapt to graph-level anomaly detection. It's unclear what are the advantages of the proposed method compared to these more advanced one-class classification methods.
- The datasets used are very different from the ones in the competing methods like GLocalKD and OCGTL. Their performance seems to be less effective than the ones reported in their original paper. What are the reasons/motivations behind? GLocalKD and OCGTL seem to work well on some commonly used graph datasets, e.g., PROTEINS, AIDS, and REDDIT. How is the performance of the proposed method on those datasets?
- The argument that "In contrast, there is little work on graph data despite the fact ..." is invalid. To my knowledge, there have been many studies on anomalous node detection; less work is on graph-level anomaly detection.
- The benefit of using the mutual information maximization loss is not examined
- To my understanding, the method can be sensitive to the setting of the percentile parameter in eq. 14, but no empirical results are given about this sensitivity.

**References**
- [1] "DROCC: Deep robust one-class classification." In International Conference on Machine Learning, pp. 3711-3721. PMLR, 2020.
- [2] "Learning and evaluating representations for deep one-class classification." arXiv preprint arXiv:2011.02578 (2020).
- [3]  "Explainable deep one-class classification." arXiv preprint arXiv:2007.01760 (2020).
- [4] "Deep one-class classification via interpolated gaussian descriptor." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, pp. 383-392. 2022.
clarity,_quality,_novelty_and_reproducibility: The overall paper clarity is fairly good. One of the components is new, but the advantages of this new one-class learning design compared to recently proposed ones are unclear.
summary_of_the_review: The paper has some merits, but the cons outweigh the pros, as discussed above.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a graph anomaly detection method through mutual information maximization. The key contributions are proposing (i) an orthogonal projection layer for the decision boundary correction and (ii) a two co-centered hyperspheres structure for estimating the normal distribution. Experimental results on multiple datasets show the effectiveness of the proposed method.
strength_and_weaknesses: Strength:
1.	The proposed method is simple yet effective. It seems possible to apply the proposed orthogonal projection layer and the bi-hypersphere compression model to many other graph anomaly detection methods.
2.	The paper is well-written and easy to follow. The main contributions are clearly summarized.
3.	Experiments on multiple datasets show the effectiveness of the proposed method.
Weaknesses:
1.	The motivation for the bi-hypersphere compression is still difficult to understand. The authors claim that anomalous data may appear in the empty inner decision region. But Figure 3 did not show this phenomenon. More importantly, the authors did not explain why bi-hypersphere compression can help. For example, if the normal area becomes more compact, how can we guarantee that anomalous data will not appear in the empty inner decision region? The authors may need to provide some comparisons to show the proposed method can indeed solve this problem.
2.	Improvements on some of the datasets seem unreasonable. For example, in Table3, compared with DOHSC, DO2HSC often improves the results by less than 2% on most of the datasets. But on class 1 of ER_MD, DO2HSC has a more than 20% improvement. Analysis needs to be provided. For example, it will be very interesting to show, on ER_MD, more anomalous data will appear in the empty inner decision region so that DO2HSC has an impressive improvement on this dataset.

clarity,_quality,_novelty_and_reproducibility: This paper is well-written and very clear. Since source codes are provided in the supp, it seems possible to reproduce the results.
summary_of_the_review: This paper proposed a simple yet effective method for graph anomaly detection. Authors may consider the weakness shown above and revise the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper first proposes a novel deep graph-level anomaly detection model, which learns the graph representation with maximum mutual information between substructure features and global structure features while exploring a hypersphere anomaly decision boundary. The numerical and visualization results on a few graph datasets demonstrate the effectiveness and superiority of the methods.
strength_and_weaknesses: I am unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.
clarity,_quality,_novelty_and_reproducibility: I am unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.
summary_of_the_review: I am unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper addresses the problem of graph-level anomaly detection, where the task is to find anomalous graphs from a collection of graphs. The proposed method learns graph embedding based on GIN and mutual information maximization, followed by performing bi-hypersphere compression to emphasize discrimination of anomalous graphs from normal graphs. The performance of the proposal is empirically examined on real-world datasets.

### Strength

- The idea of bi-hypersphere compression with its combination to GNNs is interesting. I really expect further analysis in this direction.

- The quality of presentation is high. This paper is overall clearly written and easy to follow.

### Weakness

- As reviewers pointed out, the motivation and the proposed methodology do not match very well; that is, this study is motivated by graph-level anomaly detection, while the newly proposed method is not a graph-specific technique. This issue should be resolved.

- The bi-hypersphere compression needs more analysis. Although I appreciate the authors' response, currently it is not so clear why this leads to improvement of anomaly detection. Theoretical analysis would be desirable. 

- Empirical comparison seems to be unfair. The authors mention that only normal data are available for a training dataset. However, in the fully unsupervised anomaly detection, this is not possible, and one needs to directly find anomalies from a dataset that contain both normal and abnormal data without any class labels. I think graph kernel based methods work in the latter setting, and this may be one of reasons for their inferior performance compared to GNN-based methods, although graph kernels, in particular the WL kernel, are known to be often empirically competitive to GNNs in many cases.

- Comparison is also unfair with respect to parameter tuning. Since the parameter tuning is fundamentally difficult in unsupervised learning, it should be carefully considered in experiments for fair comparison. I think the empirically optimal parameters are chosen in the proposal, then this should be applied to all the comparison partners. Please clearly state how to tune parameters for all the comparison partners. In particular, in the WL kernel, different numbers of iterations are just tested and the optimal one is not chosen. So please just present the optimal performance as the WL's result, which actually reduces the space of the paper and the authors can add more methods.
In addition, although I acknowledge an additional experiment added in the authors' response about the sensitivity, it is not convincing as the most important part is missing. In this table, there is a monotonic trend that smaller $\nu$ leads to better results. Does this mean that $\nu = 0$ leads to the best result? I do not think so, and then it is important to examine when the performance drops when $\nu$ decreases.

- There are other issues in experiments raised by the reviewers, e.g., lack of comparison with recent methods.
justification_for_why_not_higher_score: The weaknesses with respect to empirical comparison are crucial and should be addressed. Together with the fact that the empirical performance of the proposal on AIDS and PROTEINS newly added in the authors' response are not convincing, this paper is not ready for publication at the moment. I think the paper potentially contains an interesting contribution, thereby I strongly recommend carefully re-organizing empirical evaluation and also addressing other weakness points.
Decision: Reject