Deep Graph-Level Orthogonal Hypersphere Compression for Anomaly Detection
Authors: Yunhe Zhang
Yan Sun
Jinyu Cai
Jicong Fan
Venue: Submitted to ICLR 2023 (2023)
Keywords: Unsupervised learning

Abstract: Graph-level anomaly detection aims to identify abnormal samples of a set of graphs in an unsupervised manner. It is non-trivial to find a reasonable decision boundary between normal data and anomalous data without using any anomalous data in the training stage, especially for data in graphs. This paper first proposes a novel deep graph-level anomaly detection model, which learns the graph representation with maximum mutual information between substructure features and global structure features while exploring a hypersphere anomaly decision boundary. The deep orthogonal projection layer is adopted to keep the training data distribution consistent with the decision hypersphere thus avoiding erroneous evaluations. We further propose projecting the normal data into the interval region between two co-centered hyperspheres, which makes the normal data distribution more compact and effectively overcomes the issue of outliers falling close to the center of the hypersphere. The numerical and visualization results on a few graph datasets demonstrate the effectiveness and superiority of our methods in comparison to many baselines and state-of-the-art.

Under review as a conference paper at ICLR 2023
DEEP GRAPH-LEVEL ORTHOGONAL HYPERSPHERE COMPRESSION FOR ANOMALY DETECTION
Anonymous authors Paper under double-blind review
ABSTRACT
Graph-level anomaly detection aims to identify abnormal samples of a set of graphs in an unsupervised manner. It is non-trivial to find reasonable decision boundary between normal data and anomalous data without using any anomalous data in the training stage, especially for data in graphs. This paper first proposes novel deep graph-level anomaly detection model, which learns the graph rep- resentation with maximum mutual information between substructure features and global structure features while exploring a hypersphere anomaly decision bound- ary. We implement an orthogonal projection layer to keep the training data dis- tribution consistent with the decision hypersphere thus avoiding erroneous evalu- ations. More importantly, we further propose projecting the normal data into the interval region between two co-centered hyperspheres, which makes the normal data distribution more compact and effectively overcomes the issue of outliers falling close to the center of the hypersphere. The numerical and visualization results on few graph datasets demonstrate the effectiveness and superiority of our methods in comparison to many baselines and state-of-the-aat
INTRODUCTION
Anomaly detection an essential task with various applications, such as detecting abnormal patterns or actions credit-card fraud, medical diagnosis, sudden natural disasters (Aggarwal (017). etc. Usually. in anomaly detection, the training data only contain normal data and are used to train model that can distinguish unusual patterns from abnormal ones. Anomaly detection on tabular data and images has been extensively studied recently (Ruff et al.) 2018 Goyal et al.) 2020 Chen al,, 2022, Liznerski et al.) [22T]] Sohn et al.] [002T). In contrast, there little work on graph data despite the fact that graph data anomaly detection is very useful in various problems, such as identifying abnormal communities in social networks or detecting unusual protein structures in biology experiments. Compared with the other types of data, graph data is inherently complicated and rich in structural and relational information. The complexity of graph structure facilitates us to learn graph-level representations with diseriminative patterns in many supervised tasks (e.g., graph classification). As for graph-level anomaly detection, however, the intricate graph structure brings many obstacles to this unsupervised learning problem.
Graph anomaly detection usually composes four families: anomalous edge (Ouyang et al. 2020 Xu
al. 2020), node (Zhu & Zhu 2020 BBojchvski & Günnemann 2018), sub-graph (Wang al., 2018 Zheng et al. 2018). and graph-level detections (Zheng et al., 2019 Chalapathy et al. 2018). Herein, the target of the graph-level algorithms is to explore regular group pattern and distinguish the abnormal manifestations of the group. Group abnormal behaviors usually foreshadow some un- usual events and thus play an important role in practical applications.
the past five years, few approaches have focused on graph-level anomaly detection because of the difficulty of representing graphs into feature vectors without using any label information. Graph kernel can measure the sim- ilarity between graphs and regard the result as a representation non-strictly or implicitly. Based on this, graph anomaly detection task usually performs as two-stage. In our experiments (see Section 4h, also find that one-class SVM with graph kernels sometimes yields unsatisfying performances since graph kernels may not be effective enough to quantify the similarity between graphs. So there is large room for improvement regarding graph anomaly detection to our best knowledge.

---

Under review as a conference paper at ICLR 2023
Concerning end-to-end models, Ma et al. (2022) proposed global and local knowledge distillation method for graph-level anomaly detection, which learns rich global and local normal pattern infor- mation by random joint distillation of graph and node representations. The method needs to train two GCNs jointly at a high time cost. iZhao & Akoglu (202T) combined the Deep SVDD objec- tive function and graph isomorphism network to learn hypersphere of normal samples. Qiu et al.) (0022) also sought hypersphere decision boundary and optimized the representations learned by GNNs close the reference GNN while maximizing the differences between k GNNs, but did not consider the relationship between the graph-leve representation and node features. Collecting all approaches based on the hypersphere assumption in graph anomaly detection, we find that the practical decision region may be an ellipsoid instead of standard hypersphere, thus causing the error when the standard hypersphere evaluation employed. Except for that, our experiment also confirms that anomalous data may appear in decision regions that are not filled with normal data, especially near the center of the hypersphere.
GIN
Orthggnnal Pmjecter
Optiization (11)
READOUT
Global Repoesetation
Fully Connacted Network
Fully Connactod Nebwork
Input
Mutual Information Loss Optimization (4)
Optimization (15)
Figure 1: Architecture of the proposed models (right top: DOHSC; right bottom: DO2HSC).
In order to effectively explore a better representation without label information and obtain a more suitable decision boundary with high efficiency, in this paper, we propose one-class deep graph-level anomaly detection method and its improved version. The first proposed model, Deep Orthogonal Hypersphere Contraction (DOHSC), uses the mutual information of local feature maps and the global representation to learn a high-quality representation and simultancously optimizes to distribute in hypersphere area. An orthogonal projection layer then renders the decision region more hyperspherical and compact to decrease evaluation errors. With regard to phenomenon that anomalous data falling close to the hyperspherical center, improved graph-level Deep Orthogonal Bi-Hypersphere Compression (DO2HSC) for anomaly detection architecture is proposed. From a cross-scctional point of view, DO2HSC limits the decision area (of normal data) an interval en- closed by two co-centered hyperspheres and learns the ietogoonality-pooeeette representation simi- larly. The framework of the methods mentioned above shown in Figure [ correspondingly, Fur- thermore, we define new evaluation way according to DO2HSC, and comprehensive experimental results verify the effectiveness of all proposed methods. In summary, the main contributions of our work are listed as follows.
First, we present a new graph-level hypersphere contraction algorithm for anomaly detec- tion tasks, which is jointly trained via mutual information loss between local and global representations and hypersphere decision loss.
Second, we impose an orthogonal projection layer on the proposed model to promote train- ing data distribution close to the standard hypersphere, thus avoiding errors arising from inconsistencies between assessment criteria and actual conditions.
Finally, we propose an improved graph-level deep orthogonal bi-hypersphere compres- sion model to further explore a decision region enclosed by two co-centered hyperspheres, which can effectively prevent anomalous data falling close to the hyperspherical center and surpass baselines significantly in the experiments.
2

---

Under review as a conference paper at ICLR 2023
2 PROPOSED APPROACH
In this section, we first introduce a joint learning architecture in detail, named as Graph-Level Deep Orthogonal Hypersphere Contraction. Then an improved algorithm is proposed compensate for the underlying assumption's 's deficiency.
2.1
GRAPH-LEVEL DEEP ORTHOOGNNAL HYPERSPHERE CONTRACTION
2.1.1
VALLINA MODEL
Given a set of graphs G -GI,,.., Gx) with N samples, the proposed model aims to learn k- dimensional representation and then set a soft-boundary according to it. In this paper, the Graph Isomorphism Network (GIN) (Xu et al. 2019) is employed to obtain the graph representation in three stages: first, input the graph data and integrate neighbors the current node (AGGREGATE); second, combine neighbor and current node features (CONCAT); finally, integrate all node infor- mation (READOUT) into one global representation. Mathematically. the i-th node features I-th layer and the global features of its affiliated j-th graph would be denoted
2.p CONCAT ( }
Zo (G,) = READOUT (24) 1
where zig Rixk and (G;) Rixk To integrate the contained information and enhance the differentiation between node-level and global-level representations, we append additional fully con- nected layers denoted as the forms Mr(-) and To T+(+). (-), respectively, where T and V are the parameters of the added layers. So the integrated node-level and graph-level representations are obtained via
h r == (T
H o (G,) = (z,))
To better capture the local information, we utilize the batch optimization property of neural networks maximize the mutual information (MI) between local and global representations in each batch G C G, which is defined by Sun et al. (2020) as the following term:
b V f
arg max 44 L0,0,7 v, (ha r. Ha,v v.V.T
(3)
Specifically, the mutual information estimator Ta.er follows Jensen-Shannon MI estimator Nowozin et al. 2016) with positive-negative sampling method as below,
1. (ha Ho (G)) : = Σ
Is (b HG,), WEG,
Σ
G3
G,EG
E E(-o (1--o(-hã)) rx Hx,)())))
(4)
Gj
GGEG
DEG;
- B(O (h x Ha (x)))
where softplus function (2)) log(1 e²) is activated after vector multiplication between node and graph representations. For x as an input sample graph, we calculate the expected mutual infor- mation with its positive samples x and negative samples x which are generated from distribution across all graphs in a subset. Given each G = (Va, Eg) and node set Vo = [C] the positive and negative samples divided in this way:
[ Xij. O, otherwise.
if Us E Gj,
x+
(5)
And x produces the opposite result in each of the conditions above. In the next step, data- enclosing decision boundary is required for our anomaly detection task. According to the assump- tion that most normal data locate in hypersphere, the center of this decision boundary should
3

---

Under review as a conference paper at ICLR 2023
be initialized through
N
Ho.o(G.)
(6)
With this center, we expect to optimize the learned representation of normal data to be distributed as close it possible, so that the unexpected anomalous data falling out of this hypersphere would be detected. Besides, the regularization term is adopted to avoid over-fitting problems. Collectively denote the weight parameters of +, V and Y Q + U V U Y, we formulate the training loss with two joint objectives Hypersphere Contraction and MI:
G
min 0 W.V.T
G
Σ
Ho, (G,) cll² AT (bo,r, Ho, (G),(G))
Σ
IQ1
F:
QEE
where | denotes the number of graphs in batch and is a trade-off factor, the third term is network weight decay regularizer with the hyperparameter μ.
2.1.2 ORTHOGONAL, PROJECTION LAYER
Orthogonal
Projection
Figure 2: Variations in decision boundary with and without the orthogonal projection layer. In the left subfigure, the real decision region where training data are distributed may be an ellipsoid (dark grey). This contradicts with the hypersphere decision boundary (light grey) set by Optimization 7 After orthogonal projection, the ellipsoid is expected to be transformed into a standard hypersphere, which avoids the evaluation error. Note: here data are simulated only for illustration.
However, an empirical study shows that hyperellipsoid is commonly observed during deep repre- sentation learning. This phenomenon would lead to inaccuracies in the final test because the evalu- ation results are based on the hypersphere decision region. Problem (7) obviously cannot guarantee the boft-boundary of learned representation to be a standard hypersphere like Figure 2 Accord- ingly, append orthogonal projection layer after obtaining the global representation. Note that we pursue orthogonal features of latent representation rather than computing the projection onto the column standardized principal components. Our experiments also justify the necessities of this projection step and standardization process, which will be discussed further in Section 4.4 and Appendix G Specifically, the projection layer can be formulated as
space of Ho,v. This method is equivalent to performing PCA and using the
Hg v. (G) Proje (He, v (G)) = H W, subject to o Is
(8)
where (W = } are the projection parameters, I denotes an identity matrix, and k is the projected dimension.
Note that to achieve 8) one may consider adding regularization term H v.e Hg v.e - I 1 with large enough a To the objective, which is not very effective and will lead one more tuning hyperparameter. Instead, we propose to achieve g via singular value decomposition, i.c.,
UAV T H
(9)
W = VW
4

---

Under review as a conference paper at ICLR 2023
720
62
(a) Training Data
(b) Testing Data
Figure 3: Illustration of inevitable flaws in DOHSC on both the training and testing data of COX2. Left: the 2 no-m distribution of 4-dimensional distances learned from the real dataset; Right: the pseudo-layout in two-dimensional space sketched by reference to the empirical distribution.
where A diag(P1, P........... P(ci) and V are the diagonal matrix with sigular values and right- singular matrix of Ho, v. respectively. It needs to be emphasized that V [V1,-.. Vir denotes the first k' right-singular vectors and Ag := diag(pı,... .k:) In each forward propagation epoch, the original weight parameter is substituted a new matrix W in subsequent loss computations.
2.1.3 ANOMALY DETECTION
Attaching with orthogonal projection layer, the improved initialization of the center is rewritten in the following form
N
غ = N
Σ
H. H.,v,e(G.))
(10)
and the final objective function for anomaly dectection tasks in a mini-batch would become
G
min 0.
Σ
Ho (G, a012 -ā] +
Σ
Is.er (ho. H. H..)))
M
Σ QQ
GEG
QE
(11)
After the training stage, a decision boundary will be fixed, which is calculated based on the - v percentile of the training data distance distribution:
arg min P(D Sr)2w
(12)
where = ( follows sampled distribution P, and d, ||Ha,,,(G..) - -ēl. Accordingly, the anomalous score of i-th instance is defined as follows:
8; df F2
(13)
where s = (#1, $2; .......... s.). is evident that when the score is positive, the instance is identified as abnormal, and the opposite is considered normal.
The detailed procedures of algorithm are summarized into Algorithm T (see Appendix with graph representation learning and promotes the training data approximate the center of hypersphere while adding an orthogonal projection layer. Unfortunately. it can be observed from Figure3thht the anomalous data would appear partial regions of the learned decision hypersphere, which are not filled by the training data, especially the region close to the center. To handle this particular situation, an improved graph-level anomaly detection approach, termed Graph-Level Deep Orthogonal Bi- Bi-Hypersphere Compression, will be designed in the next section.
starts
2.2 GRAPH-LEVEL DEEP ORTHOGGNNLL BI-HYPERSPIIER COMPRESSION
As Figure suggests, we found peculiar phenomena in our empirical results that the learned distribu- tion of training data sometimes could not satisfy the hypersphere assumption, where anomalous data might appear within the decision region and therefore led to suboptimal detection performance. To explore the reason behind, examine the counter-intuitive behaviors of high-dimensional Gaussian
5

---

Under review as a conference paper at ICLR 2023
distributions, and the simulation results imply the soap-bubble problem, where anomalous samples could exist near the center of learned hypersphere (see Appendix B for more details). Since DOHSC cannot detect anomalies close to the center, we propose improved approach, which sets the de- cision boundary as an interval region between two co-centered hyperspheres. This can narrow the decision area's s scope and induce normal data fill the entire interval area as much possible.
After the same graph representation learning stage, we firstly utilize the DOHSC model for a few epochs and initialize the large radius mmax and the small radius min of the interval area according to the 1 // percentile and y of the sample distances distribution, respectively. The aforementioned descriptions can be denoted mathematically as below.
I' 'max
= arg min P(D S 2 v;
(14)
T min arg min P(D S 2111
After fixing the decision boundaries Tmar and T'mn: the improved training loss is also set with trade-off factor X, which implicitly emphasizes the importance of the max-min term:
G
min 0,
(max/d;, mmar] - min(de, mmin
(de,,mmnn)) + x Σ
I lav.r (he.r. Ho, Ho.
G
i=1
GEB
(15)
Q
QEQ
This decision loss has the lowest bound Tmaz Tmin and can be jointly minimized with mutual information effectively. Besides, the evaluation standard of test data is also needed to change based on this interval structure. More specifically, all instances located in the inner hypersphere and out of the outer hypersphere should be identified as anomalous graphs; only those located the interval area should be regarded as normal data. Compared with equation [3] we reset new score function to award the positive samples beyond [rmin Tmaz] and meanwhile pumishing the negative samples within the range. Accordingly, the distinctive scores calculated by
8; (d, rmaz) (d4 Tmin))
(16)
where i E {I,...,NN NJ. This way, we can also effectively identify a sample's abnormality by its score. In general, the improved deep graph-level anomaly detection algorithm changes the decision boundary and effectively makes the normal area more compact. Correspondingly. the new practical evaluation is raised to adapt the improved detection way. Eventually, summarize the detailed procedures of the optimization into Algorithm 2 (see Appendix A
3 CONNECTION WITH PREVIOUS WORK
Actually, few studies have been undertaken in graph-level anomaly detection (GAD), Existing so- lutions to GAD tasks can be categorized into two types: two-stage approach and an end-to-end one. Two-stage GAD methods first transform graphs into graph embeddings by graph neural net- works or into similarities between graphs by graph kernels, and then apply off-the-shelf anomaly detectors such as local outlier factor (LOF) (Breunig et al. 2000), one-class support vector machine (OCSVM) (Schölkopf et al. 1999), etc. The drawback of the two-stage method is that the graph fea- ture extractor and outlier detector are independent, and some graph kernels produce "hand-crafted" features that are deterministic without much space to improve. End-to-end approaches overcome this problem by utilizing deep graph learning techniques, such as graph convolutional network (GCN) (Welling& & Kipi 2016) and graph isomorphism network (GIN) (Xu et al. 2019). With an anomaly measure as the objective, end-to-end approaches jointly learn an effective graph representation for GAD task (Zhao Akoglu 2021: Qiu et al. 2022, Ma al. 2022). Zhao & Akoglu (2021) opti- mized one-class model based on deep support vector data description (Deep SVDD) as the anomaly measure. Here we clarify the differences between (Zhao & Akoglu 2021) and ours. First, the proposed model employed mutual information loss in the graph learning stage to obtain the graph representation incorporating local and global information. On the contrary. Zhao & Akoglu (202I) directly utilized the readout result of GIN. We also impose orthogonal projection representation to maintain consistency between the learned decision boundary and normal data dis- tribution. More importantly, we present new approach constructing decision boundary. The
the learned

---

Under review as a conference paper at ICLR 2023
new approach learns two hyperspheres, between which the region accommodates normal data and hence yields more space for abnormal data. As for graph kernel, we summarized the previous work in Appendix C
4 EXPERIMENTS
4.1 DATASET
In this work. we our method on six real-world graph datasets"] which contain three social networks datasets (COLLAB, COX2, and IMDB- (IDB-Binary) and three bioinformatics datasets (DD, ER MD, and MUTAG). The details of the datasets are shown in Table
Table I: Description for six datasets.
Datasets
# Graphs
Avg. # Nodes
Avg. Edges
# Classes
Graph Lahels
COLLAB
5000
74.49
2457.78
2600 / 775 1625
COX2
467
42.43
44.54
365 102
MR_MD
446
21.33
234.85
265 / 181
MUTAG
188
17.93
19.79
63 125
DD
1178
284.32
715.66
2
691 / /487
IMDB- IMDB-Binary
1000
19.77
96.53
2
500/ 500
4.2 BASELINES
We compare our method with the following unsupervised graph-level anomaly detection meth- ods: Random Walk (RW) (Gärtner al., 2003 Kashima et al. 2003), Shortest Path Kernel (SP)
(Borgwardt & Kriegel 2005), Weisfeiler-Lehmar Sub-tree Kernel (WL) (Shervashidze al. 2011), Neighborhood Hash Kernel (NH) (Hido & Kashimal 2009). Besides graph kernels, we also com- pare three graph-level representation learning methods: Deep One Class Model with GIN network (OCGIN) (Zhao Akoglu (021), Graph-level embedding Learning via Mutual Information Max- imization+Deeep SVDD GintoGrap+DDeep SVDD) (Sun al.] (2000: Ruff et al. 2018), Global and Local Knowledge Distillation for Graph-level Anomaly Detection (oLocalKD) (Ma et al. 2022)
and One Class Graph Transformation Learning (OCGTL.) (Qiu al. 2022
4.3 RESULTS
In this section, extensive experimental results are displayed to validate the effectiveness of the pro- posed models. The averages and standard deviations of the Area Under Operating Characteristic Curve (AUC) used to support the comparable experiments by repeating each algorithm ten times. The higher value of the AUC metric represents better performance. Tables 2 ric and its standard deviations. AUC values compared to other algorithms on all datasets. Both approaches outperform other state- of-the-art baselines, and DO2HSC obtains superior performance and get 5% higher performance than other algorithms on many datasets, such as MUTAG, COLLAB Class 1, and ER MD Class O, IMDB-Binary Class etc. It is woth mentioning that we defeat infoGraph+ Deep SVDD with large improvement, which is degraded version of the proposed models, thus showing that pro- moting representation learning towards the anomaly detection goal is meaningful and well targeted.
report the AUC met- be that the proposed methods basically achieve the best
The anomaly detection visualization results of DO2HSC are displayed in Figure and those of DOHSC are also shown in Appendix F We draw them by setting the projection dimension to three directly. Results different perspectives are given to avoid blind spots in the field of vision, demon- strating excellent performance. Hence, it can be concluded that the effect of the improved model is in line with our motivation and shows much potential.

---

Under review as a conference paper at ICLR 2023
Table 2: Average AUCs with standard deviation (10 trials) of different graph-level anomaly detection
algorithms. We assess models by regarding every data class as normal data, respectively. The best results are marked in bold and "_' means out of memory.
COLLAB
COX2
2
SP40CSV
0.5910 + 0.0000
0.8397 + 0.0000
0.7902 + 0.0000
0,5408 + 0.0000
0.5760 + 0.0000
WL2+OCSVM
0.5051 + 0.0000
0.7989 + 0.0000
0.6977 0.0000
0.5736 + 0.0000
0.4286 + 0.0000
WL.+0CVM
0.5079 0.0000
0.8021 0.0000
0.7884
0.5990 0.0000
0.4376 0.0000
WL:+OCSVM
0.5106 = 0.0000
0.8035 = 0.0000
0.7953 = 0.0000
0.5979 : 0.0000
0.5057 + 0.0000
0.5122 + 0.0000
0.8031 + 0.0000
0.7996 + 0.0000
0.5937 + 0.0000
0.5034 + 0.0000
NH+OCSVM
0.5976 + 0.0000
0.8054 + 0.0000
0.6414 0.0000
0.4841 +: 0.0000
0.4717 + 0.0000
RW+OCSVM
0.5243 0.0000
0.6553 0.0000
OCGIN
0,4217 + 0.0606
0.7565 + 0.2035
0.1906 0.0857
0.5964 0.0578
0,5683 1 0.0768
infoGruph+ DDep SVDD
0,5662 + 0.0597
0.7926 + 0.0986
0.4062 + 0.0978
0.4825 1 0.0624
0,5029 + 0.0700
GLocalKD
0.4638 + 0.0003
0.4330 + 0.0016
0.4792 + 0.0004
0.3861 + 0.0131
0,3143 1 0.0383
OCGTL.
0.6504 0.0433
0.8908 0.0239
0.5541 + 0.032
0.4862 + 0.0224
DOHSC (Ours)
0.9185 0.0455
0.9755
0.0030
0.5450
0.0469
0.6263 0.0333
0.6805 0.0168
DO2NSC (Ours)
0.6718 + 0.0353
0.9153 + 0.0070
0.7188 0.0260
0.6329 + 0.0292
0.6518 + 0.0481
Table 3: Average AUCs with standard deviation (10 trials) of different graph-level anomaly detection algorithms. We assess models by regarding every data class normal data, respectively. The best results are marked in bold.
ER_MD
MUTAG
0
SP+OCSVM
0.4092 0.0000
0.3824 = 0.0000
0.5917 =
0.0000 0,0000
0.2608 = 0.0000
WL2+OCSVM
0.3702 0.0000
0.3262 = 0.0000
0.5976 + 0.0000
0.2960
0.0000
OL5+OCSVM
0.4446 0.0000
0.2933 = 0.0000
0.6509 + 0,0000
0.2352 - 0.0000
WL5+OCSVM
0.4571 0.0000
0.3199 = 0.0000
+ 0
0.2176 + 0.0000
WL1++CCSV
0.4297 0.0000
0.2761 = 0.0000
0.5089
0.0000
0.2048 = 0.0000
NH+OCSVM
0.5155 0.0200
0.3648 0.0000
0.7959
= 0.0274
0.1679 0.0062
RW+OCSVM
0.4820 0.0000
0.3484 1 0.0000
0.8698 1 0.0000
0.0000 0.1504
OCGIN
0.5645 0.0323
0.4358 0.0538
0.8491 = 0.0424
0.4933 0.1525
PifoGraap+DDepp SVDD
0.5312 = 0.1545
0.5682 = 0.0704
0,8805 + 0.0448
0.6166 = 0.2052
GLocalKD
0.5781 0.97
0.7154 + 0.0000
0.3952 + 0.2258
0.2965 + 0.2641
OCGTI.
0.2755 0.0317
0.6915 + 0.0207
0.6570 + 0.0210
0.7579 0.2212
DOHSC (Ours)
0.6620 = 0.0308
0.5184 + 0.0793
0,8822 1 0.0432
0.8115 + 0.0279
DO2HSC (Ours)
0.6867 0.0226
0.7351 + 0.0159
0.9089 + 0.0609
0.8250 0.0790
Table 4: Average AUCs with standard deviation (10 trials) of different graph-level anomaly detection algorithms. We assess models by regarding every data class as normal data, respectively. The best results marked in bold and '_' means out of memory.
DD
IMDB BiDB-Binary
1
0
SP+OCSVM
0.6856 0.0000
0.4474 + 0.0000
0.4592-1 0.0000
0.4716 0.0000
WL.+
0.6225
0.0000
0.4946
0.0000
0.4294 + 0.0000
0.4607
0.0000
WL.+OCSVM
0.7127 0.0000
0.4666 + 0.0000
0.4940 + 0.0000
0.4243
0.0000
WL.+OCCVM
0.7343 + 0.0000
0.4455 + 0.0000
0.5066 + 0.0000
0.4143 + 0.0000
OCCVM
0.7397 0.0000
0.4302 + 0.0000
0.5157 + 0.0000
0.4115 0.0000
NH+OCSVM
0.7424 0.0000
0.3684 0.0000
0.5321 + 0.0000
0.4652 0.0000
RW+OCSVM
0.4951 + 0.0000
0.0000 0.5311
OCGIN
0.6659 0.0444
0.6003 + 0.0534
0.4047 + 0.1083
0.3332 0.0649
infoGraph+Dese Deepp SVDD
0.3942 0.0436
0.6484 + 0.0236
0.6353 + 0.0277
0.5836 + 0.0995
GLocalKD
0.1952 0.0000
0.2203 = 0.0001
0.5383 + 0.0124
0.4812 2 0.0101
OCGTL
0.6990 0.0260
0.6767 = 0.0280
0.6510 + 0.01 80
0.6412 0.0127
DOHSC (Ours)
0.7083 0.0188
0.7579 = 0.0154
0.6609 - 0.0033
0.7705 + 0.0045
DO2HSC (Ours)
0.7320 0.0194
0.7651 = 0.0317
0.6406 + 0.0642
0.7101 0.0429
8

---

Under review as a conference paper at ICLR 2023
40
(s) Thining Rosall.
Testing Result
Figure 4: Visualization results of the DO2HSC MUTAG Class 0 in different perspectives.
4.4 ABLATION STUDY
In this section, we display the ablation study of the orthogonal projection layer on three datasets. From quantitive comparisons, we concluded that orthogonality positively influences all performance to some extent. It also well supports our assumption discussed in Section 2.1.2
Table 5: Ablation study of the orthogonal projection layer. We test models by regarding every data class as normal data, respectively. The best performance is highlighted bold.
Orthogonal Projection Layer
Class
MUTAG
COX2
IMDB BinarB
0
0.8521
0.5817
0.5880
X
I
0,8912
0.5737
0,6082
DOHSC
0
0.8639
0.6433
0.6192
0.9088
0.6077
0.6830
0
0. 0.8934
0.6281
0.6252
1
0.7504
0.6178
0.6541
DO2HSC
0
0,9467
0.7158
0.6933
V
1
0,9184
0.7074
0.6700
Except for the aforementioned contents, please see the detailed experiment configurations, parame- ter sensitivity and robustness of the proposed models, supplemented visualizations of distance dis- tributions for anomaly detection and visualization comparison between proposed models with or- thogonal projection layer and without orthogonal projection layer in Appendix G which can also strongly support our theory and validate the effectiveness.
CONCLUSION
This paper has proposed two novel end-to-end graph-level AD methods, DOHSC and DO2HSC, which combined the effectiveness of mutual information between node-level and global features to learn graph representation and the power of hypersphere compression. DOHSC and DO2HSC mitigate the possible shortcomings of hypersphere boundary learning by applying orthogonal projection for global representation. Furthermore, DO2HSC projects normal data between the in- terval area of co-centered hyperspheres significantly alleviate the soap-bubble issue. Our comprehensive experimental results strongly demonstrate the superiority of DOHSC and DO2HSC on multifarious datasets. In future work, we will explore more efficient and excellent graph repre- sentations and refine our learning process for decision regions.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Charu C Aggarwal. An introduction to outlier analysis. 2017.
Aleksandar Bojchevski and Stephan Günnemann. Bayesian robust attributed graph clustering: Joint learning of partial anomalies and group structure. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32, 2018.
Karsten M Borgwardt and Mans-Peter Kriegel. Shortest-path kernels on graphs In Proceedings of the Fifth IEEE International Conference on Data Mining, pp. 8-pp, 2005.
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng. and Jörg Sander. Lof: Identifying density- based local outliers. In Proceedings of the ACM SIGMOD International Conference on Manage- ment of Data, pp. 93-104, 2000.
Raghavendra Chalapathy, Edward Toth, and Sanjay Chawla. Group anomaly detection using deep generative models. Joint European Conference on Machine Learning and Knowledge Discov- ery in Databases, pp. 173-189, 2018.
Yuanhong Chen, Yu Tian, Guansong Pang, and Gustavo Carneiro. Deep one-class classification via interpolated gaussian descriptor. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 383-392, 2022.
Thomas Gärtner, Peter Flach, and Stefan Wrobel. On graph kernels: Hardness results and efficient alternatives. In Learning Theory and Kernel Machines, pp. 129-143. 2003.
Sachin Goyal, Aditi Raghunathan, Moksh Jain, Harsha Vardhan Simhadri, and Prateek Jain. DROCC: deep robust one-class classification. In Proceedings of the 37th International Con- ference Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp- 3711-3721, 2020.
Shohei Hido and Hisashi Kashima. A linear-time graph kernel. In Ninth IEEE International Con- ference on Data Mining, pp. 179-188, 2009.
Hisashi Kashima, Koji Tsuda, and Akihiro Inokuchi. Marginalized kernels between labeled graphs. In Proceedings of the 20th International Conference on Machine Learning, pp. 321-328, 2003.
Nils M Kriege, Fredrik Johansson, and Christopher Morris. A survey on graph kernels. Applied Network Science, 5(1):1-42, 2020.
Beatrice Laurent and Pascal Massart. Adaptive estimation of a quadratic functional by model selce- tion. Annals of Statistics, pp. 1302-1338, 2000.
Philipp Liznerski, Lukas Ruff, Robert A. Vandermeulen, Billy Joe Franks, Marius Kloft, and Klaus- Robert Müller. Explainable deep one-class classification. In Proceedings of the 9th International Conference on Learning Representations, 2021.
Rongrong Ma, Guansong Pang, Ling Chen, and Anton van den Hengel. Deep graph-level anomaly detection by glocal knowledge distillation. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 704-714, 2022.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural sam- plers using variational divergence minimization. In Advances in Neural Information Processing Systems, volume 29, 2016.
Linshu Ouyang, Yongzheng Zhang, and Yipeng Wang. Unified graph embedding-based anomalous edge detection. In International Joint Conference Neural Networks, pp. 1-8, 2020.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Coumapeaau, M. Brucher, M. Perrot. and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2828 2011.
10

---

Under review as a conference paper at ICLR 2023
Chen Qiu, Marius Kloft, Stephan Mandt, and Maja Rudolph. Raising the bar in graph-level anomaly detection. In Lue De Raedt (ed.), Proceedings of the Thirty-First International Joint Conference 2022.
on Artificial Intelligence, pp.
Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexan- der Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International Conference on Machine Learning, pp. 4393 4402. 2018.
Bernhard Schölkopf, Robert C Williamson, Alex Smola, John Shawe-Taylor, and John Platt. Support vector method for novelty detection. Advances in Neural Information Processing Systems, 12, 1999.
Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the support of high-dimensional distribution. Neural Computation, 13(7):1443- 1471, 2001.
Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borg- wardt. Weisfeiler-Iemman graph kernels. Journal of Machine Learning Research, 12(9), 2011.
Giannis Siglidis, Giannis Nikolentzos, Stratis Limnios, Christos Giatsidis, Konstantinos Skianis, and Michalis Vazirgiannis. Grakel: A graph kernel library in Python. Journal of Machine Learning Research, 21(54):1-5, 2020.
Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister. Learning and evalu- ating representations for deep one-class classification. In Proceedings of the 9th International Conference on Learning Representations, 2021.
Fan- Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In Proceed- ings of the 8th International Conference Learning Representations, 2020.
Roman Vershynin. High- dige--iimesional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
Haibo Wang, Chuan Zhou, Jia Wu, Weizhen Dang, Xingquan Zhu, and Jilong Wang. Deep structure learning for fraud detection. In IEEE International Conference on Data Mining, pp. 567-576, 2018.
Max Welling and Thomas N Kipf. Semi-supervised classification with graph convolutional net- works. In Proceedings of the 5th International Conference on Learning Representations, 2016.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In Proceedings of the 7th International Conference on Learning Representattons, 2019.
Linchuan XU, Xiaokai Wei, Jiannong Cao, and Philip S Yu. Icane: Interaction content-aware net- work embedding via co-embedding of nodes and edges. International Journal of Data Science and Analytics, 9(4):401-414, 2020.
Lingxiao Zhao and Leman Akoglu. On using classification datasets evaluate graph outlier detec- tion: Peculiar observations and new insights. Big Data, 2021.
Mengyu Zheng, Chuan Zhou, Jia Wu, Shirui Pan, Jinqiao Shi, and Li Guo. Fraudne: A joint embedding approach for fraud detection. In International Joint Conference on Neural Networks, pp. 1-8, 2018.
Panpan Zheng, Shuhan Yuan, Xintao Wu, Jun Li, and Aidong Lu. One-class adversarial nets for fraud detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1286-1293, 2019.
Mengxiao Zhu and Haogang Zhu. Mixedad: scalable algorithm for detecting mixed anomalies in attributed graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 1274-1281, 2020.
11

---

Under review as a conference paper at ICLR 2023
A SUPPLEMENTED ALGORITHM
Algorithm summarizes the procedure of DOHSC in detail. It starts with graph representation learning and promotes the training data to approximate the center of a hypersphere while adding an orthogonal projection layer. Also, DO2HSC is recapped in Algorithm 2 and also starts with same graph representation learning. Differently, DOHSC is utilized of few epochs to initial the decision boundaries and after that, improved optimization is applied.
Algorithm Graph-Level Deep Orthogonal Hypersphere Contraction (DOHSC)
Input: The input graph set G, dimensions of GIN hidden layers k and orthogonal projection layer k', a trade-off parameter À and the coefficient of regularization term μ4, pretraining epoch T, learning rate 7)-
Output: The anomaly detection scores s. 1: Initialize the network parameters 4 V, Y and the orthogonal projection layer parameter θ;
2: for ( 7 do
3: for each batch G do
4: Calculate 4 v.Y (harr. H&, w (G)) via equation 4
5: Back-propagation GIN and update 4, v and Y, respectively;
6: end for
7: end for
8: Update the orthogonal parameter θ of orthogonal projection layer by equation
9: Obtain the global orthogonal latent representation by equation
10: Initialize the center of hypersphere by equation 10
11: repeat
12: for each batch G do
13: Calculate total loss via 11
14: Back-propagation and update +b V, Y and O, respectively;
15:
Further update the orthogonal parameter t of orthogonal projection layer by equation 9
end for
17:
until convergence
Compute decision boundary r by equation| 12
19:
Calculate the anomaly detection scores through equation 13
20:
return The anomaly detection scores s.
B RELATED PROOF OF BI-HYPERSPHENT LEARNING MOTIVATION
The traditional idea of detecting outliers is inspect the distribution's 's tails with ideal assumption that the normal data are Gaussian. Following the assumption, one may argue that an anomalous sam- ple can be distinguished by its large Euclidean distance to the data center (&2 norm ||z cll. where denotes the centroid), and accordingly, the abnormal dataset (2 : l/z cl > r} for some decision boundary r. However in high dimensional space, Gaussian distributions look like soap-bubble F which means the normal data are more likely to locate in bi-hyperspphere(Verhhhe
2018), such
as {z Tmin < c||z-||| < Tmaw ). To better understand this counterintuitive behavior, let us generate some normal samples X ~ (O, where is the data dimension in {1, 10, 50, 100, 200, 500). As Figure 6 indicates, only the univariate Gaussian has a near-zero mode, whereas other high- dimensional Gaussian distributions leave plenty of offcenter spaces in blank. The soap-bubble prob- lem in high-dimensional distributions is well demonstrated in Table 6 the higher the dimension is, the greater the quantities of data further away from the center, especially for 0.01 -qlunnnee distance. Thus, we cannot make a sanguine assumption that all of the normal data locate within some radius of hypersphere (i.e. {z ||z cl r)). Using Lemma of (Laurent & Massart 2000), we prove that proposition[] [[] which matches the values in the Table 6 that when the dimension is larger, normal data more likely lies away from center.
Proposition 1 Suppose Zı, Zz...... Zu are sampled from (0, La) independently. Then for any z; and all t 2 0, the following inequality holds:
12

---

Under review as a conference paper at ICLR 2023
Algorithm 2 Graph-Level Deep Orthogonal Bi-Hyperspherre Compression (DO2HSC)
Input:
The input graph set G, dimensions of GIN hidden layers k and orthogonal projection layer k, trade-off parameter À and the coefficient of regularization term H4, pretraining epoch Tı, iterations of initializing decision boundaries T2, learning rate 1/-
Output: The anomaly detection scores
Initialize the network parameters 4, V, and the orthogonal projection layer parameter θ
2: for ( Ti do
for each batch G do
4: Calculate 44, v.Y (he,r, H&, v (G)) via equation 4
Back-propagation GIN and update 4 V and Y, respectively; 6: end for
end for
8: Update the orthogonal parameter θ of orthogonal projection layer by equation 9
Obtain the global orthogonal latent representation by equation Initialize the center of hypersphere by equation 10 for t + To do
for each batch G do
Calculate total loss via 11
14:
Back-propagaaion and update 4, V, and θ, respectively:
Further update the orthogonal parameter O of orthogonal projection layer by equation 9
16:
end for end for
18:
Initialize decision boundaries Imaz and Tmin via equation 14 repeat
20:
for each batch G do
Calculate the improved total loss via 15
22:
Back-propagation and update 4, v, and O, respectively; Further update the orthogonal parameter of orthogonal projection layer by equation end for
24:
until convergence
26:
Calculate the anomaly detection scores through equation 16 return The anomaly detection scores
P | 2 Vd 2v dt
2 -
600
Count
0: Normal
200
1: Anomalouss
16
14
Euclidean nom from the center
Figure 5: Histogram of distances (Fuclidean norm) from the center of normal samples under 16-dimensional Gaussian distributions A( I). Three groups of anomalous data are also 16- dimensional and respectively sampled from N(u1, + 1), (M I), and A N(23, 51), where the pop- ulation means fi f2, p3 are randomized within [0, I] for each dimension.
13

---

Under review as a conference paper at ICLR 2023
Table 6: Offeenter distance under multivariate Gaussian different dimensions and quantiles.
Quantile (correspond to mmmnn)
dim=1 1
dim=10
dim=50
dim=100
dim=200
dim=500
0.01
0,0127
1.5957
5.5035
8.3817
12.5117
20.6978
0.25
0.3115
2.5829
6,5380
9.4908
13.6247
21.8542
0.50
0.6671
3.0504
7.0141
9.9662
14.1054
22.3337
0.75
1.1471
3.5399
7.5032
10.4386
14.5949
22.8200
0.99
2.5921
4.8265
8.7723
11.6049
15.7913
24.0245
We also simulate possible case of outlier detection, in which data are all sampled from 16- dimensional Gaussian with orthogonal covariance: 10,000 normal samples follow (O, and the first group of 1,000 outliers are from N(J,; To I). the second group of 500 outliers are from (12, I), and the last group of 2,000 outliers are from A (13,5T) Figure well exemplifies that abnormal data from other distribution (group-1 outliers) could fall into the small distance away from the center of the normal samples.
2000
1750
1500
1250
Count
dim-1
dim=10
2000
d0m=50
dim=100
730
dim=200
dimm550
500
20
25
Euclidean norm from the center
Figure 6: Histogram of distances (Euclidean norm) from the center of 10,000 random samples under (univariate or) high-dimensional Gaussian distributions No,,1)
C RELATED WORK ON GRAPH KERNEL
To learn with graph-structured data, graph kernels that measure the similarity between graphs be- come an established and widely-used approach (Kriege et al.] 2020). A large body of work has emerged in the past years, including kernels based on neighborhood aggregation techniques and walks and paths. (Shervashidze et al. (2011) introduced Weisfeiler-Lehmas (WL) algorithm, a well- known heuristic for graph isomorphism. In Hido & Kashima (2009). Neighborhood Hash kernel was introduced, where the neighborhood aggregation function is binary arithmetic. The most influential graph kernel for paths-based kernels the shortest-path (SP) kernel by |Borgwardt & Kriegel (2005). For walks-based kernels, Gärtner et al. (2003) and Kashima al. (2003) simultaneously proposed graph kernels based on random walks, which count number of label sequences along walks that two graphs have in common. These graph kernel methods have the desirable property that they do not rely on the vector representation of data explicitly but access data only via the Gram matrix.
D EXPERIMENT CONFIGURATION
In this part, the experiment settings are listed for reproducing. First, each dataset divided into two parts: training and testing sets. We randomly sample eighty percent of the normal data the
14

---

Under review as a conference paper at ICLR 2023
training set, and the remaining normal data together with the randomly sampled abnormal data in one-to-one ratio to form the testing set.
With regard to the experiment settings of compared graph baselines, adopted the classical AD method, One-Class SVM (OC-SVM) (Schölkopf ct al. 2001) and used 10-fold cross-validation to make fair comparison. All graph kernels via GraKel (Siglidis et al. 2020) extract a Kernel matrix and apply OC-SVM in scikit-learn (Pedregosa et al. 2011) are implemented. Specifically, we selected Floyd Warshall as the SP kernel's is algorithm and set lambda 0.01 for the RW kernel. WL kernel al algorithm is sensitive to the number of iterations, so we test four types with the iteration of {2. 5, 8, 10) and denote them as WL2. WLs, WLs, WL10- For all graph kernels, the outputs are normalized. About infoGraph+Deep SVDD, the first stage runs in 20 epochs, and the second stage pretrains 50 epochs and trains 100 epochs. In OCGIN, GLocalKD, and OCGTI, their default or reported parameter settings are adopted to reproduce experimental results. But there still exists some special situations like, due to the limited device, the relatively large-scale dataset tested with a small batch size, such as DD. This may lead to worse performance for compared algorithms.
Dilection Fumctian valum
Oolective fonetor
DOL octive Lurcitinn
Number
1a
26
36
apechs
Number spocts
(a) DOHSC
(b) DO2HSC
Figure 7: Convergence curves of the proposed models on the MUTAG dataset.
Regarding our model DOHSC, we firstly set epoch in the pretraining stage initialize the center of the decision boundary and then train the model in 500 epochs. The convergence curves are given in Figure to indicate that the final optimized results are adopted. The improved method DO2HSC is also set 1-epoch pretraining stage and trains DOHSC 5 epochs to initialize a suitable center and decision boundaries 7 T'maæ and T'mnn where the percentile of T'mex is fixed to 0.05. After initializing, the model is trained in 500 epochs. For both proposed approaches. the trade-off factor set 10 to ensure decision loss as the main optimization objective. Dimensions of the GIN hidden layer and orthogonal projection layer are fixed as 16 and 8, respectively. About the backbone network, a 4-layer GIN and 3-layer fully connected neural network are adopted. Besides, the averages and standard deviations of the Area Under Operating Characteristic Curve (AUC) are used to support the comparable experiments by repeating each algorithm ten times. The higher value of the AUC metric represents better performance. When calculating the AUC of graph-kerne baselines, we estimated the radius of the hypersphere as 99 percentile of all squared distances to the separating hyperplane and then determined the score as the difference between squared distances and its square radius. Regarding our model DOHSC, we firstly set I epoch in the pretraining stage to initialize the center of the decision boundary and then train the model in 500 epochs. The convergence curves are given in Figure 7 The improved method DO2HSC is also set 1-epoch pretraining stage and trains DOHSC 5 epochs to initialize a suitable center and decision boundaries mmaz and Tmmnn where the percentile v of Tmaax is fixed to 0.1. After initializing, the model is also trained in 500 epochs. For both proposed approaches, the trade-off factor set to 10. Dimensions of the GIN hidden layer and orthogonal projection layer are fixed as 16 and 8, respectively. About the backbone network, 4-layer GIN and a 3-layer fully connected neural network are adopted. Besides, the averages and standard deviations of the Area Under Operating Characteristic Curve (AUC) are used to support the comparable experiments by repeating each algorithm ten times. The higher value of the AUC metric represents better performance. When calculating the AUC of graph-kerne basclines, we estimated the radius of the hypersphere as 99 percentile of all squared distances to the separating hyperplane and then determined the score as the difference between squared distances and its square radius.
E DOHSC AND DO2HSC ON NON-GRAPH DATA
Since our DOHSC and DO2HSC can also be applied to non-graph data such as images, here we compare them with some state-of-the-art anomaly detections Ruff et al. 20118; Goyal et al. (202));
15

---

Under review as a conference paper at ICLR 2023
Liznerski et al. (2021) Fashion-MNIST The results are reported in Table First, DOHSC and DO2HSC obtained seven best AUCs out of ten in total. And in the remainigg three classes, the proposed models still achieved comparable performance with gaps of less than 2%. Second, Deep SVDD plays an important baseline role relative to DOHSC and DOHSC defeats it by large margin in all classes. It further verifies that the proposed orthogonal projection is meaningful and helpful. In general, bi-hypersphere learning also performs sufficiently on common datasets and is very competitive compared to these state-of-the-art anomaly detection algorithms (Deep SVDD, DROCC and FCDD). In terms of the average AUC values for all classes of the dataset in Table B our algorithm outperforms all the compared baselines reproduced above, but worth mentioning that the reported performance of the IGD (Scratch) algorithm (Chen et al. 2022) is superior to our algorithm with no more than 2% gap.
Table 7: Average AUCs in one-class anomaly detection on Fashion-MNIST.
Normal Class
Deep SVDD (Rufr al. 2018
DROCC (Goyal
2020
FCDD
( Jznerski 2011
DOHSC
DOZHSC
T-shirt
03263 D.0342
08931 D.0072
08143 0.010)
D.9153
0.0022
0.9196
0,0064
Trouser
0.9632 1 0.0072
0.9835 1 0.0054
0,9655 1: 0,0014
0 1: 0.0060
0,9839 1: 0.0020
Pullover
0.7885 0.039%
0.866 0.0140
08423 1 0.0052
0.8007 1 0.02244
0,8768
0.0122
Dress
09143
0.0230
0.9171
0.0081
Cout
0.8417 1 0.0366
0.8533 0.0143
08607 1 0.0213
0.8805 1: 0.0258
0,9038
0.0140
Sandal
0,8902 1 0.0281
0,9336 0.0123
05089 1 0.0165
0.8952 : 0.0227
0.9308 :: 0.0070
Shirt
0.7507 0.0158
0.7789 0.0188
0,7750 0.0038
0.8177
0.0124
0.8022
0.0045
Sneaker
0.0062
0.0059
0.0007
0.0050
0.9677
0.0075
Bag
0.9039
D.0355
0.7797 0.0749
08584 : 0.0222
0.9122 : 0.0258
0,9090
0.0105
Ankle Boot
0,9188
1 0.0207
0.9589 1 0.0207
0,9132 1 0.0050
0
4 0.0127
0.9785
0.0038
Table Mean AUC of all classes on Mashion-MNISST
Deep SVDD
DROCC
FCDD
IGD CCen 2022
DOHSC
DO2HSC
Mcan AUC
0.8742
0.8879
0.8890
0.9201
0.9063
0,9189
F SUPPLEMENTED VISUALIZATION
This part shows the related supplemented visualization results of the anomaly detection task.
From Section 4 we can see some DOHSC results are improved a lot through DO2HSC. For ex- ample, compared with DOHSC, DO2HSC often improves the results by less than 2% on most of the datasets. But on class of ER.MD, DO2HSC has a more than 20% improvement. Here the distance distributions of DOHSC and DO2HSC on ER ED are given in Figure 8 to prove this im- provement. In Subfigure (a), the anomalous data appear in the distance interval [0. especially in the region close to the center, and less or even none of the normal data distributes in it. On the contrary, DO2HSC divided normal data and anomalous data more specifically, and both sides of the interval have anomalous data, as we assumed before.
03
Aecrata Date
08
0.
Noma Dama
08-
0.7
Frequnny Danaly
Fequunccy Densitt
0.0
0.6
0.3
0.4
84
02
01
0.1
Distanoe
x100"
Dislance
(a) DOHSC
(b) DO2HSC
Figure& Visualizations of DOHSC and DO2HSC on ER MD dataset.
16

---

Under review as a conference paper at ICLR 2023
83
6:65
06
$450
Diaaanc
330
Ronpenry Dentyy
649
300
040
Darsace
Damsne
Darance
(a) infoCiighh+DDeyySS/V SVDD
(h) DOHSC
(c) DOZHSC
Figure 9: Distance distributions were obtained by infoGraph+Deep SVDD, the proposed model, and the improved proposed model on COX2. The first row represents the distance distribution of the training samples relation to the decision boundary. The last row indicates the distance distribution of the test data with respect to the decision boundary.
Figure 9 shows the distance distributions of the two-stage method, the proposed model DOHSC, and the improved DO2HSC. Here, the distance is defined as the distance between the sample and the center of the decision hypersphere. The distance distribution denotes the sample proportion in this distance interval to the corresponding total samples. It can be intuitively observed that most distances of instances are close to the decision boundary because of the fixed learned representation. As mentioned earlier, the jointly-trained algorithm has mitigated the situation, and the obtained representation makes many instances have smaller distances from the center of the sphere.
However, like wrote in Section 2 anomalous data may occur in regions with less training data, especially the region close to the center, which is also confirmed by (a) and (b) of Figure 9 Dif- ferently, DO2HSC effectively shrinks the decision area, and we find that the number of outliers obviously reduced due to more compact distribution of training data.
F.1 PARAMETER SENSITIVITY AND ROBUSTNESS
To claim the stability of our models, we analyze the parameter sensitivity and robustness of DOHSC and DO2HSC, respectively. Consider that the projection dimension varies in {4, 8, 16, 32, 64, 128] while the hidden layer dimension of the GIN module ranges from 4 to 128. In Figure 11 the DO2HSC model has less volatile performances than DOHSC, especially when the training dataset is sampled from COX2 class O, as Subfigure (d) shows. Noticeably, higher dimension of the GIN hidden layer usually displays better AUC result since the quality of learned graph representations improves when the embedding space is large enough.
In addition, we assess different aspects of model robustness. More specifically, the AUC results about two "ratios" are displayed: 1) Different sampling ratios for the training set; 2) Different ratios of noise disturbance for the learned representation. In Subfigures (c) (I), the purple bars regard normal data as class O, while green bars treat normal data as class 1. Notice that most AUC results are elevating along with a higher ratio of authentic data in the training stage, demonstrating our models' potential in the unsupervised setting. On the other hand, when more noise is blended into the training dataset, the AUC performances of yellow line and blue line always stay stable at a high level. This outcome verifies our models' robustness in response the alien data.
17

---

Under review as a conference paper at ICLR 2023
(a) Training Resilt
(b) Testing Result
Figure 10: Visualization results of the DOHSC with MUTAG in different perspectives.
Prodedton
Dineeaiion
130%
(a)
(b)
(c)
Deneross
Deseuon
Deransonn
Deneuord CNN
(c)
(e)
(E)
Figure 11: Parameter sensitivity and robustness of the proposed models. (a)-(b) Parameter sensitivity of DOHSC with different hidden layer dimensions of GIN and projection dimensions on COX2 with Class 0 and Class respectively. (d)-(e) Parameter sensitivity of DO2HSC with the same settings. (c) and (f) shows the performance impacts with different ratios of the training set on the IMDB- Binary dataset and added noise disturbances on the MUTAG dataset while training DOHSC and DO2HSC, respectively.
The percentile parameter sensitivity is also given in this part. It is worth mentioning that we test DOHSC with varying percentile in (0.01,0.1,.... 0.8) and test DO2HSC only in {0.01, 0.05, 0.13 because two radii of DO2HSC is obtained by percentile and 1 Two radii will be equal when = 0.5 and the interval between two co-centered hyperspheres will disappear. From the table, the performance would decrease when larger percentile set obviously. For example, on the MUTAG
18

---

Under review as a conference paper at ICLR 2023
dataset, setting the percentile 0.01 is more beneficial for DOHSC than setting it as 0.8, and setting the percentile as 0.01 is better than setting as 0.1 for DO2HSC due to the change the interval area.
Table Parameter sensitivity of the proposed methods with different percentiles (all normal data
set to Class O.
Method
Dataseu/Percentile
0.01
0.1
0.5
0.8
MUTAG
0.9527 (0.0187)
0.9497 (0.0249)
0.9112 (0.0099)
0.8675 (0. (0.1287)
DOHSC
COX2
0.6041 (0.0661)
0.6022 (0.0789)
0.5232 (0.0494)
0.5523 (0.0572)
Method
Datasct/Percentile
0.01
0.05
0.1
MUTAG
0.9089 (0.0609)
0.8041 (0.1006)
0.6769 (0. (0.1207)
DO2HSC
COX2
0.6329 (0.0292)
0.6149 (0.0187)
0.5830 (0.0713)
G SUPPLEMENTED RESULTS OF ABLATION STUDY
First, the ablation study of whether orthogonal projection needs standardization is conducted. To be more precise, we are pursuing orthogonal features, i.c., finding a projection matrix for orthog- onal latent representation (with standardization) instead of computing the projection onto the col- umn or row space of the projection matrix nonnsttandardization) though they are closely related to each other. This is equivalent performing PCA and using the standardized principal components. Therefore, we show the comparison between DOHSC with standardization and without standard- ization. From Table TO, it observed that the performance of DOHSC without standardization is acceptable and most results of it are better than the two-stage baseline, i.e., infoGraph+ SVDD. verifies the superiority of the end-to-end method over the two-stage baselines. However, the model with standardization outperforms the non- non-standardized one in almost all cases.
Table 10: Comparison of the orthogonal projection layer with or w/o standardization.
Class
infoGraph+DDccp SVDD
DOHSC (Non-Standardization)
DOHSC
0
0.8805 + 0.0448
0.8521 4: 0.0650
0.8822 1: 0.0432
MUTAG
0.6166 + 0.2052
0.6918 + 0.1467
0.8115 0.0279
0
0.4825 + 0.0624
0.5800 + 0.0473
0.6263 + 0.0333
COX2
0.5029 0.0700
0.5029 + 0.0697
0.6805 0.0168
0
0.5312 + 0.1545
0.4881 + 0.0626
0.6620 + 0.0308
ER.MD
0.5682 0.0704
0.6740 + 0.0356
0.5184 0.0793
0
0.3942 + 0.0436
0.4029 + 0.0354
0.7083 + 0.0188
DD
0.6484 0.0236
0.6903 + 0.0215
0.7579 0.0154
0.6353 1 0.0277
0.5149 1 0.0655
0.6609 1 0.0033
IMDB- IMDB-Binary Binary
0.5836 0.0995
0.6505 1 0.0585
0.7705 0.0045
0
0.5662 + 0.0597
0.6067 +: 0.1007
0.9185 +: 0.0455
COLLAB
0.7926 0.0986
0.8958 + 0.0141
0.9755 0.0030
0.4062 0.0978
0.4912 0.2000
0.5450 0.0469
Besides, the ablation study of using the mutual information maximization loss is shown in Table be intuitively concluded that mutual information loss does not always have a positive impact on all data. This also indicates that the anomaly detection optimization method and orthogonal projection we designed are effective instead of entirely due to loss of mutual information.
To demonstrate the effectiveness of the orthogonal projection layer (OPL), we conduct ablation studies and visualize the comparison 3-dimensional results produced with OPI. and without OPL,, respectively. For each model trained particular dataset class, show the result without OPL on the left side, while the result with OPL displayed on the right. As Figure [2 illustrates, OPL drastically improves the distribution of the embeddings to be more spherical rather than elliptical. Similarly, with the help of OPL, other embeddings show a more compact and rounded layout.
19

---

Under review as a conference paper at ICLR 2023
Table 11: Comparison of the loss supervision with or w/o mutual information loss (MIL).
Class
DOHSC (Non-MILL
DOHSC
DO2HSC (Non-MIL))
DO2HSC
0,9456 0.0189
0.8822 + 0.0432
0.8308 0.0548
0.9089 0.0609
MUTAG
0.7597 0.0802
0.8115 -000279
0.7915 0.0274
0.8250 0.0790
0
0,6349 + 0.0466
0.6263 + 0.0333
0.6143 + 0.0302
0.6329 + 0.0292
COX2
1
0.6231 + 0.0501
0.6805 = 0.0168
0.6576 + 0.1830
0.6518 + 0.0481
0.5837 0.0778
0.6620 + 0.0308
0.5836 0.0909
0.6867 0.0226
ER MD
0.6465 0.0600
0.5184 = 0.0793
0.7424 0.0385
0.7351 0.0159
0
0.4738 + 0.0412
0.7083 :: 0.0188
0.6882 + 0.0221
0.7320 + 0.0194
1
0.7197 + 0.0185
0.7579 + 0.0154
0.7376 + 0.0244
0.7651 + 0.0317
0.5666 0.08
0.6609 = 0.0033
0.6303 0.0538
0.6406 0.0642
IMDB-Binary
0.6827 0.0239
0.7705 + 0.0045
0.6810 0.0276
0.7101 0.0429
0
0.9330 + 0.0539
0.9185 : 0.0455
0.5415 + 0,0182
0.6718 + 0.0353
COLLAB
1
0.9744 + 0.0017
0.9755 + 0.0030
0.9293 + 0.0023
0.9153 + 0.0070
0.8275 2
+ 0.0765
0.5450 + 0.0469
0.8452 + 0.0243
0.7188 + 0.0260
(a)
DOHSC
(b) DO2HSC
Figure 12: Visualizations on the MUTAG dataset Class 0 (left: with OPL; right: without OPL).
(a) DOHSC
(b) DO2HSC
Figure 13: Visualizations on the MUTAG dataset Class (left: with OPL; right: without OPL).
DOHSC
b DO2HSC
Figure 14: Visualizations on the COX2 dataset Class 0 (left: with OPL; right: without OPL).
20

---

Under review as a conference paper at ICLR 2023
(a) DOHSC
(b) DOZHSC
Figure 15: Visualizations on the COX2 dataset Class (left: with OPL; right: without OPL).
21


Reviews:
[{"review_id": "swbgB-Bwcgo", "reviewer": "Reviewer_P4g5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The work presents a new one-class classification method for graph-level anomaly detection. The new components added to the original one-class objective include a local-global graph representation learning, a SVD-based representation projection, and a bi-hypersphere learning. The first two newly added components are directly taken from existing work. The bi-hypersphere learning is new, as far as I know. The method is evaluated on six graph datasets and compared with graph kernel and GNN-based methods.", "summary_of_the_review": "The paper has some merits, but the cons outweigh the pros, as discussed above.", "strength_and_weaknesses": "Strengths of the paper are:\n- The work tackles an important yet under-explored problem -- graph-level anomaly detection. Unlike node anomaly detection, graph-level anomaly detection methods, especially deep neural network-based methods, are relatively limited.\n- The bi-hypersphere learning objective is an improved version of popular deep one-class classifiers, which is new, to the best of my knowledge.\n- The effectiveness of the SVD-based representation projection and the bi-hypersphere learning is justified via the ablation study.\n\nThe negative aspects of the paper include:\n- Although the paper is focused on graph-level anomaly detection, the key design or the newly proposed component (i.e., the bi-hypersphere learning) is generic and does not take into account of the graph-level graph mining tasks. The only component relevant to graph-level detection is the mutual information maximization between local and global representations, which is taken directly from existing work, such as Infograph. The main concern here is that the presented method is not designed specifically for graph-level anomaly detection. \n- The bi-hypersphere learning seems to be generalizable to different types of data. Results on image/tabular data would be important  to justify whether the main argument of the bi-hypersphere learning is effective. \n- There have been many variants of deep one-class classifier for learning more meaningful one-class models, such as [1-4]. They can be easily combined with loss functions like Infograph to adapt to graph-level anomaly detection. It's unclear what are the advantages of the proposed method compared to these more advanced one-class classification methods.\n- The datasets used are very different from the ones in the competing methods like GLocalKD and OCGTL. Their performance seems to be less effective than the ones reported in their original paper. What are the reasons/motivations behind? GLocalKD and OCGTL seem to work well on some commonly used graph datasets, e.g., PROTEINS, AIDS, and REDDIT. How is the performance of the proposed method on those datasets?\n- The argument that \"In contrast, there is little work on graph data despite the fact ...\" is invalid. To my knowledge, there have been many studies on anomalous node detection; less work is on graph-level anomaly detection.\n- The benefit of using the mutual information maximization loss is not examined\n- To my understanding, the method can be sensitive to the setting of the percentile parameter in eq. 14, but no empirical results are given about this sensitivity.\n\n**References**\n- [1] \"DROCC: Deep robust one-class classification.\" In International Conference on Machine Learning, pp. 3711-3721. PMLR, 2020.\n- [2] \"Learning and evaluating representations for deep one-class classification.\" arXiv preprint arXiv:2011.02578 (2020).\n- [3]  \"Explainable deep one-class classification.\" arXiv preprint arXiv:2007.01760 (2020).\n- [4] \"Deep one-class classification via interpolated gaussian descriptor.\" In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, pp. 383-392. 2022.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "oCX7dz9aB9s", "reviewer": "Reviewer_fADg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper first proposes a novel deep graph-level anomaly detection model, which learns the graph representation with maximum mutual information between substructure features and global structure features while exploring a hypersphere anomaly decision boundary. The numerical and visualization results on a few graph datasets demonstrate the effectiveness and superiority of the methods.", "summary_of_the_review": "I am unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.", "strength_and_weaknesses": "I am unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "ba49sLtuOb", "reviewer": "Reviewer_mmsV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors deal with outlier detection on graphs.\nThey start with the paper on InfoGraph \"INFOGRAPH: UNSUPERVISED AND SEMI-SUPERVISED\nGRAPH-LEVEL REPRESENTATION LEARNING VIA MUTUAL INFORMATION MAXIMIZATION\" by Sun et al , combined with SVDD, and propose two modifications.\n\n1. DOHSC adds on top of infograph+SVDD a singular vector standardization and a projection on the first dimensions corresponding to the largest k' = 8 directions on the right side of the SVD (equation 9) . (see notes in Strength and weaknesses, that the code does actually something different)\n\n2. DO2HSC replaces the SVDD objective by a two-sided objective which aims to identify outliers by samples which have a too large or too small distances from the center.\n\nThis comes with a 2 sided loss (penalizing too large and too small distances from the center), and calculates a distance measure which is positive if the sample distance from the center is either too large or too small.\n\n", "summary_of_the_review": "The paper proposes two interesting changes. The orthogonal projection could be a mistaken implementation, see weakness 1, which is a point of lack of clarity and quality which has to be addressed.\n\n*edit* after reading the rebuttal, the reviewer is agreeable to increase his rating to marginally above the accept threshold.", "strength_and_weaknesses": "Strengths:\nthe paper does several experiments on graph datasets.\n\nWeaknesses:\n\n1. the down projection does something else than the text states,\n\nwhen applying W = V_{k'} \\Lambda^{-1}_{k'}  \nto H= U \\Lambda V^\\top \nthen the result is:\nU_{k'} (in math) aka U[:,:d]  (in python/pytorch)\n\nIt is NOT U[:,:d] I_d V^\\top[:d,:] as what would be the orthogonal projection onto the first d dimensions of a SVD decomposition.\n\nWhat did the authors want to compute  ? \n If it is U[:,:d], then the approach needs to be renamed, as that is not an orthogonal projection onto the first dimensions of U[:,:d] I_d V^\\top[:d,:] .\n\n2. It is not clear whether the singular vector standardization (by \\Lambda^{-1}_{k'}) or the downprojection onto the first k' helps to improve DOHSC over infograph+SVDD. \nTo understand this, an experiment would be useful where one only projects onto the top k' dimensions in the Orthogonal_Projector module, without the .matmul(torch.linalg.inv(S)[:d,:d]) in the weightConstraint class  (or just .matmul(torch.eye(d))  )\n\n3. the outlier detection is based on class-labels only. Depending on the dataset this might cluster too easily compared to outliers in the wild. \n\n4. an analysis is missing why prohibiting too small distances from the center is a good idea.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1au1-dId6O", "reviewer": "Reviewer_bgGj", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a graph anomaly detection method through mutual information maximization. The key contributions are proposing (i) an orthogonal projection layer for the decision boundary correction and (ii) a two co-centered hyperspheres structure for estimating the normal distribution. Experimental results on multiple datasets show the effectiveness of the proposed method.", "summary_of_the_review": "This paper proposed a simple yet effective method for graph anomaly detection. Authors may consider the weakness shown above and revise the paper.", "strength_and_weaknesses": "Strength:\n1.\tThe proposed method is simple yet effective. It seems possible to apply the proposed orthogonal projection layer and the bi-hypersphere compression model to many other graph anomaly detection methods.\n2.\tThe paper is well-written and easy to follow. The main contributions are clearly summarized.\n3.\tExperiments on multiple datasets show the effectiveness of the proposed method.\nWeaknesses:\n1.\tThe motivation for the bi-hypersphere compression is still difficult to understand. The authors claim that anomalous data may appear in the empty inner decision region. But Figure 3 did not show this phenomenon. More importantly, the authors did not explain why bi-hypersphere compression can help. For example, if the normal area becomes more compact, how can we guarantee that anomalous data will not appear in the empty inner decision region? The authors may need to provide some comparisons to show the proposed method can indeed solve this problem.\n2.\tImprovements on some of the datasets seem unreasonable. For example, in Table3, compared with DOHSC, DO2HSC often improves the results by less than 2% on most of the datasets. But on class 1 of ER_MD, DO2HSC has a more than 20% improvement. Analysis needs to be provided. For example, it will be very interesting to show, on ER_MD, more anomalous data will appear in the empty inner decision region so that DO2HSC has an impressive improvement on this dataset.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject