Title: Rethinking Reconstruction-based Graph-Level Anomaly Detection: Limitations and a Simple Remedy
Venue: NeurIPS 2024
Accepted: yes
Keywords: graph-level anomaly detection

graph neural network

graph autoencoder
Abstract: Graph autoencoders (Graph-AEs) learn representations of given graphs by aiming to accurately reconstruct them. A notable application of Graph-AEs is graph-level anomaly detection (GLAD), whose objective is to identify graphs with anomalous topological structures and/or node features compared to the majority of the graph population. Graph-AEs for GLAD regard a graph with a high mean reconstruction error (i.e. mean of errors from all node pairs and/or nodes) as anomalies. Namely, the methods rest on the assumption that they would better reconstruct graphs with similar characteristics to the majority. We, however, report non-trivial counter-examples, a phenomenon we call reconstruction flip, and highlight the limitations of the existing Graph-AE-based GLAD methods. Specifically, we empirically and theoretically investigate when this assumption holds and when it fails. Through our analyses, we further argue that, while the reconstruction errors for a given graph are effective features for GLAD, leveraging the multifaceted summaries of the reconstruction errors, beyond just mean, can further strengthen the features. Thus, we propose a novel and simple GLAD method, named MUSE. The key innovation of MUSE involves taking multifaceted summaries of reconstruction errors as graph features for GLAD. This surprisingly simple method obtains SOTA performance in GLAD, performing best overall among 14 methods across 10 datasets.

Rethinking Reconstruction-based Graph-Level Anomaly Detection: Limitations and a Simple Remedy

Abstract

Graph autoencoders (Graph-AEs) learn representations of given graphs by aiming to accurately reconstruct them. A notable application of Graph-AEs is graph-level anomaly detection (GLAD), whose objective is to identify graphs with anomalous topological structures and/or node features compared to the majority of the graph population. Graph-AEs for GLAD regard a graph with a high mean reconstruction error (i.e. mean of errors from all node pairs and/or nodes) as anomalies. Namely, the methods rest on the assumption that they would better reconstruct graphs with similar characteristics to the majority. We, however, report non-trivial counterexamples, a phenomenon we call reconstruction flip , and highlight the limitations of the existing Graph-AE-based GLAD methods. Specifically, we empirically and theoretically investigate when this assumption holds and when it fails. Through our analyses, we further argue that, while the reconstruction errors for a given graph are effective features for GLAD, leveraging the multifaceted summaries of the reconstruction errors, beyond just mean, can further strengthen the features. Thus, we propose a novel and simple GLAD method, named MUSE. The key innovation of MUSE involves taking multifaceted summaries of reconstruction errors as graph features for GLAD. This surprisingly simple method obtains SOTA performance in GLAD, performing best overall among 14 methods across 10 datasets.

1 Introduction

Graph autoencoders (Graph-AEs) are a family of graph neural networks (GNNs) that learn latent representations of given graphs by aiming to accurately reconstruct them. Representative examples of Graph-AEs include GAE [22] and GraphMAE [15], which respectively aim to accurately reconstruct graph structure and node features. Graph-AEs have a broad spectrum of applications, such as anomaly detection [38, 33, 6] and link prediction [13, 22, 24].

Graph-AEs are trained to capture the structural information of graphs used for training (i.e., training graphs). Thus, intuitively, Graph-AEs are expected to better reconstruct graphs that are similar to the training graphs. However, to our surprise, this expectation is not always true. Given Graph-AEs trained to reconstruct the graphs in Figure 1(a), which share common structural characteristics, one would expect that the Graph-AEs would reconstruct the training graphs with smaller reconstruction errors than a dissimilar counterpart (e.g., in Figure 1 (b)). We, however, report the opposite. The reconstruction error is lower for the graph in Figure 1(b) than the training graphs. While similar observations were reported in computer vision application [31], which we elaborate on in Section 2.1, the existing literature does not clearly explain the aforementioned phenomenon.

This counterintuitive phenomenon guides our analysis, implication, and the proposed method. First, we aim to understand this counterintuitive phenomenon, which we refer to as the reconstruction flip .

Error: 0.706

Error: 0.709

Error: 0.710

(a) Training graphs

Error: 0.676

(b) Unseen graph

We argue that the phenomenon can occur when (1) the training graphs and unseen graphs share a primary structural pattern and (2) the pattern is more pronounced in unseen graphs than in training graphs. We corroborate our claim with our theoretical and empirical analysis.

Second, our finding has strong implications for graph-level anomaly detection (GLAD). GLAD aims to identify graphs of different characteristics (e.g., structures and/or features) from the majority [38, 33, 39, 32]. Many existing GLAD methods leverage Graph-AEs [33, 38]. They regard a graph with a high mean reconstruction error (i.e. mean of errors from all node pairs and/or nodes) as anomalies, assuming trained Graph-AEs would struggle to reconstruct the graphs with structural patterns (and/or node attributes) that deviate from most training graphs. However, our finding suggests that this assumption does not always hold. Specifically, anomalous graphs with structural characteristics distinct from the training graphs may exhibit similar or even lower mean reconstruction errors. In such cases, the existing methods would struggle to detect anomalous graphs. We argue that this issue arises from how the existing methods leverage reconstruction errors, yet these errors remain valuable graph features for GLAD. We propose using multifaceted summaries of the reconstruction errors, rather than relying solely on their mean, to further enhance the features.

Third, based on the analysis and implication, we propose a simple and novel GLAD method, MUSE ( Mu ltifacted S ummarization of Reconstruction E rrors). MUSE, like other Graph-AE-based GLAD methods, reconstructs given graphs. However, MUSE employs a new way of using reconstruction errors: representing a graph with multifaceted summaries (e.g., mean, standard deviation, etc.) of the graph's reconstruction errors for GLAD. This simple innovation allows MUSE to obtain SOTA performance in the GLAD tasks. Through our extensive experiments including 13 baseline methods and 10 benchmark datasets, MUSE performs overall best, achieving up to 28.1% performance gain (in terms of AUROC) compared to the best competitor. We summarize our contribution as follows:

Analysis (Section 3) with implications (Section 4): We investigate the reconstruction flip phenomenon both theoretically and empirically, yielding practical implications for GLAD.

Effective method (Section 5): Motivated by our analysis, we propose MUSE, a simple yet effective GLAD method that represents a graph as multifaceted summaries of its reconstruction errors.

Extensive experiments (Section 6): Our experiments on 10 datasets demonstrate the superiority of MUSE over prior GLAD methods. Our code and datasets are available at https://github. com/kswoo97/GLAD_MUSE .

2 Related Work and Preliminaries

In this section, we provide a brief literature review on graph-level anomaly detection (GLAD) and peculiar observations in GLAD. We then present the preliminaries of our work.

2.1 Related work

Graph-level anomaly detection. Graph-level anomaly detection (GLAD) aims to identify graphs whose characteristics deviate from those of the majority of the graph population [38, 33, 39, 32]. GLADhas various applications, including brain diagnosis [25] and drug discovery [46]. While several GLADmethods require anomaly label supervision to train a detector [54, 35], we focus on the methods that do not require anomaly labels [38, 33, 39, 32], since anomaly labels are often not available in real-world scenarios. A GLAD method typically trains a model to perform certain pretext tasks on given graphs, and the graphs are regarded as anomalies if the trained model shows poor pretext task

performance on them. Notable pretext tasks include graph reconstruction [38, 33], graph embedding hypersphere minimization [39, 57], and cross-view mutual information maximization [34, 32].

Peculiar observations in graph-level anomaly detection. The most relevant literature with our work is a study of Zhao and Akoglu [56]. In graph classification datasets, they regarded graphs belonging to a particular class as normal graphs and otherwise as anomalies to benchmark several GLAD methods. In this setting, they observed that several GLAD methods, especially kernel-based ones, occasionally perform worse than random guessing. However, their analysis focused on graph kernels [37, 41], without covering graph autoencoders, which are our focus. In computer vision, Liu et al. [31] suggested that certain anomalous images can be easier to reconstruct than normal ones, presenting a method that can overcome these undesirable circumstances. However, neither their analysis nor the method can be trivially extended to the graph domain, as elaborated in Appendix F.1.

2.2 Preliminary

Graphs. A graph G = ( V , E ) is defined by a node set V = { v 1 , . . . , v |V| } and an edge set E = { e 1 , . . . , e |E| } , where each edge e i ∈ E is defined by a node pair. We assume that each node v i ∈ V is equipped with a feature vector x i ∈ R d , which forms a node feature matrix X ∈ R |V|× d where each i -th row X i, : = x i . Moreover, E can be represented by an adjacency matrix A ∈ { 0 , 1 } |V|×|V| , where A i,j = 1 if and only if { v i , v j } ∈ E . Therefore, the graph can also be defined as G = ( X , A ) .

Graph neural networks. Graph neural networks (GNNs) are a category of neural networks designed for processing graph data. GNNs primarily leverage message passing schemes [49, 14, 45, 11, 28, 30]. In this work, we consider GNNs as parameterized functions f θ that generate node embeddings Z = f θ ( X , A ) ∈ R |V|× d ′ for each graph G = ( X , A ) . 1

Graph autoencoders. Graph autoencoders (Graph-AEs) are a family of GNNs that learn graph latent representations by graph reconstruction. Specifically, Graph-AEs reconstruct a given graph's structure [22, 43] and/or node features [15, 16]. For example, GAE [22] reconstructs the adjacency matrix of a given graph. Specifically, GAE first uses an encoder GNN f θ to generate node embeddings Z = f θ ( X , A ) of a given graph G = ( X , A ) . Then, GAE obtains a reconstructed adjacency matrix ˆ A ∈ R |V|×|V| through the inner-product of node embeddings and entry-wise sigmoid activation σ (i.e., ˆ A = σ ( ZZ T ) ). Lastly, it minimizes the difference between A and ˆ A , for which one can use the binary cross-entropy (BCE) loss L BCE ( G | f θ ) := -∑ |V| i,j =1 ( A i,j log ˆ A i,j +(1 -A i,j ) log (1 -ˆ A i,j ) ) or the squared Frobenius-norm loss L SFN ( G | f θ ) := ∥ A -ˆ A ∥ 2 F = ∑ |V| i,j =1 ( A i,j -ˆ A i,j ) 2 . Given a set of training graphs G = {G 1 , G 2 , · · · , G | G | } , GAE is trained (i.e., the encoder parameter θ is updated) to minimize ∑ G∈ G L ( G | f θ ) (or equivalently, E G∈ G L ( G | f θ ) ), where L is L BCE or L SFN . In our analysis below, we focus on methods that reconstruct graph structures, while we describe and investigate node-feature-reconstruction methods in Appendix E.1.

3 Analysis of Graph Autoencoders

In this section, we explore reconstruction flip phenomena (e.g., the phenomenon described in Figure 1), where graph autoencoders (Graph-AEs) reconstruct some unseen graphs that are dissimilar to training graphs better than training graphs.

Our analysis focuses on evidencing the claims below. We assume GAE [22] trained on graphs sharing a primary pattern P of strength S .

Claims

Reconstruction flip tends to occur when unseen (test) graphs have the same primary pattern P but with a greater strength S ′ > S .

̸

Reconstruction flip tends not to occur when unseen (test) graphs have a different primary pattern P ′ = P .

BCE loss

Epochs

Epochs

Epochs

Epochs

Com-Com dataset

Cycle-Cycle dataset

BCE loss

Unseen graphs

(same pattern but

stronger strength)

Training graphs

(same pattern but

weaker strength)

BCE loss

Epochs

Epochs

Epochs

Epochs

Com-Cycle dataset

Cycle-Com dataset

BCE loss

Unseen graphs

(distinct pattern)

Training graphs

(same pattern)

̸

That is, when Graph-AEs are trained on graphs with a primary pattern P of strength S , the trained Graph-AEs tend to exhibit lower errors in reconstructing graphs with P of a greater strength S ′ > S . However, the trained Graph-AEs tend to exhibit higher errors in reconstructing graphs with a different pattern P ′ = P . For demonstration, we elaborate on P and S via synthetic datasets (Section 3.1) and present both empirical (Section 3.2) and theoretical (Section 3.3) investigations.

3.1 Synthetic graphs

To elaborate on the concepts of a primary pattern P and a pattern strength S , we employ two types of synthetic graphs: (1) graphs with community structures and (2) graphs with cycles.

Community type. Syn-Com graphs have community structures, a pervasive pattern in real-world graphs [12], as the primary pattern P . Graphs in Figure 1 are instances of such graphs. We control the strength S of community structures through a parameter τ ∈ [0 , 1] . The 10 nodes in each graph are evenly divided into two communities. Each intra- and inter-community edge is formed with probability (1 + τ ) / 2 and (1 -τ ) / 2 , respectively. In our empirical analysis, we consider two graph classes with different τ 's. Graphs in one class G com τ =0 . 4 are generated with τ = 0 . 4 (weaker community structure; the graphs in Figure 1(a)) and those in the other class G com τ =0 . 8 are generated with τ = 0 . 8 (stronger community structure; the graph in Figure 1(b)).

Cycle type. Syn-Cycle graphs contain a cycle of nodes (i.e., a closed chain of nodes), which is commonly observed in realworld graphs [8]. In Syn-Cycle, the primary pattern P corresponds to the node cycle. Graphs in Figure 4 are instances of Syn-Cycle. In Syn-Cycle graphs, the pattern strength S is stronger in the clean-cycle graph consisting only of a cycle with n nodes (the left graph in Figure 4), and it is weaker in noisy-cycle graphs, where a node within the cycle of ( n -1 )

Clean cycle

Noisy cycle

nodes is linked to an extra node (the right graph in Figure 4). The number of nodes n is fixed to 10 .

We provide detailed descriptions of both graph types in Appendix C.

3.2 Empirical analysis

Setting. We consider two scenarios. In Scenario 1, we aim to validate our first claim. To this end, we leverage (1) the Com-Com dataset, where the training graphs are sampled from G com τ =0 . 4 and the unseen graphs are sampled from G com τ =0 . 8 , and (2) the Cycle-Cycle dataset, where the training graphs are sampled from the noisy-cycle graphs and the unseen graph is the clean-cycle graph.

In Scenario 2, we aim to validate our second claim. To this end, we leverage (3) the Com-Cycle dataset, where the training graphs are sampled from G com τ =0 . 4 and the unseen graphs are sampled from the noisy-cycle graphs, and (4) the Cycle-Com dataset, where the training graphs are sampled from the noisy-cycle graphs and the unseen graphs are sampled from G com τ =0 . 4 .

For each dataset, we train GAE [22] equipped with GIN [49] as the graph encoder. We use either the (1) binary cross-entropy (BCE) loss or the (2) squared Frobenius-norm (Frobenius) loss, which are formalized in Section 2.2, for the reconstruction loss. We compare the reconstruction losses for the training graphs and unseen graphs. Further details are provided in Appendix D.2.

Note. While we focus on adjacency reconstruction in this section, we provide an analysis of featurereconstruction methods in Appendix E.1, where the results are consistent with those below.

Empirical results. In Scenario 1, for both Com-Com and Cycle-Cycle datasets, reconstruction losses are lower for the graphs with stronger pattern strengths, which are unseen graphs (see Figure 2). That is, all graphs share a primary pattern P (either node community or cycle) with two different strengths S < S ′ , and GAE trained on graphs of pattern strength S reconstructs those of pattern strength S ′ with lower losses, demonstrating our first claim. In Scenario 2, for both Com-Cycle and Cycle-Com datasets, reconstruction losses become lower for the training graphs after some training epochs (see Figure 3). That is, given training graphs with a primary pattern P (either node communities or a cycle), the trained GAE reconstructs graphs with the same pattern P with lower losses than those with a different pattern P ′ , demonstrating our second claim.

3.3 Theoretical analysis

In this subsection, we theoretically analyze the empirical observations in Section 3.2, focusing on the occurrence of reconstruction flip. Informally speaking, we shall show that when GAE is trained on graphs with a primary pattern P of strength S , the following holds:

( A1) Reconstruction losses decrease for graphs with the same P of various strengths.

( A2) Reconstruction losses decrease more rapidly for graphs with the same P of greater strengths.

Setting. For theoretical analysis, we use a single-layer linear GAE. Formally, for a given graph G = ( X , A ) , the linear GAE reconstructs the given graph's adjacency matrix as follows: ˆ A = AXW ( AXW ) T , where W ∈ R n × n is a learnable weight matrix and n is the number of nodes. We use the squared Frobenius norm as the reconstruction loss L ( G , W ) = ∥ A -ˆ A ∥ 2 F . We consider Syn-Com graphs G com τ (Section 3.1). To formalize the training of linear GAE, we define the expected gradient descent update of W as follows:

where γ > 0 is a learning rate and E G [ ·| τ ] takes the expectation over all the graphs G ∈ G com τ . We assume graphs of strength τ 1 are used as the training graphs and graphs of strength τ 2 are used as the unseen (test) graphs.

Theoretical results. We now present our theoretical results, with proof of each theorem in Appendix A. We first demonstrate that ( A1) holds by expectation.

Theorem 1 (Generalization across τ ) . For any n ≥ 12 and 0 < τ 1 ≤ 1 , there exists ϵ > 0 such that for any 0 < γ ≤ ϵ and any 0 < τ 2 ≤ 1 (recall that 0 and 1 are the lower and upper bounds of τ , respectively) with the initial weight matrix W (0) = I n ,

Roughly, Theorem 1 states that regardless of the values of τ 1 and τ 2 , updating W with graphs of strength τ 1 reduces the reconstruction losses for graphs of strength τ 2 . Intuitively, this suggests that GAE learns the primary pattern P , regardless of the strength S (here, τ 1 ) of the training graphs.

Now, we demonstrate that ( A2) holds by expectation.

(a) Graph

G

1

(b) Graph

G

2

Reconstruction error (BCE)

Density

distributions of

G

1

(blue) and

G

2

(red).

Theorem 2 (Correlation with τ ) . For any n ≥ 34 and 0 < τ 1 ≤ 0 . 98 , there exists ϵ > 0 such that for any 0 < γ ≤ ϵ and any τ 1 +0 . 02 ≤ τ 2 ≤ 1 with the initial weight W (0) = I n ,

E G [ L ( G , W (0) ) -L ( G , W ( τ 1 , W (0) , γ )) | τ 1 ] ︸ ︷︷ ︸ the decrease in loss on graphs of strength τ 1 (training graphs) < E G [ L ( G , W (0) ) -L ( G , W ( τ 1 , W (0) , γ )) | τ 2 ] ︸ ︷︷ ︸ the decrease in loss on graphs of strength τ 2 (unseen graphs) .

Roughly, Theorem 2 states that updating W using graphs of pattern strength τ 1 results in a greater reduction of reconstruction losses for graphs of higher strength τ 2 > τ 1 than for the graphs used in W update (i.e., graphs of strength τ 1 ). Intuitively, this result suggests that even if GAE is trained on graphs with a primary pattern P of strength S (here, τ 1 ), the trained GAE tends to reconstruct graphs of a greater strength S ′ (here, τ 2 ) with smaller losses.

4 Implications in Graph-Level Anomaly Detection

In this section, we investigate the practical implications of our analysis in Section 3 for graph-level anomaly detection (GLAD). Specifically, we discuss the limitations of existing graph-autoencoderbased GLAD methods and propose an improved approach to using graph reconstruction errors.

4.1 Limitations of existing Graph-AEs in GLAD

Anomalous graphs may have low mean reconstruction errors. As outlined in Section 2.1, many methods for graph-level anomaly detection (GLAD) use graph autoencoders (Graph-AEs), aiming to identify graphs whose characteristics deviate from the majority. Typically, such GLAD methods [33, 38] consider graphs with higher reconstruction losses (i.e. mean (or sum) of reconstruction errors from all node pairs and/or nodes) as anomalies. This is based on the intuition that Graph-AEs would struggle to reconstruct anomalous graphs that are dissimilar to the majority of training graphs. However, our empirical (Section 3.2) and theoretical (Section 3.3) analyses reveal that Graph-AEs can exhibit lower mean reconstruction errors for graphs that are less similar to training graphs. In the context of GLAD, when anomalous graphs have the same primary pattern as training graphs but have stronger strengths, Graph-AEs tend to exhibit lower mean reconstruction errors, compared to the training graphs. Thus, such anomalies tend not to be detected by existing Graph-AE-based GLAD methods, revealing their severe limitations.

Mean is not all you need. In addition in practice, we find that in some cases, dissimilar graphs can have similar mean reconstruction errors. As shown in Figure 5, although the graph G 1 in (a) and the graph G 2 in (b) have significantly different numbers of edges, their mean reconstruction errors are similar, making it difficult to distinguish between the two graphs (refer to the caption of Figure 5). On the other hand, Figure 5(c) shows that the reconstruction error distributions over individual node pairs 2 of G 1 (blue distribution) and G 2 (red distribution) differ significantly in shape. Other descriptive statistics, such as standard deviation, can effectively distinguish the two graphs in such cases. Thus, incorporating additional statistical measures beyond the mean is beneficial for Graph-AEs in GLAD.

4.2 Enhancing the use of reconstruction errors

Based on the implications discussed in Section 4.1, we argue that a graph's reconstruction errors aggregated in various ways can provide features that effectively distinguish anomalous graphs from normal graphs. We provide detailed descriptions of this claim below.

̸

Reconstruction errors serve as good features for GLAD. Consider the following scenarios where Graph-AEs are trained on graphs sharing a pattern P of strength S . First, for an unseen graph of a pattern P with higher strength S ′ > S , Graph-AEs tend to return lower mean reconstruction errors compared to those of the training graphs (Figure 2). Second, for an unseen graph of a different pattern P ′ = P , Graph-AEs tend to give higher mean reconstruction error than those of the training graphs (Figure 3). In both cases, the error distributions of anomalous graphs differ from those of normal graphs, serving as effective features for distinguishing them.

Using multifaceted summaries of errors is important for GLAD. In practice, we need a fixed-size representation of the error distribution to be used with anomaly classifiers, which generally require a fixed-size input vector. To achieve this, we propose using various summary statistics (e.g., mean and standard deviation) to represent the given error distribution. Specifically, rather than relying on a single statistic, we use multifaceted summaries that capture various aspects of the error distributions. This is because distinct error distributions may share certain similar summary statistics, as shown in Figure 5. Therefore, multifaceted summaries of the error distribution can act as an effective feature vector for distinguishing anomalous graphs from normal graphs.

5 Proposed Method: MUSE

Based on the discussions provided in Section 4.2, we present MUSE ( Mu ltifacted S ummarization of Reconstruction E rrors), a simple yet effective graph-level anomaly detection (GLAD) method. MUSE aggregates a graph's reconstruction errors into multiple summary statistics and uses a vector of these aggregated errors as the graph's representation for GLAD. We describe how we train the reconstruction model in Section 5.1 and how we obtain the error representation (i.e., a multifaceted summary of reconstruction errors) in Section 5.2.

5.1 Step 1: Training a reconstruction model

We first train MUSE to reconstruct graphs in training data. Step 1 is composed of three parts: (1) augmentation, (2) encoding/decoding, and (3) reconstruction loss minimization. For an input graph G = ( X , A ) , MUSE first augments the input graph by randomly dropping some edges, mitigating potential overfitting problems. Specifically, MUSE randomly samples ⌈ p |E|⌉ edges from E for some 0 ≤ p < 1 . Let E ′ denote the set of sampled edges. We mask the corresponding entries in A by setting them to zero to generate the augmented adjacency matrix A ′ ∈ { 0 , 1 } |V|×|V| (i.e., A ′ i,j = 0 if { v i , v j } ∈ E ′ , otherwise A ′ i,j = A i,j, , ∀ i, j ∈ [ |V| ] ).

MUSE then encodes the augmented graph G ′ = ( X , A ′ ) to obtain node embeddings Z ∈ R |V|× d ′ by using a graph neural network (GNN) encoder f θ (i.e., Z = f θ ( X , A ′ ) ). Subsequently, MUSE decodes the node embeddings Z to obtain reconstructed node features ˆ X ∈ R |V|× d and reconstructed adjacency matrix ˆ A ∈ (0 , 1) |V|×|V| . For node feature reconstruction, MUSE uses a node feature decoder g ψ : R d ′ ↦→ R d (i.e., ˆ X = g ψ ( Z ) ). Similarly, for adjacency matrix reconstruction, MUSE uses an adjacency matrix decoder h ϕ : R d ′ ↦→ R d ′′ and the inner product of its output (i.e., ˆ A = σ ( Z ′ Z ′ T ) , where σ is a sigmoid function and Z ′ = h ϕ ( Z ) ).

Lastly, for training, we leverage reconstruction losses regarding the node features and the adjacency matrix. For node feature reconstruction, we use the cosine-similarity loss as in GraphMAE [15]. Formally, the loss is defined as follows:

For adjacency-matrix reconstruction, we use the binary cross-entropy (BCE) loss as in GAE [22]. Since real-world graphs are often sparse (i.e., the adjacency matrix contains significantly more zero

entries than ones), we weight the loss terms associated with non-zero entries to stabilize model training in the presence of this imbalance. Formally, the loss is defined as follows:

where ω = ( |V| 2 ∑ |V| i =1 ∑ |V| j =1 A i,j -1) τ and τ is a scale hyperparameter. Then, the total reconstruction loss L ( G ) is defined as the mean of L X ( G ) (Eq. (2)) and L A ( G ) (Eq. (3)) (i.e., L ( G ) = ( L X ( G ) + L A ( G )) / 2 ). For a set of training graphs G = {G 1 , · · · , G K } , we update the parameters (i.e., θ, ψ, and ϕ ) by using gradient descent aiming to minimize 1 K ∑ t ∈ [ K ] L ( G t ) .

5.2 Step 2: Generating error representation

After training the reconstruction model, we represent each graph using its reconstruction errors, specifically those from all node pairs and/or nodes. Consider obtaining the reconstruction errors for a graph G = ( X , A ) with the trained reconstruction model (i.e., f θ , g ψ , and h ϕ ). To this end, we first obtain reconstructed features ˆ X and a reconstructed adjacency matrix ˆ A by using the encoding and decoding scheme described in Section 5.1. Note that we do not perform augmentation for Step 2 (i.e., the original graph ( X , A ) is the input of the encoder f θ ).

After that, we obtain node-feature reconstruction errors L X ( G | f θ , g ψ , h ϕ ) on each node and adjacency-matrix reconstruction errors L A ( G | f θ , g ψ , h ϕ ) on each node pair. Formally,

Lastly, based on the motivation described in Section 4, we aggregate reconstruction errors (i.e., ' summarize ' vectors into scalars) using T different aggregation functions Agg t 's as follows:

Any aggregation functions that provides a representative summary of L X ( G ) and L A ( G ) are applicable, such as mean and standard deviation. As a result, we represent a graph G as a 2 T -dimensional vector Err ( G ) (Eq. (6)), which we refer to as the final error representation of G . We present a time complexity analysis of MUSE in Appendix F.3.

After obtaining error representations of graphs, we leverage one-class classifiers on these representations to finally perform anomaly detection. Note that Steps 1 and 2 are decoupled from the application of one-class classifiers (i.e., one-class classifier is trained after completing Steps 1 and 2).

6 Experiments

In this section, we evaluate the effectiveness of MUSE in graph-level anomaly detection (GLAD) by addressing four key research questions.

RQ1. How accurately does MUSE detect anomalous graphs?

RQ2. How robust is MUSE against contamination of the training set?

RQ3. Can the error representations of MUSE distinguish anomalies from normal graphs in the representation space?

RQ4. Are the key components of MUSE essential for its performance?

6.1 Experimental settings

Datasets. Given the absence of benchmark datasets with ground-truth graph-level anomalies, following existing GLAD studies [39, 32, 33, 55, 34, 38], we use graph classification benchmark datasets

for evaluation. Specifically, we use 10 datasets from diverse domains, such as chemical molecules, bioinformatics, and social networks. Detailed descriptions of the datasets are in Appendix B.

Training and evaluation. Following Qiu et al. [39], for a dataset with C graph classes, we use C experimental configurations. In each configuration, graphs with one class are treated as normal, while graphs with the other classes are considered anomalies. For each configuration, the normal graphs are split into training, validation, and test sets in an 80%/10%/10% ratio. Additionally, 5% of anomalies are sampled for the validation set (only for hyperparameter tuning) and 5% for the test set. We conduct five trials for each configuration, with each trial using different data splits and model initializations. Each model's performance on each dataset is evaluated by averaging the test mean AUROC across all configurations.

Baseline methods and MUSE. We compare MUSE against 13 baseline methods, including 7 GLAD methods [39, 32, 33, 55, 34, 38, 6] and 6 graph self-supervised learning (SSL) methods [15, 22, 53]. Among SSL methods, those with '-1' in their names are one-stage SSL methods that use SSL losses as anomaly scores. Those with '-2' are two-stage SSL methods that (1) obtain graph embeddings the models learn via SSL and (2) employ a one-class classifier. For aggregation functions of MUSE (Eq. (6)), we use two aggregation functions: mean and standard deviation. Therefore, we represent each graph with a 4 -dimensional vector, which is formalized in Appendix D.4. All methods use GIN [49] as a graph encoder. All two-stage approaches use an MLP autoencoder as a one-class classifier. Details of two-stage methods and the MLP autoencoder are in Appendix D.3. In addition, we provide further experimental details, including hyperparameter configurations, in Appendix D.

6.2 RQ1. Performance in graph-level anomaly detection

As shown in Table 1, MUSE outperforms all baseline methods in terms of average ranking. Two points stand out. First, in 8 out of 10 datasets, MUSE outperforms all other GLAD methods. Compared to the second-best GLAD method (OCGTL), MUSE achieves up to 16.2% performance gain (in the NCI1 dataset). Second, in 8 out of 10 datasets, MUSE outperforms all the SSL-based two-stage baselines (GraphCL-2, GAE-2, and GraphMAE-2). Given that they use the same one-class classifier with MUSE, this result demonstrates the effectiveness of the proposed error representation (Eq. (6)).

6.3 RQ2. Robustness against the training set contamination

In real-world scenarios, training data may also contain anomalous graphs, making the robustness of GLAD methods against training set contamination crucial for their practical effectiveness. To assess this robustness, we inject anomalous graphs into the training set at rates of 10%, 20%, and 30%. 3 As shown in Figure 6, MUSE exhibits the least performance drop, among the three strongest GLAD methods, demonstrating its robustness.

AIDS

IMDB

NCI1

ER

Test AUROC

Anomaly injection ratio

MuSE

(ours)

OCGTL

(baseline)

GLAM

(baseline)

Anomaly injection ratio

Anomaly injection ratio

Anomaly injection ratio

6.4 RQ3. Error representation visualization

We evaluate whether the error representation offered by MUSE can effectively distinguish graphs of different classes. We sample half of the graphs belonging to the same class to train MUSE. Then, we visualize the error representations generated by the trained MUSE for the rest half and graphs sampled from another class. As shown in Figure 7, the error representations produced by MUSE are distinguishable across different classes.

6.5 RQ4. Ablation study

AIDS

Trained on class 0

Trained on class 1

Class 0 graphs

Class 1 graphs

DD

Reddit

Protein

Lastly, we provide an ablation study of MUSE. We aim to justify the two key components of MUSE: (1) the reconstruction errors L X and L A , and (2) multiple aggregation functions (specifically, mean and standard deviation). To this end, we leverage the following four variants of MUSE:

( V1) MUSE w/o L X : A variant that only leverages adjacency matrix reconstruction,

( V2) MUSE w/o L A : A variant that only leverages node feature reconstruction,

( V3) MUSE w/o STD : A variant that only leverages mean aggregation,

( V4) MUSE w/o AVG : A variant that only leverages standard-deviation aggregation.

As shown in Table 1, MUSE with all components outperforms all of its variants in 8 out of 10 datasets, justifying the design choice of MUSE.

7 Discussions

In this work, we report and analyze an intriguing phenomenon, reconstruction flip of graph autoencoders (Graph-AEs). We investigate the phenomenon theoretically and empirically (Section 3) and further claim their implication in graph-level anomaly detection (GLAD; Section 4). Based on our analysis, we propose a novel GLAD method, MUSE (Section 5), and report its superior performance in GLAD via extensive experiments (Section 6). Below, we conclude the paper by discussing some potential limitations of our research, which could be addressed in future work.

More diverse and general primary patterns. In our analysis, we focus on the two patterns: community structures and a node cycle (Section 3). Although our claims hold for these patterns, a wide range of graph patterns remains unexplored in this work. Furthermore, a unified framework that can incorporate various graph patterns into a single pattern may exist. Further investigation into these patterns may improve our analysis, leading to new potential applications.

Scalability improvement. Since MUSE reconstructs the entire adjacency matrix of a graph (Eq. (3)), its time complexity for reconstruction is O ( n 2 ) , where n is the number of nodes in the graph. While this complexity is manageable for many real-world graph-level datasets (refer to Table 2), leveraging MUSE for large-scale graphs can be challenging. A simple technique that reconstructs only a sampled subset of an adjacency matrix results in a performance drop (refer to Appendix E.5). Therefore, improving the scalability of MUSE while preserving its detection performance is a non-trivial task, making it a promising direction for future research.

Acknowledgements. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2024-00406985) (90%). This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2019-II190075, Artificial Intelligence Graduate School Program (KAIST)) (10%).

References

K. M. Borgwardt, C. S. Ong, S. Schönauer, S. Vishwanathan, A. J. Smola, and H.-P. Kriegel. Protein function prediction via graph kernels. Bioinformatics , 21(1):i47-i56, 2005.

Z. Chen, Y. Fu, Y.-X. Wang, L. Ma, W. Liu, and M. Hebert. Image deformation meta-networks for one-shot learning. In CVPR , 2019.

W.-L. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, and C.-J. Hsieh. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In KDD , 2019.

M. Choe, S. Kim, J. Yoo, and K. Shin. Classification of edge-dependent labels of nodes in hypergraphs. In KDD , 2023.

J. Dai, H. Qi, Y . Xiong, Y . Li, G. Zhang, H. Hu, and Y . Wei. Deformable convolutional networks. In ICCV , 2017.

K. Ding, J. Li, R. Bhanushali, and H. Liu. Deep anomaly detection on attributed networks. In SDM , 2019.

P. D. Dobson and A. J. Doig. Distinguishing enzyme structures from non-enzymes without alignments. Journal of molecular biology , 330(4):771-783, 2003.

I. J. Farkas, I. Derényi, A.-L. Barabási, and T. Vicsek. Spectra of 'real-world' graphs: Beyond the semicircle law. Physical Review E , 64(2):026704, 2001.

S. Freitas, Y . Dong, J. Neil, and D. H. Chau. A large-scale database for graph representation learning. In NeurIPS , 2021.

H. Gao and S. Ji. Graph u-nets. In ICML , 2019.

J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chemistry. In ICML , 2017.

M. Girvan and M. E. Newman. Community structure in social and biological networks. Proceedings of the national academy of sciences , 99(12):7821-7826, 2002.

Z. Guo, F. Wang, K. Yao, J. Liang, and Z. Wang. Multi-scale variational graph autoencoder for link prediction. In WSDM , 2022.

W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. In NeurIPS , 2017.

Z. Hou, X. Liu, Y. Cen, Y. Dong, H. Yang, C. Wang, and J. Tang. Graphmae: Self-supervised masked graph autoencoders. In KDD , 2022.

Z. Hou, Y. He, Y. Cen, X. Liu, Y. Dong, E. Kharlamov, and J. Tang. Graphmae2: A decodingenhanced masked self-supervised graph learner. In WWW , 2023.

J. Kazius, R. McGuire, and R. Bursi. Derivation and validation of toxicophores for mutagenicity prediction. Journal of medicinal chemistry , 48(1):312-320, 2005.

S. Kim, D. Lee, Y. Kim, J. Park, T. Hwang, and K. Shin. Datasets, tasks, and training methods for large-scale hypergraph learning. Data Mining and Knowledge Discovery , 37(6):2216-2254, 2023.

S. Kim, S. Kang, F. Bu, S. Y. Lee, J. Yoo, and K. Shin. Hypeboy: Generative self-supervised representation learning on hypergraphs. In ICLR , 2024.

S. Kim, S. Y. Lee, Y. Gao, A. Antelmi, M. Polato, and K. Shin. A survey on hypergraph neural networks: an in-depth and step-by-step guide. In KDD , 2024.

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR , 2015.

T. N. Kipf and M. Welling. Variational graph auto-encoders. In NeurIPS Worksop of Bayesian Deep Learning , 2016.

T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICLR , 2017.

G. Kollias, V. Kalantzis, T. Idé, A. Lozano, and N. Abe. Directed graph auto-encoders. In AAAI , 2022.

T. Lanciano, F. Bonchi, and A. Gionis. Explainable classification of brain networks via contrast subgraphs. In KDD , 2020.

S. Laue, M. Mitterreiter, and J. Giesen. Computing higher order derivatives of matrix and tensor expressions. In NeurIPS , 2018.

J. Lee, S. Kim, and K. Shin. Slade: Detecting dynamic anomalies in edge streams without labels via self-supervised learning. In KDD , 2024.

S. Y. Lee, F. Bu, J. Yoo, and K. Shin. Towards deep attention in graph neural networks: Problems and remedies. In ICML , 2023.

S. Y. Lee, S. Kim, F. Bu, J. Yoo, J. Tang, and K. Shin. Feature distribution on graph topology mediates the effect of graph convolution: Homophily perspective. In ICML , 2024.

L. Liang, S. Kim, K. Shin, Z. Xu, S. Pan, and Y. Qi. Sign is not a remedy: Multiset-to-multiset message passing for learning on heterophilic graphs. In ICML , 2024.

W. Liu, H. Chang, B. Ma, S. Shan, and X. Chen. Diversity-measurable anomaly detection. In CVPR , 2023.

Y. Liu, K. Ding, Q. Lu, F. Li, L. Y. Zhang, and S. Pan. Towards self-interpretable graph-level anomaly detection. In NeurIPS , 2023.

X. Luo, J. Wu, J. Yang, S. Xue, H. Peng, C. Zhou, H. Chen, Z. Li, and Q. Z. Sheng. Deep graph level anomaly detection with contrastive learning. Scientific Reports , 12(1):19867, 2022.

R. Ma, G. Pang, L. Chen, and A. van den Hengel. Deep graph-level anomaly detection by glocal knowledge distillation. In WSDM , 2022.

X. Ma, J. Wu, J. Yang, and Q. Z. Sheng. Towards graph-level anomaly detection via deep evolutionary mapping. In KDD , 2023.

C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. In Workshop on Graph Representation Learning and Beyond (GRL+) , 2020.

M. Neumann, R. Garnett, C. Bauckhage, and K. Kersting. Propagation kernels: efficient graph kernels from propagated information. Machine learning , 102:209-245, 2016.

C. Niu, G. Pang, and L. Chen. Graph-level anomaly detection via hierarchical memory networks. In ECML-PKDD , 2023.

C. Qiu, M. Kloft, S. Mandt, and M. Rudolph. Raising the bar in graph-level anomaly detection. In IJCAI , 2022.

K. Riesen and H. Bunke. Iam graph database repository for graph based pattern recognition and machine learning. In Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshop , pages 287-297. Springer, 2008.

N. Shervashidze, P. Schweitzer, E. J. Van Leeuwen, K. Mehlhorn, and K. M. Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research , 12(9), 2011.

J. J. Sutherland, L. A. O'brien, and D. F. Weaver. Spline-fitting with a genetic algorithm: A method for developing classification structure- activity relationships. Journal of chemical information and computer sciences , 43(6):1906-1915, 2003.

Q. Tan, N. Liu, X. Huang, S.-H. Choi, L. Li, R. Chen, and X. Hu. S2gae: self-supervised graph autoencoders are generalizable learners with graph masking. In WSDM , 2023.

W. Tu, Q. Liao, S. Zhou, X. Peng, C. Ma, Z. Liu, X. Liu, Z. Cai, and K. He. Rare: Robust masked graph autoencoder. IEEE Transactions on Knowledge and Data Engineering , 36(10): 5340-5353, 2024.

P. Veliˇ ckovi´ c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio. Graph attention networks. In ICLR , 2017.

H. G. Vogel. Drug discovery and evaluation: pharmacological assays . Springer Science & Business Media, 2002.

N. Wale, I. A. Watson, and G. Karypis. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems , 14:347-375, 2008.

J. Xia, L. Wu, J. Chen, B. Hu, and S. Z. Li. Simgrace: A simple framework for graph contrastive learning without data augmentation. In WWW , 2022.

K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? In ICLR , 2019.

X. Yan, H. Cheng, J. Han, and P. S. Yu. Mining significant graph patterns by leap search. In SIGMOD , 2008.

P. Yanardag and S. Vishwanathan. Deep graph kernels. In KDD , 2015.

Z. Ying, J. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec. Hierarchical graph representation learning with differentiable pooling. In NeurIPS , 2018.

Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen. Graph contrastive learning with augmentations. In NeurIPS , 2020.

G. Zhang, Z. Yang, J. Wu, J. Yang, S. Xue, H. Peng, J. Su, C. Zhou, Q. Z. Sheng, L. Akoglu, et al. Dual-discriminative graph neural network for imbalanced graph-level anomaly detection. In NeurIPS , 2022.

Z. Zhang and L. Zhao. Unsupervised deep subgraph anomaly detection. In ICDM , 2022.

L. Zhao and L. Akoglu. On using classification datasets to evaluate graph outlier detection: Peculiar observations and new insights. Big Data , 11(3):151-180, 2023.

L. Zhao, S. Sawlani, A. Srinivasan, and L. Akoglu. Graph anomaly detection with unsupervised gnns. In ICDM , 2022.

Y. Zhou, H. Huo, Z. Hou, L. Bu, J. Mao, Y. Wang, X. Lv, and F. Bu. Co-embedding of edges and nodes with deep graph convolutional neural networks. Scientific Reports , 13(1):16966, 2023.

A Proof

In this section, we provide proof of each theorem presented in the paper.

A.1 Proof of Theorem 1

Theorem 1 (Suspicious generalization across τ ) . For any n ≥ 12 and 0 < τ 1 ≤ 1 , there exists ϵ > 0 such that for any 0 < γ ≤ ϵ and any 0 < τ 2 ≤ 1 with the initial weight matrix W (0) = I n ,

.

Proof. Note that the reconstruction loss is defined as follows:

Here, since X is identity, Eq. (7) is equal to L := ∥ A -AWWA ∥ 2 F . The derivative of Eq. (7) and its expectation are as follows [26]:

Given n (the total number of nodes) and τ (the community strength), let N = n 2 (the number of nodes in each community) and p = 1+ τ 2 (the inner-community edge probability), we have

(1) If i = j :

and

(2) If i and j are in the same group:

and

(3) If i and j are in different groups

and

where

Hence, the entries in

̸

̸

have three different values: (1) i and j are equal, (2) i = j and v i and v j belong to the same group, and (3) i = j and v i and v j belong to the different groups. Thus, we can express E A [ A 4 -A 3 ] as a I + b P + c U , where a, b, c ∈ R and P and U are defined as:

where 1 ∈ { 1 } N × N and 0 ∈ { 0 } N × N .

Specifically,

For any training graphs with 0 < τ 1 ≤ 1 , i.e., 0 . 5 < p ≤ 1 , we have

For any N ≥ 4 , a > 0 ,

For any N ≥ 6 , b > 0 ,

For any N ≥ 4 , c > 0 .

Thus, a, b, c > 0 for any N ≥ 6 (i.e., n ≥ 12 ) hold.

Let W ′ = W ( τ 1 , W (0) , γ ) be the updated W after one step of gradient descent with learning rate γ trained on graphs with τ 1 (and n ). Now, we derive the reconstruction loss on test graphs computed with W ′ . Formally, reconstruction loss by using W ′ is defined as follows:

We further simplify ( W ′ ) 2 as follows:

To simplify Eq. (10), we use the property that P 2 = N P , U 2 = N U , and PU = UP = N U . Using these facts, we rewrite Eq. (10) as below:

1 2 3 1 2 3 1 2 1 3 2 3 where

Now, by using the derived results, we rewrite the reconstruction loss term Eq. (9) as below:

where

We can decompose the loss by

Given i and j , let x = A ij , y = ( A 2 ) ij , z = ( APA ) ij , and w = ( AUA ) ij , we have

Let ¯ L ij = ( A ij -( A 2 ) ij ) 2 = ( x -y ) 2 . By using this notation, we derive the following result:

We shall show that

As γ → 0 + , ignoring higher-order terms and keeping the term linear w.r.t. γ , it suffices to show that

Note: In the following derivation, p is from test graphs (i.e., p = τ 2 +1 ).

2 (1) If i = j

Hence,

(2) Else if i and j are in the same group

Hence,

(3) Else if i and j are in different groups

Hence,

Conclusion.

Recall that a, b, c > 0 for any n ≥ 12 and 0 < τ 1 ≤ 1 and we aim to show that

It suffices to show that, for any 0 . 5 < p ≤ 1 ,

with at least one inequality being strict. Since there are

2 N pairs of ( i, j ) such that i = j ,

2 N ( N -1) pairs of ( i, j ) such that i = j and i and j are in the same group, and

̸

̸

2 N 2 pairs of ( i, j ) such that i = j and i and j are in different groups,

it is equivalent to showing that

and at least for one i value, the inequality is strict.

Indeed,

for any N ≥ 4 ;

for any N ≥ 3 ;

for any N ≥ 2 .

In conclusion, when trained with any 0 . 5 < p train ≤ 1 (i.e., 0 < τ 1 ≤ 1 ) and N train ≥ 6 (i.e., n ≥ 12 ), as γ approaches 0 + , for any 0 . 5 < p test ≤ 1 (i.e., 0 < τ 2 ≤ 1 ),

completing the proof.

A.2 Proof of Theorem 2

Theorem 2 (Correlation with τ ) . For any n ≥ 34 and 0 < τ 1 ≤ 0 . 98 , there exists ϵ > 0 such that for any 0 < γ ≤ ϵ and any τ 1 +0 . 02 ≤ τ 2 ≤ 1 with the initial weight W (0) = I n ,

Proof. Following the notations in the proof of Theorem 1 above, regarding the 'speed' of gradient,

Trained with p ′ = τ 1 +1 2 and testing with p = τ 2 +1 2 , we have

By analyzing the above equation, we can find that when N ≥ 17 (i.e., n ≥ 34 ), 0 . 5 < p 1 ≤ 0 . 99 (i.e., 0 < τ 1 ≤ 0 . 98 ), and p 1 +0 . 01 ≤ p 2 ≤ 1 (i.e., τ 1 +0 . 02 ≤ τ 2 ≤ 1 ), the above equation is negative, i.e., f ( p 2 , p 1 , N ) -f ( p 1 , p 1 , N ) < 0 or equivalently

completing the proof.

B Datasets

In this section, we describe the details of the benchmark graph classification datasets utilized in our experiments. For experiments, we use datasets that are preprocessed by Morris et al. [36]. We provide descriptive statistics of each dataset in Table 2.

The DD dataset [7] is a bioinformatics dataset that contains protein graphs. Each graph represents a particular protein. Each node represents an individual amino acid. Each edge represents two nodes that satisfy certain spatial proximity. Each node feature represents a type of the corresponding node's amino acid type. Each graph binary label represents whether the corresponding compound is an enzyme or not.

The Protein dataset [1] is a bioinformatics dataset that contains protein graphs. Each graph represents a particular protein. Each node represents a secondary structure element of a protein. Each edge

represents two nodes that are neighbors along the amino acid sequence or are top-K neighbors (K=3) at the protein space. Each node feature represents a type of the corresponding node's secondary structure element type. Each graph binary label represents whether the corresponding protein is an enzyme or not.

The NCI1 dataset [47] is a chemical dataset that contains chemical molecular compound graphs. Each graph represents a particular chemical molecular compound. Each node represents an atom. Each edge represents a chemical bond between two atoms. Each node feature represents a type of atom (e.g., carbon, and hydrogen). Each graph binary label represents whether the corresponding compound is an anti-cancer chemical or not.

The AIDS dataset [40] is a chemical dataset that contains chemical molecular compound graphs. Each graph represents a particular chemical molecular compound. Each node represents an atom. Each edge represents a chemical bond between two atoms. Each node feature represents a type of atom (e.g., carbon, and hydrogen). Each graph binary label represents whether the corresponding compound shows an anti-HIV activity or not.

The Reddit dataset [51] is a social-network dataset that contains social network graphs. Among Reddit-Binary, Reddit-Multi-5K, and Reddit-Multi-12K, we utilized Reddit-Binary. Each graph represents a thread on Reddit. Each node represents a user. Each edge represents a comment between two users. There are no available node features, thus, we utilize the one-hot encoded degree of the node as its feature, following existing literature in GLAD [39]. Each graph binary label represents whether the corresponding thread is a discussion-based thread or question-answer-based thread.

The IMDB dataset [51] is an actor-ego-network dataset that contains social network graphs. Each graph represents the ego network of a particular actor. Each node represents an actor. Each edge represents a co-appearance of two actors in (a) movies. There are no available node features, thus, we utilize the one-hot encoded degree of the node as its feature, following existing literature in GLAD [39]. Each graph binary label represents whether the corresponding ego-user particularly acts in the romance genre or action genre.

The MUTAG (formally, mutagenicity) dataset [17] is a chemical dataset that contains chemical molecular compound graphs. Each graph represents a particular chemical molecular compound. Each node represents an atom. Each edge represents a chemical bond between two atoms. Each node feature represents a type of atom (e.g., carbon, and hydrogen). Each graph binary label represents whether the corresponding compound is mutagenic or not.

The DHFR dataset [42] is a chemical dataset that contains chemical molecular compound graphs. Each graph represents a particular chemical molecular compound. Each node represents an atom. Each edge represents a chemical bond between two atoms. Each node feature represents a type of atom (e.g., carbon, and hydrogen). Each graph binary label represents whether the corresponding compound works as a dihydrofolate reductase inhibitor or not.

The BZR [42] dataset is a chemical dataset that contains chemical molecular compound graphs. Each graph represents a particular chemical molecular compound. Each node represents an atom. Each edge represents a chemical bond between two atoms. Each node feature represents a type of atom (e.g., carbon, and hydrogen). Each graph binary label represents whether the corresponding compound is active against the benzodiazepine receptor.

The ER dataset is a chemical dataset that contains chemical molecular compound graphs. Specifically, it contains compounds that are NR-estrogen receptor (ER)-LBD. Each graph represents a particular chemical molecular compound. Each node represents an atom. Each edge represents a chemical bond

between two atoms. Each node feature represents a type of atom (e.g., carbon, and hydrogen). Each graph binary label represents whether the corresponding compound has toxicity or not.

The source of the ER dataset is https://tripod.nih.gov/tox21/challenge/data.jsp .

C Formal Expression of Synthetic Datasets

In this section, we formally describe the leveraged synthetic graphs: Syn-Com and Syn-Cycle. We elaborate on how each graph is generated.

C.1 Syn-Com

We formalize the data generation process of Syn-Com. For a given even number of nodes n > 2 , assume two disjoint node communities V a and V b (WLOG, V a = { v i : 1 ≤ i ≤ n/ 2 } and V b = { v j : n/ 2 + 1 ≤ j ≤ n } ). For a given community parameter τ ∈ [0 , 1] , a graph of Syn-Com G = ( X , A ) satisfies the following: A i,j = A j,i ∼ b (1 , ( τ + 1) / 2) for { v i , v j } ∈ ( V a 2 ) ∪ ( V b 2 ) , A i,j = A j,i ∼ b (1 , (1 -τ ) / 2) for v i ∈ V a , v j ∈ V b , and A i,i = 0 , ∀ i ∈ [ n ] , where b (1 , p ) is a Bernoulli sampling with a parameter p . We let X = I n , where I n is a n -byn identity matrix.

C.2 Syn-Cycle

We formalize the data generation process of Syn-Cycle. For a given number of nodes n > 2 , we consider a clean-cycle graph G cyc ∗ = ( V , E ∗ ) ≡ ( X , A ∗ ) where E ∗ = { v i , v i +1 : i ∈ [ n ] } ∪ { v 1 , v n } and A ∗ is the adjacency matrix that represents E ∗ .

We also consider a set of noisy-cycle graphs (i.e., pan graphs) G cyc ′ , constructed by randomly relocating an edge from G cyc ∗ to form a ( n -1 )-size cycle, (i.e., ( X , { v i , v j | i } ∪ E ∗ \ { e k } ) , ( X , { v j , v i | j } ∪ E ∗ \{ e k } ) ∈ G cyc ′ , ∀ e k = { v i , v j } ∈ E ∗ , where v j | i is a v j 's neighbor except for v i ). We let X = I n . Note that for n nodes, there exist 2 n noisy-cycle graphs in G cyc ′ and a single clean-cycle graph.

D Experimental Details

In this section, we provide experimental details of the conducted experiments in our work.

D.1 Machines

All experiments of this work are performed on a machine with NVIDIA RTX 8000 D6 GPUs (48GB VRAM) and two Intel Xeon Silver 4214R processors.

D.2 Details of empirical analysis

For the Com-Com dataset , we sample 500 graphs from G com τ =0 . 4 for training graphs and sample 500 graphs from G com τ =0 . 8 for unseen graphs.

For the Cycle-Cycle dataset , we sample half of G cyc ′ for training graphs and employ the clean-cycle graph as the unseen graph.

For the Com-Cycle dataset , we sample 500 graphs from G com τ =0 . 4 for training graphs and sample half of G cyc ′ for unseen graphs.

For the Cycle-Com dataset , we sample half of G cyc ′ for training graphs and sample 500 graphs from G com τ =0 . 4 for unseen graphs.

Training and evaluation We train GAEs for 200 epochs. The results are similar for 500 epochs, but we present the 200-epoch results for better visualization since the error lines flatten after 200 epochs. For every 10 epochs of training, we measure the mean reconstruction errors for each graph and report the mean of these values for graphs within the same class.

D.3 Details of two-stage GLAD methods

We provide an overview of two-stage anomaly detection methods. First, by performing certain pretext tasks, they learn representations of data. Then, upon the obtained representations, they utilize a oneclass classifier to detect anomalies. Below, (1) we describe graph representations of utilized GLAD methods, including MUSE, and (2) describe the utilized one-class classifier, an MLP autoencoder.

Note . MUSE is a two-stage method.

Graph representation. As described in Section 5, MUSE obtains a representation of each graph with the proposed error representation Eq (6). On the other hand, utilized baseline two-stage approaches, which are GraphCL [53], GAE [23], and GraphMAE [15], obtain graph representations from the graph or node embeddings. Specifically, they first train graph neural network encoders with their pretext tasks. Specifically, GraphCL performs graph contrastive learning, GAE performs adjacency matrix reconstruction, and GraphMAE performs node feature reconstruction. After finalizing selfsupervised learning, we utilize the trained encoder to obtain embeddings of graphs or nodes. Details of each method are as follows:

GraphCL: It learns graph embeddings to perform its contrastive learning task. Thus, we directly utilize the graph embeddings as representations of graphs.

GAE: It only learns node embeddings. Thus, we pool node embeddings by using the elementwisemean readout function.

GraphMAE: It only learns node embeddings. Thus, we pool node embeddings by using the elementwise-mean readout function.

One-class classifier. In our experiments, every two-stage method leverages an MLP autoencoder as its one-class classifier. In a nutshell, the MLP autoencoder detects anomalies by using the reconstruction loss of a graph. We first elaborate on how we train the MLP autoencoder and then describe how the trained MLP autoencoder is leveraged to detect anomalies.

We start with the training of the MLP autoencoder. Formally, for a given data representation vector z ∈ R d , the MLP autoencoder MLP ξ : R d ↦→ R d generates a reconstructed representation ˆ z ∈ R d (i.e., ˆ z = MLP ξ ( z ) ). Then, we compute a L2-norm reconstruction loss L as follows: L := ∥ z -ˆ z ∥ 2 . We update the parameters of the MLP autoencoder ξ by using gradient descent to minimize L .

After training the MLP autoencoder MLP ξ , we detect anomalies by using a reconstruction loss as an anomaly score. However, since the scale of each representation dimension significantly varies, the reconstruction loss is often dominated by dimensions with larger scales, overshadowing those with smaller values. To mitigate this undesirable phenomenon, in the anomaly inference step, we leverage dimensionwise-weighted L2 scores. Importantly, we apply different weights to different dimensions w l ∈ R , ∀ l ∈ [ d ] (note that d is the dimension of an input vector). Specifically, we utilize the dimensionwise standard deviation of train data's representations, which is defined as follows:

where z i,l is a l -th element of i -th graph's representation vector and D train is a set of training graphs. By using the normalization weight of Eq (13), we compute the anomaly score. Formally, the anomaly detection score of G t , denoted by s T , is defined as follows:

Note that the higher s t indicates the t -th data point is normal.

D.4 Details of MUSE

In this subsection, we provide details of MUSE.

Reconstruction models. As described in Section 6, we use GIN [49] as our backbone encoder, which is the same as other leveraged GLAD baseline methods. For the node feature decoder and adjacency matrix decoder, we use a 2-layer MLP with a ReLU activation function.

Error representation. For aggregation functions of error representation, we leverage a mean function and a standard deviation function. Formally, for a set A where A ⊂ R , a mean aggregation function Agg mean : 2 R ↦→ R and a standard deviation aggregation function Agg std : 2 R ↦→ R ≥ 0 are defined as follows:

Thus, we represent each graph G by using a R 4 vector as follows:

D.5 Hyperparameters and Details of Baselines

For every method, we tune the hyperparameters and choose the hyperparameter configuration that gives the best validation AUROC. We provide details of the search space of each model category and its hyperparameters.

Fixed settings. For all the methods, we fix the dropout probability and weight decay as 0 . 3 and 1 e -6 , respectively. In addition, all the methods are trained with Adam optimizer [21].

One-stage learning-based GLAD methods. We tune the following hyperparameters for one-stage learning-based GLAD methods, which are DOMINANT, OCGTL, GLADC, GLocalKD, GLAM, HIMNET, and SIGNET:

Training learning rate γ ∈ { 10 -3 , 10 -4 }

Models hidden dimension d ′ ∈ { 16 , 32 , 64 , 128 , 256 }

Number of GNN layers K ∈ { 3 , 4 , 5 }

Number of model training epochs L ∈ { 30 , 60 , · · · 300 }

For other hyperparameters, we follow the default configuration provided in their official Github.

Two-stage learning-based GLAD methods. We tune the following hyperparameters for two-stage learning-based GLAD methods, which are GraphCL, GAE, GraphMAE, and MUSE.

Note. Our proposed method MUSE is also included in this category and tuned hyperparameters according to the below

Pretext task learning rate γ ∈ { 10 -3 , 10 -4 }

Models hidden dimension d ′ ∈ { 16 , 32 , 64 , 128 , 256 }

Number of GNN layers K ∈ { 3 , 4 , 5 }

Number of pretext task training epochs L ∈ { 20 , 40 , · · · , 200 }

In addition to these hyperparameters, we also tune the positive weight coefficient for our adjacency matrix reconstruction, which is denoted as τ in Section 5, within { 0 . 0 , 1 . 0 , 2 . 0 } .

Here, models indicate both the encoder model and decoder model that are utilized to perform pretext tasks. Note that as described in Appendix D.3, all two-stage methods utilize MLP autoencoder for their one-class classifier. We tune the hyperparameter of the MLP autoencoder as follows:

MLP hidden dimension d ′ ∈ { 32 , 64 , 128 }

MLP learning rate γ ∈ { 10 -2 . 10 -3 , 10 -4 }

We fix the MLP auteoncoder training epochs and the number of MLP layers as 500 and 3 , respectively.

Two-stage baselines details. For GraphCL, we obtain graph views by randomly dropping 50% of edges and adding Gaussian noise to node features sampled from N (0 , 0 . 1) and use a two-layer MLP projection head. For GraphMAE, we mask 50% of node features with the zero mask, which is empirically demonstrated to be more effective than a learnable mask in our preliminary study. For GAE, we utilize a binary cross-entropy loss and a two-layer MLP decoder. 4

Cosine loss

Epochs

Epochs

Epochs

Epochs

Com-Com dataset

Cycle-Cycle dataset

Unseen

graphs

Training

graphs

Cosine loss

Cosine loss

Epochs

Epochs

Epochs

Epochs

Com-Cycle dataset

Cycle-Com dataset

Unseen

graphs

Training

graphs

Cosine loss

E Additional Experimental Results

E.1 Node feature reconstruction methods

Note that in Section 3, we investigate Graph-AEs that reconstruct the adjacency matrix of a given graph. In this subsection, we analyze another large branch of Graph-AEs: node feature reconstruction methods [15, 16, 44].

Feature reconstruction model. We first formalize the node feature reconstruction method. Consider a graph G = ( X , A ) , where X ∈ R |V|× d . First, a GNN encoder f θ is utilized to generates node embeddings Z ∈ R |V|× d ′ (i.e., f θ ( X , A ) = Z ). Then, by using the feature decoder g ϕ : R d ′ ↦→ R d , the reconstructed node features ˆ X ∈ R |V|× d are obtained. Lastly, we compute the feature reconstruction loss, which is either (1) squared Frobenius norm loss [38] (i.e., L := ∥ X -ˆ X ∥ 2 F ) or (2) cosine similarity loss [15] (i.e., L := 1 |V| ∑ |V| i =1 (1 -X T i ˆ X i ∥ X i ∥ 2 ·∥ ˆ X i ∥ 2 ) ). Parameters of the GNN encoder f θ and the node feature decoder g ϕ are updated by gradient descent to minimize L .

Setting. The dataset setting is equal to that of empirical analysis provided in Section 3.2. Regarding the model setting, we utilize 3-layer GIN as the GNN encoder f θ and 2 -layer MLP as the feature decoder g ϕ . We utilize either squared Frobenius norm or cosine similarity loss for the reconstruction loss. We train each reconstruction model for 200 epochs.

Results. In a nutshell, we can observe the same results as those of GAE. In Scenario 1, as shown in Figure 8, one can observe that the reconstruction flip occurs, which is also observed in GAE. In Scenario 2, as shown in Figure 9, one can observe that the reconstruction flip does not occur. Thus, we verify that our findings in Section 3.2 are not only restricted to the adjacency matrix reconstruction methods but also hold for the node feature reconstruction methods.

E.2 Experimental results on each setting.

We present experimental results in each setting of our main experiment. Specifically, experimental results for the case where class 0 graphs are anomalies and class 1 graphs are normal are provided in Table 3. In addition, experimental results for the case where class 1 graphs are anomalies and class 0 graphs are normal are provided in Table 4. Notably, in both cases, MUSE obtains the best average ranking among 18 methods (refer to Table 3 and Table 4).

E.3 Experimental results on additional evaluation metrics

In this section, we compare the GLAD performance of MUSE against those of two strongest competitors, OCGTL and GLAM. To this end, we leverage two metrics: average precision score (AP score) and precision@K. As shown in Table 5 and Table 6, MUSE outperforms the competitors in 7/10 and 8/10 in terms of AP score and precision@K, respectively. These results demonstrate the effectiveness of MUSE is not limited to a particular evaluation metric.

E.4 Experimental results on additional ablation study

In this section, we demonstrate the effectiveness of the following two components of MUSE: graph augmentation and cosine-similarity feature reconstruction loss. To this end, we use two variants of MUSE. MUSE w/o Aug. is a variant where the input graph augmentation is removed. MUSE w/o Cos. is a variant that uses the Frobenious norm loss for the feature reconstruction loss, instead of the

cosine similarity loss. Note that MUSE uses both input graph augmentation and cosine similarity loss. As shown in Table 7, MUSE outperforms its variants in 8 out of 10 datasets, demonstrating the effectiveness of our graph augmentation and cosine-similarity loss.

E.5 Experimental results from the scalability analysis

In this section, we study the scalability and effectiveness of GLAD methods in large-scale graphs.

Datasets. We use two large-scale real-world graphs, which are MalNetTiny [9] and OVCAR-8 [50]. Specifically, the MalNetTiny dataset consists of graphs, each containing a large number of nodes and edges. In contrast, the OVCAR-8 dataset consists of a large number of graphs.

Scalable MUSE. Note that MUSE may not be scalable for graphs with a large number of nodes, since MUSE reconstructs all the entries of an adjacency matrix, which results in the complexity of O ( |V| 2 ) . Thus, we present a scalable version of MUSE, which we call MUSE-Sample . Specifically, MUSE-Sample samples K number of entries from the adjacency matrix, and reconstructs only the sampled entries. Therefore, the time complexity of computing reconstruction loss becomes

Experimental results. We compare the (1) GLAD performance and (2) inference runtime of our proposed methods (i.e., MUSE and MUSE-Sample) against the two strongest baseline methods: OCGTL and GLAM. Regarding the performance comparison, as shown in Table 8, MUSE and MUSE-Sample outperform the baseline methods in both datasets. Regarding the runtime analysis, as shown in Table 9, MUSE is the second-fastest method among the four methods.

F Further Analysis and Discussion

In this section, we provide our further analyses and discussions.

F.1 Reconstruction flip in computer vision

Analogous to our observations of the reconstruction flips in graphs, certain images are more easily reconstructed. For instance, Liu et al. [31] noted that " anomalies with colors close to the background may yield unreliable reconstruction errors ". Moreover, in the MNIST dataset, a reconstruction model trained on class-7 images sometimes reconstructs class-4 images better than class-7 images [31].

To mitigate this undesirable phenomenon, Liu et al. [31] proposed a deformation-based anomaly detection method. In this method, an input image is first deformed in a way that makes a reconstruction model easy to reconstruct. Here, a deformation method that maintains the semantics of the input image is employed [5]. To identify whether an image is an anomaly, they first deform the image and then let a reconstruction model reconstruct it. Their underlying intuition is that the deformation model would drastically change the image dissimilar to the majority of the training images, and this would

make the deformed anomalous image different from its original image. Finally, when the deformed image is reconstructed by the reconstruction model, the reconstructed anomalous image would be significantly different from the original anomalous image, resulting in a large reconstruction error.

While the method is intuitive in computer vision, the extension of the suggested method to the graph domain is non-trivial. First, the graph deformation that maintains semantics is non-trivial. It is hard to know the semantics of a graph without sufficient domain knowledge, especially without any external graph labels. Furthermore, naive graph augmentations, such as node dropping and edge perturbation, may harm the semantics of a graph [48]. Second, while the deformation techniques are widely studied in computer vision [5, 2], those of a graph is an underexplored region. Therefore, employing an adequate technique for graph deformation is also practically infeasible.

Considering these challenges, we consider that the method proposed in Liu et al. [31] is hard to be trivially extended to the graph domain.

F.2 Reconstruction flip in real-world graph datasets

We provide our in-depth analysis result of reconstruction flip in Section 3 by using the synthetic datasets . In this section, we analyze whether the reconstruction flip phenomena occur in real-world graph datasets .

Setting. We focus on the anomaly detection task performance of GAE [22], which reconstructs the adjacency matrix of a given graph. GAE uses reconstruction loss as an anomaly score, where an anomalous graph is likely to have high reconstruction errors. We measure the AUROC score on test graphs, based on the setting described in Section 6.1. Here, the AUROC score lower than 0.5 indicates the corresponding model performs worse than random guessing, implying the reconstruction flip has occurred.

Results. As shown in Table 3, in the protein dataset, GAE [22] tends to perform worse than random guessing (spec., average AUROC is 0.458), which indicates that the reconstruction flip occurs. Thus, we demonstrate that the reconstruction flip is not limited to synthetic scenarios but also occurs in real-world graph datasets.

F.3 Complexity analysis of MUSE

In this section, we provide a time complexity analysis for the forward pass of MUSE, concerning the size of a graph (i.e., number of nodes |V| and edges |E| ).

Encoding. We first encode a graph G = ( X ∈ R |V|× d , A ∈ { 0 , 1 } |V|×|V| ) with a GNN encoder. The time complexity of encoding G with L -layer GCN [23] is O ( L |V| d 2 + L |E| d ) [3]. Given that d and L are constants, the complexity is equivalent to O ( |V| + |E| ) . 5

Decoding. We decode node embeddings by using a node decoder and edge decoder, which are MLPs. Since the hidden dimensions and the number of layers of MLPs are all constants, the complexity of decoding is equivalent to O ( |V| ) .

Reconstruction loss. We compute adjacency matrix reconstruction loss and node feature reconstruction loss. Each loss computation has the time complexity of O ( |V| 2 ) and O ( |V| ) , respectively. Thus, the overall time complexity of the reconstruction loss computation becomes O ( |V| 2 ) .

Error representation. We utilize average pooling and standard deviation pooling, which both have linear time complexity. Thus, the time complexity is equivalent to O ( |V| 2 ) .

Overall. Thus, the time complexity of the forward pass of MUSE is as follows:

While MUSE has a quadratic complexity to the number of nodes, our task is a graph-level task where real-world graphs typically have an affordable number of nodes (as shown in Table 2, the dataset that has the largest average number of nodes has smaller than 430 average number of nodes).

F.4 Broader impacts of our work

In this section, we discuss the broader impacts of our research. While our focus lies in the graph-level task, similar observations can be found in node-level tasks within various types of graphs, such as dynamic graphs [27], heterophilic graphs [29, 30], and hypergraphs [19, 20, 4, 18]. We anticipate that our findings and the proposed MUSE will be widely utilized in applications requiring graphlevel anomaly detection, such as drug discovery and brain diagnosis. In addition, as described in Section 2.1, the phenomenon of the reconstruction flip is not limited to the graph domain, and it is also observed in other domains such as computer vision. To our knowledge, we are the first to utilize summarized reconstruction errors as a representation feature of data. This approach can be directly applied to other fields, such as computer vision.

F.5 Generalized categorization of primary patterns

Expectation. In this work, as a primary pattern, we use community structure and node cycles. As a future direction, more general and formal categorization of graph patterns can be considered. This categorization would enable our analysis (Section 3) to be more systematic and generalized.

Challenge. However, real-world datasets may exhibit a wider variety of patterns, including those related to node attributes. For example, in the protein dataset, both normal and anomalous graphs exhibit homophilic patterns (i.e., edges tend to join nodes with the same attribute), with anomalies displaying a stronger pattern than normal graphs. This diversity makes the categorization not straightforward. Thus, adequately considering real-world graph patterns would play a crucial role in this categorization.

G Checklist

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We clarify our goal, analyzing reconstruction flip, and provide analysis results in Section 3. We present a method (Section 5) and demonstrate its superiority (Section 6).

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We provide limitations of the proposed method MUSE in Section 7.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the full statements for all theorems in Section 3.3 and provide full proofs in Appendix A.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide architectures of MUSE in Section 5, details regarding hyperparameters in Appendix D, and model code in https://github.com/kswoo97/GLAD_MUSE .

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide all the utilized datasets, full code for models, and instructions to run the code in https://github.com/kswoo97/GLAD_MUSE .

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide the key setting in Section 6, and further details in Appendix D.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We describe the number of experimental runs and standard deviation of the performance in Section 6.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We describe machines utilized for the experiments in Appendix D.1.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We strictly follow the NeurIPS Code of Ethics, particularly in terms of anonymity that we do not reveal our nationality and institutes, etc.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the broader impact of our work in Appendix F.4

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We believe we do not have risks of misusing.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We properly add citations for all the mentioned existing literature or methods.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release any new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We do not utilize any crowdsourcing or human-related tasks.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA] .

Justification: We do not utilize any crowdsourcing or human-related tasks.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the reconstruction flip phenomena in graph autoencoder which may be beneficial for graph-level anomaly detection.\nThen the paper proposes a novel graph autoencoder model MUSE, which simply represents a graph as multifaceted summaries of its reconstruction errors and achieves appreciable improvement.\n\nSTRENGTHS:\nThis article takes an in-depth look at the problem of reconstruction loss in graph autoencoder and presents an impressive theoretical analysis.\n\nThe logic of this paper is quite clear. The theoretical analysis is also very rigorous and inspiring. The similarity of the graph structure in the test set to the training set leads to lower reconstruction errors in the test set, which is consistent with common sense.\n\nWEAKNESSES:\nI think the title is a bit long and redundant with information. Please change the title if you can.\n\nThe primary patterns mentioned in line 122 currently contain community structures and circles. Could there be a more systematic definition or categorization based on graph theory? I believe this may be a future direction for the research. As it stands, the community structure can probably be summarized as the case where the degree distribution is uneven, and the circle is the case where the degree is uniformly 2.\n\nUsing AUROC as a metric alone may be biased. This is because, for anomaly detection tasks, there tend to be fewer anomaly labels, which will lead to large differences in precision and recall rates, which in turn will lead to an inflated AUROC. At the very least, the reasons for using AUROC can be explained.\n\nQUESTIONS:\nPlease see the Weaknesses. Thanks!\n\nLIMITATIONS:\nThe authors mention the greater complexity of MUSE and the challenges of its operation on large-scale graphs.\n\nIn addition, MUSE is limited by the error distribution. However, this limitation may not occur in practice since current real-world datasets generally do not have error distributions similar to those of the majority.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new GLAD method. The authors identify and analyze a phenomenon termed \"reconstruction flip\", where Graph-AEs sometimes reconstruct unseen graphs (with certain patterns more accurately than training graphs) better than the training graphs themselves. Based on this analysis, the authors propose MUSE (Multifacted Summarization of Reconstruction Errors), which leverages multifaceted summaries of reconstruction errors to enhance anomaly detection performance. MUSE has been shown to outperform existing methods in various datasets.\n\nSTRENGTHS:\n1. This paper is well-organized, and identifying the reconstruction flip phenomenon is novel and provides new insights into the existing GLAD methods.\n\n2. This paper provides relevant theoretical analysis to elaborate on the reconstruction flip phenomenon.\n\n3. Authors conducted extensive experiments in comparison with several SOTA baselines, and demonstrated the effectiveness of the proposed MUSE. Besides, the code is released for reproducibility.\n\nWEAKNESSES:\n1. The primary contribution of this paper is the introduction of multifaceted summaries of reconstruction errors for anomaly detection. However, the methods employed for error representation and summarization are conventional and do not demonstrate significant innovation. Furthermore, the authors utilize this feature to train a one-class classifier, which is also a traditional two-stage method. Consequently, the originality and novelty of this contribution appear to be rather limited.\n\n2. Another important concern of the reviewer is the theoretical analysis provided in the paper, which primarily demonstrates the existence of the reconstruction flip phenomenon. However, it does not significantly contribute to understanding or improving the anomaly detection capabilities of the proposed method.  The practical relevance of the theoretical findings to the performance improvements in anomaly detection is not convincingly demonstrated.\n\n3. This paper does not evaluate MUSE on large-scale graph benchmarks to demonstrate the feasibility of large-scale applications.\n\n4. There is a lack of analysis of the computational complexity of MUSE compared to other methods. Such an analysis would help establish the practical advantages of the proposed method in real-world scenarios.\n\nQUESTIONS:\nPlease refer to the weaknesses.\n\nLIMITATIONS:\nThis paper does not have any potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel approach to enhance feature representation by leveraging multifaceted summaries beyond the mere average of reconstruction errors, addressing a limitation prevalent in current reconstruction-based methods. This motivation is timely and relevant, aiming to enrich the feature space for improved anomaly detection capabilities. The experimental setup is comprehensive, and the theoretical framework is solid; however, these two parts are somewhat disjointed.\n\nSTRENGTHS:\n**Reconstruction Flip Phenomenon**: The illustration of the reconstruction flip in Figure 3 is particularly interesting, offering new insights into the behavior of reconstruction-based models under varying conditions.\n\n**Enlightening Visualization in Figure 5**: This figure effectively demonstrates the inadequacy of relying solely on the average reconstruction error, highlighting cases where two distributions may have identical averages yet exhibit completely different characteristics. Such visualizations are crucial for understanding the nuances of reconstruction-based methods and their limitations.\n\n**Robust Experiments and Theoretical Foundation**: The experimental setup is comprehensive, and the theoretical framework is solid.\n\nWEAKNESSES:\n**Disjointed Analysis and Methodology Presentation**: There is a noticeable disconnect between the insightful analysis presented in Section 3 and the methodology described in Section 5. While the former provides a deep dive into the limitations of current approaches, the latter fails to directly reflect or build upon these findings. It is recommended that the authors integrate the analytical insights more coherently into the development of their method, ensuring a seamless flow of ideas from problem identification to solution proposal.\n\n**Characterization of Primary Patterns in Datasets**: The paper utilizes ten datasets, but lacks a detailed exploration of the **primary patterns** that characterize these datasets. Specifically, is it possible that so-called primary patterns are indeed distinct across anomaly and normal graphs? Including an analysis of these patterns would enhance the necessity and the effectiveness of the proposed approach.\n\nQUESTIONS:\n1. What's the connection between the analysis and the method? Is the purpose of the analysis just to prove that the average reconstruction error is not enough to detect anomalies? Does the analysis provide some insights on how to solve this issue?\n\n2. What are the primary patterns for anomaly and normal graphs in ten datasets?\n\nLIMITATIONS:\nThe authors have included the limitation and broader impact section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGraph autoencoders aim to learn graph representations by reconstructing them, with a key application in graph-level anomaly detection. GLAD identifies graphs with unusual structures or node features by considering those with high reconstruction errors as anomalies. However, the paper reports counter-examples where this assumption fails, a phenomenon named reconstruction flip. The authors investigate when this assumption holds and its limitations. Based on the conversations, the authors proposed MUSE, a novel method that uses these multifaceted summaries for better anomaly detection. MUSE outperforms 14 methods across 10 datasets, achieving state-of-the-art performance in GLAD.\n\nSTRENGTHS:\n+ This paper proposes a new method for graph anomaly detection which is from empirical studies and with theoretical analysis.\n\n+ The paper reports an interesting reconstruction flip phenomenon that conflicts with previous assumptions on graph reconstructions in the graph anomaly detection problem. \n\n+ The proposed method is simple and effective in detecting anomalies compared to previous methods using extensive experiments.\n\nWEAKNESSES:\n- The proposed method is two-stage instead of end-to-end, so a straightforward question is how to guarantee that the noises/mistakes from the first stage, i.e., representation learning, will not impact the second classification stage. A follow-up question is what is the impact of each stage in detecting anomalies? \n\n- Only AUROC is used as the evaluation metric. Previous methods, e.g., [37], also used other metrics to verify anomaly detection performance, including Precision, Recall, or F1.\n\n- Although the computational complexity has been analyzed, since the complexity is quadratic to the number of nodes, it will be helpful to understand the efficiency by comparing the running time of the proposed MUSE method.\n\nQUESTIONS:\n1. Question about the two-stage model architecture (see details above).\n\n2. Performance w.r.t other evaluation metrics.\n\n3. Running time comparison.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}