Original: Dual-discriminative Graph Neural Network for Imbalanced Graph-level Anomaly Detection
Revised: Dual-discriminative Graph Neural Network for Imbalanced Graph-level Anomaly Detection

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Graph-level anomaly detection aims to distinguish anomalous graphs in a graphdataset from normal graphs.
Revised: Graph-level anomaly detection aims to distinguish anomalous graphs in a graph dataset from normal graphs.

Original: Anomalous graphs represent  very few but essentialpatterns in the real world.
Revised: Anomalous graphs represent a very few but essential patterns in the real world.

Original: The anomalous property of a graph may be referable toits anomalous attributes of particular nodes and anomalous substructures referringto a subset of nodes and edges in the graph.
Revised: The anomalous property of a graph may be referable to its anomalous attributes of particular nodes and anomalous substructures that refer to a subset of nodes and edges in the graph.

Original: In addition, due to the imbalancenature of  anomaly problem,the anomalous information will be diluted by normalgraphs with overwhelming quantities.
Revised: In addition, due to the imbalance nature of the anomaly problem, anomalous information will be diluted by normal graphs with overwhelming quantities.

Original: Various anomaly notions in the attributesand/or substructures and the imbalance nature together make detecting anomalousgraphs a non-trivial task.
Revised: Various anomaly notions in the attributes and/or substructures and the imbalance nature together make detecting anomalous graphs a non-trivial task.

Original: In this paper, we propose a dual-discriminative graphneural network for graph-level anomaly detection, namely iGAD.
Revised: In this paper, we propose a dual-discriminative graph neural network for graph-level anomaly detection, namely iGAD.

Original: Specifically,an anomalous graph attribute-aware graph convolution and an anomalous graphsubstructure-aware deep Random Walk Kernel (deep RWK) are welded into agraph neural network to achieve a dual-discriminative ability on anomalous attributes and substructures.
Revised: Specifically, an anomalous graph attribute-aware graph convolution and an anomalous graph substructure-aware deep Random Walk Kernel (deep RWK) are welded into a graph neural network to achieve the dual-discriminative ability on anomalous attributes and substructures.

Original: The deep RWK in iGAD makes up for the deficiencyof graph convolution in distinguishing structural information caused by the simple neighborhood aggregation mechanism.
Revised: The deep RWK in iGAD makes up for the deficiency of graph convolution in distinguishing structural information caused by the simple neighborhood aggregation mechanism.

Original: Further, we propose a Point Mutual Information-based loss function to address the  imbalance nature of anomaly problem.
Revised: Further, we propose a Point Mutual Information (PMI)-based loss function to target the problems caused by imbalance distributions.

Original: The loss function enables iGAD to capture the essential correlation betweeninput graphs and their anomalous/normal properties.
Revised: The loss function enables iGAD to capture the essential correlation between input graphs and their anomalous/normal properties.

Original: We evaluate iGAD on fourreal-world graph datasets.
Revised: We evaluate iGAD on four realworld graph datasets.

Original: Extensive experiments demonstrate the superiority ofiGAD on the graph-level anomaly detection task.
Revised: Extensive experiments demonstrate the superiority of iGAD on the graph-level anomaly detection task.

Original:  Introduction
Revised:  Introduction

Original: Graphs have been used as an indispensable tool to describe relational data, such as chemical compounds, transport networks and social networks.
Revised: Graphs have been used as an indispensable tool to describe relational data, such as chemical compounds, transport networks, and social networks.

Original: For  a graph that models a chemical compound,nodes represent atoms and edges describethe chemical bonds between atoms.
Revised: For instance, in a graph that models a chemical compound, nodes represent atoms and edges describe chemical bonds between atoms.

Original: Learning graphs tounderstand the world has been drawing considerable attention in recent years [1, 2, 3, 4].
Revised: Learning graphs to understand the world has been drawing considerable attention in recent years [5, 14, 15, 50].

Original: Given agraph dataset containing a set of graphs, the graph-level anomaly detection task aims to detect graphshaving anomalous information.
Revised: Given a graph dataset containing a set of graphs, the graph-level anomaly detection task aims to detect graphs with anomalous information.

Original: Generally, these anomalous graphs with distinct characteristics orfunctions are the minority in a graph dataset.
Revised: Generally, these anomalous graphs with special characteristics or functions are the minority in a graph dataset.

Original: For example, among all antibody molecules, only asmall part of molecules exhibitthe anti-cancer activity on a specific  cancer cell line [5].
Revised: For example, among all antibody molecules, only a few of them exhibit anti-cancer activity on a specific type of cancer cell line [54].

Original: Graph-levelanomaly detection is a research topic of great practical value, say, it can support cancer drug discoveryby detecting molecules with anti-cancer activity from other antibody molecules [6].
Revised: Graph-level anomaly detection is a research topic of great practical value, say, it can support cancer drug discovery by identifying molecules with anti-cancer activity from other antibody molecules [47].

Original: Also, the researchcan help to reveal pathogenic brain mechanisms by distinguishing the brain structures of patientswith neurological disorders from healthy individuals [7].
Revised: Also, the research can help to reveal pathogenic brain mechanisms by distinguishing the brain structures of patients with neurological disorders from healthy individuals [21].

Original: In practice, the attributes of particular nodes are a contributing factor for a graph’s anomalousproperties.
Revised: In practice, the attributes of particular nodes are a contributing factor to a graph’s anomalous properties.

Original: For example, the hydrogen and oxygen atoms at the endside of molecules are thekey to discriminating between Alcohol and Alkane.
Revised: For example, the hydrogen and oxygen atoms at the end of molecules are the key to discriminating between Alcohol and Alkane [44].

Original: Graph substructures composed of a subset ofnodes and edges are another factor for a graph’s anomalous properties.
Revised: Graph substructures can be another factor for a graph’s anomalous properties.

Original: For instance, moleculartoxicity can be induced by the pharmacophores consisting of some graph substructures, such astriangles and cycle-based graphlets.
Revised: For instance, some pharmacophores composed of substructures (e.g., triangle and cycle-based graphlets) can induce molecular toxicity.

Original: These graph attributes and substructures that are essential fordistinguishing anomalous graphs from normal graphs are regarded as anomalous attributes andanomalous substructures .
Revised: These graph attributes and substructures that are essential for distinguishing anomalous graphs from normal graphs are regarded as anomalous attributes and anomalous substructures , respectively.

Original: Most existing graph mining methods, especially those graph convolutionbased models which have achieved state-of-the-art performance [8, 9, 10, 11], focus on the aggregationof the neighbor-structure information.
Revised: Most existing graph mining methods, especially those graph convolution-based models which have achieved state-of-theart performance [53, 58, 20, 3], focus on the aggregation of neighborhood-structured information.

Original: However, the anomalous structure information could be allpossible substructures.
Revised: However, the anomalous structure information could be all possible substructures.

Original: In addition, the anomalous attributes and substructures will be diluted by theinformation of normal graphs with overwhelming quantities, which is the imbalance nature of theanomaly problem.
Revised: In addition, the anomalous attributes and substructures will be diluted by the information of normal graphs with overwhelming quantities, which is the imbalance nature of the anomaly problem.

Original: In summary, various anomaly notions in attributes and/or substructures and theimbalance nature makethe graph-level anomaly detection an important yet challenging problem.
Revised: In summary, various anomaly notions in attributes and/or substructures and the imbalance nature make graph-level anomaly detection an important yet challenging problem.

Original: In this paper, we propose a dual-discriminative graph neural network for i mbalanced G raph-level
Revised: In this paper, we propose a dual-discriminative graph neural network for i mbalanced G raph-level

Original: A nomaly D etection ( iGAD ).
Revised: A nomaly D etection ( iGAD ).

Original: Specifically, iGAD investigates a novel anomalous attribute-aware graphconvolution, which separates self-representation and neighborhood-representation when representinga node in a graph.
Revised: Specifically, iGAD investigates a novel anomalous attribute-aware graph convolution. The final node representation composed of the independent encoding on a node itself and the independent encoding on its neighborhoods from different hops.

Original: In this way, theanomalous attribute can be exposed in the graph convolutionwithout being overshadowed by its neighbors, and the attributes of each node in a graph can evolveindividually.
Revised: In this way, the attributes of each node in a graph can evolve independently over the graph convolution process and the anomalous attributes on particular nodes can be exposed.

Original: To enable the anomalous substructure awareness ability for iGAD, solely relying ongraph convolution is tricky.
Revised: However, it is intractable to enable graph convolution to be aware of anomalous substructures.

Original: The graph convolution performs message passing depending on theneighborhood structure only and cannot explore all possible substructures in a graph.
Revised: Graph convolution performs message passing depending on multi-hop neighborhoods only and cannot explore all possible substructures in a graph.

Original: Even the simplesubstructures like triangles, graph convolution is unable to count them in a graph [11].
Revised: Even the simple substructures like triangles, graph convolution is unable to count them in a graph [3].

Original: Graph Kernelslike the Random Walk Kernel (RWK) [12], as another manner to learn graphs, naturally involvethe substructure comparison  among graphs.
Revised: One of the Graph Kernels (GKs), RWK [46], as another way to learn graphs, involves the comparison of any substructures among graphs.

Original: RWK learns graphs by exploring common randomwalk sequences among all input graphs.
Revised: RWK learns graphs by exploring common random walk sequences among all input graphs.

Original: To encode anomalous structural information  into graphrepresentations, we propose to compare random walk sequences of input graphs against learnableanomalous substructures.
Revised: To embed the information of anomalous substructures into graph representations, we propose to compare random walk sequences of input graphs against anomalous substructures.

Original: Anomalous substructuresare modeled as  learnable parameters that can beupdated through back-propagation.
Revised: By modeling anomalous substructures as parameters, they became learnable and can get updated through back-propagation.

Original: We call this design anomalous substructure-aware deep RWK.
Revised: We entitle this design anomalous substructure-aware deep RWK.

Original: Inaddition, we propose a new loss function to target the imbalance nature of  anomaly problem causedby the great difference in the number of normal and anomalous graphs.
Revised: In addition, we propose a new loss function to target the imbalance nature of the anomaly problem caused by the great difference in the number of normal and anomalous graphs.

Original: The newly proposed lossfunction enables iGAD to capture anomalous properties from an imbalanced graph dataset by fittingthe Point Mutual Information (PMI) between graphs and their normal/anomalous properties.
Revised: The newly proposed loss function models the PMI between graphs and their normal/anomalous properties and enables iGAD to capture anomalous information from an imbalanced graph dataset.

Original: Weldingthree designs mentioned above into a graph neural network, the graph-level anomaly detection modeliGAD is achieved.
Revised: Welding three designs mentioned above into a graph neural network, the graph-level anomaly detection model iGAD is achieved.

Original: We conclude the contributions of iGAD as follows:
Revised: We conclude the contributions of iGAD as follows:

Original: • The anomalous attribute-aware graph convolution highlights the anomalous attributesofparticular nodes by keeping the individual evolution of nodes’ attributes.•
Revised: • The anomalous attribute-aware graph convolution exposes anomalous attributes by keeping the independent evolution of each node’s attribute. • Anomalous substructures are modeled as learnable parameters.

Original: The  anomalous substructure-aware deep RWK learns anomalous substructures, and compares the input graphs’ and  learned anomalous substructures’ random walk sequences.•
Revised: The proposed anomalous substructure-aware deep RWK embeds the information of anomalous substructures into graph representations by comparing the random walk sequences in the input graphs and the learned anomalous substructures.

Original:  The PMI-based loss function addresses  the imbalance nature of the anomaly problem andenables iGAD to fit anomalous graphs in the minority effectively.
Revised: • The PMI-based loss function addresses issues brought by the imbalance nature of the anomaly problem and enables iGAD to learn anomalous graphs as a minority effectively.

Original: • Extensive experiments  on four real-world graph datasets about chemical compounds demonstrate the superiority of iGAD.
Revised: • Comprehensive experiments demonstrate the effectiveness of iGAD, which outperforms state-of-the-art graph convolution-based graph classification models and graph-level anomaly detection algorithms on four real-world graph datasets.

Original: The code is available at Anonymous GitHub.
Revised: The code is available at https://github.com/graph-level-anomalies/iGAD.

Original:  Related Work
Revised:  Related Work

Original: Anomaly Detection.
Revised: Anomaly Detection.

Original: The task aims to identify samples that deviate from the majority of samples.
Revised: The task aims to identify anomalies that deviate from the majority of samples.

Original: Early anomaly detection algorithms employ quantitative indicators [13], such as distance  and density,to describe the degree of deviation between samples to detectthe anomalous ones.
Revised: Early anomaly detection algorithms use indicators, such as distance [18] and density [7], to describe the degree of deviation among samples to detect anomalous ones.

Original: In addition, as oneof the representatives, isolation forest-based algorithms [14, 15] detect the anomalous by building thebinary search tree structure to isolate samples.
Revised: For example, isolation forest-based algorithms [25, 26] identify the anomalous by building the binary search tree structure to isolate samples.

Original: Due to the small number of anomalous samples and thehigh cost of labeling, anomaly detection algorithms based on one-class classification [16, 17, 18, 19] and autoencoder [20, 21] also draw the attention of practitioners.
Revised: In addition, anomaly detection algorithms based on one-class classification [39, 38, 36, 62] and autoencoder [2, 63] have also been in the spotlight. For the graph-structured data, most graph anomaly detection algorithms dedicate to detect anomalies (e.g., anomalous nodes [43, 13, 57] and edges [35, 56]) in a single graph [1]. The anomaly notions of graph-level anomalies are more diversified than those of node-level anomalies.

Original: For the anomaly detection task onthe graph-structured data, most algorithms dedicate to the  anomaly detection task in a single graph[22, 23], such as identifying anomaly nodes [24, 25], edges [26, 27], and communities [28].
Revised: For example, node-level anomaly detection methods usually regard nodes that betray network homophily 1 as the anomalies [43]. However, at the graph level, nodes that can achieve an anomalous graph may include but are not limited to the nodes that do not meet homophily. Node-level anomaly detection methods identify anomaly nodes from a micro perspective (e.g., neighborhood structure) while identifying anomalous graphs needs learning the graph dataset at the macro level.

Original:  Graph-level Anomaly detection.
Revised: Employing node-level anomaly detection methods to detect graph-level anomalies could capture some anomalies, but only a few. Moreover, the graph-level anomaly detection algorithms need to consider anomaly notions related to all possible substructures. Hence, graph anomaly detection methods that identify anomalies in a single graph cannot adapt to the graph-level anomaly detection problem through simple modifications. Graph-level Anomaly Detection.

Original: To the best of our knowledge,there is only a few preliminaryworks for this task. 
Revised: To the best of our knowledge, only several preliminary research is proposed for the graph-level anomaly detection task. For example, Zhao et al.

Original: [29] investigated a research question of “should we use  graph classification datato evaluate the anomaly detection model?”
Revised: [61] investigated a research question of “should we use the data for the graph classification task to evaluate graph-level anomaly detection models?”, followed by which two graph-level anomaly detection algorithms [27, 34]

Revised: combined Graph Neural Networks (GNNs) with knowledge distillation and one-class classification to detect anomalous graphs, respectively.

Original: and this task can be regarded as a special case of graphclassification.
Revised: Graph-level anomaly detection can be regarded as a special case of graph classification.

Revised: There is a huge gap between the number of graphs belonging to the normal class versus the anomalous ones.

Original: But graph classification algorithms  assume that the number of graphs in each classis balanced.
Revised: But, graph classification algorithms usually assume that the number of graphs in each class is balanced.

Original: Graph kernels (GKs) [12, 30, 31, 32, 33] used to be the mainstream method for graphclassification.
Revised: GKs [46, 16, 41, 6, 40] used to be the mainstream graph classification methods.

Original: Generally, they calculatepairwise kernel values ( i.e. , similarity) of graphs through welldefined kernel functions to obtain the Gram matrix.
Revised: Generally, GKs use the kernel function to calculate kernel values (i.e., similarity) between pair-wised graphs.

Original: Then, an off-the-shelf classifier can be applied toclassify graphs.
Revised: Afterward, an off-the-shelf classifier is applied to the graph similarity matrix to perform the classification task.

Original: In recent years, researchers pay more attention to GraphNeural Networks (GNNs)[34, 35, 36, 37] since graph classification algorithms based on GKs cannot learn graph representationsexplicitly and be optimized in an end-to-end fashion. 
Revised: Graph classification algorithms based on GKs cannot learn graph representations explicitly and be optimized in an end-to-end fashion. In recent years, graph mining practitioners pay more attention to GNNs [8, 11, 4, 37].

Original: GNNs based on spatial  convolution [10, 38, 39]are summarized as message-passing framework (MPNNs)
Revised: GNNs with spatial graph convolution [20, 45, 19] are summarized as messagepassing framework (MPNNs)

Original: In general, MPNNsupdates the representation of each node by  aggregating information coming from their neighbors andapplying the readout function or pooling  on nodes to obtain graph representations.
Revised: In general, MPNNs update the representation of each node by iteratively aggregating information coming from their neighbors. To represent a graph, MPNNs apply the readout function or pooling operation on node embeddings.

Original: The spatial graphconvolution that mix the representation of nodes and their neighborhoods to represent nodes willsmooth node attributes [43].
Revised: The spatial graph convolution that mix the representation of nodes and their neighborhoods to represent nodes will smooth node attributes [60, 4].

Original: In addition, [44, 45, 46] proved that spatial graph convolution has limitedpower in distinguishing graph structures.
Revised: In addition, [31, 28, 59] proved that spatial graph convolution has limited power in distinguishing graph structures.

Original: Performing message passing between subgraphs can enablespatial graph convolution to capture more graph structural information.
Revised: Performing message passing between subgraphs can enable spatial graph convolution to capture more graph structural information.

Original: But such a method consumesexponentially increases memory and computation complexities [11].
Revised: But such a solution consumes exponentially increases memory and computation complexities [3].

Original: All the abovealgorithms do notconsider anomalous attributes and substructures. Moreover, they are likely to under-fit the anomalousgraphs since they do not propose any solutions for theissue of imbalanced data distribution.
Revised: Graph classification methods mentioned above are likely to underfit the anomalous graphs since they do not have any intrinsic mechanism to target the imbalanced distributions. In iGAD, a novel loss is presented for the imbalanced distributions by modeling PMI.

Revised: In addition, different from existing graph-level anomaly methods [61, 27, 34] that employ the off-the-shelf MPNNs (e.g., GIN [53]) to represent graphs, iGAD can embed the information of anomalous attributes and substructures into graph representations.

Original: Imbalanced Learning.
Revised: Imbalanced Learning.

Original: Learning data with imbalanced distribution can be divided into the dataand algorithm-level approaches.
Revised: Learning data with imbalanced distribution can be divided into the data- and algorithm-level approaches.

Original: The former alleviates the imbalanced distribution of the originaldata by either under-sampling the majority [47] or over-sampling the minority [48].
Revised: The former alleviates the imbalanced distribution of the original data by either under-sampling the majority [48] or over-sampling the minority [9].

Original: However,modifying the original distribution of data will bring a lot of drawbacks inevitably, such as over-fittingand discarding important samples.
Revised: However, modifying the original distribution of data will bring a lot of drawbacks inevitably, such as over-fitting and discarding important samples.

Original: There are different types of approaches  address the issue inalgorithm-level.
Revised: There are different types of approaches to address the issue at the algorithm-level.

Original: The typical approaches include employing post-hoc correction [49], aligning datadistribution, [50] and modifying the loss function [51, 52].
Revised: The typical approaches include employing post-hoc correction [30], aligning data distribution, [52] and modifying the loss function [24, 22].

Original: Considering the different consequencesof false-positive identification ( e.g. , predicting an inactive chemical compound to be active) andfalse-negative identification ( e.g. , predicting a cancer patient as a healthy individual), the situationwill be more complicated in the real-world application [53].
Revised: In the real-world application, considering the different consequences of false-positive identification (e.g., predicting an inactive chemical compound to be active) and false-negative identification (e.g., predicting a cancer patient as a healthy individual), the situation will be more complicated [33]. In this paper, iGAD introduces PMI to the cross-entropy loss function to solve this problem.

Original:  Preliminaries
Revised:  Preliminaries

Original: Notations.
Revised: Notations.

Original: The graph dataset consisting of N graphs is denoted as G =
Revised: The graph dataset consisting of N graphs can be denoted as G =

Original: { G 1 , ..., G N } .
Revised: { G 1 , ..., G N } , where

Original: G i =
Revised: G i =

Original: { V i , E i } describes a graph in G .
Revised: { V i , E i } is an unweighted and undirected graph in G , V i

Original: V i and E
Revised: and E

Original: i are the node and  edge set, respectively.
Revised: i are the node and the edge set, respectively.

Original: x u ∈ R c is theattribute vector of node u ∈ V .

Original: The graph structure of G i can be described by an adjacency matrix 
Revised: The topology of G i can be modeled as an adjacency matrix A i ∈ { 0 , 1 } n × n , where

Original: A ∈ { 0 , 1 } n × n ( A uu = 0 ), where A uv = 1 if there is an edge between nodes u and v , otherwise 0. 
Revised: A i ( u, v ) = 1 if there is an edge between nodes u and v , otherwise 0. A i does not consider the self-loop, A i ( u, u ) = 0 , ∀ u ∈ V i . We use x u ∈ R c to denote the attribute vector of node u ∈ V i . X i ∈ R n × c is the attribute matrix of G i .

Original: The graph G  i has a label y i ∈ { 0 , 1 } ∈ Y , where 1 represents anomalous and 0 represents normal .
Revised: G ′ i = { V ′ i , E ′ i } can be regarded as a substructure in G i , iff V ′ i ⊆ V i and E ′ i ⊆

Revised: E i .

Revised: Y is the label set of G .

Revised: G i is labeled by y i ∈ { 0 , 1 } , where 1 represents anomalous and 0 represents normal .

Original: Problem Definition.
Revised: Problem Definition.

Original: This paper concentrates on the supervised graph-level anomaly detectionproblem.
Revised: This paper concentrates on the supervised graph-level anomaly detection problem.

Original: Given the training set T = { ( G 1 , y 1 ) , ( G 2 , y 2 ) , ... } , we aim to train a model to predict thenormal/anomalous proprieties of unseen graphs.
Revised: Given the training set T = { ( G 1 , y 1 ) , ( G 2 , y 2 ) , ... } , we aim to train a model to predict the normal/anomalous proprieties of unseen graphs.

Original: T contains N 1 anomalous graphs and N 0 normalgraphs ( N 1 ≪ N 0 ).
Revised: T contains N 1 anomalous graphs and N 0 normal graphs ( N 1 ≪ N 0 ).

Original: The imbalanced ratio of T is defined as δ ( δ =
Revised: The imbalanced ratio of T is defined as δ ( δ =

Original: N 1 N 0 + N 1 ).
Revised: N 1 N 0 + N 1 ).

Original: Spatial Graph Convolution Layer.
Revised: Spatial Graph Convolution.

Original: It updates each node’s representation in a graph by aggregatingthe features coming from its one-hop neighbors iteratively.
Revised: It updates each node’s representation in a graph by iteratively aggregating the features coming from its one-hop neighborhood.

Original: The formulation is:
Revised: The formulation is:

Original: [Equation]
Revised: [Equation]

Original: where  N ( u ) denotes the one-hop neighborhood of node u .
Revised: where 0 ≤ k ≤ K , N ( u ) denotes the 1-hop neighborhood of node u .

Original: f
Revised: f

Original: Aggregate denotes how to the aggregateneighborhood information of node u , and f  is the embedding function. 
Revised: Aggregate denotes how to aggregate neighbors of node u , and f combine defines how to combine the immediate representations of node u and its neighborhood. σ is the activation function. Generally, the final representation of the node u is h ( K ) ( u ) . Random Walk Kernels (RWK).

Original: A Graph Dataset … (cid:29)(cid:3) Normal graph : Anomalous graph 
Revised: RWK measures graph similarities by counting common random walk sequences among graphs. There are two graphs G 1 = { V 1 , E 1 } and G 2 = { V 2 , E 2 } . G ×

Revised: = { V × , E × } denotes the direct product graph about G 1 and G 2 , where V × =

Revised: { ( u, u ′ )

Revised: | u ∈ V 1 , u ′ ∈

Revised: V 2 } and E × = { (( u, u ′ ) , ( v, v ′ ))

Revised: | ( u, v ) ∈

Revised: E 1 , ( u ′ , v ′ )

Revised: ∈

Revised: E 2 } .

Revised: Performing random walking on G × acts as doing random walking on G 1 and G 2 simultaneously.

Revised: The adjacency matrix of direct product graph

Revised: A × can be obtained by the Kronecker product operation ⊗ on A 1 and A 2 , that is A × =

Revised: A 1 ⊗ A 2 .

Revised: The number of l -length common walk sequences between G 1 and G 2 is κ ( l ) ( G 1 , G 2 ) =

Revised: q T × ( A × ) l p × , where p × and q × are the vector about the starting and stopping probabilities of random walks on G × , respectively.

Revised: The elements in p × and q × are all ones when there is no prior information.

Revised: Overall, the

Revised: RWK about G 1 and G 2 can be defined as:

Revised: [Equation]

Original: A n o m a l o u s a tt r i b
Revised: In this section, we will describe iGAD in detail (see Figure 1 for an illustration). iGAD contains three key designs: (1) anomalous graph attribute-aware graph convolution; (2) anomalous substructureaware deep RWK; and (3) PMI-based loss function. The first two designs encode the anomalous information on graph attributes and substructures into graph embeddings.

Original: u t e
Revised: The loss function enables iGAD to get rid of the domination from normal graphs. 4.1 iGAD: Anomalous Attribute-aware Graph Convolution As aforementioned in Section 1, the attributes of particular nodes can achieve an anomalous graph. The iterative graph convolution operations which update node representations by performing feature transformation on the mixed or concatenated information about nodes and their neighborhoods will

Revised: A Graph Dataset … (cid:29)(cid:3) Normal Graph : Anomalous Graph

Original: A n o m a l o u s  s  u b s t r u c t u r e
Revised: A n o m a l o u s A tt r i b u t e A n o m a l o u s S u b s t r u c t u r e

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: R ea do u t
Revised: R ea do u t

Original: MLP with the PMI-based loss

Original: I. Anomalous Attribute-aware Graph Convolution 
Revised: I. Anomalous Attribute-aware Graph Convolution (K=2)

Original: II.
Revised: II.

Original: Anomalous Substructure-aware Deep RWK 
Revised: Anomalous Substructure-aware Deep RWK (M=2, n’=4, L=5)

Original: Updatethrough back-propagation A
Revised: Update A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: Updatethrough back-propagation A
Revised: Update A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G …(cid:29)(cid:3) Normal graph …
Revised: G … …

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D E
Revised: D E

Original: F G: :
Revised: F G

Original: C o n ca t e n a t e
Revised: C o n ca t e n a t i o n

Original: Graph representation of … … …
Revised: A n o m a l o u s III.

Original: Predicting the anomalous/normal properties of III.
Revised: PMI-based Loss Function Prediction F E G

Original: Random-walk Graph Kernels (RWK).
Revised: E F G C … … A MLP

Original: Given two graphs G 1 = { V 1 , E 1 } and G 2 = { V 2 , E 2 } , thedirect product graph is defined as G × =
Revised: Figure Taking learning the graph G N in a graph dataset as an example, we illustrate the framework of iGAD with three designs.

Original: { V × , E × } , where V × = { ( u, u ′ ) | u ∈ V 1 , u ′ ∈ V 2 } and 
Revised: I. Anomalous Attribute-aware Graph Convolution, which concatenates the independent encoding on nodes and their neighborhoods to represent nodes in G N .

Original: E × = { (( u, u ′ ) , ( v, v ′ )) | ( u, v ) ∈ E 1 , ( u ′ , v ′ ) ∈
Revised: To obtain the graph representation h ConvG N , a readout function is applied to all node representations.

Original: E 2 } .
Revised: II.

Original: Performing the random walks on G × acts asdoing random walks on G 1 and G 2 simultaneously. The adjacency matrix of direct product graph 
Revised: The Anomalous Substructure-aware Deep RWK, which performs random walks on two learnable anomalous substructures (i.e., S 1 and S 2 ) and the input graph G N simultaneously.

Original: A × can be obtain by the Kronecker product operation, that is A × =
Revised: Deep RWK obtains h KerG N based on kernel values κ ( l ) ( G N , S m ) , 1 ≤ l ≤ L, 1 ≤ m ≤ M .

Original: A 1 ⊗ A 2 .
Revised: III.

Original: The number ofl -length common walk sequence between G 1 and G 2 is κ ( l ) ( G 1 , G 2 ) = q T × ( A × ) l p × , where p × andq × are the vector about the starting and stopping probabilities, respectively. They should be all oneswhen there is no prior information. The RWK about G 1 and G 2 can be defined as:
Revised: The PMI-based

Original: [Equation]
Revised: Loss Function.

Original: In this section, we will describe iGAD in detail (see Figure 1 for the illustration).
Revised: iGAD concatenates h ConvG N and h KerG N to obtain the final graph representation h G N .

Original: iGAD contains threekey designs: (1) anomalous graph attribute-aware graph convolution; (2) anomalous substructureaware deep RWK; and(3) the PMI-based loss function. The first two designs encode the anomalousinformation on graph attributes and substructure into graph embeddings, respectively. The lossfunction empowers iGAD to get rid of the domination of normal graphs with overwhelming quantities.
Revised: Finally, a MLP and the PMI-based loss function are applied to h G N to predict G N .

Original: As aforementioned in Section 1, the attributes of particular nodes can makea graph to be anomalous. A graph convolution that has anomalous graph attribute awareness should violate the basic assumptionin most graph convolution, that a node has similar properties to its neighborhood in a graph. Followingthis assumption and mixing the information of nodes with their neighborhoods together will smooththose node attributes that achieve an anomalous graph [43].
Revised: make node representations over-smoothed [60].

Original: And the anomalous attribute informationwill become more obscure in the anomalous graphs’ embeddings.
Revised: Under such a graph convolution mechanism, only a limited anomalous information can be embedded into corresponding node representations.

Original: We propose that making eachnode’s attribute can evolve independently during the graph convolution process is critical for capturinganomalous attributes.
Revised: We argue that making each node’s attributes evolve independently over the graph convolution process is critical for capturing anomalous attributes.

Original: To achieve that, the graph convolution in iGAD separates the  encoding on anode itself from its  neighborhood when representing nodes, that is, h u = f Combine (cid:16)
Revised: To achieve that, anomalous attribute-aware graph convolution concatenates the independent encoding on the node itself and its k -hop neighborhood (1 ≤ k ≤ K ) to represent each node in a graph. Formally,

Original: [Equation]
Revised: [Equation]

Original: (3) where h u is the  representation ofnode u ∈ V i ,N k ( u ) represents the k th-hop neighborhood of thenode u , h (N k  ( u ) is the representation of node u ’s k th-hop neighborhood, and f Combine denotes theconcatenation operator.
Revised: where h u is the final node representation of u ∈ V i , h ( k ) ( u ) is the representation of node u ’s k -hop neighborhood, and f Combine is the concatenation operation.

Original: h  ( u ) and h (N k  ( u )) are formulated as:
Revised: h (0) ( u ) and h ( k ) ( u ) are formulated as:

Original: [Equation]
Revised: [Equation]

Original: (4) and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: 5) respectively, where  σ represents the activation function,f Aggregate can be the sum or average aggregator, and W ( k ), k ∈ { 0 , ..., K } is the trainable matrices.
Revised: respectively, where H (0) i = X i , MLP denotes a multi-layer perceptron, σ represents the activation function, and W ( k ) is the trainable weight matrix.

Original: To obtain the entire graph representationh ConvG i , we apply the permutation invariant Readout function  on all nodes in the graph G i ,
Revised: To obtain the entire graph representation h Conv G i , we apply the permutation invariant Readout function R ( · ) on all nodes in the graph G i . That is,

Original: [Equation]
Revised: [Equation]

Original: (6) where Readout can be the sum or mean operation.
Revised: (6) where R ( · ) can be the sum or mean operation.

Original: 4.2 iGAD: Anomalous Substructure-aware Deep RWK
Revised: 4.2 iGAD: Anomalous Substructure-aware Deep RWK

Original: The neighborhood aggregation mechanism in graph convolution  utilizes the neighbour-structure asthe clue for message passing, and cannot figure out all possible substructures in a graph [46, 54].
Revised: The neighborhood aggregation mechanism in graph convolution only utilizes the neighborhood structure as the clue for message passing, and cannot explore all possible substructures in a graph [59, 10].

Original: Tomake iGAD to be  anomalous substructure-aware, we turn to the RWK that can explore all possiblesubstructures by performing random walks in a graph.
Revised: To make iGAD to be aware of anomalous substructures, we turn to the RWK which can explore all possible substructures by performing random walks in a graph.

Original: [55] proposed the pioneering work thatrepresents graphs through doing random walks on input graphs and some substructures simultaneously.
Revised: There is a pioneering work [32] that represents graphs by the RWK values about input graphs and some substructures.

Original: This work inspires us to encode  anomalous substructures into graph representations by comparinginput graphs and  anomalous substructures.
Revised: It inspires us to encode information of anomalous substructures into graph representations by performing random walks in input graphs and learnable anomalous substructures simultaneously.

Original: Hence, we propose an anomalous substructure-aware deep RWK in which  the  anomalous substructures are modeled by learnable parameters that can be updatedthrough back-propagation.
Revised: We present an anomalous substructure-aware deep RWK, which models each element in the adjacency matrices of anomalous substructures as a learnable parameter.

Original: Supposed that { S 1 , ..., S M } is the set of M anomalous substructures, andthe size of S m is n ′ .
Revised: Supposed that { S 1 , ..., S M } is the set of M anomalous substructures, and the size of S m is n ′ .

Original: P m ∈ R n ′ × n ′ is the adjacency matrix of S m , where P m ( u ′ , v ′ ) denotes thepossibility that an edge exists between nodes u ′ and v ′ in S m .
Revised: P m ∈ R n ′ × n ′ is the adjacency matrix of S m , where P m ( u ′ , v ′ ) denotes the possibility that an edge exists between nodes u ′ and v ′ in S m .

Original: Due to the symmetry of the adjacencymatrix, P m contains n ′ ( n ′ − 1)learnable parameters.
Revised: Due to the symmetry of the adjacency matrix, P m contains n ′ ( n ′ − 1)parameters.

Original: Refer to Eq. 2 and suppose that the starting andthe stopping possibilities on each node are all one, the number of l -length common random walksequences between the input graph G i and the anomalous substructure S m is:
Revised: Referring to Eq.and supposing that the starting and the stopping possibilities on each node are all ones, the number of l -length common random walk sequences between the input graph G i and the anomalous substructure S m is:

Original: [Equation]
Revised: [Equation]

Original: where 1 ∈ R n ′ n is a column vector full of 1, and E ∈ R n ′ × n is a matrix full of 1.
Revised: where 1 ∈ R n ′ n is a column vector full of 1, and E ∈ R n ′ × n is a matrix full of 1.

Original: Randomwalk kernels with the setting of infinite walk length will stuck in the halting issue [56].

Original: Therefore,iGAD only considers the random walk length from 1 to L .

Original: The proof of Eq.
Revised: The proof of Eq.

Original: 7 can be found in Appendix A.1 .
Revised: 7 can be found in Appendix A.1.

Original: To improve the algorithm efficiency, we can calculate E ( A i ) l through the sparsematrix multiplications, that is E
Revised: Random walk kernels with infinite random walk lengths will often meet the halting issue [42]. Therefore, iGAD only considers the random walk length fromto L . As the size of anomalous substructures ( n ′ ) is small, optimizing ( P m ) l is not necessary. A i is a sparse matrix, and we compute ( A i ) l by the sparse matrix multiplication in PyTorch.

Original:  ( A si ) l
Revised: That is ( A si ) l

Original: = (( EA si ) A si) ... ) A si  ) , where A si is the sparse representationof A i .
Revised: = A si ( A si ... ( A si ( A i )) ... ) , where A si is the sparse version of A i .

Original: As the size of anomalous substructures ( n ′ ) is usually small, optimizing ( P m ) l is not necessary.

Original: We put all kernel values about G i and S m , m ∈ { 1 , .., M }  into a vector, that is,
Revised: Afterward, all deep RWK values about G i and S m , m ∈ { 1 , .., M } will be assembled into a vector a G i . That is,

Original: [Equation]
Revised: [Equation]

Original: where a G i ∈ R ML .
Revised: where a G i ∈ R ML .

Original: The graph representation h kerG i based on deep RWK is:
Revised: The graph representation h kerG i based on deep RWK is:

Original: [Equation]
Revised: [Equation]

Original: where MLP is the multi-layer perceptrons.

Original: To get the final graph embedding h G i for G i ∈ G , we concatenate the h ConvG i and h KerG i together.
Revised: The final graph representation h G i for G i ∈ G can be obtained by concatenating h ConvG i and h KerG i together.

Original: Based on h G i , the anomalous attribute-aware graph convolution and anomalous substructure-awaredeep RWK are welded into a graph neural network, and these two parts can be updated through theback-propagation simultaneously.
Revised: Based on h G i , the anomalous attribute-aware graph convolution and anomalous substructureaware deep RWK are welded into a graph neural network.

Original: 4.3 iGAD: PMI-based Loss Function
Revised: 4.3 iGAD: PMI-based Loss Function

Original: The number of normal graphs is much more than that of anomalous graphs.
Revised: The number of normal graphs is much more than that of anomalous graphs.

Original: Under the imbalancenature of  anomaly problem, normal graphs will dominate the machine learning model.
Revised: Under the imbalance nature of the anomaly problem, normal graphs will dominate the machine learning model.

Original: To addressthis issue, we propose a new loss function that can model the Pointwise Mutual Information (PMI)between input graphs and their anomalous/normal properties.
Revised: Targeting this issue, we propose a new loss function that can model the PMI between input graphs and their anomalous/normal properties.

Original: Specifically, we use fy i ( Gi ; Θ) todenote the proposed model iGAD, where Θ represents model parameters.
Revised: Specifically, we use f ( G ; Θ) to denote the proposed model iGAD, where Θ represents model parameters.

Original: iGAD models thePMIbetween input graph G i and its anomalous/normal properties can be formulated as:
Revised: Modeling the PMI between the input graph G i and its anomalous/normal properties can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: 10) where the right part represents the PMI between G i and y i , P ( y i ) is the proportion of anomalousgraphs or normal graphs in the training set T .
Revised: where the right part represents the PMI between G i and y i , P ( y i ) is the proportion of anomalous graphs or normal graphs in the training set T .

Original: Compared withthe most existingloss, especiallythe cross-entropy loss  that models the conditional distribution (that is f y 
Revised: Compared with most existing loss functions, especially the loss function that models the conditional distribution about y i given G i (i.e., f y i

Original:  i 
Revised: ( G i ; Θ) ∼

Original: ( G i ; Θ) ∼ P Θ ( y i | G i ) ),modeling PMI can make the model give priority to learning the information that reveals the essentialcorrelation between graphs and their anomalous/normal proprieties.
Revised: P Θ ( y i | G i ) ), PMI enables iGAD to give priority to learning the information that reveals the essential correlation between graphs and their labels.

Original: Eq.
Revised: Eq.

Original: 10 can be reformatted as:
Revised: 10 can be reformatted as:

Original: [Equation]
Revised: [Equation]

Original:  After softmax normalization,we can obtain the distribution of  given G i concerning y i :
Revised: [Equation] After softmax normalization, the distribution of y i given G i is formulated as:

Original: [Equation]
Revised: [Equation]

Original: Incorporating P Θ ( y i | G i ) in to the framework of  cross-entropy loss, we can obtain a new loss function L :
Revised: Incorporating P Θ ( y i | G i ) in to the framework of the cross-entropy loss function, we can obtain the PMI-based loss function L :

Original: [Equation]
Revised: [Equation]

Original: Specifically, iGAD predict the graph G i through:
Revised: Specifically, iGAD predict the graph G i : [Table] [Equation]

Revised: testing stage

Original: [Equation]
Revised: [Equation]

Original: The reason that we use argmax y i f ∗ y
Revised: The reason that we use argmax y i f ∗ y

Original: i
Revised: i

Original: ( G i ; Θ) to predict a graph in the testing phase is in Appendix A.2 .
Revised: ( G i ; Θ) to predict a graph in the testing stage can be found in Appendix A.2.

Original: There are otheralternative loss functions to solve the imbalanced data distribution of anomalyproblem, the typical representative is the cost-sensitive loss function that assigns or learns differentweights to the majority and the minority in data [57, 58, 59].
Revised: There are other loss functions targeting the difficulties brought by the imbalanced distributions, such as the cost-sensitive loss function that assigns or learns different weights to the majority and the minority samples [29, 49, 12].

Original: Compared with them, the PMI-basedloss function does not introduce any new parameters that need to be carefully tuned.
Revised: Compared with them, the PMI-based loss function does not introduce any new parameters that need to be carefully tuned.

Original: It comes in theform of the most widely used cross-entropy loss function but outputs the prediction results that canrealize the max mutual information with the input data.
Revised: It comes in the form of the most widely used cross-entropy loss function and gives prediction results that can realize the maximized PMI between inputs and prediction results.

Original: 4.4 Algorithm Description&Time Complexity
Revised: 4.4 Algorithm Description & Computational Complexity

Original: We include the pseudocode of iGAD  inappendix (see Algorithm 1).
Revised: The pseudocode of iGAD is included in Algorithm 1 in Appendix A.1.

Original: We employ the mini-batchtraining techniques [60].
Revised: We divide graphs into different mini-batches to train iGAD [23].

Original: But for illustrating how iGAD predicts a graph more clearly,in thepseudocode, we suppose that the batch size is  1.
Revised: But for illustrating how iGAD predicts a graph more clearly, the batch size is set as 1 in Algorithm 1.

Original: Given a set of labeled graphs, we initialize anomaloussubstructures and other parameters (Line 1), and calculate the proportion of normal and anomalousgraphs (Line 2) at the beginning.
Revised: Given the training set, we initialize anomalous substructures and other parameters (Line 1 in Algorithm 1) and calculate the proportion of normal and anomalous graphs (Line 2).

Original: For each graph, iGAD learns its graph representation (Lines 3 to15).
Revised: For each graph, iGAD learns its graph representation (Lines 3 to 15).

Original: Based on the graph representation, an MLP equipped with the PMI-based loss function predictsthe graph (Lines 16 and 17).
Revised: Based on the graph representation, a MLP equipped with the PMI-based loss function predicts the graph (Lines 16 and 17).

Original: At the back-propagation stage, anomalous substructures and otherparameters will be updated.
Revised: At the back-propagation stage, anomalous substructures and other parameters will get updated.

Original: In the next epoch, deep RWK performs random walk on the input graphsand the updated anomalous substructures.
Revised: At the next epoch, deep RWK performs random walking on the input graphs and the updated anomalous substructures simultaneously.

Original: Lemma 1{+
+} The time complexity of iGAD is O ( n + | E | ) , where n and | E | is the number of nodes andedges in the largest graph.
Revised: Lemma {+
+} The computational complexity of iGAD is O ( n + | E | ) , where n and | E | are the number of nodes and edges, respectively.

Original: (We include the proof in Appendix A.3 ). [Table]
Revised: (The detailed analysis can be found in Appendix A.3).

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct a series of experiments to study the performance of iGAD on the graphlevel anomaly detection task.
Revised: In this section, we conduct a series of experiments to study the performance of iGAD on the graphlevel anomaly detection task.

Original: Specifically, we investigate the following  questions:
Revised: Specifically, we investigate the following six questions:

Original: Q1.
Revised: Q1.

Original: How efficient is iGAD on the graph-level anomaly detection task?
Revised: How about the performance of iGAD on the graph-level anomaly detection task?

Original: Q2.
Revised: Q2.

Original: How efficient is the anomalous graph attribute-aware graph convolution in iGAD?
Revised: Does iGAD benefit from the PMI-based loss function?

Original: Q3.
Revised: Q3.

Original: How efficient is iGAD when it utilizes pre-defined anomalous substructures?Q4. Can the PMI-based loss function improve models’ fitting on imbalanced graph data distribution?
Revised: How about the performance of the anomalous graph attribute-aware graph convolution in iGAD? Q4. How about the performance of iGAD when it utilizes pre-defined anomalous substructures?

Original: Q5.
Revised: Q5.

Original: Can the anomalous substructure-aware deep RWK in iGAD learn the anomalous substructures?
Revised: Can the anomalous substructure-aware deep RWK in iGAD learn the anomalous substructures?

Original: Q6.
Revised: Q6.

Original: How does the max length of random walk sequences influence the performance of iGAD?
Revised: How does the maximum length of random walk sequences influence the performance of iGAD?

Original: The questions from Q2 to Q4 can be regarded as the ablation study on our three main design elements,respectively: (
Revised: The questions from Q2 to Q4 can be regarded as the ablation study on our three main design elements, they are (a).

Original: a
Revised: cross-entropy loss function versus the proposed PMI-based loss function; (b).

Original: regular convolution versus the proposed graph convolution; (b).
Revised: general graph convolution versus the proposed graph convolution; (c).

Original: RWK versus theproposed deep RWK; (c).
Revised: RWK versus the proposed deep RWK. [Table] [Figure] [Table]

Revised: [Figure]

Original: cross-entropy loss function versus the proposed loss function.
Revised: [Table] [Figure] [Table] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: 5.1 Experimental Setup
Revised: 5.1 Experimental Setup

Original: Datasets.
Revised: Datasets.

Original: SW-610, MOLT-4, PC -3, and MCF-7 are four real-world graph datasets of chemicalcompounds.
Revised: SW-610, MOLT-4, PC-3, and MCF-7 are four real-world graph datasets.

Original: These datasets are collected fromthe PubChem website. PubChem  records the anticancer activity testing result (“active" or “inactive") of compounds on different types of cancer celllines.
Revised: These datasets are collected from PubChem 2 , which records a tremendous amount of chemical compounds and their anti-cancer activity testing results (“active” or “inactive”) on different types of cancer cell lines.

Original: We regard the compound with the outcome “active" as anomalous, and “inactive" as normal.
Revised: We regard the chemical compounds with the outcome “active” as anomalous [Table] and “inactive” as normal.

Original: For example, the graph dataset SW-620 is about the anti-cancer activity of a series of compounds onthe colon cancer cell line.
Revised: For example, the graph dataset SW-620 is about 40,532 chemical formulas and their anti-cancer activity testing results on the colon cancer cell line.

Original: Compounds that exhibit antibody activity against colon cancer are labeledas anomalous  graphs.
Revised: Chemical compounds that exhibit antibody activity against colon cancer are labeled as anomalous graphs, otherwise normal graphs.

Original: The statistical information about these four datasets is shown in Table 1. 
Revised: The statistical information about these four datasets is shown in Table 1. [Table] [Table]

Original: Baselines&Experimental Settings.
Revised: Baselines & Experimental Settings.

Original: To investigate Q1 , we compare iGAD with the following baselinemodels: (1) GCN+readout (abbreviated GCN) and GIN [8], these two algorithms employ messagepassing graph convolution to learn node representation and apply  readout on node representations toobtain graph representations and detect anomalous graphs; (2)DGCNN [9] and SOPool [42].
Revised: To investigate Q1, we compare iGAD with the following baseline models: (1) GCN[20] + Readout function (abbreviated GCN) and GIN [53], these two algorithms employ spatial graph convolution to learn node representation and apply the readout function to node representations to obtain graph representations and classify graphs; (2)

Revised: DGCNN [58] and SOPool [51].

Original: In thesetwo models, message passing-based graph convolution and  graph pooling  are employed to completethe graph-level anomaly detection; (3) RWGNN [55] employs random walk sequences to representgraphs and then detect anomalous graphs based on graph representations.
Revised: In these two models, spatial graph convolution and the graph pooling operation are employed to downsize graphs and complete the graph classification task; (3) RWGNN [32] employs random walk sequences to represent graphs and then classify graphs based on graph representations.

Original: All the above baselinesuse the cross-entropy loss.
Revised: All the above baselines are supervised and use the cross-entropy loss function; (4) The graph-level anomaly detection methods GLocalKD [27] and OCGTL [34]. The former adopts random distillation to detect the locally- and globally-anomalous graphs and the latter combines the one-class classification and graph transformation learning to identify anomalous graphs. These two algorithms only use labeled normal graphs as supervision information.

Original: For the fairness of experiments and to answer Q2 , we also comparetheperformance of iGAD with the above baselines in which the loss is substituted by the proposed
Revised: For the fairness of experiments and to answer Q2, we also compare iGAD with GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD.

Original: PMI-based loss function. We denote these baselines with the  PMI-based loss function as GCN-GAD, GIN-GAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD.
Revised: These algorithms are the graph anomaly detection (GAD) variants of GCN, GIN, DGCNN, SOPool, and RWGNN. They replace the crossentropy loss in the original graph classification algorithms with the proposed PMI-based loss function.

Original: To answer Q3 , we consider a variantiGAD-1 that only utilizes the anomalous graph attribute-aware graph convolution to  detect anomalousgraphs.
Revised: To answer Q3, we consider a variant of iGAD —iGAD-1 that only utilizes the anomalous graph attribute-aware graph convolution to represent graphs and detect anomalous graphs.

Original: To answer Q4 , we consider the other variant  iGAD-2 in which the random walk comparesthe input graphs against the pre-defined anomalous graph substructures.
Revised: To answer Q4, we consider the other variant named iGAD-2 in which the random walking performs on the input graphs and the pre-defined anomalous graph substructures simultaneously.

Original: Specifically, 3-star,triangle,tailed triangle, and 4-cycle graphlets are pre-defined graph anomalous substructures (see Figure 4in Appendix for these graphlets).
Revised: Specifically, 3-star,

Original: The parameter setting and the implementation about all the abovemodels are shown in Appendix B.1. We use 5-fold cross-validation to conduct experiments.
Revised: [Figure]

Original: We take the stratified sampling to divide thetraining and the testing set concerning the division ratio 80% to 20%. Stratified sampling ensures thattraining and testing sets have a consistent ratio between normal and anomalous graphs. The description about evaluation metrics AUC , Recall , Recall(A) , and F - score is in Appendix B.1. [Table]
Revised: the setting M =5 and n ′ =8.

Original: [Figure] [Table] [Figure] [Table]
Revised: The second row reports the performance of iGAD under the setting M =5 and n ′ =10.

Original: [Figure]

Original: [Table] [Figure] [Table] [Figure]
Revised: triangle, tailed triangle, and 4-cycle graphlets are pre-defined graph anomalous substructures (see

Original: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: Figure 4 in Appendix B for these substructures).

Original: [Figure] [Table] [Figure] [Table]
Revised: The parameter setting and the implementation of all the above models can be found in Appendix B.1.

Original: [Figure]

Original: [Table] [Figure] [Table] [Figure]
Revised: We report the experimental results under 5-fold cross-validation.

Original: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: 5.2 Experimental Results

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure] [Table] [Figure]
Revised: This section answers Q1 - Q6 by A1 - A6, respectively.

Original: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: A1.

Original: [Figure] [Table] [Figure] [Table]
Revised: iGAD demonstrates effective performance in identifying graph-level anomalies.

Original: [Figure] [Table] [Figure] [Table]
Revised: The performance of iGAD exceeds all graph classification baselines, their corresponding

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table] 5.2 Experimental Results A1. iGAD is effective in graph-level anomaly detection.
Revised: GAD variants, and the state-of-the-art graph-level anomaly detection methods.

Original: Observed from Table 2 that: 1) Comparedwith GCN, GIN, DGCNN, SOPool, and RWGNN, iGAD achievesthe superior performance on alldatasets in terms of all evaluation metrics; 2) iGAD also beats GCN-GAD, GIN-GAD, DGCNNGAD, SOPool-GAD, and RWGNN-GAD concerning all evaluation metrics. Overall, iGAD exceedsall baselines including the models that are extended with the proposed PMI-based loss function. [Figure]
Revised: Observed from Table 2: 1) Compared with GCN, GIN, DGCNN, SOPool, and RWGNN, iGAD achieves superior performance on all datasets in terms of all evaluation metrics; 2) iGAD also beats GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD in terms of all evaluation metrics. In addition, as shown in Table 3, iGAD outperforms two graph-level anomaly detection methods GLocalKD and OCGTL with obvious advantages in terms of AUC.

Original: A2.
Revised: A2.

Revised: The PMI-based loss function can capture essential correlations between graphs and their anomalous/normal proprieties even if the data distribution is imbalanced.

Revised: As shown in Table 2, compared with the baseline models with the cross-entropy loss function (GCN, GIN, DGCNN,

Revised: SOPOOL, and RWGNN), their GAD variants equipped with the newly proposed PMI-based loss function (GCN-GAD, GIN-GAD, DGCNN-GAD, SOPOOL-GAD, and RWGNN-GAD) witness a significant performance improvement regarding Recall and Recall(A) on all datasets.

Revised: For example, on the dataset SW-620, the Recall and Recall(A) value achieved by DGCNN are 56.02 ± 1.16 and 12.74 ± 2.

Original:  The experiment results indicate that the anomalous attributeaware graph convolution that separates the nodes and their neighborhood when representing nodes is effective in highlightinganomalous attributes.
Revised: respectively. In contrast, 72.65 ± 0.63 and 67.84 ± 1.78 are reported by DGCNN-GAD. A3. The experiment results indicate that the anomalous attribute-aware graph convolution achieves competitive performance in encoding anomalous attributes.

Original: Observed from Table 2, iGAD-1 significantly outperforms all baseline algorithms on all datasets interms of AUC , Recall , and F-score .
Revised: As shown in Table 2, iGAD-1 significantly outperforms all baseline algorithms on four graph datasets in terms of AUC , Recall , and F-score .

Original: Concerning Recall(A) , on thedataset MCF-7, iGAD-1 has a little performance degradation compared with DGCNN-GAD.
Revised: Concerning Recall(A) , on the dataset MCF-7, iGAD-1 has a little performance degradation compared with DGCNN-GAD.

Original: Specifically, DGCNN-GAD achieves69.37 ± 3.05 on MCF-7, while iGAD-1 gets 67.67 ± 2.54 .
Revised: Specifically, DGCNN-GAD achieves 69.37 ± 3.05 on MCF-7, while iGAD-1 gets 67.67 ± 2.54 .

Original: But iGAD-1beats DGCNN-GAD on other datasets in terms of Recall(A) .
Revised: But iGAD-1 beats DGCNN-GAD on other datasets in terms of Recall(A) .

Original: A3.
Revised: A4.

Revised: iGAD is also effective when it is equipped with pre-defined anomalous substructures.

Original: As shown in Table 2, iGAD-2 achieves obvious advantagesagainst all baseline models.
Revised: As shown in Table 2, iGAD-2 demonstrates obvious advantages against all baseline models.

Original: Moreover, iGAD-2 also beats iGAD-1on MOLT-4, PC-3, and MCF-7 regarding all evaluation metrics.
Revised: Moreover, iGAD-2 also beats iGAD-1 concerning all metrics on MOLT-4, PC-3, and MCF-7.

Original: Not surprisingly, iGAD beats iGAD-2  on all datasets concerning AUC , Recall(A) , and F-score .
Revised: iGAD-2 does not outperform than iGAD on all datasets concerning AUC , Recall(A) , and F-score .

Original: While on PC-3 and MCF-7, iGAD-2has a slight lead over iGAD regarding Recall .
Revised: But, on PC-3 and MCF-7, iGAD-2 has a slight lead over iGAD regarding Recall .

Original: The experiment results demonstrate that in most cases,the performance of iGAD equipped with a deep random kernel is better than  using pre-definedanomalous substructures and algorithms that do not consider anomalous substructures.
Revised: The experiment results demonstrate that, in most cases, the performance of iGAD equipped with a deep RWK is better than iGAD-2 using pre-defined anomalous substructures and algorithms that do not consider anomalous substructures.

Original: Based onthe above analysis, we can conclude that comparing thesame number of random walk sequencesbetween the input graph and pre-defined substructure can improve anomalous graph detection to acertain extent, but the algorithm that can  learn the anomalous substructures is better.A4.
Revised: Based on the above analysis, we can conclude that comparing the random walk sequences between the input graph and pre-defined substructure can improve graph-level anomaly detection to a certain extent, but the algorithm that can automatically learn the anomalous substructures is better.

Original: The PMI-based loss function can capture essential correlations between graphs and anomalous/normal proprieties even if the data distribution is extremely imbalanced.

Original: As shown in Table 2,compared with the baseline models with the cross-entropy loss (GCN, GIN, DGCNN, SOPOOL, and

Original: RWGNN)

Original: , the baselines equipped with the newly proposed PMI-based loss function (GCN-GAD,

Original: GIN-GAD, DGCNN-GAD, SOPOOL-GAD, and RWGNN-GAD) witness significant experimentalresult increases regarding Recall and Recall(A) on all datasets.

Original: For example, on the dataset SW-620,

Original: [Figure]

Original: Recall and Recall-A scores of DGCNN on SW-620 are 56.02 ± 1.16 and 12.74 ± 2.41 , in contrast, 72.65 ± 0.63and 67.84 ± 1.78 are achieved by DGCNN-GAD.

Original: A5.
Revised: A5.

Original: The anomalous substructuresin four datasets learned by iGAD are demonstrated in Figure 2.
Revised: The anomalous substructures learned by iGAD are demonstrated in Figure 2.

Original: Wecan observe that anomalous graph substructures on four datasets are composed of the triangle, thetailed triangle, and the 4-cycle graphlets.
Revised: We can observe that anomalous graph substructures on four datasets are composed of the triangle, the tailed triangle, and the 4-cycle graphlets.

Original: It explains why iGAD-2 can achieve comparable resultswith iGAD (as shown in Table 2).
Revised: It explains why iGAD-2 can achieve competitive results compared with iGAD (as shown in Table 2 and A4 ).

Original: We can draw a preliminary conclusion that the graph substructureshaving the triangle pattern, the tailed triangle pattern, and the 4-cycle graphlets are important fordistinguishing anomalous graphs from normal graphs on these four graph datasets.
Revised: We can draw a preliminary conclusion that the graph substructures having the triangle, the tailed triangle, and the 4-cycle graphlets are important for distinguishing anomalous graphs from normal graphs on these four graph datasets.

Original: A6.
Revised: A6.

Original: As shown in Figure 3, we report the performance of iGAD under different max random walklengths.
Revised: The performance of iGAD under different maximum random walk lengths L is shown in Figure 3.

Original: Under the setting that the anomalous substructures contain 8 nodes, on different datasets, thebest F-score valuecan be achieved when the max random walk length ranges from 4 to 6. In addition,under the setting that the anomalous substructures contain 10 nodes, when the max random walklength ranges from 5 to 7, the best F-score value on different datasets can be reached.
Revised: Under the setting that iGAD with 5 anomalous substructures and each of substructures has 8 nodes, on these four graph datasets SW-620, MOLT-4, PC-3, and MCF-7, iGAD can achieve the highest F-score value when the maximum random walk length ranges from 4 to 8. In addition, when each anomalous substructure contains 10 nodes, iGAD can reach the best F-score score at the maximum random walk length from 5 to 9.

Original: The resultsverify that appropriately increasing the  length of the maximum step can improve the effectiveness ofiGAD since  longer random walk sequences can capture more structural information.
Revised: The results verify that appropriately increasing the maximum length of random walking can improve the performance of iGAD since more much longer random walk sequences can capture more graph structure information.

Original: Overall, the above experiments demonstrate the effectiveness of iGAD in graph-level anomalydetection.
Revised: Overall, the above experiments demonstrate the effectiveness of iGAD in graph-level anomaly detection.

Original: In addition, we put moreexperiments on parameter analysis  in Appendix B.2.
Revised: We put more parameter analysis experiments in Appendix B.2.

Original: Conclusion
Revised: Finally, we discuss the limitations of the proposed graph-level anomaly detection model iGAD. In practice, the nodes and/or substructures at a particular position in a graph can be the reason for the graph’s anomalous properties. For example, an Alcohol compound can be obtained by replacing the hydrogen atom at the end of the Alkane compound with a hydroxy group (i.e., -OH). But, iGAD cannot capture the position information of nodes and substructures in graphs.

Original: In this paper, we proposed iGAD, a dual-discriminative graph neural network for imbalanced graphlevel anomaly detection. To address various anomaly notions referring to anomalous attributes and/orsubstructures, and the imbalance nature of anomaly problem, iGAD proposed three key designs.
Revised: In addition, iGAD is a supervised algorithm. Its performance on identifying the graph-level anomalies that are completely different from the training anomalies may not be superior. Our future work is to explore graph-level anomaly detection algorithms that can identify more extensive anomalous graphs.

Revised:  Conclusion

Original: They are anomalous attribute-aware graph convolution, anomalous substructure-aware deep RWK,and the PMI-based loss function. Experiment results onfour real-world datasets demonstrate theeffectiveness of our proposed model in the graph-level anomaly detection task.
Revised: This paper focused on graph-level anomaly detection which is a research topic of great practical value but less explored. We highlighted the challenges brought by the imbalanced distribution and various anomaly notions in graph attributes and substructures. In addition, the significant differences between detecting anomalous graphs in a graph dataset and identifying anomalous nodes in a single graph were also discussed.

Original: The limitation of iGAD is that it cannot capture the position/location information. In practice, theparticular position/location of nodes and/or substructures can be thereason for anomalous propertiesof a graph.
Revised: The proposed method iGAD regarded graph-level anomaly detection as a special case of graph classification. In iGAD, the anomalous attribute-aware graph convolution and anomalous substructure-aware deep RWK embed graph-level anomaly information into graph representations.

Original: [61] did a preliminary study on enabling GNNs to capture the position/location informationof nodes in a graph by considering anchor nodes.
Revised: A MLP with the PMI-based loss function is employed to distinguish anomalous samples from normal samples based on graph representations. We did extensive experiments on four graph datasets, and iGAD achieved superior performance on the graph-level anomaly detection task. Acknowledge

Original: We leave exploring the anomalous caused by theposition/location of nodes and/or substructures to future work.
Revised: We greatly appreciate the insightful and constructive reviews from reviewers. This work is funded by the Australian Research Council Discovery Early Career Researcher Award (ARC DECRA) Project (No. DE200100964).

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] see section 6.
Revised: [Yes] see section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] see Appendix A.1 and

Original: A.2. (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] see Appendix A.1 and A.2.
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] see section 1.

Original:  (b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?[Yes] see section 5 and Appendix B.
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original:  (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We report standard deviation error bars over different data splits.

Original:  (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs,internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] see Appendix B.
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whetherand how consent was
Revised: [N/A] (d) Did you discuss whether

Original: obtained

Original:  from people whose data you’re using/curating
Revised: and how consent was obtained from people whose data you’re us- ing/curating?

Original: N

Original: /A] [Table] ) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?

Original: [N/A
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... [Table] ) Did you include the full text of instructions given to participants and screenshots, if applicable?[Table]
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: N

Original: [Table]

Original: [Table]

Original: [Table]

Original: you

Original: [Table]

Original:  any potential participant [Table] , with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A [Table] [Table] ) Did you include the estimated hourly wage paid to participants and the total amount spent on 
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: participant compensation?

Original: [N/A
Revised: [N/A]

Original: [Table]