Original: Dual-discriminative Graph Neural Network for Imbalanced Graph-level Anomaly Detection
Revised: Graph Neural Networks for Imbalanced Graph-level Anomaly Detection

Original: Abstract
Revised: Abstract

Original: Graph-level anomaly detection aims to distinguish anomalous graphs in a graph dataset from normal graphs.
Revised: Graph-level anomaly detection aims to distinguish anomalous graphs in a graph dataset from normal graphs.

Original: Anomalous graphs represent a very few but essential patterns in the real world.
Revised: Anomalous graphs represent a very few but essential patterns in the real world.

Original: The anomalous property of a graph may be referable to its anomalous attributes of particular nodes and anomalous substructures that refer to a subset of nodes and edges in the graph.
Revised: The anomalous property of a graph may be referable to its anomalous attributes of particular nodes and anomalous substructures that refer to a subset of nodes and edges in the graph.

Original: In addition, due to the imbalance nature of anomaly problem, anomalous information will be diluted by normal graphs with overwhelming quantities.
Revised: In addition, due to the imbalance nature of anomaly problem, anomalous information will be diluted by normal graphs with overwhelming quantities.

Original: Various anomaly notions in the attributes and/or substructures and the imbalance nature together make detecting anomalous graphs a non-trivial task.
Revised: Various anomaly notions in the attributes and/or substructures and the imbalance nature together make detecting anomalous graphs a non-trivial task.

Original: In this paper, we propose adual-discriminative graph neural network for graph-level anomaly detection, namely iGAD.
Revised: In this paper, we propose a graph neural network for graph-level anomaly detection, namely iGAD.

Original: Specifically, an anomalous graph attribute-aware graph convolution and an anomalous graph substructure-aware deep Random Walk Kernel (deep RWK) are welded into a graph neural network to achieve the dual-discriminative ability on anomalous attributes and substructures. 
Revised: Specifically, an anomalous graph attribute-aware graph convolution and an anomalous graph substructure-aware deep Random Walk Kernel (deep RWK) are welded into a graph neural network to achieve the discriminative ability on anomalous attributes and substructures. Deep

Original: The deep RWK in iGAD makes up for the deficiency of graph convolution in distinguishing structural information caused by the simple neighborhood aggregation mechanism. 
Revised: RWK in iGAD makes up for the deficiency of graph convolution in distinguishing structural information caused by the simple neighborhood aggregation mechanism. Further we

Revised: propose

Original: Further, we propose a Point Mutual Information (PMI)-based loss function to target the problems caused by imbalance distributions. 
Revised: a Point Mutual Information (PMI)-based loss function to target the problems caused by imbalance distributions. PMI-based loss function

Revised: enables

Original: The loss function enables iGAD to capturethe essential correlation between inputgraphs and their anomalous/normal properties.
Revised: iGAD to capture essential correlation between input

Revised: graphs and

Revised: their anomalous/

Revised: normal properties.

Original: We evaluate iGAD on four realworld graph datasets.
Revised: We evaluate iGAD on four real-world graph datasets.

Original: Extensive experiments demonstrate the superiority of iGAD on the graph-level anomaly detection task.
Revised: Extensive experiments demonstrate the superiority of iGAD on the graph-level anomaly detection task

Original:  Introduction
Revised:  Introduction

Original: Graphs have been used as an indispensable tool to describe relational data, such as chemical compounds, transport networks, and social networks.
Revised: Graphs have been used as an indispensable tool to describe relational data, such as chemical compounds, transport networks, and social networks.

Original: For instance, in a graph that models a chemical compound, nodes represent atoms and edges describe chemical bonds between atoms.
Revised: For instance, in a graph that models a chemical compound, nodes represent atoms and edges describe chemical bonds between atoms.

Original: Learning graphs to understand the world has been drawing considerable attention in recent years [5, 14, 15, 50].
Revised: Learning graphs to understand the world has been drawing considerable attention in recent years [5, 14, 15, 51].

Original: Given a graph dataset containing a set of graphs,the graph-level anomaly detectiontask aims to detect graphs with anomalous information.
Revised: Given a graph dataset containing a set of graphs, graph-level anomaly detection aims to identify graphs with anomalous information.

Original: Generally, these anomalous graphs with special characteristics or functions are the minority in a graph dataset.
Revised: Generally, these anomalous graphs with special characteristics or functions are the minority in a graph dataset.

Original: For example, among all antibody molecules, only a few of them exhibit anti-cancer activity on a specific type of cancer cell line [54].
Revised: For example, among all antibody molecules, only a few of them exhibit anti-cancer activity on a specific type of cancer cell line [55].

Original: Graph-level anomaly detection is a research topic of great practical value, say, it can support cancer drug discovery by identifying molecules with anti-cancer activity from other antibody molecules [47].
Revised: Graph-level anomaly detection is a research topic of great practical value, say, it can support cancer drug discovery by identifying molecules with anti-cancer activity from other antibody molecules [48].

Original: Also, the research can help to reveal pathogenic brain mechanisms by distinguishing the brain structures of patients with neurological disorders from healthy individuals [21].
Revised: Also, the research can help to reveal pathogenic brain mechanisms by distinguishing the brain structures of patients with neurological disorders from healthy individuals [21].

Original: In practice, the attributes of particular nodes are a contributing factor to a graph’s anomalous properties.
Revised: In practice, the attributes of particular nodes are a contributing factor to a graph’s anomalous properties.

Original: For example, the hydrogen and oxygen atoms at the end of molecules are the key to discriminating between Alcohol and Alkane [44].
Revised: For example, the hydrogen and oxygen atoms at the end of molecules are the key to discriminating between Alcohol and Alkane [45].

Original: Graph substructures can be another factor for a graph’s anomalous properties.
Revised: Graph substructures can be another factor for a graph’s anomalous properties.

Original: For instance, pharmacophores composed of some particular substructures can induce molecular toxicity.
Revised: For instance, pharmacophores composed of some particular substructures can induce molecular toxicity.

Original: These graph attributes and substructures that are essential for distinguishing anomalous graphs from normal graphs are regarded as anomalous attributes and anomalous substructures , respectively.
Revised: These graph attributes and substructures that are essential for distinguishing anomalous graphs from normal graphs are regarded as anomalous attributes and anomalous substructures , respectively.

Original: Most existing graph mining methods, especially those graph convolution-based models which have achieved state-of-the-art performance [53, 58, 20, 3], focus on the aggregation of neighborhood-structured information.
Revised: Most existing graph mining methods, especially those graph convolution-based models which have achieved state-of-the-art performance [54, 59, 20, 3], focus on the aggregation of neighborhood-structured information.

Original: However, the anomalous structure information could be all possible substructures.
Revised: However, the anomalous structure information could be all possible substructures.

Original: In addition,the anomalous attributes and substructures will be diluted by the information of normal graphs with overwhelming quantities, which is the imbalance nature of the anomaly problem.
Revised: In addition, anomalous attributes and substructures will be diluted by the information of normal graphs with overwhelming quantities, which is the imbalance nature of the anomaly problem.

Original: In summary, various anomaly notions in attributes and/or substructures and the imbalance nature make graph-level anomaly detection an important yet challenging problem.
Revised: In summary, various anomaly notions in attributes and/or substructures and the imbalance nature make graph-level anomaly detection an important yet challenging problem.

Original: In this paper, we propose adual-discriminative graph neural network for i mbalanced G raph-level A nomaly D etection ( iGAD ).
Revised: In this paper, we propose a graph neural network for imbalanced Graph-level Anomaly Detection (iGAD).

Original: Specifically, iGAD investigates a novel anomalous attribute-aware graph convolution.
Revised: Specifically, iGAD investigates a novel anomalous attribute-aware graph convolution.

Original: The final node representation is the concatenation of the independent encoding on a node itself and the independent encoding on its neighborhoods from different hops.
Revised: The final node representation is the concatenation of the independent encoding on a node itself and the independent encoding on its neighborhoods from different hops.

Original: In this way, the attributes of each node in a graph can evolve independently over the graph convolution process and the anomalous attributes on particular nodes can be exposed.
Revised: In this way, the attributes of center nodes can evolve independently over the graph convolution process and the anomalous attributes on particular nodes can get exposure.

Original: However, it is intractable to enable graph convolution to be aware of anomalous substructures.
Revised: However, it is intractable to enable graph convolution to be aware of anomalous substructures.

Original: Due to the high dependence on neighborhood aggregation, graph convolution can only capture very limited substructures in graphs.
Revised: Due to the high dependence on neighborhood aggregation, graph convolution can only capture very limited substructures in graphs.

Original: Moreover, even the simple substructures such as triangles, graph convolution cannot count them in a graph [3].
Revised: Moreover, even the simple substructures such as triangles, graph convolution cannot count them in a graph [3].

Original: RWK [46], as one type of Graph Kernels (GKs), learns graphs by exploring common random walk sequences among all input graphs, which naturally involves the comparison about diverse substructures in different graphs.
Revised: RWK [47], as one type of Graph Kernels (GKs), learns graphs by exploring common random walk sequences among all input graphs, which naturally involves the comparison about diverse substructures in different graphs.

Original: To embed the information of anomalous substructures into graph representations, we propose to compare random walk sequences of input graphs against anomalous substructures.
Revised: To embed the information of anomalous substructures into graph representations, we propose to compare random walk sequences of input graphs against anomalous substructures.

Original: By modeling anomalous substructures as parameters, they became learnable and can get updated through back-propagation.
Revised: By modeling anomalous substructures as parameters, they became learnable and can get updated through back-propagation.

Original: We entitle this design anomalous substructure-aware deep RWK.
Revised: We entitle this design anomalous substructure-aware deep RWK.

Original: In addition, we propose a new loss function to target the imbalance distribution caused by the great difference in the number of normal and anomalous graphs.
Revised: In addition, we propose a new loss function to target the imbalance distribution caused by the great difference in the number of normal and anomalous graphs.

Original: The newly proposed loss function models the PMI between graphs and their normal/anomalous properties and enables iGAD to capture anomalous information from an imbalanced graph dataset.
Revised: The newly proposed loss function models the PMI between graphs and their normal/anomalous properties and enables iGAD to capture anomalous information from an imbalanced graph dataset.

Original: Welding three designs mentioned above into a graph neural network, the graph-level anomaly detection model iGAD is achieved.
Revised: Welding three designs mentioned above into a graph neural network, the graph-level anomaly detection model iGAD is achieved.

Original: We conclude the contributions of iGAD as follows:
Revised: We conclude the contributions of iGAD as follows:

Original: • The anomalous attribute-aware graph convolution exposes anomalous attributes by keeping the independent evolution of each node’s attributes.
Revised: • The anomalous attribute-aware graph convolution exposes anomalous attributes by keeping the independent evolution of center node’s attributes.

Original: • Anomalous substructures are modeled as learnable parameters.
Revised: • Anomalous substructures are modeled as learnable parameters.

Original: Substructure-aware deep RWK embeds the information of anomalous substructures into graph representations by comparing the random walk sequences in input graphs andlearned anomalous substructures.
Revised: Substructure-aware deep RWK embeds the information of anomalous substructures into graph representations by comparing the random walk sequences from input graphs and anomalous substructures.

Original: • The PMI-based loss function addresses issues brought by the imbalance nature of the anomaly problem and enables iGAD to learn anomalous graphs as a minority effectively.
Revised: • The PMI-based loss function addresses issues brought by the imbalance nature of the anomaly problem and enables iGAD to learn anomalous graphs as a minority effectively.

Original: • Comprehensive experiments demonstrate the effectiveness of iGAD, which outperforms state-of-the-art graph convolution-based graph classification models and graph-level anomaly detection algorithms on four real -world graph datasets.
Revised: • Comprehensive experiments demonstrate the effectiveness of iGAD, which outperforms state-of-the-art graph convolution-based graph classification models and graph-level anomaly detection algorithms on four real-world graph datasets.

Original: The code is available at https://github.com/graph-level-anomalies/iGAD.
Revised: The code is available at https://github .com/graph-level-anomalies/iGAD.

Original:  Related Work
Revised:  Related Work

Original: Anomaly Detection.
Revised: Anomaly Detection.

Original: The task aims to identify anomalies that deviate from the majority of samples.
Revised: The task aims to identify anomalies that deviate from the majority of samples.

Original: Early anomaly detection algorithms use indicators, such as distance [18] and density [7], to describe thedegree of deviation  among samples to detect anomalous ones.
Revised: Early anomaly detection algorithms use different metrics, such as distance-based [18] and density- based [7], to measure the deviation degree among samples to detect anomalies.

Original: For example, isolation forest-based algorithms [25, 26] identifythe anomalous by building the binary search tree structure to isolate samples.
Revised: For example, isolation forest-based algorithms [25, 26] identify anomalous by building the binary search tree structure to isolate samples.

Original: In addition, anomaly detection algorithms based on one-class classification [39, 38, 36, 62] and autoencoder [2, 63] have also been in the spotlight.
Revised: In addition, anomaly detection algorithms based on one-class classification [40, 39, 37, 63] and autoencoder [2, 64] have also been in the spotlight.

Original: For the graph-structured data, most graph anomaly detection algorithms dedicate to detect anomalies (e.g., anomalous nodes [43, 13, 57] and edges [35, 56]) in a single graph [1].
Revised: For the graph-structured data, most graph anomaly detection algorithms dedicate to detect anomalies (e.g., anomalous nodes [44, 13, 58] and edges [36, 57]) in a single graph [1].

Original: The anomaly notions of graph-level anomalies are more diversified than those of node-level anomalies.
Revised: The anomaly notions of graph-level anomalies are more diversified than those of node-level anomalies.

Original: For example, node-level anomaly detection methods usually regard nodes that betray network homophily 1 asthe anomalies [43].
Revised: For example, node-level anomaly detection methods usually regard nodes that betray network homophily 1 as anomalies [44].

Original: However, at the graph level, nodes that can achieve an anomalous graph go far beyond the nodes that do not meet homophily.
Revised: However, at the graph level, nodes that can achieve an anomalous graph go far beyond the nodes that do not meet homophily.

Original: Node-level anomaly detection methods identify anomaly nodes from a micro perspective (e.g., neighborhood structures) while identifying anomalous graphs needs learning graphs at the macro level.
Revised: Node-level anomaly detection methods identify anomaly nodes from a micro perspective (e.g., neighborhood structures) while identifying anomalous graphs needs learning graphs at the macro level.

Original: Employing node-level anomaly detection methods to detect graph-level anomalies could capture some anomalies, but only a small part.
Revised: Employing node-level anomaly detection methods to detect graph-level anomalies could capture some anomalies, but only a small part.

Original: Moreover,the graph-level anomaly detection algorithms need to consider anomaly notions related to graph substructures.
Revised: Moreover, graph-level anomaly detection algorithms need to consider anomaly notions related to graph substructures.

Original: Hence, graph anomaly detection methods that identify anomalies in a single graph cannot adapt to the graph-level anomaly detection problem through simple modifications.
Revised: Hence, it is necessary to propose anomaly detection methods that are specialized in detecting graph-level anomalies.

Original: Graph-level Anomaly Detection.
Revised: Graph-level Anomaly Detection.

Original: To the best of our knowledge, there are only several preliminary research onthe topic of graph-level anomaly detection.
Revised: To the best of our knowledge, there are only several preliminary research on graph-level anomaly detection.

Original: Zhao et al.
Revised: Zhao et al.

Original: [61] investigated a research question of “should we use the data for the graph classification task to evaluate graph-level anomaly detection models?”, followed by which two graph-level anomaly detection algorithms [27, 34]
Revised: [62] investigate a research question of “should we use the data for the graph classification task to evaluate graph-level anomaly detection models?”, followed by which two graph-level anomaly detection algorithms [27, 35] combine Graph Neural

Original: combined Graph Neural Networks (GNNs) with knowledge distillation and one-class classification respectively to detect anomalous graphs.
Revised: Networks (GNNs) with knowledge distillation and one-class classification respectively to detect anomalous graphs.

Original: Graph-level anomaly detection can be regarded as a special case of graph classification.
Revised: Graph-level anomaly detection can be regarded as a special case of graph classification.

Original: There is a huge gap between the number of graphs belonging to the normal class versus the anomalous ones.
Revised: There is a huge gap between the number of graphs belonging to the normal class versus the anomalous ones.

Original: But, graph classification algorithms usually assume that the number of graphs in each class is balanced.
Revised: But, graph classification algorithms usually assume that the number of graphs in each class is close.

Original: GKs [46, 16, 41, 6, 40] used to be the mainstream graph classification methods.
Revised: GKs [47, 16, 42, 6, 41] used to be the mainstream graph classification methods.

Original: Generally, GKs use the kernel function to calculate kernel values (i.e., similarity) between pair-wised graphs.
Revised: Generally, GKs use the kernel function to calculate kernel values (i.e., similarity) between pair-wised graphs.

Original: Afterward, an off-the-shelf classifier is applied to the graph similarity matrix to perform the classification task.
Revised: Afterward, an off-the-shelf classifier is applied to the graph similarity matrix to perform the classification task.

Original: Graph classification algorithms based on GKs cannot learn graph representations explicitly and be optimized in an end-to-end fashion.
Revised: Graph classification algorithms based on GKs cannot learn graph representations explicitly and be optimized in an end-to-end fashion.

Original: In recent years, graph mining practitioners pay more attention to
Revised: In recent years, graph mining practitioners pay more attention to

Original: GNNs [8, 11, 4, 37].
Revised: GNNs [8, 11, 4, 38].

Original: GNNs with spatial graph convolution [20, 45, 19] are summarized as messagepassing framework (MPNNs)
Revised: GNNs with spatial graph convolution [20, 46, 19] are summarized as messagepassing framework (MPNNs)

Original: In general, MPNNs update the representation of each node by iteratively aggregating information coming from their neighbors.
Revised: In general, MPNNs update the representation of each node by iteratively aggregating information coming from their neighbors.

Original: To represent a graph,
Revised: To represent a graph,

Original: MPNNs apply the readout function or pooling operation on node embeddings.
Revised: MPNNs apply the readout function or pooling operation on node embeddings.

Original: The spatial graph convolution that mix the representation of nodes and their neighborhoods to represent nodes will smooth node attributes [60, 4].
Revised: The spatial graph convolution that mix the representation of nodes and their neighborhoods to represent nodes will smooth node attributes [61, 4].

Original: In addition, [31, 28, 59] proved that spatial graph convolution has limited power in distinguishing graph structures.
Revised: In addition, [31, 28, 60] proved that spatial graph convolution has limited power in distinguishing graph structures.

Original: Performing message passing between subgraphs can enable spatial graph convolution to capture more graph structural information.
Revised: Performing message passing between subgraphs can enable spatial graph convolution to capture more graph structural information.

Original: But such a solutionconsumes exponentially increases memory  and computation complexities [3].
Revised: But such a solution exponentially consumes memory resources and computation complexities [3].

Original: Graph classification methods mentioned above are likely to underfitthe anomalous graphs since they do not have any intrinsic mechanism to target the imbalanced distributions.
Revised: Graph classification methods mentioned above are likely to underfit anomalous graphs since they do not have any intrinsic mechanism to target the imbalanced distributions.

Original: In iGAD, a novel loss is presented for the imbalanced distributions by modeling PMI.
Revised: In iGAD, a novel loss is presented for the imbalanced distributions by modeling PMI.

Original: In addition, different from existing graph-level anomaly methods [61, 27, 34] that employ the off-the-shelf MPNNs (e.g., GIN [53]) to represent graphs, iGAD can embed the information of anomalous attributes and substructures into graph representations.
Revised: In addition, different from existing graph-level anomaly methods [62, 27, 35] that employ the off-the-shelf MPNNs (e.g., GIN [54]) to represent graphs, iGAD can embed the information of anomalous attributes and substructures into graph representations.

Original: Imbalanced Learning.
Revised: Imbalanced Learning.

Original: Learning data with imbalanced distribution can be divided intothe data- and algorithm-level approaches.
Revised: Learning data with imbalanced distribution can be divided into data- and algorithm-level approaches.

Original: The former alleviates the imbalanced distribution of the original data by either under-sampling the majority [48] or over-sampling the minority [9].
Revised: The former alleviates the imbalanced distribution of the original data by either under-sampling the majority [49] or over-sampling the minority [9].

Original: However, modifying the original distribution of data will bring a lot of drawbacks inevitably, such as over-fitting and discarding important samples.
Revised: However, modifying the original distribution of data will bring a lot of drawbacks inevitably, such as over-fitting and discarding important samples.

Original: There are different types of approaches to address the issue at the algorithm-level.
Revised: There are different types of approaches to address the issue at the algorithm level.

Original: The typical approaches include employing post-hoc correction [30], aligning data distribution, [52] and modifying the loss function [24, 22].
Revised: The typical approaches include employing post-hoc correction [30], aligning data distribution, [53], and modifying the loss function [24, 22].

Original: In the real-world application, considering the different consequences of false-positive identification (e.g., predicting an inactive chemical compound to be active) and false-negative identification (e.g., predicting a cancer patient as a healthy individual), the situation will be more complicated [33].
Revised: In the real-world application, considering the different consequences of false-positive identification (e.g., predicting an inactive chemical compound to be active) and false-negative identification (e.g., predicting a cancer patient as a healthy individual), the situation will be more complicated [33].

Original: In this paper, iGAD introduces PMI to the cross-entropy loss function to solve this problem.
Revised: In this paper, iGAD introduces PMI to the cross-entropy loss function to solve this problem.

Original:  Preliminaries
Revised:  Preliminaries

Original: Notations.
Revised: Notations.

Original: The graph dataset consisting of N graphs can be denoted as G =
Revised: A graph dataset consisting of N graphs can be denoted as G =

Original: { G 1 , ..., G N } , where
Revised: { G 1 , ..., G N } , where

Original: G i =
Revised: G i =

Original: { V i , E i } is an unweighted and undirected graph in G , V i
Revised: { V i , E i } is an unweighted and undirected graph in G , V i

Original: and
Revised: and

Original: E
Revised: E

Original: i are the node and the edge set, respectively.
Revised: i are the node and the edge set, respectively.

Original: The topology of G i can be modeled as an adjacency matrix A i ∈ { 0 , 1 } n × n , where
Revised: The topology of G i can be modeled as an adjacency matrix A i ∈ { 0 , 1 } n × n , where

Original: A i ( u, v ) = 1 if there is an edge between nodes u and v , otherwise 0.
Revised: A i ( u, v ) = 1 if there is an edge between nodes u and v , otherwise 0.

Original: A i does not consider the self-loop, A i ( u, u ) = 0 , ∀ u ∈ V i .
Revised: A i does not consider the self-loop, A i ( u, u ) = 0 , ∀ u ∈ V i .

Original: We use x u ∈ R c to denote the attribute vector of node u ∈ V i .
Revised: We use x u ∈ R c to denote the attribute vector of node u ∈ V i .

Original: X i ∈ R n × c is the attribute matrix of G i .
Revised: X i ∈ R n × c is the attribute matrix of G i .

Original: G ′
Revised: G ′

Original: i =
Revised: i =

Original: { V ′ i , E ′ i } can be regarded as a substructure in G i , iff V ′ i ⊆
Revised: { V ′ i , E ′ i } can be regarded as a substructure in G i , iff V ′ i ⊆

Original: V i and E ′ i ⊆
Revised: V i and E ′ i ⊆

Original: E i .
Revised: E i .

Original: Y is the label set of G .
Revised: Y is the label set of G .

Original: G i is labeled by y i ∈ { 0 , 1 } , where 1 represents anomalous and 0 represents normal .
Revised: G i is labeled by y i ∈ { 0 , 1 } , where 1 represents anomalous and 0 represents normal .

Original: Problem Definition.
Revised: Problem Definition.

Original: This paper concentrates on the supervised graph-level anomaly detection problem.
Revised: This paper concentrates on the supervised graph-level anomaly detection problem.

Original: Given the training set T = { ( G 1 , y 1 ) , ( G 2 , y 2 ) , ... } , we aim to train a model to predict the normal/anomalous proprieties of unseen graphs.
Revised: Given the training set T = { ( G 1 , y 1 ) , ( G 2 , y 2 ) , ... } , we aim to train a model to predict the normal/anomalous proprieties of unseen graphs.

Original: T contains N 1 anomalous graphs and N 0 normal graphs ( N 1 ≪ N 0 ).
Revised: T contains N 1 anomalous graphs and N 0 normal graphs ( N 1 ≪ N 0 ).

Original: The imbalanced ratio of T is defined as δ ( δ =
Revised: The imbalanced ratio of T is defined as δ ( δ =

Original: N 1 N 0 + N 1 ).
Revised: N 1 N 0 + N 1 ).

Original: Spatial Graph Convolution.
Revised: Spatial Graph Convolution.

Original: It updates each node’s representation in a graph  by iteratively aggregating the features coming from its one-hop neighborhood.
Revised: A graph convolution layer updates node representations by iteratively aggregating information coming from its one-hop neighborhood.

Original: The formulation is:
Revised: The formulation is:

Original: [Equation]

Original: [Equation]

Original: ) , f Aggregate (

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: v

Original: [Equation]
Revised: [Equation]

Original: N

Original: [Equation]

Original: } )) (cid:17) ,

Original: [Equation]

Original: [Equation] u [Equation] where 0 ≤ k ≤ K − 1 , N ( u ) denotes the 1-hop neighborhood of node u .
Revised: where 0 ≤ k ≤ K − 1 , N ( u ) denotes the 1-hop neighborhood of node u .

Original: f
Revised: f

Original: Aggregate denotes how to aggregate neighbors of node u , and f combine defines how to combine the immediate representations of node u and its neighborhood.
Revised: A GGREGATE denotes how to aggregate neighbors of node u , and f C OMBINE defines how to combine the immediate representations of node u and its neighbors.

Original: σ is the activation function.
Revised: σ is the activation function.

Original: Generally, the final representation of the node u is h ( K ) ( u ) .
Revised: Generally, the final representation of the node u is h ( K ) ( u ) .

Original: Random Walk Kernels (RWK).
Revised: Random Walk Kernels (RWK).

Original: RWK measures graph similarities by counting common random walk sequences among graphs.
Revised: RWK measures graph similarities by counting common random walk sequences among graphs.

Original: There are two graphs G 1 = { V 1 , E 1 } and G 2 =
Revised: There are two graphs G 1 = { V 1 , E 1 } and G 2 =

Original: { V 2 , E 2 } .
Revised: { V 2 , E 2 } .

Original: G ×
Revised: G ×

Original: = { V × , E × } denotes the direct product graph about G 1 and G 2 , where V × =
Revised: = { V × , E × } denotes the direct product graph about G 1 and G 2 , where V × =

Original: { ( u, u ′ )
Revised: { ( u, u ′ )

Original: | u ∈ V 1 , u ′ ∈
Revised: | u ∈ V 1 , u ′ ∈

Original: V 2 } and E × = { (( u, u ′ ) , ( v, v ′ ))
Revised: V 2 } and E × = { (( u, u ′ ) , ( v, v ′ ))

Original: | ( u, v ) ∈
Revised: | ( u, v ) ∈

Original: E 1 , ( u ′ , v ′ )
Revised: E 1 , ( u ′ , v ′ )

Original: ∈
Revised: ∈

Original: E 2 } .
Revised: E 2 } .

Original: Performing random walking on G × acts as doing random walking on G 1 and G 2 simultaneously.
Revised: Performing random walking on G × is equivalent to doing random walking on G 1 and G 2 simultaneously.

Original: The adjacency matrix of direct product graph A × can be obtained by the Kronecker product operation ⊗ on A 1 and A 2 , that is A × =
Revised: The adjacency matrix of direct product graph A × can be obtained by the Kronecker product operation ⊗ on A 1 and A 2 , that is A × =

Original: A 1 ⊗ A 2 .
Revised: A 1 ⊗ A 2 .

Original: The number of l -length common walk sequences between G 1 and G 2 is κ ( l ) ( G 1 , G 2 ) =
Revised: The number of l -length common walk sequences between G 1 and G 2 is κ ( l ) ( G 1 , G 2 ) =

Original: q T × ( A × ) l p × , where p × and q × are the vector about the starting and stopping probabilities of random walks on G × , respectively.
Revised: q T × ( A × ) l p × , where p × and q × are the vector about the starting and stopping probabilities of random walks on G × , respectively.

Original: The elements in p × and q × are all ones when there is no prior knowledge.
Revised: The elements in p × and q × are all ones when there is no prior knowledge.

Original: Overall, the RWK about G 1 and G 2 can be defined as:
Revised: Overall, the RWK about G 1 and G 2 can be defined as:

Original: [Equation]
Revised: [Equation]

Original:  iGAD
Revised:  iGAD

Original: In this section, we will describe iGAD in detail (see Figure 1 for an illustration).
Revised: In this section, we will describe iGAD in detail (see Figure 1 for an illustration).

Original: iGAD contains three key designs: (1) anomalous graph attribute-aware graph convolution; (2) anomalous substructureaware deep RWK; and (3) PMI-based loss function.
Revised: iGAD contains three key designs: (1) anomalous graph attribute-aware graph convolution; (2) anomalous substructureaware deep RWK; and (3) PMI-based loss function.

Original: The first two designs encode the anomalous information on graph attributes and substructures into graph embeddings.
Revised: The first two designs encode the anomalous information on graph attributes and substructures into graph embeddings.

Original: The loss function enables iGAD to get rid of the domination from normal graphs.
Revised: The loss function makes the training on iGAD not completely dominated by normal graphs.

Original: 4.1 iGAD: Anomalous Attribute-aware Graph Convolution
Revised: 4.1 iGAD: Anomalous Attribute-aware Graph Convolution

Original: As aforementioned in Section 1, the attributes of particular nodes can achieve an anomalous graph.
Revised: As aforementioned in Section 1, the attributes of particular nodes can achieve an anomalous graph.

Original: The graph convolution which iteratively updates node representations by performing feature transformation on themixed or concatenated information about nodes and their neighborhoods will make node representations over-smoothed [60].
Revised: The graph convolution which iteratively updates node representations by performing feature transformation on the concatenated information about nodes and their neighborhoods will make node representations over-smoothed [61].

Original: Under such a graph convolution mechanism, only limited 
Revised: Under such a graph convolution mechanism, only limited anomalous information can be manifest in corresponding node representations. We argue that making each

Original: A Graph Dataset … (cid:29)(cid:3) Normal Graph : Anomalous Graph
Revised: A Graph Dataset … (cid:29)(cid:3) Normal Graph : Anomalous Graph

Original: A n o m a l o u s
Revised: A n o m a l o u s

Original: A tt r i b u t e
Revised: A tt r i b u t e

Original: A n o m a l o u s
Revised: A n o m a l o u s

Original: S u b s t r u c t u r e
Revised: S u b s t r u c t u r e

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: R ea do u t
Revised: R ea do u t

Original: I. Anomalous Attribute-aware Graph Convolution (K=2)
Revised: I. Anomalous Attribute-aware Graph Convolution (K=2)

Original: II.
Revised: II.

Original: Anomalous Substructure-aware Deep RWK (M=2, n’=4, L=5)
Revised: Anomalous Substructure-aware Deep RWK (M=2, n’=4, L=5)

Original: Update A
Revised: Update A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: Update A
Revised: Update A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G … …
Revised: G … …

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D E
Revised: D E

Original: F G
Revised: F G

Original: C o n ca t e n a t
Revised: C o n ca t e n a t

Original: i o n
Revised: i o n

Original: A n o m a l o u s
Revised: A n o m a l o u s

Original: III.
Revised: III.

Original: PMI-based Loss Function
Revised: PMI-based Loss Function

Original: Prediction
Revised: Prediction

Original: F E
Revised: F E

Original: G
Revised: G

Original: E
Revised: E

Original: F
Revised: F

Original: G C … …
Revised: G C … …

Original: A MLP
Revised: A MLP

Original: The
Revised: The

Original:  Deep  which performs random walks two learnable
Revised: Substructure-aware Deep , which performs random walks two learnable

Original: (i.e
Revised: (i.e

Original: , Sand S
Revised: , S S

Original: G N h  based on kernel  κ (l
Revised: G N h G based on kernel values κ (

Original: ( G N , S m
Revised: ( G N , S m

Original: = { 
Revised: = { to

Original: L and
Revised: Finally a MLP is applied to

Original: G
Revised: G

Original: Finally, aMLP and the PMI-based loss function are applied to h G N to predict G N .
Revised: The model training is guided by a PMI-based loss function.

Original: anomalous information can be manifest in corresponding node representations.

Original: We argue that making each node’s attributes evolve independently over the graph convolution process is critical for capturing anomalous attributes.
Revised: node’s attributes evolve independently over the graph convolution process is critical for capturing anomalous attributes.

Original: To achieve it, anomalous attribute-aware graph convolution concatenates the independent encoding on the node itself and the independent encoding on its k -hop neighborhood (1 ≤ k ≤ K ) to represent each node in a graph.
Revised: To achieve it, anomalous attribute-aware graph convolution concatenates the independent encoding on the node itself and the independent encoding on its k -hop neighborhood (1 ≤ k ≤ K ) to represent each node in a graph.

Original: Formally,
Revised: Formally,

Original: [Equation]
Revised: [Equation]

Original:  where h u is the final node representation of  u∈ V i , h ( k ) ( u ) is the representation of node u ’s k -hop neighborhood, and f Combine is the concatenation operation.
Revised: ) where h u is the final node representation of the node u , h ( k ) ( u ) is the representation of node u ’s khop neighborhood, and f C OMBINE is the concatenation operation.

Original: h (0) ( u ) and h ( k ) ( u ) are formulated as:
Revised: h (0) ( u ) and h ( k ) ( u ) are formulated as:

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: respectively, where H (0) i
Revised: respectively, where H (0) i

Original: =
Revised: =

Original: X i , MLP denotes a multi-layer perceptron, σ represents the activation function, and W ( k ) is the trainable weight matrix.
Revised: X i , M LP denotes a multi-layer perceptron, σ represents the activation function, and W ( k ) is the trainable weight matrix.

Original: To obtain the entire graph representation h ConvG i , we apply the permutation invariant Readout function R ( · ) on all nodes in the graph G i .
Revised: To obtain the entire graph representation h C ONV G i , we apply the permutation invariant Readout function R ( · ) on all nodes in the graph G i .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: where R ( · ) can be the sum or mean operation.
Revised: where R ( · ) can be the sum-based or mean-based global pooling operation.

Original: 4.2 iGAD: Anomalous Substructure-aware Deep RWK
Revised: 4.2 iGAD: Anomalous Substructure-aware Deep RWK

Original: Generally, the neighborhood aggregation mechanism  only utilizes the local neighborhood structure to build the rooted subtree around each node and performs message passing [59]), which affects the ability of graph convolution in distinguishing substructures [3, 10].
Revised: Generally, the neighborhood aggregation mechanism adopted by existing GNNs only depends on local neighborhood structures to perform message passing [60]), which leads to the deficiency of graph convolution in distinguishing substructures [3, 10].

Original: To make iGAD to be aware of anomalous substructures, we resort tothe RWK which can explore diverse substructures through random walking in graphs.
Revised: To make iGAD to be aware of anomalous substructures, we resort to RWK that can explore diverse substructures through random walking sequences.

Original: There is a pioneering work [32] that represents graphs by the RWK values about input graphs and some substructures.
Revised: There is a pioneering work [32] that represents graphs by calculating RWK values between input graphs and some substructures.

Original: It inspires us to embed information of anomalous substructures into graph representations by performing random walks in input graphs and learnable anomalous substructures simultaneously.
Revised: It inspires us to embed information about anomalous substructures into graph representations by performing random walks in input graphs and learnable anomalous substructures simultaneously.

Original: We present an anomalous substructure-aware deep RWK, which models each element in the adjacency matrices of anomalous substructures asa learnable parameter.
Revised: We present an anomalous substructure-aware deep RWK, which models elements in the adjacency matrices of anomalous substructures as learnable parameters.

Original: Supposed that { S 1 , ..., S M } is the set of M anomalous substructures, and the size of S m is n ′ .
Revised: Supposed that { S 1 , ..., S M } is the set of M anomalous substructures, and the size of S m is n ′ .

Original: P m ∈ R n ′ × n ′ is the adjacency matrix of S m , where  P m ( u ′ , v ′ ) denotes the possibility that an edge exists between nodes u ′ and v ′ in S m .
Revised: P m ∈ R n ′ × n ′ is the adjacency matrix of S m , where the matric element P m ( u ′ , v ′ ) denotes the possibility that an edge exists between nodes u ′ and v ′ in S m .

Original: Due to the symmetry of the adjacency matrix, P m contains n ′ ( n ′ − 1)parameters.
Revised: As the adjacency matrix is symmetric, P m contains n ′ ( n ′ − 1)parameters.

Original: Referring to Eq.and supposing that the starting and the stopping possibilities on each node are all ones, the number of l -length common random walk sequences between the input graph G i and the anomalous substructure S m is:
Revised: Referring to Eq. 2 and supposing that the starting and the stopping possibilities on each node are all ones, the number of l -length common random walk sequences between the input graph G i and the anomalous substructure S m is:

Original: [Equation]
Revised: [Equation]

Original: where 1 ∈ R n ′ n is a column vector full of 1, and E ∈ R n ′ × n is a matrix full of 1.
Revised: where 1 ∈ R n ′ n is a column vector full of 1, and E ∈ R n ′ × n is a matrix in which all elements are 1.

Original: The proof of Eq.
Revised: The proof of Eq.

Original: 7 can be found in Appendix A.1.
Revised: 7 can be found in Appendix A.1.

Original: Random walk kernels with infinite random walk lengths will often meet the halting issue [42].
Revised: RWK with infinite random walk lengths will often meet the halting issue [43].

Original: Therefore, iGAD only considers the random walk length fromto L .
Revised: Therefore, iGAD only considers the random walk length from 1 to L .

Original: As the size of anomalous substructures ( n ′ ) is small, optimizing ( P m ) l is not necessary.
Revised: As the size of anomalous substructures ( n ′ ) is small, optimizing ( P m ) l is not necessary.

Original:  A i is a sparse matrix, and we compute ( A i ) l by the sparse matrix multiplication in PyTorch.
Revised: The adjacency matrix A i is sparse, and we compute ( A i ) l by the sparse matrix multiplication in PyTorch [34].

Original: That is ( A si ) l
Revised: That is ( A si ) l

Original: = A si ( A si ... ( A si ( A i )) ... ) , where A si is the sparse version of A i .
Revised: = A si ( A si ... ( A si ( A i )) ... ) , where A si is the sparse version of A i .

Original: Afterward,all deep RWK values  about G i and S m , m = { 1 , .., M }  will be assembled into a vector a G i .
Revised: Afterward, deep RWK values κ ( l ) ( G i , S m ) about G i and S m , m = { 1 , .., M } , l = { 1 , .., L } will be assembled into a vector a G i .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: where a G i ∈ R ML .
Revised: where a G i ∈ R ML .

Original: The graph representation h kerG i based on deep RWK is:
Revised: The graph representation h KER G i based on deep RWK is:

Original: [Equation]
Revised: [Equation]

Original: The final graph representation h G i for G i ∈ G can be obtained by concatenating h ConvG i and h KerG i together.
Revised: The final graph representation h G i for G i ∈ G can be obtained by concatenating h C ONV G i and h K ER G i together.

Original: Based on h G i , the anomalous attribute-aware graph convolution and anomalous substructureaware deep RWK are welded into a graph neural network.
Revised: The anomalous attribute-aware graph convolution and anomalous substructure-aware deep RWK are welded into a graph neural network together through h G i .

Original: 4.3 iGAD: PMI-based Loss Function
Revised: 4.3 iGAD: PMI-based Loss Function

Original: The number of normal graphs is much more than that of anomalous graphs.
Revised: The number of normal graphs is much more than that of anomalous graphs.

Original: Under the imbalance nature of the anomaly problem, normal graphs will dominate the machine learning model.
Revised: Under the imbalance nature of the anomaly problem, normal graphs will dominate model training.

Original: Targeting this issue, we propose a new loss function that can model the PMI between input graphs and their anomalous/normal properties.
Revised: Targeting this issue, we propose a new loss function that can model the PMI between input graphs and their anomalous/normal properties.

Original: Specifically, we use f ( G ; Θ) to denote the proposed model iGAD, where Θ represents model parameters.
Revised: Specifically, we use f ( G ; Θ) to denote the proposed model iGAD, where Θ represents model parameters.

Original: Modeling the PMI between the input graph G i and its anomalous/normal properties can be formulated as:
Revised: Modeling the PMI between the input graph G i and its anomalous/normal properties can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where the right part represents the PMI between G i and y i , P ( y i ) is the proportion of anomalous graphs or normal graphs in the training set T .
Revised: where the right part represents the PMI between G i and y i , P ( y i ) is the proportion of anomalous graphs or normal graphs in the training set T .

Original: Compared with most existing loss functions, especially the loss function that models the conditional distribution about y i given G i (i.e., f y i
Revised: Compared with most existing loss functions, especially the loss function that models the conditional distribution about y i given G i (i.e., f y i

Original: ( G i ; Θ) ∼
Revised: ( G i ; Θ) ∼

Original: P Θ ( y i | G i ) ), PMI enables iGAD to give priority to learning the information that reveals the essential correlation between graphs and their labels.
Revised: P Θ ( y i | G i ) ), PMI enables iGAD to give priority to learning the information that reveals the essential correlation between graphs and their labels.

Original: Eq.
Revised: Eq.

Original: 10 can be reformatted as:
Revised: 10 can be reformatted as:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: After softmax normalization, the distribution of y i given G i is formulated as:
Revised: After softmax normalization, the distribution of y i given G i is formulated as:

Original: [Equation]
Revised: [Equation]

Original: Incorporating P Θ ( y i | G i ) in to the framework of the cross-entropy loss function, we can obtain the
Revised: Incorporating P Θ ( y i | G i ) in to the framework of the cross-entropy loss function, we can obtain the

Original: PMI-based loss function L :
Revised: PMI-based loss function L :

Original: [Equation]
Revised: [Equation]

Original: Specifically, iGAD predict the graph G i :
Revised: Specifically, iGAD predict the graph G i :

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: testing stage
Revised: training [Table] [Equation] during testing.

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: The reason that we use argmax y i f ∗ y
Revised: The reason that we use argmax y i f ∗ y

Original: i
Revised: i

Original: ( G i ; Θ) to predict a graph in the testing stage can be found in 
Revised: ( G i ; Θ) to predict a graph in test time can be found in Appendix

Original: Appendix A.2.
Revised: A.2.

Original: There are other loss functions targeting the difficulties brought bythe imbalanced distributions, such as the cost-sensitive loss function that assigns or learns different weights to the majority and the minority samples [29, 49, 12].
Revised: Many other loss functions target the difficulties brought by imbalanced distributions, such as the cost-sensitive loss function that assigns or learns different weights to the majority and the minority samples [29, 50, 12].

Original: Compared with them, the PMI-based loss function does not introduce any new parameters that need to be carefully tuned.
Revised: Compared with them, the PMI-based loss function does not introduce any new parameters that need to be carefully tuned.

Original: It comes in the form of the most widely used cross-entropy loss function and gives prediction results that can realize the maximized PMI between inputs and prediction results.
Revised: It comes in the form of the most widely used cross-entropy loss function and gives prediction results that can achieve maximized PMI between inputs and predictions.

Original: 4.4 Algorithm Description & Computational Complexity
Revised: 4.4 Algorithm Description & Computational Complexity

Original: The pseudocode of iGAD is included in Algorithm 1 in Appendix A.1.
Revised: The pseudocode of iGAD is included in Algorithm 1 in Appendix A.1.

Original: We divide graphs  into different mini-batches to train iGAD [23].
Revised: We divide graphs in G into different mini-batches to train iGAD [23].

Original: But for illustrating how iGAD predicts a graph more clearly, the batch size is set as 1 in Algorithm 1.
Revised: But for illustrating how iGAD predicts a graph more clearly, the batch size is set as 1 in Algorithm 1.

Original: Given the training set, we initialize anomalous substructures and other parameters (Line 1 in Algorithm 1) and calculate the proportion of normal and anomalous graphs (Line 2).
Revised: Given the training set, we initialize anomalous substructures and other parameters (Line 1 in Algorithm 1) and calculate the proportion of normal and anomalous graphs (Line 2).

Original: For each graph, iGAD learns its graph representation (Lines 3 to 15).
Revised: For each graph, iGAD learns its graph representation (Lines 3 to 15).

Original: Based on the graph representation, a MLP equipped with the PMI-based loss function predicts the graph (Lines 16 and 17).
Revised: Based on the graph representation, a MLP equipped with the PMI-based loss function gives predictions to graphs (Lines 16 and 17).

Original: At the back-propagation stage, anomalous substructures and other parameters will get updated.
Revised: At the back-propagation stage, anomalous substructures and other parameters will get updated.

Original: At the next epoch, deep RWK performs random walking onthe input graphs andthe updated anomalous substructures simultaneously.
Revised: At each epoch, deep RWK performs random walking on input graphs and anomalous substructures that get updated during the previous epoch.

Original: Lemma 

Revised: Lemma

Original: The computational complexity of iGAD is O ( n + | E | ) , where n and | E | are the number of nodes and edges, respectively.
Revised: The computational complexity of iGAD is O ( n + | E | ) , where n and | E | are the number of nodes and edges, respectively.

Original: (The detailed analysis can be found in Appendix A.3).
Revised: (The detailed analysis can be found in Appendix A.3).

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct a series of experiments to study the performance of iGAD on the graphlevel anomaly detection task.
Revised: In this section, we conduct a series of experiments to study the performance of iGAD on the graphlevel anomaly detection task.

Original: Specifically, we investigate the following six questions:
Revised: Specifically, we investigate the following six questions:

Original: Q1.
Revised: Q1.

Original: How about the performance of iGAD on the graph-level anomaly detection task?
Revised: How about the performance of iGAD on the graph-level anomaly detection task?

Original: Q2.
Revised: Q2.

Original: Does iGAD benefit from the PMI-based loss function?
Revised: Does iGAD benefit from the PMI-based loss function?

Original: Q3.
Revised: Q3.

Original: How about the performance of the anomalous graph attribute-aware graph convolution in iGAD? Q4.
Revised: How about the performance of the anomalous graph attribute-aware graph convolution in iGAD? Q4.

Original: How about the performance of iGAD when it utilizes pre-defined anomalous substructures?
Revised: How about the performance of iGAD when it utilizes pre-defined anomalous substructures?

Original: Q5.
Revised: Q5.

Original: Can the anomalous substructure-aware deep RWK in iGAD learnthe anomalous substructures?
Revised: Can the anomalous substructure-aware deep RWK in iGAD learn anomalous substructures?

Original: Q6.
Revised: Q6.

Original: How does the maximum length of random walk sequences influence the performance of iGAD?
Revised: How does the maximum length of random walk sequences affect the performance of iGAD?

Original: The questions from Q2 to Q4 can be regarded as the ablation study on our three main design elements, they are (a).
Revised: The questions from Q2 to Q4 can be regarded as the ablation study on our three main design elements, they are: (a).

Original: cross-entropy loss function versus the proposed PMI-based loss function; (b).
Revised: cross-entropy loss function versus the proposed PMI-based loss function; (b).

Original: general graph convolution versus the proposed graph convolution; (c).
Revised: general graph convolution versus the proposed graph convolution; (c).

Original: RWK versus the proposed deep RWK.
Revised: RWK versus the proposed deep RWK.

Original: [Table]
Revised: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: 5.1 Experimental Setup
Revised: 5.1 Experimental Setup

Original: Datasets.
Revised: Datasets.

Original: SW-610, MOLT-4, PC-3, and MCF-7 are four real-world graph datasets.
Revised: SW-610, MOLT-4, PC-3, and MCF-7 are four real-world graph datasets.

Original: These datasets are collected from PubChem 2 , which records a tremendous amount of chemical compounds and their anti-cancer activity testing results (“active” or “inactive”) on different types of cancer cell lines.
Revised: These datasets are collected from PubChem 2 , which records a tremendous amount of chemical compounds and their anti-cancer activity testing results (“active” or “inactive”) on different types of cancer cell lines.

Original: We regard the chemical compounds with the outcome “active” as anomalous [Table] and “inactive” as normal.
Revised: We regard the chemical compounds with the outcome “active” as anomalous, and “inactive” as normal.

Original: For example, the graph dataset SW-620 is about 40,532 chemical formulas and their anti-cancer activity testing results on the colon cancer cell line.
Revised: For example, the graph dataset SW-620 records 40,532 chemical formulas and their anti-cancer activity testing results on the colon cancer cell line.

Original: Chemical compounds that exhibit antibody activity against colon cancer are labeled as anomalous graphs, otherwise normal graphs.
Revised: Chemical compounds that exhibit antibody activity against colon cancer are labeled as anomalous graphs, otherwise normal graphs.

Original: The statistical information about these four datasets is shown in Table 1.
Revised: The statistical information about these four datasets is shown in Table 1.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Baselines & Experimental Settings.
Revised: Baselines & Experimental Settings.

Original: To investigate
Revised: To investigate

Original: Q1, we compare iGAD with the following baseline models: (1) GCN[20]
Revised: Q1, we compare iGAD with the following baseline models: (1) GCN[20]

Original: + Readout function (abbreviated
Revised: + Readout function (abbreviated

Original: GCN) and GIN [53], these two algorithms employ spatial graph convolution to learn node representation and apply the readout function to node representations to obtain graph representations and classify graphs; (2)
Revised: GCN) and GIN [54]. These two algorithms employ spatial graph convolution to learn node representation and apply the readout function to node representations to obtain graph representations and classify graphs; (2)

Original: DGCNN [58] and SOPool [51].
Revised: DGCNN [59] and SOPool [52].

Original: In these two models, spatial graph convolution and the graph pooling operation are employed to downsize graphs and complete the graph classification task; (3) RWGNN [32] employs random walk sequences to represent graphs and then classify graphs based on graph representations.
Revised: In these two models, spatial graph convolution and the graph pooling operation are employed to downsize graphs and complete the graph classification task; (3) RWGNN [32]. It employs random walk sequences to represent graphs and then classify graphs based on graph representations.

Original: All the above baselines are supervised and use the cross-entropy loss function; (4) The graph-level anomaly detection methods GLocalKD [27] and OCGTL [34].
Revised: All the above baselines are supervised and use the cross-entropy loss function; (4) The graph-level anomaly detection methods GLocalKD [27] and OCGTL [35].

Original: The former adopts random distillation to detect the locally- and globally-anomalous graphs and the latter combines the one-class classification and graph transformation learning to identify anomalous graphs.
Revised: The former adopts random distillation to detect the locally- and globally-anomalous graphs and the latter combines the one-class classification and graph transformation learning to identify anomalous graphs.

Original: These two algorithms only uselabeled normal graphs as supervision information.
Revised: Both of them only use normal graphs to train models.

Revised: They learn anomaly scores for graphs to measure each graph’ anomaly degree and do not provide any criterion to determine whether a graph is anomalous or not.

Original: For the fairness of experiments and to answer Q2, we also compare iGAD with GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD.
Revised: For the fairness of experiments and to answer Q2, we also compare iGAD with GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD.

Original: These algorithms are the graph anomaly detection (GAD) variants of GCN, GIN, DGCNN, SOPool, and RWGNN.
Revised: These algorithms are the graph anomaly detection (GAD) variants of GCN, GIN, DGCNN, SOPool, and RWGNN.

Original: They replace the crossentropy loss in the original graph classification algorithms with the proposed PMI-based loss function.
Revised: We replace the crossentropy loss in these algorithms with the proposed PMI-based loss function.

Original: To answer Q3, we consider a variant of iGAD —iGAD-1 that only utilizes the anomalous graph attribute-aware graph convolution to represent graphs and detect anomalous graphs.
Revised: To answer Q3, we consider a variant of iGAD —iGAD-1 that only utilizes the anomalous graph attribute-aware graph convolution to represent graphs and detect anomalous graphs.

Original: To answer

Original:  Q4, we consider the other variant named iGAD-2 in which the random walking performs onthe input graphs andthe pre-defined anomalous graph substructures simultaneously.
Revised: To answer Q4, we consider the other variant named iGAD-2 in which the random walking performs on input graphs and pre-defined anomalous graph substructures simultaneously.

Original: Specifically,  3-star, 
Revised: Specifically, we use 3-star, triangle, tailed triangle,

Original: [Figure]
Revised: [Figure]

Original: the setting M =5 and n ′ =8. The second row reports the performance ofiGAD under the setting M =5 and n ′ =10.
Revised: performance of

Original: triangle, tailed triangle, and 4-cycle graphlets are pre-defined graph anomalous substructures (see Figure 4 in Appendix B for these substructures).
Revised: The and 4-cycle graphlets as pre-defined graph anomalous substructures (see Figure 4 in Appendix B for these substructures).

Original: The parameter setting and the implementationof all the above models can be found in Appendix B.1.
Revised: Information about parameter setting and algorithm implementation can be found in Appendix B.1.

Original: We reportthe experimental results under 5-fold cross-validation.
Revised: We report experimental results under 5-fold cross-validation.

Original: 5.2 Experimental Results
Revised: 5.2 Experimental Results

Original: [Figure]
Revised: [Figure]

Original: This section answers Q1 - Q6 by A1 - A6, respectively.
Revised: This section answers Q1 - Q6 by A1 - A6, respectively.

Revised: We put extra experiments in Appendix B.2.

Original: A1.
Revised: A1.

Original: iGAD demonstrates effective performance in identifying graph-level anomalies.
Revised: iGAD demonstrates superior performance in identifying graph-level anomalies.

Original: The performance of iGAD exceeds all graph classification baselines, their corresponding GAD variants, and the state-of-the-art graph-level anomaly detection methods. Observed from Table 2: 1) Compared with  GCN, GIN, DGCNN, SOPool, and RWGNN, iGAD achieves superior performance on all datasets in terms of all evaluation metrics; 2) iGAD also beats  GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD in terms of all evaluation metrics.
Revised: Observed from Table 2: 1) Compared with the graph classification baselines GCN, GIN, DGCNN, SOPool, and RWGNN, iGAD demonstrates significant advantages on four real world datasets in terms of all evaluation metrics; 2) iGAD also beats the above baselines’ GAD variants, GCN-GAD, GIN-GAD, DGCNNGAD, SOPool-GAD, and RWGNN-GAD in terms of all evaluation metrics.

Original: In addition, as shown in Table 3, iGAD outperforms two graph-level anomaly detection methods GLocalKD and OCGTL with obvious advantages in terms of AUC.
Revised: In addition, as shown in Table 3, iGAD outperforms two graph-level anomaly detection methods GLocalKD and OCGTL concerning AUC.

Original: A2.
Revised: A2.

Original: The PMI-based loss function can capture essential correlations between graphs and their anomalous/normal proprieties even if the data distribution is imbalanced.
Revised: PMI-based loss function can capture essential correlations between graphs and their anomalous/normal proprieties even if the data distribution is imbalanced.

Original: As shown in Table 2, compared with the baseline models with the cross-entropy loss function (GCN, GIN, DGCNN, 
Revised: As shown in Table 2, compared with GNNs with the crossentropy loss function (GCN, GIN, DGCNN, SOPOOL, and

Original: SOPOOL, and RWGNN), their GAD variants equipped with the newly proposed PMI-based loss function (GCN-GAD, GIN-GAD, DGCNN-GAD, SOPOOL-GAD, and RWGNN-GAD) witness a significant performance improvement regarding Recall and Recall(A) on all datasets.
Revised: RWGNN), their GAD variants equipped with the newly proposed PMI-based loss function (GCN-GAD, GIN-GAD, DGCNN-GAD, SOPOOL-GAD, and RWGNN-GAD) witness a significant performance improvement regarding Recall and Recall(A) over all datasets.

Original: For example,on the dataset SW-620, the  Recall and Recall(A)value achieved by DGCNN  are 56.02 ± 1.16 and 12.74 ± 2. respectively.
Revised: For example, the value of Recall and Recall(A) achieved by DGCNN over the dataset SW-620 are 56.02 ± 1.16 and 12.74 ± 2.41 respectively.

Original: In contrast, 72.65 ± 0.63 and 67.84 ± 1.78 are reported by DGCNN-GAD.
Revised: In contrast, 72.65 ± 0.63 and 67.84 ± 1.78 are reported by DGCNN-GAD.

Revised: PMI-based loss function can greatly improve GNN-based graph classification model in datasets with highly imbalanced class distributions.

Original: A3.
Revised: A3.

Original: Theexperiment results indicate that the anomalous attribute-aware graph convolution achieves competitive performance in encoding anomalous attributes.
Revised: The anomalous attribute-aware graph convolution can embed anomalous attributes into graph representations and improve graph-level anomaly detection.

Original: As shown in Table 2, iGAD-1 significantly outperforms all baseline algorithms on four graph datasets in terms of AUC , Recall , and F-score .
Revised: As shown in Table 2, iGAD-1 significantly outperforms all baseline algorithms on four graph datasets in terms of AUC , Recall , and F-score .

Original: Concerning Recall(A) , on the dataset MCF-7, iGAD-1 has a little performance degradation compared with DGCNN-GAD.
Revised: Concerning Recall(A) , on the dataset MCF-7, iGAD-1 has a little performance degradation compared with DGCNN-GAD.

Original: Specifically, DGCNN-GAD achieves 69.37 ± 3.05 on MCF-7, while iGAD-1 gets 67.67 ± 2.54 .
Revised: Specifically, DGCNN-GAD achieves 69.37 ± 3.05 on MCF-7, while iGAD-1 gets 67.67 ± 2.54 .

Original: But iGAD-1 beats DGCNN-GAD on other datasets in terms of Recall(A) .
Revised: But iGAD-1 beats DGCNN-GAD on other datasets in terms of Recall(A) .

Original: A4.
Revised: A4.

Original: iGAD is also effective when it is equipped with pre-defined anomalous substructures.
Revised: iGAD is also effective when it is equipped with pre-defined anomalous substructures.

Original: As shown in Table 2, iGAD-2 demonstrates obvious advantages against all baseline models.
Revised: As shown in Table 2, iGAD-2 demonstrates significant advantages against all baseline models.

Original: Moreover, iGAD-2 also beats iGAD-1 concerning all metrics on MOLT-4, PC-3, and MCF-7.
Revised: Moreover, iGAD2 also beats iGAD-1 concerning all metrics over datasets MOLT-4, PC-3, and MCF-7.

Original: iGAD-2 does not outperform than iGAD on all datasets concerning AUC , Recall(A) , and F-score .
Revised: iGAD-2 does not outperform than iGAD on all datasets concerning AUC , Recall(A) , and F-score .

Original: But, on PC-3 and
Revised: But, on PC-3 and

Original: MCF-7, iGAD-2 has a slight lead over iGAD regarding Recall .
Revised: MCF-7, iGAD-2 has a slight lead over iGAD regarding Recall .

Original: The experiment results demonstrate that, in most cases, the performance of iGAD equipped with a deep RWK is better than iGAD-2 using pre-defined anomalous substructures and algorithms that do not consider anomalous substructures.
Revised: The experiment results demonstrate that, in most cases, the performance of iGAD equipped with a deep RWK is better than iGAD-2 using pre-defined anomalous substructures and algorithms that do not consider anomalous substructures.

Original: Based on the above analysis, we can conclude that comparing the random walk sequences between the input graph and pre-defined substructure can improve graph-level anomaly detection to a certain extent, but the algorithm that can automatically learn the anomalous substructures is better.
Revised: Based on the above analysis, we can conclude that comparing the random walk sequences between the input graph and pre-defined substructure can improve graph-level anomaly detection to a certain extent, but the algorithm that can automatically learn the anomalous substructures is better.

Original: A5.
Revised: A5.

Original: The anomalous substructures learned by iGAD are demonstrated in Figure 2.
Revised: The anomalous substructures learned by iGAD are demonstrated in Figure 2.

Original: We can observe that anomalous graph substructures on four datasets are composed of the triangle, the tailed triangle, and the 4-cycle graphlets.
Revised: We can observe that anomalous graph substructures on four datasets are composed of the triangle, the tailed triangle, and the 4-cycle graphlets.

Original: It explains why iGAD-2 can achieve competitive results compared with iGAD (as shown in Table 2 and A4 ).
Revised: It explains why iGAD-2 can achieve competitive results compared with iGAD (as shown in Table 2 and A4 ).

Original: We can draw a preliminary conclusion that the graph substructures having the triangle, the tailed triangle, and the 4-cycle graphlets are important for distinguishing anomalous graphs from normal graphs on these four graph datasets.
Revised: We can draw a preliminary conclusion that the graph substructures having the triangle, the tailed triangle, and the 4-cycle graphlets are important for distinguishing anomalous graphs from normal graphs on these four graph datasets.

Original: A6.
Revised: A6.

Original: The performance of iGAD under different maximum random walk lengths L is shown in Figure 3.
Revised: The performance of iGAD under different maximum random walk lengths L is shown in Figure 3.

Original: Under the setting that iGAD with 5 anomalous substructures and each of substructures has 8 nodes, on these four graph datasets SW-620, MOLT-4, PC-3, and MCF-7, iGAD can achieve the highest
Revised: Under the setting that iGAD with 5 anomalous substructures and each of substructures has 8 nodes, on these four graph datasets SW-620, MOLT-4, PC-3, and MCF-7, iGAD can achieve the highest

Original: F-score value when the maximum random walk length ranges from 4 to 8.
Revised: F-score value when the maximum random walk length ranges from 4 to 8.

Original: In addition, when each anomalous substructure contains 10 nodes, iGAD can reach the best F-score score at the maximum random walk length from 5 to 9.
Revised: In addition, when each anomalous substructure contains 10 nodes, iGAD can reach the best F-score score at the maximum random walk length from 5 to 9.

Original: The results verify that appropriately increasing the maximum length of random walking can improve the performance of iGAD since more much longer random walk sequences can capture more graph structure information.
Revised: The results verify that appropriately increasing the maximum length of random walking can improve the performance of iGAD since more much longer random walk sequences can capture more graph structure information.

Original: Overall, the above experiments demonstrate the effectiveness of iGAD in graph-level anomaly detection.

Original: We put more parameter analysis experiments in Appendix B.2.

Original: Finally, we discuss the limitations of the proposed graph-level anomaly detection model iGAD.
Revised: Finally, we discuss the limitations of the proposed graph-level anomaly detection model iGAD.

Original: In practice, the nodes and/or substructures at a particular position in a graph can also be the reason for the graph’s anomalous properties.
Revised: In practice, the nodes and/or substructures at a particular position in a graph can also be the reason for the graph’s anomalous properties.

Original: For example, an Alcohol compound can be obtained by replacing the hydrogen atom at the end of the Alkane compound with a hydroxy group (i.e., -OH).
Revised: For example, an Alcohol compound can be obtained by replacing the hydrogen atom at the end of the Alkane compound with a hydroxy group (i.e., -OH).

Original: But, iGAD cannot capture the position information of nodes and substructures in graphs.
Revised: But, iGAD cannot capture the position information of nodes and substructures in graphs.

Original: In addition, iGAD is a supervised algorithm.
Revised: In addition, iGAD is a supervised algorithm.

Original: It may not be able to achieve superior performance in identifying the graph-level anomalies that do not have any common characteristics with the training  anomalies.
Revised: It may not be able to achieve superior performance in identifying the graph-level anomalies that follow the anomaly distributions that have never been exposed in the training graph anomalies.

Original: In the future, we will propose specific solutions to this problem.
Revised: In the future, we will propose specific solutions to this problem.

Original:  Conclusion
Revised:  Conclusion

Original: This paper focused on graph-level anomaly detection which is a research topic of great practical value but less explored.
Revised: This paper proposes a method named iGAD for graph-level anomaly detection.

Original: We highlighted the challenges brought bythe imbalanced distribution and various anomaly notions in graph attributes and substructures.
Revised: We highlight the challenges brought by imbalanced distributions and various anomaly notions in graph attributes and substructures.

Original: In addition, the significant differences between detecting anomalous graphs in a graph dataset and identifying anomalous nodes in a single graph were also discussed.
Revised: In addition, the significant differences between detecting anomalous graphs in a graph dataset and identifying anomalous nodes in a single graph are also introduced in detail.

Original: The proposed method iGAD regarded graph-level anomaly detection as a special case of graph classification.
Revised: iGAD treats graph-level anomaly detection as a special case of graph classification and learns binary labels for graphs.

Original: In iGAD, the anomalous attribute-aware graph convolution and anomalous substructure-aware deep RWK embed graph-level anomaly information into graph representations.
Revised: In iGAD, the anomalous attribute-aware graph convolution and anomalous substructure-aware deep RWK embed graph-level anomaly information into graph representations.

Original: AMLP with the PMI-based loss function is employed to distinguish anomalous samples from  normal samples based on graph representations.
Revised: A PMI-based loss function guides iGAD to identify anomalous samples from a large number of normal samples.

Original: We did extensive experiments on four  graph datasets, and iGAD achieved superior performance on the graph-level anomaly detection task.
Revised: We do extensive experiments on four real-world graph datasets, and iGAD achieves superior performance on the graph-level anomaly detection task.

Original: Acknowledge
Revised: Acknowledge

Original: We greatly appreciate the constructive and insightful comments from reviewers.
Revised: We greatly appreciate the constructive and insightful comments from reviewers.

Original: This work is funded by the Australian Research Council Discovery Early Career Researcher Award (ARC DECRA)
Revised: This work is funded by the Australian Research Council Discovery Early Career Researcher Award (ARC DECRA)

Original: Project (No. DE200100964) and the NSFC (No. 61872360).
Revised: Project (No. DE200100964) and the NSFC (No. 61872360).

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] see section 5.

Original:  (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] 3.
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether
Revised: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re us- ing/curating?
Revised: and how consent was obtained from people whose data you’re us- ing/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable? 
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N [Table]

Original: [N/A] (b) Did youdescribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: A] (b you any potential participant risks, with links to Institutional Review Board IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]