Original: Dual-discriminative Graph Neural Network for Imbalanced Graph-level Anomaly Detection
Revised: Dual-discriminative Graph Neural Network for Imbalanced Graph-level Anomaly Detection

Original: Abstract
Revised: Abstract

Original: Graph-level anomaly detection aims to distinguish anomalous graphs in a graph dataset from normal graphs.
Revised: Graph-level anomaly detection aims to distinguish anomalous graphs in a graph dataset from normal graphs.

Original: Anomalous graphs represent a very few but essential patterns in the real world.
Revised: Anomalous graphs represent a very few but essential patterns in the real world.

Original: The anomalous property of a graph may be referable to its anomalous attributes of particular nodes and anomalous substructures that refer to a subset of nodes and edges in the graph.
Revised: The anomalous property of a graph may be referable to its anomalous attributes of particular nodes and anomalous substructures that refer to a subset of nodes and edges in the graph.

Original: In addition, due to the imbalance nature ofthe anomaly problem, anomalous information will be diluted by normal graphs with overwhelming quantities.
Revised: In addition, due to the imbalance nature of anomaly problem, anomalous information will be diluted by normal graphs with overwhelming quantities.

Original: Various anomaly notions in the attributes and/or substructures and the imbalance nature together make detecting anomalous graphs a non-trivial task.
Revised: Various anomaly notions in the attributes and/or substructures and the imbalance nature together make detecting anomalous graphs a non-trivial task.

Original: In this paper, we propose a dual-discriminative graph neural network for graph-level anomaly detection, namely iGAD.
Revised: In this paper, we propose a dual-discriminative graph neural network for graph-level anomaly detection, namely iGAD.

Original: Specifically, an anomalous graph attribute-aware graph convolution and an anomalous graph substructure-aware deep Random Walk Kernel (deep RWK) are welded into a graph neural network to achieve the dual-discriminative ability on anomalous attributes and substructures.
Revised: Specifically, an anomalous graph attribute-aware graph convolution and an anomalous graph substructure-aware deep Random Walk Kernel (deep RWK) are welded into a graph neural network to achieve the dual-discriminative ability on anomalous attributes and substructures.

Original: The deep RWK in iGAD makes up for the deficiency of graph convolution in distinguishing structural information caused by the simple neighborhood aggregation mechanism.
Revised: The deep RWK in iGAD makes up for the deficiency of graph convolution in distinguishing structural information caused by the simple neighborhood aggregation mechanism.

Original: Further, we propose a Point Mutual Information (PMI)-based loss function to target the problems caused by imbalance distributions.
Revised: Further, we propose a Point Mutual Information (PMI)-based loss function to target the problems caused by imbalance distributions.

Original: The loss function enables iGAD to capture the essential correlation between input graphs and their anomalous/normal properties.
Revised: The loss function enables iGAD to capture the essential correlation between input graphs and their anomalous/normal properties.

Original: We evaluate iGAD on four realworld graph datasets.
Revised: We evaluate iGAD on four realworld graph datasets.

Original: Extensive experiments demonstrate the superiority of iGAD on the graph-level anomaly detection task.
Revised: Extensive experiments demonstrate the superiority of iGAD on the graph-level anomaly detection task.

Original:  Introduction
Revised:  Introduction

Original: Graphs have been used as an indispensable tool to describe relational data, such as chemical compounds, transport networks, and social networks.
Revised: Graphs have been used as an indispensable tool to describe relational data, such as chemical compounds, transport networks, and social networks.

Original: For instance, in a graph that models a chemical compound, nodes represent atoms and edges describe chemical bonds between atoms.
Revised: For instance, in a graph that models a chemical compound, nodes represent atoms and edges describe chemical bonds between atoms.

Original: Learning graphs to understand the world has been drawing considerable attention in recent years [5, 14, 15, 50].
Revised: Learning graphs to understand the world has been drawing considerable attention in recent years [5, 14, 15, 50].

Original: Given a graph dataset containing a set of graphs, the graph-level anomaly detection task aims to detect graphs with anomalous information.
Revised: Given a graph dataset containing a set of graphs, the graph-level anomaly detection task aims to detect graphs with anomalous information.

Original: Generally, these anomalous graphs with special characteristics or functions are the minority in a graph dataset.
Revised: Generally, these anomalous graphs with special characteristics or functions are the minority in a graph dataset.

Original: For example, among all antibody molecules, only a few of them exhibit anti-cancer activity on a specific type of cancer cell line [54].
Revised: For example, among all antibody molecules, only a few of them exhibit anti-cancer activity on a specific type of cancer cell line [54].

Original: Graph-level anomaly detection is a research topic of great practical value, say, it can support cancer drug discovery by identifying molecules with anti-cancer activity from other antibody molecules [47].
Revised: Graph-level anomaly detection is a research topic of great practical value, say, it can support cancer drug discovery by identifying molecules with anti-cancer activity from other antibody molecules [47].

Original: Also, the research can help to reveal pathogenic brain mechanisms by distinguishing the brain structures of patients with neurological disorders from healthy individuals [21].
Revised: Also, the research can help to reveal pathogenic brain mechanisms by distinguishing the brain structures of patients with neurological disorders from healthy individuals [21].

Original: In practice, the attributes of particular nodes are a contributing factor to a graph’s anomalous properties.
Revised: In practice, the attributes of particular nodes are a contributing factor to a graph’s anomalous properties.

Original: For example, the hydrogen and oxygen atoms at the end of molecules are the key to discriminating between Alcohol and Alkane [44].
Revised: For example, the hydrogen and oxygen atoms at the end of molecules are the key to discriminating between Alcohol and Alkane [44].

Original: Graph substructures can be another factor for a graph’s anomalous properties.
Revised: Graph substructures can be another factor for a graph’s anomalous properties.

Original: For instance,some pharmacophores composed of  substructures(e.g., triangle and cycle-based graphlets) can induce molecular toxicity.
Revised: For instance, pharmacophores composed of some particular substructures can induce molecular toxicity.

Original: These graph attributes and substructures that are essential for distinguishing anomalous graphs from normal graphs are regarded as anomalous attributes and anomalous substructures , respectively.
Revised: These graph attributes and substructures that are essential for distinguishing anomalous graphs from normal graphs are regarded as anomalous attributes and anomalous substructures , respectively.

Original: Most existing graph mining methods, especially those graph convolution-based models which have achieved state-of-theart performance [53, 58, 20, 3], focus on the aggregation of neighborhood-structured information.
Revised: Most existing graph mining methods, especially those graph convolution-based models which have achieved state-of-the-art performance [53, 58, 20, 3], focus on the aggregation of neighborhood-structured information.

Original: However, the anomalous structure information could be all possible substructures.
Revised: However, the anomalous structure information could be all possible substructures.

Original: In addition, the anomalous attributes and substructures will be diluted by the information of normal graphs with overwhelming quantities, which is the imbalance nature of the anomaly problem.
Revised: In addition, the anomalous attributes and substructures will be diluted by the information of normal graphs with overwhelming quantities, which is the imbalance nature of the anomaly problem.

Original: In summary, various anomaly notions in attributes and/or substructures and the imbalance nature make graph-level anomaly detection an important yet challenging problem.
Revised: In summary, various anomaly notions in attributes and/or substructures and the imbalance nature make graph-level anomaly detection an important yet challenging problem.

Original: In this paper, we propose a dual-discriminative graph neural network for i mbalanced G raph-level
Revised: In this paper, we propose a dual-discriminative graph neural network for i mbalanced G raph-level

Original: A nomaly D etection ( iGAD ).
Revised: A nomaly D etection ( iGAD ).

Original: Specifically, iGAD investigates a novel anomalous attribute-aware graph convolution.
Revised: Specifically, iGAD investigates a novel anomalous attribute-aware graph convolution.

Original: The final node representation composed of the independent encoding on a node itself and the independent encoding on its neighborhoods from different hops.
Revised: The final node representation is the concatenation of the independent encoding on a node itself and the independent encoding on its neighborhoods from different hops.

Original: In this way, the attributes of each node in a graph can evolve independently over the graph convolution process and the anomalous attributes on particular nodes can be exposed.
Revised: In this way, the attributes of each node in a graph can evolve independently over the graph convolution process and the anomalous attributes on particular nodes can be exposed.

Original: However, it is intractable to enable graph convolution to be aware of anomalous substructures.
Revised: However, it is intractable to enable graph convolution to be aware of anomalous substructures.

Original: Graph convolution performs message passing depending on multi-hop neighborhoods only and cannot explore all possible substructures in a graph.
Revised: Due to the high dependence on neighborhood aggregation, graph convolution can only capture very limited substructures in graphs.

Original: Even the simple substructures like triangles, graph convolution is unable to count them in a graph [3].
Revised: Moreover, even the simple substructures such as triangles, graph convolution cannot count them in a graph [3].

Original: One of the Graph Kernels (GKs), RWK [46], as another way to learn graphs, involves the comparison of any substructures among graphs. RWK learns graphs by exploring common random walk sequences among all input  graphs.
Revised: RWK [46], as one type of Graph Kernels (GKs), learns graphs by exploring common random walk sequences among all input graphs, which naturally involves the comparison about diverse substructures in different graphs.

Original: To embed the information of anomalous substructures into graph representations, we propose to compare random walk sequences of input graphs against anomalous substructures.
Revised: To embed the information of anomalous substructures into graph representations, we propose to compare random walk sequences of input graphs against anomalous substructures.

Original: By modeling anomalous substructures as parameters, they became learnable and can get updated through back-propagation.
Revised: By modeling anomalous substructures as parameters, they became learnable and can get updated through back-propagation.

Original: We entitle this design anomalous substructure-aware deep RWK.
Revised: We entitle this design anomalous substructure-aware deep RWK.

Original: In addition, we propose a new loss function to target the imbalance nature of the anomaly problem caused by the great difference in the number of normal and anomalous graphs.
Revised: In addition, we propose a new loss function to target the imbalance distribution caused by the great difference in the number of normal and anomalous graphs.

Original: The newly proposed loss function models the PMI between graphs and their normal/anomalous properties and enables iGAD to capture anomalous information from an imbalanced graph dataset.
Revised: The newly proposed loss function models the PMI between graphs and their normal/anomalous properties and enables iGAD to capture anomalous information from an imbalanced graph dataset.

Original: Welding three designs mentioned above into a graph neural network, the graph-level anomaly detection model iGAD is achieved.
Revised: Welding three designs mentioned above into a graph neural network, the graph-level anomaly detection model iGAD is achieved.

Original: We conclude the contributions of iGAD as follows:
Revised: We conclude the contributions of iGAD as follows:

Original: • The anomalous attribute-aware graph convolution exposes anomalous attributes by keeping the independent evolution of each node’s attribute.
Revised: • The anomalous attribute-aware graph convolution exposes anomalous attributes by keeping the independent evolution of each node’s attributes.

Original: • Anomalous substructures are modeled as learnable parameters.
Revised: • Anomalous substructures are modeled as learnable parameters.

Original: The proposed anomalous substructure-aware deep RWK embeds the information of anomalous substructures into graph representations by comparing the random walk sequences inthe input graphs andthe learned anomalous substructures.
Revised: Substructure-aware deep RWK embeds the information of anomalous substructures into graph representations by comparing the random walk sequences in input graphs and learned anomalous substructures.

Original: • The PMI-based loss function addresses issues brought by the imbalance nature of the anomaly problem and enables iGAD to learn anomalous graphs as a minority effectively.
Revised: • The PMI-based loss function addresses issues brought by the imbalance nature of the anomaly problem and enables iGAD to learn anomalous graphs as a minority effectively.

Original: • Comprehensive experiments demonstrate the effectiveness of iGAD, which outperforms state-of-the-art graph convolution-based graph classification models and graph-level anomaly detection algorithms on 
Revised: • Comprehensive experiments demonstrate the effectiveness of iGAD, which outperforms state-of-the-art graph convolution-based graph classification models and graph-level anomaly detection algorithms on four real

Original: four real-world graph datasets.
Revised: -world graph datasets.

Original: The code is available at https://github.com/graph-level-anomalies/iGAD.
Revised: The code is available at https://github.com/graph-level-anomalies/iGAD.

Original:  Related Work
Revised:  Related Work

Original: Anomaly Detection.
Revised: Anomaly Detection.

Original: The task aims to identify anomalies that deviate from the majority of samples.
Revised: The task aims to identify anomalies that deviate from the majority of samples.

Original: Early anomaly detection algorithms use indicators, such as distance [18] and density [7], to describe the degree of deviation among samples to detect anomalous ones.
Revised: Early anomaly detection algorithms use indicators, such as distance [18] and density [7], to describe the degree of deviation among samples to detect anomalous ones.

Original: For example, isolation forest-based algorithms [25, 26] identify the anomalous by building the binary search tree structure to isolate samples.
Revised: For example, isolation forest-based algorithms [25, 26] identify the anomalous by building the binary search tree structure to isolate samples.

Original: In addition, anomaly detection algorithms based on one-class classification [39, 38, 36, 62] and autoencoder [2, 63] have also been in the spotlight.
Revised: In addition, anomaly detection algorithms based on one-class classification [39, 38, 36, 62] and autoencoder [2, 63] have also been in the spotlight.

Original: For the graph-structured data, most graph anomaly detection algorithms dedicate to detect anomalies (e.g., anomalous nodes [43, 13, 57] and edges [35, 56]) in a single graph [1].
Revised: For the graph-structured data, most graph anomaly detection algorithms dedicate to detect anomalies (e.g., anomalous nodes [43, 13, 57] and edges [35, 56]) in a single graph [1].

Original: The anomaly notions of graph-level anomalies are more diversified than those of node-level anomalies.
Revised: The anomaly notions of graph-level anomalies are more diversified than those of node-level anomalies.

Original: For example, node-level anomaly detection methods usually regard nodes that betray network homophily 1 as the anomalies [43].
Revised: For example, node-level anomaly detection methods usually regard nodes that betray network homophily 1 as the anomalies [43].

Original: However, at the graph level, nodes that can achieve an anomalous graph may include but are not limited to the nodes that do not meet homophily.
Revised: However, at the graph level, nodes that can achieve an anomalous graph go far beyond the nodes that do not meet homophily.

Original: Node-level anomaly detection methods identify anomaly nodes from a micro perspective (e.g., neighborhood structure) while identifying anomalous graphs needs learning the graph dataset at the macro level.
Revised: Node-level anomaly detection methods identify anomaly nodes from a micro perspective (e.g., neighborhood structures) while identifying anomalous graphs needs learning graphs at the macro level.

Original: Employing node-level anomaly detection methods to detect graph-level anomalies could capture some anomalies, but only a few.
Revised: Employing node-level anomaly detection methods to detect graph-level anomalies could capture some anomalies, but only a small part.

Original: Moreover, the graph-level anomaly detection algorithms need to consider anomaly notions related to all possible substructures.
Revised: Moreover, the graph-level anomaly detection algorithms need to consider anomaly notions related to graph substructures.

Original: Hence, graph anomaly detection methods that identify anomalies in a single graph cannot adapt to the graph-level anomaly detection problem through simple modifications.
Revised: Hence, graph anomaly detection methods that identify anomalies in a single graph cannot adapt to the graph-level anomaly detection problem through simple modifications.

Original: Graph-level Anomaly Detection.
Revised: Graph-level Anomaly Detection.

Original: To the best of our knowledge,  only several preliminary research is proposed for the  graph-level anomaly detection task.
Revised: To the best of our knowledge, there are only several preliminary research on the topic of graph-level anomaly detection.

Original: For example, Zhao et al.
Revised: Zhao et al.

Original: [61] investigated a research question of “should we use the data for the graph classification task to evaluate graph-level anomaly detection models?”, followed by which two graph-level anomaly detection algorithms [27, 34]
Revised: [61] investigated a research question of “should we use the data for the graph classification task to evaluate graph-level anomaly detection models?”, followed by which two graph-level anomaly detection algorithms [27, 34]

Original: combined Graph Neural Networks (GNNs) with knowledge distillation and one-class classification  to detect anomalous graphs, respectively.
Revised: combined Graph Neural Networks (GNNs) with knowledge distillation and one-class classification respectively to detect anomalous graphs.

Original: Graph-level anomaly detection can be regarded as a special case of graph classification.
Revised: Graph-level anomaly detection can be regarded as a special case of graph classification.

Original: There is a huge gap between the number of graphs belonging to the normal class versus the anomalous ones.
Revised: There is a huge gap between the number of graphs belonging to the normal class versus the anomalous ones.

Original: But, graph classification algorithms usually assume that the number of graphs in each class is balanced.
Revised: But, graph classification algorithms usually assume that the number of graphs in each class is balanced.

Original: GKs [46, 16, 41, 6, 40] used to be the mainstream graph classification methods.
Revised: GKs [46, 16, 41, 6, 40] used to be the mainstream graph classification methods.

Original: Generally, GKs use the kernel function to calculate kernel values (i.e., similarity) between pair-wised graphs.
Revised: Generally, GKs use the kernel function to calculate kernel values (i.e., similarity) between pair-wised graphs.

Original: Afterward, an off-the-shelf classifier is applied to the graph similarity matrix to perform the classification task.
Revised: Afterward, an off-the-shelf classifier is applied to the graph similarity matrix to perform the classification task.

Original: Graph classification algorithms based on GKs cannot learn graph representations explicitly and be optimized in an end-to-end fashion.
Revised: Graph classification algorithms based on GKs cannot learn graph representations explicitly and be optimized in an end-to-end fashion.

Original: In recent years, graph mining practitioners pay more attention to
Revised: In recent years, graph mining practitioners pay more attention to

Original: GNNs [8, 11, 4, 37].
Revised: GNNs [8, 11, 4, 37].

Original: GNNs with spatial graph convolution [20, 45, 19] are summarized as messagepassing framework (MPNNs)
Revised: GNNs with spatial graph convolution [20, 45, 19] are summarized as messagepassing framework (MPNNs)

Original: In general, MPNNs update the representation of each node by iteratively aggregating information coming from their neighbors.
Revised: In general, MPNNs update the representation of each node by iteratively aggregating information coming from their neighbors.

Original: To represent a graph,
Revised: To represent a graph,

Original: MPNNs apply the readout function or pooling operation on node embeddings.
Revised: MPNNs apply the readout function or pooling operation on node embeddings.

Original: The spatial graph convolution that mix the representation of nodes and their neighborhoods to represent nodes will smooth node attributes [60, 4].
Revised: The spatial graph convolution that mix the representation of nodes and their neighborhoods to represent nodes will smooth node attributes [60, 4].

Original: In addition, [31, 28, 59] proved that spatial graph convolution has limited power in distinguishing graph structures.
Revised: In addition, [31, 28, 59] proved that spatial graph convolution has limited power in distinguishing graph structures.

Original: Performing message passing between subgraphs can enable spatial graph convolution to capture more graph structural information.
Revised: Performing message passing between subgraphs can enable spatial graph convolution to capture more graph structural information.

Original: But such a solution consumes exponentially increases memory and computation complexities [3].
Revised: But such a solution consumes exponentially increases memory and computation complexities [3].

Original: Graph classification methods mentioned above are likely to underfit the anomalous graphs since they do not have any intrinsic mechanism to target the imbalanced distributions.
Revised: Graph classification methods mentioned above are likely to underfit the anomalous graphs since they do not have any intrinsic mechanism to target the imbalanced distributions.

Original: In iGAD, a novel loss is presented for the imbalanced distributions by modeling PMI.
Revised: In iGAD, a novel loss is presented for the imbalanced distributions by modeling PMI.

Original: In addition, different from existing graph-level anomaly methods [61, 27, 34] that employ the off-the-shelf MPNNs (e.g., GIN [53]) to represent graphs, iGAD can embed the information of anomalous attributes and substructures into graph representations.
Revised: In addition, different from existing graph-level anomaly methods [61, 27, 34] that employ the off-the-shelf MPNNs (e.g., GIN [53]) to represent graphs, iGAD can embed the information of anomalous attributes and substructures into graph representations.

Original: Imbalanced Learning.
Revised: Imbalanced Learning.

Original: Learning data with imbalanced distribution can be divided into the data- and algorithm-level approaches.
Revised: Learning data with imbalanced distribution can be divided into the data- and algorithm-level approaches.

Original: The former alleviates the imbalanced distribution of the original data by either under-sampling the majority [48] or over-sampling the minority [9].
Revised: The former alleviates the imbalanced distribution of the original data by either under-sampling the majority [48] or over-sampling the minority [9].

Original: However, modifying the original distribution of data will bring a lot of drawbacks inevitably, such as over-fitting and discarding important samples.
Revised: However, modifying the original distribution of data will bring a lot of drawbacks inevitably, such as over-fitting and discarding important samples.

Original: There are different types of approaches to address the issue at the algorithm-level.
Revised: There are different types of approaches to address the issue at the algorithm-level.

Original: The typical approaches include employing post-hoc correction [30], aligning data distribution, [52] and modifying the loss function [24, 22].
Revised: The typical approaches include employing post-hoc correction [30], aligning data distribution, [52] and modifying the loss function [24, 22].

Original: In the real-world application, considering the different consequences of false-positive identification (e.g., predicting an inactive chemical compound to be active) and false-negative identification (e.g., predicting a cancer patient as a healthy individual), the situation will be more complicated [33].
Revised: In the real-world application, considering the different consequences of false-positive identification (e.g., predicting an inactive chemical compound to be active) and false-negative identification (e.g., predicting a cancer patient as a healthy individual), the situation will be more complicated [33].

Original: In this paper, iGAD introduces PMI to the cross-entropy loss function to solve this problem.
Revised: In this paper, iGAD introduces PMI to the cross-entropy loss function to solve this problem.

Original:  Preliminaries
Revised:  Preliminaries

Original: Notations.
Revised: Notations.

Original: The graph dataset consisting of N graphs can be denoted as G =
Revised: The graph dataset consisting of N graphs can be denoted as G =

Original: { G 1 , ..., G N } , where
Revised: { G 1 , ..., G N } , where

Original: G i =
Revised: G i =

Original: { V i , E i } is an unweighted and undirected graph in G , V i
Revised: { V i , E i } is an unweighted and undirected graph in G , V i

Original: and
Revised: and

Original: E
Revised: E

Original: i are the node and the edge set, respectively.
Revised: i are the node and the edge set, respectively.

Original: The topology of G i can be modeled as an adjacency matrix A i ∈ { 0 , 1 } n × n , where
Revised: The topology of G i can be modeled as an adjacency matrix A i ∈ { 0 , 1 } n × n , where

Original: A i ( u, v ) = 1 if there is an edge between nodes u and v , otherwise 0.
Revised: A i ( u, v ) = 1 if there is an edge between nodes u and v , otherwise 0.

Original: A i does not consider the self-loop, A i ( u, u ) = 0 , ∀ u ∈ V i .
Revised: A i does not consider the self-loop, A i ( u, u ) = 0 , ∀ u ∈ V i .

Original: We use x u ∈ R c to denote the attribute vector of node u ∈ V i .
Revised: We use x u ∈ R c to denote the attribute vector of node u ∈ V i .

Original: X i ∈ R n × c is the attribute matrix of G i .
Revised: X i ∈ R n × c is the attribute matrix of G i .

Original: G ′
Revised: G ′

Original: i =
Revised: i =

Original: { V ′ i , E ′ i } can be regarded as a substructure in G i , iff V ′ i ⊆
Revised: { V ′ i , E ′ i } can be regarded as a substructure in G i , iff V ′ i ⊆

Original: V i and E ′ i ⊆
Revised: V i and E ′ i ⊆

Original: E i .
Revised: E i .

Original: Y is the label set of G .
Revised: Y is the label set of G .

Original: G i is labeled by y i ∈ { 0 , 1 } , where 1 represents anomalous and 0 represents normal .
Revised: G i is labeled by y i ∈ { 0 , 1 } , where 1 represents anomalous and 0 represents normal .

Original: Problem Definition.
Revised: Problem Definition.

Original: This paper concentrates on the supervised graph-level anomaly detection problem.
Revised: This paper concentrates on the supervised graph-level anomaly detection problem.

Original: Given the training set T = { ( G 1 , y 1 ) , ( G 2 , y 2 ) , ... } , we aim to train a model to predict the normal/anomalous proprieties of unseen graphs.
Revised: Given the training set T = { ( G 1 , y 1 ) , ( G 2 , y 2 ) , ... } , we aim to train a model to predict the normal/anomalous proprieties of unseen graphs.

Original: T contains N 1 anomalous graphs and N 0 normal graphs ( N 1 ≪ N 0 ).
Revised: T contains N 1 anomalous graphs and N 0 normal graphs ( N 1 ≪ N 0 ).

Original: The imbalanced ratio of T is defined as δ ( δ =
Revised: The imbalanced ratio of T is defined as δ ( δ =

Original: N 1 N 0 + N 1 ).
Revised: N 1 N 0 + N 1 ).

Original: Spatial Graph Convolution.
Revised: Spatial Graph Convolution.

Original: It updates each node’s representation in a graph by iteratively aggregating the features coming from its one-hop neighborhood.
Revised: It updates each node’s representation in a graph by iteratively aggregating the features coming from its one-hop neighborhood.

Original: The formulation is:
Revised: The formulation is:

Revised: [Equation]

Revised: [Equation]

Revised: ) , f Aggregate (

Revised: [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Revised: v

Revised: [Equation]

Revised: N

Revised: [Equation]

Revised: } )) (cid:17) ,

Revised: [Equation]

Original:  where 0 ≤ k ≤ K  , N ( u ) denotes the 1-hop neighborhood of node u .
Revised: [Equation] u [Equation] where 0 ≤ k ≤ K − 1 , N ( u ) denotes the 1-hop neighborhood of node u .

Original: f
Revised: f

Original: Aggregate denotes how to aggregate neighbors of node u , and f combine defines how to combine the immediate representations of node u and its neighborhood.
Revised: Aggregate denotes how to aggregate neighbors of node u , and f combine defines how to combine the immediate representations of node u and its neighborhood.

Original: σ is the activation function.
Revised: σ is the activation function.

Original: Generally, the final representation of the node u is h ( K ) ( u ) .
Revised: Generally, the final representation of the node u is h ( K ) ( u ) .

Original: Random Walk Kernels (RWK).
Revised: Random Walk Kernels (RWK).

Original: RWK measures graph similarities by counting common random walk sequences among graphs.
Revised: RWK measures graph similarities by counting common random walk sequences among graphs.

Original: There are two graphs G 1 = { V 1 , E 1 } and G 2 =
Revised: There are two graphs G 1 = { V 1 , E 1 } and G 2 =

Original: { V 2 , E 2 } .
Revised: { V 2 , E 2 } .

Original: G ×
Revised: G ×

Original: = { V × , E × } denotes the direct product graph about G 1 and G 2 , where V × =
Revised: = { V × , E × } denotes the direct product graph about G 1 and G 2 , where V × =

Original: { ( u, u ′ )
Revised: { ( u, u ′ )

Original: | u ∈ V 1 , u ′ ∈
Revised: | u ∈ V 1 , u ′ ∈

Original: V 2 } and E × = { (( u, u ′ ) , ( v, v ′ ))
Revised: V 2 } and E × = { (( u, u ′ ) , ( v, v ′ ))

Original: | ( u, v ) ∈
Revised: | ( u, v ) ∈

Original: E 1 , ( u ′ , v ′ )
Revised: E 1 , ( u ′ , v ′ )

Original: ∈
Revised: ∈

Original: E 2 } .
Revised: E 2 } .

Original: Performing random walking on G × acts as doing random walking on G 1 and G 2 simultaneously.
Revised: Performing random walking on G × acts as doing random walking on G 1 and G 2 simultaneously.

Original: The adjacency matrix of direct product graph
Revised: The adjacency matrix of direct product graph

Original: A × can be obtained by the Kronecker product operation ⊗ on A 1 and A 2 , that is A × =
Revised: A × can be obtained by the Kronecker product operation ⊗ on A 1 and A 2 , that is A × =

Original: A 1 ⊗ A 2 .
Revised: A 1 ⊗ A 2 .

Original: The number of l -length common walk sequences between G 1 and G 2 is κ ( l ) ( G 1 , G 2 ) =
Revised: The number of l -length common walk sequences between G 1 and G 2 is κ ( l ) ( G 1 , G 2 ) =

Original: q T × ( A × ) l p × , where p × and q × are the vector about the starting and stopping probabilities of random walks on G × , respectively.
Revised: q T × ( A × ) l p × , where p × and q × are the vector about the starting and stopping probabilities of random walks on G × , respectively.

Original: The elements in p × and q × are all ones when there is no prior information.
Revised: The elements in p × and q × are all ones when there is no prior knowledge.

Original: Overall, the RWK about G 1 and G 2 can be defined as:
Revised: Overall, the RWK about G 1 and G 2 can be defined as:

Original: [Equation]
Revised: [Equation]

Original:  iGAD
Revised:  iGAD

Original: In this section, we will describe iGAD in detail (see Figure 1 for an illustration).
Revised: In this section, we will describe iGAD in detail (see Figure 1 for an illustration).

Original: iGAD contains three key designs: (1) anomalous graph attribute-aware graph convolution; (2) anomalous substructureaware deep RWK; and (3) PMI-based loss function.
Revised: iGAD contains three key designs: (1) anomalous graph attribute-aware graph convolution; (2) anomalous substructureaware deep RWK; and (3) PMI-based loss function.

Original: The first two designs encode the anomalous information on graph attributes and substructures into graph embeddings.
Revised: The first two designs encode the anomalous information on graph attributes and substructures into graph embeddings.

Original: The loss function enables iGAD to get rid of the domination from normal graphs.
Revised: The loss function enables iGAD to get rid of the domination from normal graphs.

Original: 4.1 iGAD: Anomalous Attribute-aware Graph Convolution
Revised: 4.1 iGAD: Anomalous Attribute-aware Graph Convolution

Original: As aforementioned in Section 1, the attributes of particular nodes can achieve an anomalous graph.
Revised: As aforementioned in Section 1, the attributes of particular nodes can achieve an anomalous graph.

Original: Theiterative graph convolutionoperations which update node representations by performing feature transformation on the mixed or concatenated information about nodes and their neighborhoods will 
Revised: The graph convolution which iteratively updates node representations by performing feature transformation on the mixed or concatenated information about nodes and their neighborhoods will make node representations over-smoothed [60]. Under such a graph convolution mechanism, only limited

Original: A Graph Dataset … (cid:29)(cid:3) Normal Graph : Anomalous Graph
Revised: A Graph Dataset … (cid:29)(cid:3) Normal Graph : Anomalous Graph

Original: A n o m a l o u s
Revised: A n o m a l o u s

Original: A tt r i b u t e
Revised: A tt r i b u t e

Original: A n o m a l o u s
Revised: A n o m a l o u s

Original: S u b s t r u c t u r e
Revised: S u b s t r u c t u r e

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: R ea do u t
Revised: R ea do u t

Original: I. Anomalous Attribute-aware Graph Convolution (K=2)
Revised: I. Anomalous Attribute-aware Graph Convolution (K=2)

Original: II.
Revised: II.

Original: Anomalous Substructure-aware Deep RWK (M=2, n’=4, L=5)
Revised: Anomalous Substructure-aware Deep RWK (M=2, n’=4, L=5)

Original: Update A
Revised: Update A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G
Revised: G

Original: Update A
Revised: Update A

Original: B
Revised: B

Original: C
Revised: C

Original: D
Revised: D

Original: E
Revised: E

Original: F
Revised: F

Original: G … …
Revised: G … …

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: D E
Revised: D E

Original: F G
Revised: F G

Original: C o n ca t e n a t
Revised: C o n ca t e n a t

Original: i o n
Revised: i o n

Original: A n o m a l o u s
Revised: A n o m a l o u s

Original: III.
Revised: III.

Original: PMI-based Loss Function
Revised: PMI-based Loss Function

Original: Prediction
Revised: Prediction

Original: F E
Revised: F E

Original: G
Revised: G

Original: E
Revised: E

Original: F
Revised: F

Original: G C … …
Revised: G C … …

Original: A MLP
Revised: A MLP

Original: Figure
Revised: The

Original: Taking learning the graph G N in a graph dataset as an example, we illustrate the framework of iGAD with three designs. I. Anomalous Attribute-aware Graph Convolution, which concatenates the independent encoding on nodes and their neighborhoods to represent nodes in G N . To obtain the graph representation h ConvG N , a readout function is applied to all node representations.
Revised: Deep which performs random walks two learnable

Original: II.
Revised: (i.e

Original: The Anomalous Substructure-aware Deep RWK, which performs random walks on two learnable anomalous substructures (i.e., S1 and S2 ) and the input graph G N simultaneously.
Revised: , S and S

Original: Deep RWK obtains h KerG N  based on kernelvalues κ ( l) ( G N , S m ) , 1 ≤ l ≤ L, 1 ≤ m ≤ M .
Revised: G N h based on kernel κ ( l ( G N , S m = {

Original: III.
Revised: L and

Original: The PMI-based Loss Function. iGAD concatenates h ConvG N and h KerG N to obtain the final graph representation h GN .
Revised: G

Original: Finally, a MLP and the PMI-based loss function are applied to h G N to predict G N .
Revised: Finally, a MLP and the PMI-based loss function are applied to h G N to predict G N .

Original: make node representations over-smoothed [60]. Under such a graph convolution mechanism, only a limited anomalous information can be embedded into corresponding node representations.
Revised: anomalous information can be manifest in corresponding node representations.

Original: We argue that making each node’s attributes evolve independently over the graph convolution process is critical for capturing anomalous attributes.
Revised: We argue that making each node’s attributes evolve independently over the graph convolution process is critical for capturing anomalous attributes.

Original: To achieve that, anomalous attribute-aware graph convolution concatenates the independent encoding on the node itself and  its k -hop neighborhood (1 ≤ k ≤ K ) to represent each node in a graph.
Revised: To achieve it, anomalous attribute-aware graph convolution concatenates the independent encoding on the node itself and the independent encoding on its k -hop neighborhood (1 ≤ k ≤ K ) to represent each node in a graph.

Original: Formally,
Revised: Formally,

Original: [Equation]
Revised: [Equation]

Original: where h u is the final node representation of u ∈ V i , h ( k ) ( u ) is the representation of node u ’s k -hop neighborhood, and f Combine is the concatenation operation.
Revised: where h u is the final node representation of u ∈ V i , h ( k ) ( u ) is the representation of node u ’s k -hop neighborhood, and f Combine is the concatenation operation.

Original: h (0) ( u ) and h ( k ) ( u ) are formulated as:
Revised: h (0) ( u ) and h ( k ) ( u ) are formulated as:

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: respectively, where H (0) i
Revised: respectively, where H (0) i

Original: =
Revised: =

Original: X i , MLP denotes a multi-layer perceptron, σ represents the activation function, and W ( k ) is the trainable weight matrix.
Revised: X i , MLP denotes a multi-layer perceptron, σ represents the activation function, and W ( k ) is the trainable weight matrix.

Original: To obtain the entire graph representation h Conv G i , we apply the permutation invariant Readout function R ( · ) on all nodes in the graph G i .
Revised: To obtain the entire graph representation h ConvG i , we apply the permutation invariant Readout function R ( · ) on all nodes in the graph G i .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: (6) where R ( · ) can be the sum or mean operation.
Revised: where R ( · ) can be the sum or mean operation.

Original: 4.2 iGAD: Anomalous Substructure-aware Deep RWK
Revised: 4.2 iGAD: Anomalous Substructure-aware Deep RWK

Original: The neighborhood aggregation mechanismin graph convolution only utilizes the  neighborhood structure as the clue for message passing, and cannot explore all possible substructures in a graph [59, 10].
Revised: Generally, the neighborhood aggregation mechanism only utilizes the local neighborhood structure to build the rooted subtree around each node and performs message passing [59]), which affects the ability of graph convolution in distinguishing substructures [3, 10].

Original: To make iGAD to be aware of anomalous substructures, we turn to the RWK which can explore all possible substructures by performing random walks in a graph.
Revised: To make iGAD to be aware of anomalous substructures, we resort to the RWK which can explore diverse substructures through random walking in graphs.

Original: There is a pioneering work [32] that represents graphs by the RWK values about input graphs and some substructures.
Revised: There is a pioneering work [32] that represents graphs by the RWK values about input graphs and some substructures.

Original: It inspires us to encode information of anomalous substructures into graph representations by performing random walks in input graphs and learnable anomalous substructures simultaneously.
Revised: It inspires us to embed information of anomalous substructures into graph representations by performing random walks in input graphs and learnable anomalous substructures simultaneously.

Original: We present an anomalous substructure-aware deep RWK, which models each element in the adjacency matrices of anomalous substructures as a learnable parameter.
Revised: We present an anomalous substructure-aware deep RWK, which models each element in the adjacency matrices of anomalous substructures as a learnable parameter.

Original: Supposed that { S 1 , ..., S M } is the set of M anomalous substructures, and the size of S m is n ′ .
Revised: Supposed that { S 1 , ..., S M } is the set of M anomalous substructures, and the size of S m is n ′ .

Original: P m ∈ R n ′ × n ′ is the adjacency matrix of S m , where P m ( u ′ , v ′ ) denotes the possibility that an edge exists between nodes u ′ and v ′ in S m .
Revised: P m ∈ R n ′ × n ′ is the adjacency matrix of S m , where P m ( u ′ , v ′ ) denotes the possibility that an edge exists between nodes u ′ and v ′ in S m .

Original: Due to the symmetry of the adjacency matrix, P m contains n ′ ( n ′ − 1)parameters.
Revised: Due to the symmetry of the adjacency matrix, P m contains n ′ ( n ′ − 1)parameters.

Original: Referring to Eq.and supposing that the starting and the stopping possibilities on each node are all ones, the number of l -length common random walk sequences between the input graph G i and the anomalous substructure
Revised: Referring to Eq.and supposing that the starting and the stopping possibilities on each node are all ones, the number of l -length common random walk sequences between the input graph G i and the anomalous substructure

Original: S m is:
Revised: S m is:

Original: [Equation]
Revised: [Equation]

Original: where 1 ∈ R n ′ n is a column vector full of 1, and E ∈ R n ′ × n is a matrix full of 1.
Revised: where 1 ∈ R n ′ n is a column vector full of 1, and E ∈ R n ′ × n is a matrix full of 1.

Original: The proof of
Revised: The proof of

Original: Eq.
Revised: Eq.

Original: 7 can be found in Appendix A.1.
Revised: 7 can be found in Appendix A.1.

Original: Random walk kernels with infinite random walk lengths will often meet the halting issue [42].
Revised: Random walk kernels with infinite random walk lengths will often meet the halting issue [42].

Original: Therefore, iGAD only considers the random walk length fromto L .
Revised: Therefore, iGAD only considers the random walk length fromto L .

Original: As the size of anomalous substructures ( n ′ ) is small, optimizing ( P m ) l is not necessary.
Revised: As the size of anomalous substructures ( n ′ ) is small, optimizing ( P m ) l is not necessary.

Original: A i is a sparse matrix, and we compute ( A i ) l by the sparse matrix multiplication in PyTorch.
Revised: A i is a sparse matrix, and we compute ( A i ) l by the sparse matrix multiplication in PyTorch.

Original: That is ( A si ) l
Revised: That is ( A si ) l

Original: = A si ( A si ... ( A si ( A i )) ... ) , where A si is the sparse version of A i .
Revised: = A si ( A si ... ( A si ( A i )) ... ) , where A si is the sparse version of A i .

Original: Afterward, all deep RWK values about G i and S m , m ∈ { 1 , .., M } will be assembled into a vector a G i .
Revised: Afterward, all deep RWK values about G i and S m , m = { 1 , .., M } will be assembled into a vector a G i .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: where a G i ∈ R ML .
Revised: where a G i ∈ R ML .

Original: The graph representation h kerG i based on deep RWK is:
Revised: The graph representation h kerG i based on deep RWK is:

Original: [Equation]
Revised: [Equation]

Original: The final graph representation h G i for G i ∈ G can be obtained by concatenating h ConvG i and h KerG i together.
Revised: The final graph representation h G i for G i ∈ G can be obtained by concatenating h ConvG i and h KerG i together.

Original: Based on h G i , the anomalous attribute-aware graph convolution and anomalous substructureaware deep RWK are welded into a graph neural network.
Revised: Based on h G i , the anomalous attribute-aware graph convolution and anomalous substructureaware deep RWK are welded into a graph neural network.

Original: 4.3 iGAD: PMI-based Loss Function
Revised: 4.3 iGAD: PMI-based Loss Function

Original: The number of normal graphs is much more than that of anomalous graphs.
Revised: The number of normal graphs is much more than that of anomalous graphs.

Original: Under the imbalance nature of the anomaly problem, normal graphs will dominate the machine learning model.
Revised: Under the imbalance nature of the anomaly problem, normal graphs will dominate the machine learning model.

Original: Targeting this issue, we propose a new loss function that can model the PMI between input graphs and their anomalous/normal properties.
Revised: Targeting this issue, we propose a new loss function that can model the PMI between input graphs and their anomalous/normal properties.

Original: Specifically, we use f ( G ; Θ) to denote the proposed model iGAD, where Θ represents model parameters.
Revised: Specifically, we use f ( G ; Θ) to denote the proposed model iGAD, where Θ represents model parameters.

Original: Modeling the PMI between the input graph G i and its anomalous/normal properties can be formulated as:
Revised: Modeling the PMI between the input graph G i and its anomalous/normal properties can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where the right part represents the PMI between G i and y i , P ( y i ) is the proportion of anomalous graphs or normal graphs in the training set T .
Revised: where the right part represents the PMI between G i and y i , P ( y i ) is the proportion of anomalous graphs or normal graphs in the training set T .

Original: Compared with most existing loss functions, especially the loss function that models the conditional distribution about y i given G i (i.e., f y i
Revised: Compared with most existing loss functions, especially the loss function that models the conditional distribution about y i given G i (i.e., f y i

Original: ( G i ; Θ) ∼
Revised: ( G i ; Θ) ∼

Original: P Θ ( y i | G i ) ), PMI enables iGAD to give priority to learning the information that reveals the essential correlation between graphs and their labels.
Revised: P Θ ( y i | G i ) ), PMI enables iGAD to give priority to learning the information that reveals the essential correlation between graphs and their labels.

Original: Eq.
Revised: Eq.

Original: 10 can be reformatted as:
Revised: 10 can be reformatted as:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: After softmax normalization, the distribution of y i given G i is formulated as:
Revised: After softmax normalization, the distribution of y i given G i is formulated as:

Original: [Equation]
Revised: [Equation]

Original: Incorporating P Θ ( y i | G i ) in to the framework of the cross-entropy loss function, we can obtain the
Revised: Incorporating P Θ ( y i | G i ) in to the framework of the cross-entropy loss function, we can obtain the

Original: PMI-based loss function L :
Revised: PMI-based loss function L :

Original: [Equation]
Revised: [Equation]

Original: Specifically, iGAD predict the graph G i :
Revised: Specifically, iGAD predict the graph G i :

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: testing stage
Revised: testing stage

Original: [Equation]
Revised: [Equation]

Original: The reason that we use argmax y i f ∗ y
Revised: The reason that we use argmax y i f ∗ y

Original: i
Revised: i

Original: ( G i ; Θ) to predict a graph in the testing stage can be found in
Revised: ( G i ; Θ) to predict a graph in the testing stage can be found in

Original: Appendix A.2.
Revised: Appendix A.2.

Original: There are other loss functions targeting the difficulties brought by the imbalanced distributions, such as the cost-sensitive loss function that assigns or learns different weights to the majority and the minority samples [29, 49, 12].
Revised: There are other loss functions targeting the difficulties brought by the imbalanced distributions, such as the cost-sensitive loss function that assigns or learns different weights to the majority and the minority samples [29, 49, 12].

Original: Compared with them, the PMI-based loss function does not introduce any new parameters that need to be carefully tuned.
Revised: Compared with them, the PMI-based loss function does not introduce any new parameters that need to be carefully tuned.

Original: It comes in the form of the most widely used cross-entropy loss function and gives prediction results that can realize the maximized PMI between inputs and prediction results.
Revised: It comes in the form of the most widely used cross-entropy loss function and gives prediction results that can realize the maximized PMI between inputs and prediction results.

Original: 4.4 Algorithm Description & Computational Complexity
Revised: 4.4 Algorithm Description & Computational Complexity

Original: The pseudocode of iGAD is included in Algorithm 1 in Appendix A.1.
Revised: The pseudocode of iGAD is included in Algorithm 1 in Appendix A.1.

Original: We divide graphs into different mini-batches to train iGAD [23].
Revised: We divide graphs into different mini-batches to train iGAD [23].

Original: But for illustrating how iGAD predicts a graph more clearly, the batch size is set as 1 in Algorithm 1.
Revised: But for illustrating how iGAD predicts a graph more clearly, the batch size is set as 1 in Algorithm 1.

Original: Given the training set, we initialize anomalous substructures and other parameters (Line 1 in Algorithm 1) and calculate the proportion of normal and anomalous graphs (Line 2).
Revised: Given the training set, we initialize anomalous substructures and other parameters (Line 1 in Algorithm 1) and calculate the proportion of normal and anomalous graphs (Line 2).

Original: For each graph, iGAD learns its graph representation (Lines 3 to 15).
Revised: For each graph, iGAD learns its graph representation (Lines 3 to 15).

Original: Based on the graph representation, a MLP equipped with the PMI-based loss function predicts the graph (Lines 16 and 17).
Revised: Based on the graph representation, a MLP equipped with the PMI-based loss function predicts the graph (Lines 16 and 17).

Original: At the back-propagation stage, anomalous substructures and other parameters will get updated.
Revised: At the back-propagation stage, anomalous substructures and other parameters will get updated.

Original: At the next epoch, deep RWK performs random walking on the input graphs and the updated anomalous substructures simultaneously.
Revised: At the next epoch, deep RWK performs random walking on the input graphs and the updated anomalous substructures simultaneously.

Original: Lemma 

Revised: Lemma

Original: The computational complexity of iGAD is O ( n + | E | ) , where n and | E | are the number of nodes and edges, respectively.
Revised: The computational complexity of iGAD is O ( n + | E | ) , where n and | E | are the number of nodes and edges, respectively.

Original: (The detailed analysis can be found in Appendix A.3).
Revised: (The detailed analysis can be found in Appendix A.3).

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct a series of experiments to study the performance of iGAD on the graphlevel anomaly detection task.
Revised: In this section, we conduct a series of experiments to study the performance of iGAD on the graphlevel anomaly detection task.

Original: Specifically, we investigate the following six questions:
Revised: Specifically, we investigate the following six questions:

Original: Q1.
Revised: Q1.

Original: How about the performance of iGAD on the graph-level anomaly detection task?
Revised: How about the performance of iGAD on the graph-level anomaly detection task?

Original: Q2.
Revised: Q2.

Original: Does iGAD benefit from the PMI-based loss function?
Revised: Does iGAD benefit from the PMI-based loss function?

Original: Q3.
Revised: Q3.

Original: How about the performance of the anomalous graph attribute-aware graph convolution in iGAD? Q4.
Revised: How about the performance of the anomalous graph attribute-aware graph convolution in iGAD? Q4.

Original: How about the performance of iGAD when it utilizes pre-defined anomalous substructures?
Revised: How about the performance of iGAD when it utilizes pre-defined anomalous substructures?

Original: Q5.
Revised: Q5.

Original: Can the anomalous substructure-aware deep RWK in iGAD learn the anomalous substructures?
Revised: Can the anomalous substructure-aware deep RWK in iGAD learn the anomalous substructures?

Original: Q6.
Revised: Q6.

Original: How does the maximum length of random walk sequences influence the performance of iGAD?
Revised: How does the maximum length of random walk sequences influence the performance of iGAD?

Original: The questions from Q2 to Q4 can be regarded as the ablation study on our three main design elements, they are (a).
Revised: The questions from Q2 to Q4 can be regarded as the ablation study on our three main design elements, they are (a).

Original: cross-entropy loss function versus the proposed PMI-based loss function; (b).
Revised: cross-entropy loss function versus the proposed PMI-based loss function; (b).

Original: general graph convolution versus the proposed graph convolution; (c).
Revised: general graph convolution versus the proposed graph convolution; (c).

Original: RWK versus the proposed deep RWK.
Revised: RWK versus the proposed deep RWK.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: 5.1 Experimental Setup
Revised: 5.1 Experimental Setup

Original: Datasets.
Revised: Datasets.

Original: SW-610, MOLT-4, PC-3, and MCF-7 are four real-world graph datasets.
Revised: SW-610, MOLT-4, PC-3, and MCF-7 are four real-world graph datasets.

Original: These datasets are collected from PubChem 2 , which records a tremendous amount of chemical compounds and their anti-cancer activity testing results (“active” or “inactive”) on different types of cancer cell lines.
Revised: These datasets are collected from PubChem 2 , which records a tremendous amount of chemical compounds and their anti-cancer activity testing results (“active” or “inactive”) on different types of cancer cell lines.

Original: We regard the chemical compounds with the outcome “active” as anomalous
Revised: We regard the chemical compounds with the outcome “active” as anomalous

Original: [Table]
Revised: [Table]

Original: and “inactive” as normal.
Revised: and “inactive” as normal.

Original: For example, the graph dataset SW-620 is about 40,532 chemical formulas and their anti-cancer activity testing results on the colon cancer cell line.
Revised: For example, the graph dataset SW-620 is about 40,532 chemical formulas and their anti-cancer activity testing results on the colon cancer cell line.

Original: Chemical compounds that exhibit antibody activity against colon cancer are labeled as anomalous graphs, otherwise normal graphs.
Revised: Chemical compounds that exhibit antibody activity against colon cancer are labeled as anomalous graphs, otherwise normal graphs.

Original: The statistical information about these four datasets is shown in Table 1.
Revised: The statistical information about these four datasets is shown in Table 1.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Baselines & Experimental Settings.
Revised: Baselines & Experimental Settings.

Original: To investigate
Revised: To investigate

Original: Q1, we compare iGAD with the following baseline models: (1) GCN[20]
Revised: Q1, we compare iGAD with the following baseline models: (1) GCN[20]

Original: + Readout function (abbreviated
Revised: + Readout function (abbreviated

Original: GCN) and GIN [53], these two algorithms employ spatial graph convolution to learn node representation and apply the readout function to node representations to obtain graph representations and classify graphs; (2)
Revised: GCN) and GIN [53], these two algorithms employ spatial graph convolution to learn node representation and apply the readout function to node representations to obtain graph representations and classify graphs; (2)

Original: DGCNN [58] and SOPool [51].
Revised: DGCNN [58] and SOPool [51].

Original: In these two models, spatial graph convolution and the graph pooling operation are employed to downsize graphs and complete the graph classification task; (3) RWGNN [32] employs random walk sequences to represent graphs and then classify graphs based on graph representations.
Revised: In these two models, spatial graph convolution and the graph pooling operation are employed to downsize graphs and complete the graph classification task; (3) RWGNN [32] employs random walk sequences to represent graphs and then classify graphs based on graph representations.

Original: All the above baselines are supervised and use the cross-entropy loss function; (4) The graph-level anomaly detection methods GLocalKD [27] and OCGTL [34].
Revised: All the above baselines are supervised and use the cross-entropy loss function; (4) The graph-level anomaly detection methods GLocalKD [27] and OCGTL [34].

Original: The former adopts random distillation to detect the locally- and globally-anomalous graphs and the latter combines the one-class classification and graph transformation learning to identify anomalous graphs.
Revised: The former adopts random distillation to detect the locally- and globally-anomalous graphs and the latter combines the one-class classification and graph transformation learning to identify anomalous graphs.

Original: These two algorithms only use labeled normal graphs as supervision information.
Revised: These two algorithms only use labeled normal graphs as supervision information.

Original: For the fairness of experiments and to answer Q2, we also compare iGAD with GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD.
Revised: For the fairness of experiments and to answer Q2, we also compare iGAD with GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD.

Original: These algorithms are the graph anomaly detection (GAD) variants of GCN, GIN, DGCNN, SOPool, and RWGNN.
Revised: These algorithms are the graph anomaly detection (GAD) variants of GCN, GIN, DGCNN, SOPool, and RWGNN.

Original: They replace the crossentropy loss in the original graph classification algorithms with the proposed PMI-based loss function.
Revised: They replace the crossentropy loss in the original graph classification algorithms with the proposed PMI-based loss function.

Original: To answer Q3, we consider a variant of iGAD —iGAD-1 that only utilizes the anomalous graph attribute-aware graph convolution to represent graphs and detect anomalous graphs.
Revised: To answer Q3, we consider a variant of iGAD —iGAD-1 that only utilizes the anomalous graph attribute-aware graph convolution to represent graphs and detect anomalous graphs.

Original: To answer
Revised: To answer

Original: Q4, we consider the other variant named iGAD-2 in which the random walking performs on the input graphs and the pre-defined anomalous graph substructures simultaneously.
Revised: Q4, we consider the other variant named iGAD-2 in which the random walking performs on the input graphs and the pre-defined anomalous graph substructures simultaneously.

Original: Specifically, 3-star,
Revised: Specifically, 3-star,

Original: [Figure]
Revised: [Figure]

Original: the setting M =5 and n ′ =8.
Revised: the setting M =5 and n ′ =8.

Original: The second row reports the performance of iGAD under the setting M =5 and n ′ =10.
Revised: The second row reports the performance of iGAD under the setting M =5 and n ′ =10.

Original: triangle, tailed triangle, and 4-cycle graphlets are pre-defined graph anomalous substructures (see
Revised: triangle, tailed triangle, and 4-cycle graphlets are pre-defined graph anomalous substructures (see

Original: Figure 4 in Appendix B for these substructures).
Revised: Figure 4 in Appendix B for these substructures).

Original: The parameter setting and the implementation of all the above models can be found in Appendix B.1.
Revised: The parameter setting and the implementation of all the above models can be found in Appendix B.1.

Original: We report the experimental results under 5-fold cross-validation.
Revised: We report the experimental results under 5-fold cross-validation.

Original: 5.2 Experimental Results
Revised: 5.2 Experimental Results

Original: [Figure]
Revised: [Figure]

Original: This section answers Q1 - Q6 by A1 - A6, respectively.
Revised: This section answers Q1 - Q6 by A1 - A6, respectively.

Original: A1.
Revised: A1.

Original: iGAD demonstrates effective performance in identifying graph-level anomalies.
Revised: iGAD demonstrates effective performance in identifying graph-level anomalies.

Original: The performance of iGAD exceeds all graph classification baselines, their corresponding
Revised: The performance of iGAD exceeds all graph classification baselines, their corresponding

Original: GAD variants, and the state-of-the-art graph-level anomaly detection methods.
Revised: GAD variants, and the state-of-the-art graph-level anomaly detection methods.

Original: Observed from Table 2: 1) Compared with GCN, GIN, DGCNN, SOPool, and RWGNN, iGAD achieves superior performance on all datasets in terms of all evaluation metrics; 2) iGAD also beats GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD in terms of all evaluation metrics.
Revised: Observed from Table 2: 1) Compared with GCN, GIN, DGCNN, SOPool, and RWGNN, iGAD achieves superior performance on all datasets in terms of all evaluation metrics; 2) iGAD also beats GCN-GAD, GINGAD, DGCNN-GAD, SOPool-GAD, and RWGNN-GAD in terms of all evaluation metrics.

Original: In addition, as shown in Table 3, iGAD outperforms two graph-level anomaly detection methods GLocalKD and OCGTL with obvious advantages in terms of AUC.
Revised: In addition, as shown in Table 3, iGAD outperforms two graph-level anomaly detection methods GLocalKD and OCGTL with obvious advantages in terms of AUC.

Original: A2.
Revised: A2.

Original: The PMI-based loss function can capture essential correlations between graphs and their anomalous/normal proprieties even if the data distribution is imbalanced.
Revised: The PMI-based loss function can capture essential correlations between graphs and their anomalous/normal proprieties even if the data distribution is imbalanced.

Original: As shown in Table 2, compared with the baseline models with the cross-entropy loss function (GCN, GIN, DGCNN,
Revised: As shown in Table 2, compared with the baseline models with the cross-entropy loss function (GCN, GIN, DGCNN,

Original: SOPOOL, and RWGNN), their GAD variants equipped with the newly proposed PMI-based loss function (GCN-GAD, GIN-GAD, DGCNN-GAD, SOPOOL-GAD, and RWGNN-GAD) witness a significant performance improvement regarding Recall and Recall(A) on all datasets.
Revised: SOPOOL, and RWGNN), their GAD variants equipped with the newly proposed PMI-based loss function (GCN-GAD, GIN-GAD, DGCNN-GAD, SOPOOL-GAD, and RWGNN-GAD) witness a significant performance improvement regarding Recall and Recall(A) on all datasets.

Original: For example, on the dataset SW-620, the Recall and Recall(A) value achieved by DGCNN are 56.02 ± 1.16 and 12.74 ± 2.
Revised: For example, on the dataset SW-620, the Recall and Recall(A) value achieved by DGCNN are 56.02 ± 1.16 and 12.74 ± 2.

Original: respectively.
Revised: respectively.

Original: In contrast, 72.65 ± 0.63 and 67.84 ± 1.78 are reported by DGCNN-GAD.
Revised: In contrast, 72.65 ± 0.63 and 67.84 ± 1.78 are reported by DGCNN-GAD.

Original: A3.
Revised: A3.

Original: The experiment results indicate that the anomalous attribute-aware graph convolution achieves competitive performance in encoding anomalous attributes.
Revised: The experiment results indicate that the anomalous attribute-aware graph convolution achieves competitive performance in encoding anomalous attributes.

Original: As shown in Table 2, iGAD-1 significantly outperforms all baseline algorithms on four graph datasets in terms of AUC , Recall , and F-score .
Revised: As shown in Table 2, iGAD-1 significantly outperforms all baseline algorithms on four graph datasets in terms of AUC , Recall , and F-score .

Original: Concerning Recall(A) , on the dataset MCF-7, iGAD-1 has a little performance degradation compared with DGCNN-GAD.
Revised: Concerning Recall(A) , on the dataset MCF-7, iGAD-1 has a little performance degradation compared with DGCNN-GAD.

Original: Specifically, DGCNN-GAD achieves 69.37 ± 3.05 on MCF-7, while iGAD-1 gets 67.67 ± 2.54 .
Revised: Specifically, DGCNN-GAD achieves 69.37 ± 3.05 on MCF-7, while iGAD-1 gets 67.67 ± 2.54 .

Original: But iGAD-1 beats DGCNN-GAD on other datasets in terms of Recall(A) .
Revised: But iGAD-1 beats DGCNN-GAD on other datasets in terms of Recall(A) .

Original: A4.
Revised: A4.

Original: iGAD is also effective when it is equipped with pre-defined anomalous substructures.
Revised: iGAD is also effective when it is equipped with pre-defined anomalous substructures.

Original: As shown in Table 2, iGAD-2 demonstrates obvious advantages against all baseline models.
Revised: As shown in Table 2, iGAD-2 demonstrates obvious advantages against all baseline models.

Original: Moreover, iGAD-2 also beats iGAD-1 concerning all metrics on MOLT-4, PC-3, and MCF-7.
Revised: Moreover, iGAD-2 also beats iGAD-1 concerning all metrics on MOLT-4, PC-3, and MCF-7.

Original: iGAD-2 does not outperform than iGAD on all datasets concerning AUC , Recall(A) , and F-score .
Revised: iGAD-2 does not outperform than iGAD on all datasets concerning AUC , Recall(A) , and F-score .

Original: But, on PC-3 and
Revised: But, on PC-3 and

Original: MCF-7, iGAD-2 has a slight lead over iGAD regarding Recall .
Revised: MCF-7, iGAD-2 has a slight lead over iGAD regarding Recall .

Original: The experiment results demonstrate that, in most cases, the performance of iGAD equipped with a deep RWK is better than iGAD-2 using pre-defined anomalous substructures and algorithms that do not consider anomalous substructures.
Revised: The experiment results demonstrate that, in most cases, the performance of iGAD equipped with a deep RWK is better than iGAD-2 using pre-defined anomalous substructures and algorithms that do not consider anomalous substructures.

Original: Based on the above analysis, we can conclude that comparing the random walk sequences between the input graph and pre-defined substructure can improve graph-level anomaly detection to a certain extent, but the algorithm that can automatically learn the anomalous substructures is better.
Revised: Based on the above analysis, we can conclude that comparing the random walk sequences between the input graph and pre-defined substructure can improve graph-level anomaly detection to a certain extent, but the algorithm that can automatically learn the anomalous substructures is better.

Original: A5.
Revised: A5.

Original: The anomalous substructures learned by iGAD are demonstrated in Figure 2.
Revised: The anomalous substructures learned by iGAD are demonstrated in Figure 2.

Original: We can observe that anomalous graph substructures on four datasets are composed of the triangle, the tailed triangle, and the 4-cycle graphlets.
Revised: We can observe that anomalous graph substructures on four datasets are composed of the triangle, the tailed triangle, and the 4-cycle graphlets.

Original: It explains why iGAD-2 can achieve competitive results compared with iGAD (as shown in Table 2 and A4 ).
Revised: It explains why iGAD-2 can achieve competitive results compared with iGAD (as shown in Table 2 and A4 ).

Original: We can draw a preliminary conclusion that the graph substructures having the triangle, the tailed triangle, and the 4-cycle graphlets are important for distinguishing anomalous graphs from normal graphs on these four graph datasets.
Revised: We can draw a preliminary conclusion that the graph substructures having the triangle, the tailed triangle, and the 4-cycle graphlets are important for distinguishing anomalous graphs from normal graphs on these four graph datasets.

Original: A6.
Revised: A6.

Original: The performance of iGAD under different maximum random walk lengths L is shown in Figure 3.
Revised: The performance of iGAD under different maximum random walk lengths L is shown in Figure 3.

Original: Under the setting that iGAD with 5 anomalous substructures and each of substructures has 8 nodes, on these four graph datasets SW-620, MOLT-4, PC-3, and MCF-7, iGAD can achieve the highest
Revised: Under the setting that iGAD with 5 anomalous substructures and each of substructures has 8 nodes, on these four graph datasets SW-620, MOLT-4, PC-3, and MCF-7, iGAD can achieve the highest

Original: F-score value when the maximum random walk length ranges from 4 to 8.
Revised: F-score value when the maximum random walk length ranges from 4 to 8.

Original: In addition, when each anomalous substructure contains 10 nodes, iGAD can reach the best F-score score at the maximum random walk length from 5 to 9.
Revised: In addition, when each anomalous substructure contains 10 nodes, iGAD can reach the best F-score score at the maximum random walk length from 5 to 9.

Original: The results verify that appropriately increasing the maximum length of random walking can improve the performance of iGAD since more much longer random walk sequences can capture more graph structure information.
Revised: The results verify that appropriately increasing the maximum length of random walking can improve the performance of iGAD since more much longer random walk sequences can capture more graph structure information.

Original: Overall, the above experiments demonstrate the effectiveness of iGAD in graph-level anomaly detection.
Revised: Overall, the above experiments demonstrate the effectiveness of iGAD in graph-level anomaly detection.

Original: We put more parameter analysis experiments in Appendix B.2.
Revised: We put more parameter analysis experiments in Appendix B.2.

Original: Finally, we discuss the limitations of the proposed graph-level anomaly detection model iGAD.
Revised: Finally, we discuss the limitations of the proposed graph-level anomaly detection model iGAD.

Original: In practice, the nodes and/or substructures at a particular position in a graph can  be the reason for the graph’s anomalous properties.
Revised: In practice, the nodes and/or substructures at a particular position in a graph can also be the reason for the graph’s anomalous properties.

Original: For example, an Alcohol compound can be obtained by replacing the hydrogen atom at the end of the Alkane compound with a hydroxy group (i.e., -OH).
Revised: For example, an Alcohol compound can be obtained by replacing the hydrogen atom at the end of the Alkane compound with a hydroxy group (i.e., -OH).

Original: But, iGAD cannot capture the position information of nodes and substructures in graphs.
Revised: But, iGAD cannot capture the position information of nodes and substructures in graphs.

Original: In addition, iGAD is a supervised algorithm.
Revised: In addition, iGAD is a supervised algorithm.

Original: Its performance on identifying the graph-level anomalies that are completely different from the training anomalies may not be superior.
Revised: It may not be able to achieve superior performance in identifying the graph-level anomalies that do not have any common characteristics with the training anomalies.

Original: Our future work is to explore graph-level anomaly detection algorithms that can identify more extensive anomalous graphs.
Revised: In the future, we will propose specific solutions to this problem.

Original:  Conclusion
Revised:  Conclusion

Original: This paper focused on graph-level anomaly detection which is a research topic of great practical value but less explored.
Revised: This paper focused on graph-level anomaly detection which is a research topic of great practical value but less explored.

Original: We highlighted the challenges brought by the imbalanced distribution and various anomaly notions in graph attributes and substructures.
Revised: We highlighted the challenges brought by the imbalanced distribution and various anomaly notions in graph attributes and substructures.

Original: In addition, the significant differences between detecting anomalous graphs in a graph dataset and identifying anomalous nodes in a single graph were also discussed.
Revised: In addition, the significant differences between detecting anomalous graphs in a graph dataset and identifying anomalous nodes in a single graph were also discussed.

Original: The proposed method iGAD regarded graph-level anomaly detection as a special case of graph classification.
Revised: The proposed method iGAD regarded graph-level anomaly detection as a special case of graph classification.

Original: In iGAD, the anomalous attribute-aware graph convolution and anomalous substructure-aware deep RWK embed graph-level anomaly information into graph representations.
Revised: In iGAD, the anomalous attribute-aware graph convolution and anomalous substructure-aware deep RWK embed graph-level anomaly information into graph representations.

Original: A MLP with the PMI-based loss function is employed to distinguish anomalous samples from normal samples based on graph representations.
Revised: A MLP with the PMI-based loss function is employed to distinguish anomalous samples from normal samples based on graph representations.

Original: We did extensive experiments on four graph datasets, and iGAD achieved superior performance on the graph-level anomaly detection task.
Revised: We did extensive experiments on four graph datasets, and iGAD achieved superior performance on the graph-level anomaly detection task.

Original: Acknowledge
Revised: Acknowledge

Original: We greatly appreciate theinsightful and constructive reviews from reviewers.
Revised: We greatly appreciate the constructive and insightful comments from reviewers.

Original: This work is funded by the Australian Research Council Discovery Early Career Researcher Award (ARC DECRA)Project
Revised: This work is funded by the Australian Research Council Discovery Early Career Researcher Award (ARC DECRA)

Original:  (No. DE200100964).
Revised: Project (No. DE200100964) and the NSFC (No. 61872360).

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] see section 5.
Revised: [Yes] see section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] 3.
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether
Revised: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re us- ing/curating?
Revised: and how consent was obtained from people whose data you’re us- ing/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]