Learning-Based Link Anomaly Detection in Continuous-Time Dynamic Graphs
Authors: Tim Postuvan
Claas Grohnfeldt
Michele Russo
Giulio Lovisotto
Venue: Accepted by TMLR ()

Abstract: Anomaly detection in continuous-time dynamic graphs is an emerging field yet under-explored in the context of learning algorithms. In this paper, we pioneer structured analyses of link-level anomalies and graph representation learning for identifying categorically anomalous graph links. First, we introduce a fine-grained taxonomy for edge-level anomalies leveraging structural, temporal, and contextual graph properties. Based on these properties, we introduce a method for generating and injecting typed anomalies into graphs. Next, we introduce a novel method to generate continuous-time dynamic graphs featuring consistencies across either or combinations of time, structure, and context. To enable temporal graph learning methods to detect specific types of anomalous links rather than the bare existence of a link, we extend the generic link prediction setting by: (1) conditioning link existence on contextual edge attributes; and (2) refining the training regime to accommodate diverse perturbations in the negative edge sampler. Comprehensive benchmarks on synthetic and real-world datasets -- featuring synthetic and labeled organic anomalies and employing six state-of-the-art link prediction methods -- validate our taxonomy and generation processes for anomalies and benign graphs, as well as our approach to adapting methods for anomaly detection. Our results reveal that different learning methods excel in capturing different aspects of graph normality and detecting different types of anomalies. We conclude with a comprehensive list of findings highlighting opportunities for future research.

Published in Transactions on Machine Learning Research (10/2024)
Learning-Based Link Anomaly Detection in Continuous- Time Dynamic Graphs
Tim Poštuvan" EPFL
twm.postuwon@eeepf.cc
Claas Grohnfeldt Huawei Technologies
cloas. clossggrdnfflldi@aaaaascccccc
Michele Russo
michele.nusoohhuawc.ccom
Huawei Technologies
Giulio Lovisotto* Huawei Technologies
giutioboisotto@@@aallcoom
Reviewed on OpenReview: https: // openreview. net/ forum? id- 8tmVCizVEu
Abstract
Anomaly detection continuous-time dynamic graphs an emerging field yet under- explored in the context of learning algorithms. In this paper, we pioneer structured anal- yses of link-level anomalies and graph representation learning for identifying categorically anomalous graph links. First, introduce fine-graing grained taxonomy for edge-level anomalies leveraging structural, temporal, and contextual graph properties. Based on these properties, we introduce method for generating and injecting typed anomalies into graphs. Next, we introduce novel method to generale continuous-lime dynamic gruphs featuring consisten- cies across either or combinations of time, structure, and context. To enable temporal graph learning methods defect specific types of anomalous links rather than the bare existence of a link, extend the generic link prediction setting by: (1) conditioning link existence contextual edge attributes; and (2) refining the training regime to accommodate diverse perturbations in the negative edge sampler. Comprehensive benchmarks synthetic and real-world datascts - featuring synthetic and labeled organic anomalies and employing six state-of-the-arl link prediction methods validate our laxonomy and generation processes for anomalies and benign graphs, as well as our approach to adapting methods for anomaly detection. Our results reveal that different learning methods excel in capturing different aspects of graph normality and detecting different types of anomalies. We conclude with comprehensive list of findings highlighting opportunities for future research. The code is available at httts://githie com wttp://iaiittttaa/oooooaaaaaaaaa
1 Introduction
Anomaly detection in graphs the task of identifying abnormal behavior, with applications in spam detection financial fraud identification (Dou et al. Although extensive literature exists on
Yee & Akoglu 2015), sensor fault detection (Gaddam et al. 2020
2020), and cybersecurity (Zipperle et al. 2022 Reha et al.
2023
anomaly detection in static graphs (Peng et al. 2018 Ding et al. 2019 Fan et al.] 2020 [aadyopadhyay
et al. 2019 Yuan et al.) 2021 Xu et al. 2022)
most real-world networks evolve over time, requiring a focus on dynamic graphs. The most general dynamic graph formulation is the continuous- time dynamic graph (CTDG), where interactions occur at irregular times and carry information as edge altribules.
"Work done while at Huawei Tochnologies. 1 "Correspondigg author.

---

Published in Transactions on Machine Learning Research (10/2024)
Despite the prevalence of dynamic networks, most baariing-based graph anomaly detection methods operate
static or discrete-time representations (Goyal et al. 2018 Zheng et al.
2019
Cai et al.
2021b, p, limiting their
practical applicability (Rossi et al.] 2020 Souza et al. 2022). Other research addresses streaming anomaly
detection in dynamic graphs using iketch-basee algorithms [Eswaran et al.,
2018 Bhatia et al. 2020 2022)
which excel at delecting strong cumulative anomalies (e.g,; bursts of edges)
rather than individ ua link-level
anomalies, trading granular detection capabilities for efficiency. [Reha et al.
(0223) study lcarning-base based link
anomaly detection in CTDGs but only considers one method, one dataset and generic link anomalies.
While there is limited work anomaly detection in CTDGs, there also exists no categorization of edge- level anomalics in these graphs. Paired with the lack of public CTDG datascts with labeled anomalies, this makes it difficult to analyze learning methods for abilities to detect typed anomalies. On the other hand, the desire of practitioners for detection methods tailored to the specific anomalies of interest in their usecases (Han et al.
2022) highlights the need for ways to characterize and generate anomalies of specific types in CTDGs. For these reasons, a number of works have proposed catogorizations of anomalies in various
domains (Zhao al. 2019: Steinbuss & Böhm 2021 Han et al.] 2022), including anomaly classifications for static graph (Liu et al. 2022 and contimuous- time discrete graphs [Ma et al., (021). While these works build their taxonomics for anomalics on structural and contextual graph properties, they lack the explicit temporal perspective necessary for modeling the contimuous- time dynamics of CTDGs.
Addressing these limitations, we present the first structured analysis on Baaning-basee graph representation methods in CTDGs applied to anomalous edge detection. We introduce a fine-grained taxonomy of edge anomaly types in CTDGs and methods to generate and inject these anomalies into any dataset. In doing so, we generalize and extend the work on static graphs {Liu al. 2022 Ma et al. 2021 to include time.
To validate our anomaly calegorization and anomaly generation procedure, we propose process for gen- erating synthetic dynamic graphs that are consistent in time, structure, and context. To address the task of detecting typed anomalous in CTDG, we build upon the link prediction literature and adopt graph- representation learning methods for typed link anomaly detection. We show that the direct application of link prediction results in sub-optimal performance in anomaly detection. We identify and mitigate two shortcomings of the general link prediction approach: To make models context-aware, we devise a modi- fication of existing temporal graph methods to prodict the existence of link conditioned on context and time. Further, we improve the training regime using lyped edge-level perturbations that improve models prediction capabilities w.r.t. the individual dimensions of time, context, and structure.
We benchmark six existing temporal graph mothods, spanning from non-parametric memorization, sequential models, and random walk models to graph convolution methods, on the link anomaly detection task. We assess their detection performance on eight datasets for benchmarking, including synthetic and real graphs while using synthetically generated and organic anomalies. We find that the considerations made on synthetic graphs designed with structural, contextual and temporal consistencies generalize to real-world datasets, validating the soundness of our taxonomy. On the other hand, experiments reveal some real graphs do not exhibit strong consistencies along some of these dimensions. We also find that different lcarning methods are more suited to capturing different aspects of graph normality, hence detecting different types of anomalies, highlighting the benefit of tailoring the choice of model to the dataset. Finally, we present a list of future directions and rescarch opportunities in the field of anomaly detection for CTDG.
In this paper, we make the following contributions:
We present the first comprehensive analysis of link anomaly detection for continuous-tine dynamic graphs.
We introduce a novel taxonomy for edge-level anomalies in continuous-time dynamic graphs along with corresponding procedures for generating and injecting typed anomalies ($3.1 and $3.2). We also propose synthetic graph generation process, where individual anomaly types are distinguishable ($3.3).
We propose two steps to adapt graph-reppreentltion learning methods to the more subtle task of link anomaly detection 4.2 and
We assess state-of-tho-art temporal graph learning methods for detecting typed synthetic and organic anomalies in synthetic and real graphs ($5) highlight future research opportunities ($6)
2

---

Published in Transactions on Machine Learning Research (10/2024)
2 Problem Definition
We address the problems of categorizing and detecting anomalous edges in dynamic graphs. continuous- time dynamic graph (CTDG) extends the concept a static graph by incorporating timestamps and at- tributes (also referred lo as message) on edges:
dynamic graph is sequence of ( (u2, U2 t2, m2) .......} with 0 0 t1 S tt ..., where u, E V denote the source node and destination node of the jth edge at timestamp t and V is the set of all nodes. Each interaction (u, v,t,m, m)) has edge message features m E RD, where D the number of altribules. For ann-attributed graphs, edge features are often set zero vectors, i.e., m 0.
Definition 1 (Continuuous-lime) dynamic graph) A continuuus- non-decreann non-decreasing chronological interactions G = {[u1, 21,4 m,),
We define the task of link anomaly detection in CTDGs as follows:
Definition 2 (Link anomaly detection in CTDGs) Given a continuousttimg dynamic graph 9, the gool of link anomaly detection learn a function: f V x R TR. - R that maps a node pair (u, v), timestamp and edge message m a real-valued anomaly score for the edge e (,,,
3 Anomaly Taxonomy and Synthesis
Background. The categorization and synthesis of realistic anomalies are fundamental for designing, ana- lyzing, and benchmarking anomaly detection methods across domains (Zhao et al. 2019 Steinbuss & Böhm, 2021 Han et al. 2022 Liu et al., 2022; Ma et al. 2021), especially when labeled data is scarce costly (Han ct al. 2022). For static graphs, Liu ct al. (2022) introduce a taxonomy and generation method for struc- lural and contertual anomalies. Ma et al. (2021) brielly discuss another categorization of anomalies in static graphs and re-use it for discrete-ting time dynamic graphs, i.e., graphs represented as timestamped snapshots, while disregarding the effect of time. To date, there exists no taxonomy for anomalics in CTDGs.
Challenges in CTDG taxonomies. We identify three challenges when expanding related work from static and discrete-time dynamic graphs to the CTDG domain:
1. With the added temporal dimension, edge and node histories co-define "normal" graph behavior, which constantly evolues over time (Liu et al. 2020). Such evolving normality is not captured by definitions of anomalies in static graphs (Liu et al. 2022).
2. Recent categorizations ions such as the following ones from Liu et al. 2022 and Ma et al. 2021 either lack completeness or are too general for our aim:
structural anomalies, defined anomalously densely connected nodes in static graphs Liu et al. 2022 disregard structural anomalies relative to the graphs' history as well as the aspect of communities conteztual anomalies, defined for static graphs nodes with attributes different to those of their neighbors (Liu el al. 2022), are limited to local anomalies;
The definition of anomalous edges for static and discreto-time dynamic graphs (i.e., graph snap- shots) (aa et al. 2021) regards only structural properties explicitly, but not edge-specific histories attributes. Moreover, this broad definition does not inform about the nature of the edge anomaly.
3. There exist neither real-world nor synthetic CTDGs suitable to validate a fine-grained taxonomy for anomalies in CTDGs.
Here, propose fine-grain taxonomy for edge-level anomalies in CTDGs by re-interpreting contezt and structure, and incorporating time explicitly. We then describe procedures to generate into graphs edges that are anomalous w.r.t. one or multiple of these dimensions. Lastly, we introduce a graph generation process for CTDGs that assures reliable and largely independent consistencies along the structural, temporal, and contextual dimensions. This process is used in $ to: validate our proposed taxonomy and procedures for synthesizing and injecting edge-level anomalies, and comparatively assess the capabilities of link prediction approaches in ing the five anomaly types defined by our taxonomy.
3

---

Published in Transactions on Machine Learning Research (10/2024)
The proposed taxonomy of syuthetic anomaly types with their acronyms. We exemplify each anomaly type using scenario of package distribution network, where nodes are either warchouses customers, and edge between a warehouse and customer marks the delivery of a package.
Table 1:
Anomaly type
Acronym
Example
Benign
The ordered package is delivered the customer time.
Temporal
T
The ordered package is delivered to the customer with delay. package with wrong content is delivered to the customer on time.
Contextual
C
T-C
package with wrong content is delivered to the customer with delay.
Structural-comtee contextual
S-C
The ordered package is delivered to a wrong address on time.
Tenpporalstrruutrralonntttttutt
T-S-C
A random package is delivered to randon address at randon time.
Continnous time
Normal Edges
Anomalous Behavior Temsporal- cuntextual
Temporsk structural- contectioal
Struetural- contextual
Dynamic Graph (CTDG)
Temporal
Contestual
C
E T-C
S.C
T-S-C
Time
fn2
0
Temporal
Activity
Figure 1: A CTDG with the proposed synthetic anomaly types, visualized in snapshots for clarity. Times- tamps to to tx-5 represent the normal graph behavior, characterized by the properties discussed in $3.1 (1) two node communities define the structure, i.c., (2) there are two types of message conterts
tied to these communities, (3) the timing of edges follows the Temporal Activity at the bottom, i.e.,
AL indicate the node is active at this time. From timestamp tx-4 to t, we highlight and categorize edge anomalous behavior. (T): Edge appears at an unexpected time. (C): Edge message differs from the expected message between the two nodes. (T-C): Edge appears at an unexpected time and its message differs from the expected message between the two nodes. (S-C): Edge connects two nodes that expected to be active this time but do not usually connect with each other, its message is out-of-distribution (m). (T-S-C): Edge connects two nodes that do not usually connect with each other, the nodes are not expected to be aclive at this time and its message is
Icons _ JL mark temporal anomalies.
3.1 A Taxonomy for Anomaly Types in CTDGs
Table shows our proposed taxonomy for edge-level anomalics in contimuous-time dynamic graphs. The individual anomaly lypes are described in the following.
Context. For CTDGs, refer to edge attributes as context. The predictability of altributes in many dynamic graphs motivates the individual consideration of corresponding anomalies. Contextual normal- ity may be learned from different distributions of edges: the node pair's historical edges, their neighbors, communities, and all available historical edges (Liu et al. 2022
Definition 3 (Conteztual Anomaly (c)) In CTDG, a contertual anomaly an edge whose attributes (message) signéficantly differ from their expected values.
4

---

Published in Transactions on Machine Learning Research (10/2024)
Structure. Graph structure, sometimes referred to the graph's topology (Jin et al. 2022 describes the interconnectivity of nodes through edges. Common structures include the multipartite structure, where no two endpoints have the same class (color), and the community structure where nodes belonging to the same community are more likely to connect. For example, in collaboration or citation networks, nodes (researchers) are more likely to connect. (co-author a paper) with other researchers working in the same research area (from the same community) (aa et al. 2021]. Other real-world dynamic graphs such as proximity networks (Poursafaei et al., 2022), political networks (Poursafaei et al., 2022). ; and interaction
networks (Panzarasa et al. 2009 Poursafaei al. 2022 incorporate similar structural properties.
Definition 4 (Structural Anomaly (s)) In CTDG, structural anomaly is edge connecting two nodes that are structurally unerpected to connect.
Time. In dynamicc graphs, nodes may interconnect following specific temporul patterns. These patterns create dynamics where nodes tend to communicate predictable times, e.g., due to periodicity, other kinds of regularity, periods of high activity. For example, traffic in industrial arcas occurs most likely during rush hours, friends exchange private messages most likely outside of working hours (Panzarasa el. al. 2009
and corporate e-mails may be sent most likely between 9-10 am (Shetty & Adibi 2004). Other networks with temporal consistencies include flights [Huang et al. 2023a), transactions (Huang et al., 2023a), and interactions (Kumar et al. 2019 Nadiri & Takes 2022).
Definition 5 (Temporal Anomaly (r)) In a CTDG, a temporal anomaly is an edgge that appears al an unexpected time.
Combinations of anomaly types. The three dimensions of CTDGs, i.e., structure, context, and time, are mutually complementary yet not completely orthogonal, meaning not all combinations of anomalies can be considered independently. While an edge can be anomalous w.r.t. time, context, or both time and context, we argue that in CTDGs, structural anomalies imply contextual anomalies: When an edge connects two nodes that never connected before and are otherwise structurally unlikely to connect, this implies a contextual anomaly, as there exists no meaningful distribution of edges messages to learn contextual normality from. Overall, the following five out of seven possible combinations of anomalies are considered in our taxonomy: C, T-C, S-C, T-S-C. Table 1 and Figure Securreexxee eexemplifies and illustrates these categorical anomalies, respectively.
3.2
Generation of Anomalies
Here, present efficient strategies to synthesize - for given CTDG anomalous edges that fall into the five categories of anomalies defined above. Anomalies of categories T, C, T-C, and S-C are generated by: (1) sampling a benign reference edge, and (2) applying category-specific perturbations to create abnormal properties; Only T-S-C anomalies do not necessitate benign sampling. In compliance with Def. 3
5
we use
randomness as a proxy for abnormality. For a given observed edge, we assume that: (i) randomizing the destination node creates a structural anomaly, (ii) randomizing the attributes creates a contextual anomaly, and (iii) randomizing the edge's timestamp creates lemporal anomaly. Note that random sampling is a highly efficient and common approach to generating anomalies yet it might produce non-anomalous samples at negligible error rate. We refer to (u, t, m) as the sampled reference benign edge, with its source destination timestamp t, and attributes m. We inject anomalies as follows:
(T): create è = (u, F, m), where f is a randomly selected timestamp within the time span of the data.
(c): create é (u, u. f, m) where in is the result of m. argmaxm, d(m, my) with mj E {mo,-... mx), where d is pair-wise distance metric between multi-dimensional vectors, and mo..., m are K randomly sampled edge messages from the entire message set.
(T-C): create é (u, v, I, m) where i and mn use the above procedures for and anomalies.
(s-c): sample a sccond benign reference edge 62 = (r, m'm) within a temporal window of size W (i.e., one of W temporally closest edges). Create an edge (u, q, t, m) where g is the destination of eg and v is selected with the procedure for injection of contextual anomalies above.
5

---

Published in Transactions on Machine Learning Research (10/2024)
(T-S-C): create è = (û, 6, m)) where û, 0 are randomly sampled nodes, f is randomly sampled times- tamp, and in is randomly sampled edge message.
3.3 Synthetic Dynamic Graph Generation
We introduce a synthetic CTDG generation process with clear temporal, structural, and contextual con- sistencies. The generation starts by randomly creating a static graph from node set V and number of temporal edges E. Temporal edges are then sampled from existing static edges, ensuring consistency in context, structure, and time. Algorithm 1 outlines the complete process. To enforce consistencies in the generated CTDG, we do the following:
Structural consistency: We generate the initial static graph with the Stochastic Block Model (Abbe 2018), using . predefined communities (Line 1 ensuring structure as randomly sampled nodes likely belong different communities.
Contextual consistency: We uniformly assign nodes to fixed number of classes C, and create expected odge message attributes "i between class pairs (Lines 2 3 Expected messages are orthogonal for perfect separation, so randomly sampled edge attributes likely differ from these expectations.
Temporal consistency: We uniformly sample time windows for each edge's activity (Line For the same edge, we ensure uniform time intervals between its multiple occurrences (Line 10 Randomly sampled timestamps are unlikely to match these predefined windows.
Generation Process. The graph is generated iteratively: for each static edge (u, v) in the STATICGRAPH output, number of temporal edge occurencce oe is sampled from Poisson distribution. Then, an active time window of activity [*5, 45 + Ar is sampled based time-span of the temporal graph tıax- Then each edge occurrence's timestamp is evenly spaced within the active time window (within some Gaussian noise), and its message is sampled from a Multivariate Normal distribution based on the nodes u and e respective classes, See further details and default parameters in App. A.1 The graph consistencies allow us use random sampling (along structure, time, or context) generate anomalies that fall under eractly one of the categories in $3.1 We use Alg. I to generate CTDG for experiments in $5
Lognormal distribution and the maximum
4
Link Anomaly Detection
4.1
Link Prediction vs. Link Anomaly Detection
A prrominent learning task in CTDGs is link prediction (Nguyen al. 2018 Kumar et al., 2019 Rossi al. Yu et al., 2023 Huang et al. 2023a). While such task resembles that of link anomaly detection in Def. there are significant differences that outline here.
2020
Link prediction setting. When learning parametric methods for link prediction, it is common to assume that the observed edges in 9 are legitimate, i.c., they represent normal graph behavior. In this setting, models can be trained with self-supervised loss function based on maximizing the probability of occurrence of observed edges
(i.e., positive edges), while minimizing the probability of occurrence of unseen edges ē G (i.e,, negative edges). To create the negative set G, one uses random negative sampling: take all sources of the positive edges in a batch and re-wire them to randomly sampled destinations. Doing so, exactly one negative edge ē is sampled for each positive edge e = (u, v, f, m) (Huang et al. 2023a Poursafaci et al. 2022). where the negative one only differs in the destination:
ē == (u, 2, 1, m) with E~ u(V)
(1)
The edge pairs e, ē are used to solve the problem optimizing So with a binary loss such as cross-entropy:
(c) + ( = log fo( v, t)) + log(1 fo( f)))
(2)
The optimization still uses edge attributes of past neighbors u, through Eq.

---

Published in Transactions on Machine Learning Research (10/2024)
Algorithm 1: Synthetic Dynamic Graph Generation
Input Nodeset V, n. temporal edges E, n. of communities M, avg. n. of edge occurrences Aoce: graph time span baxx, mean Pts and st.d. Ots for sampling edge time spans, st.d. of timestamps perturbations olt), of node classes C, edge message dimension D, st.d. for messages perturbations o(m).
Output: Synthetic temporal graph G.
& + STATICGRAPH((M)) Auce MM)
2 t (Cu, (Gu,:Cuz..... . where Cu,, u(1,2......C [}
H j.k EXPECTEDMESSAGE D), Vj E fl....,cp C)2
[m)
@
for e (u, E & do
Oe ~ Pois()ace)
At" Lognooraal(pss 0ts)
16 ~ (0, luxx - A1
for i 1 to 0g do
of + 1o + (i (i Ar" ef, where ef NN o(1)
10
m NW (m)) (na(m)) where j Cu, k Co
11
12
g 9 (u,
13
end
end
15
return G
To benefit from the graph structure, the network uses a neighbor sampler which stores node neighbors at each time step. For a node u, its past neighbors are used aggregate information at time f:
No = [v](u, v,t ,m) G and t < t}.
(3)
After training, the trained model fo is able to provide a likelihood of existence for any arbitrary link.
Link anomaly detection setting. Certain aspects differ in the task of link anomaly detection (Def. 2 First, (1) while random negative sampling of Eq. models structural anomalics well ($3.1) by sampling random destinations, it disregards anomalies in the temporal and contextual dimension: L and are never perturbed. This limits the model's ability to capture non non-structural abnormalities. Secondly, (2) the message attributes m are informative, which models should now use in the prediction. Unlike in link prediction, the goal is not to predict the existence an arbitrary unobserved link, but to estimate the likelihood of occurrence of an observed link. Lastly, (3) the assumption that the observed edges are legitimate (i.e., evaluation time as a portion of edges in the validation and test set of can be
normal) is invalidated anomalous but their normality is unknown. The graph can be scen as the union of normal and anomalous edges g U g While the problem can still be formulated negative sampling, this leads to anomalous information entering the neighbor sampler A of Eq. 3 N² {ul(u, v, " m) E gb U g and t- < t}. While the challenges stemming from (3) can not be addressed directly (see G we address problems (1) and (2) in the following.
self self-supervised learning problem with
4.2 Conditioning on Context
The optimization problem defined for link prediction (Eq. 2 is sub-optimal in link anomaly detection since the information carried in the edge attributes (m) can be used for edge scoring.
Problem.
Solution. We reformulate the optimization problem with the inclusion of the edge attributes as follows:
(() + ( - loggooo(((() fo((u, & m)) + log(1 - So((u, 2, m))).
(4)
Note that during training the assumption still holds.

---

Published in Transactions on Machine Learning Research (10/2024)
In practice, to incorporate the contextual information of an edge e (u, v,t, m), project the edge at- tributes with a linear layer and concatenate them to the node embeddings before feeding the concatenation into the link anomaly detection head. Given the temporal node embeddings at time t, h² h the predic- tion head receives h WW m as the input, W E Dxd2, d2 < D is learned with the rest of the model parameters. Note that we propose straightforward yet versatile and model-agnostic method for incor- porating contextual information. Alternatively, contextual attributes could be directly integrated into the cresage-passing process of each model, enabling the generation of context-aware temporal embeddings.
4.3 Improved Training Regime
Problem. Because the random negative sampling described in Eq. only modifies edges' destinations, the negative edge set only contains structural anomalics but not temporal or contextual ones.
Solution. We propose novel negative sampler to be used in training. The sampler applies one of the following three perturbations to positive edge (u, v,t,m, to create negative edge
(u, 2, m)
with ā ~ u(V) or
E (u, v I, m)
with [ U1o mMAx] or with m 21 (m,. m1....!
(u, v, m)
These perturbations are universal and do not rely on the proposed injection strategies, In the implementation, the latter sampler samples messages uniformly among the messages in the batch for efficiency reasons.
5 Experiments
We design our experimental setup to answer the following questions: RQ1 ($5.2) How effective are our methods for adapting link prediction methods to the link anomaly detection lask? RQ2 ($5.3) How valid is our taxonomy and generation process of typed anomalies when applied to synthetic and real-world CTDGs and how discernible are these anomalies? RQ3 ($5.3 $5.4p How do capabilitics of different link prediction methods differ at detecting typed anomalies in synthetic and real datasets? RQ4 (555) How do hods perform at detecting organic anomalies in real datasets?
5.1 Experimental Setup
We provide here information about the experimental setup. Complementary details can be found in the appendix: datasets (App. A.1 model descriptions (App. A.2 hyper-parameter configurations (App. A.3
runtimes (App. and extensive prediction results (App.
Datasets. We run experiments on eight different datasets. The first one is TEMPSYNTHGRAPI CTDG we generated using Alg. with 10,000 nodes (see $3.3). We use five benign real-world datasets: WIKIPEDIA, REDDIT, MOOC, ENRON, and UCT from the work of Pouussfaaei et al. (2022). Finally, we experiment
on two CTDGs from the cybersecurity domain with labeled organic anomalies: LANL (Kent) 2016 and DARPA-THEIA (Reha el al. 2023
Methods. We compare seven representative CTDGs learning methods that are based on graph convolutions, memory networks, random walks, and sequential models: TGN Rossi et al. 2020), CAWN (Wagg et al. 2020), TCL (Wang et al., 2021), GraphMixer (Cong et al. 2022). DyGFormer (Yu et al. 2023), and two variants EdgeBank (Poursafaei et al. 2022), namely EdgeBa and EdgeBankm..
Experimental setting. We train models for the task of link anomaly detection of Def. 2 Following our considerations from - we use binary cross-entropy loss as in Eq. 4 and the negative sampler in Eq. 5 For each method, the same MLP prediction head takes the edge (including time and edge attributes) as input and predicts the probability p(e) of the link's existence. The link anomaly score is then computed as 1-p(e). We use the Area Under the Receiver Operating Characteristic curve (AUC) as the main evaluation metric, but also report Average Precision (AP) and Recall@k metrics in App. C Tn datasets with synthetic anomalies.
8

---

Published in Transactions on Machine Learning Research (10/2024)
chronologically divide each dataset into training (70%), validation (15%), and testing (15%). For the two datasets with organic anomalies LANL, and DARPA-THEIA, we use all edges that occurred before the first anomalous edge for training, while the remaining data is split so that validation and testing sets contain an equal number of anomalous edges. For testing, we select models based on their AUC on the validation set.
Synthetic anomaly injection. In TEMPSYNTHGRAPH, WIKIPEDIA, REDDIT, UCI, ENRON and MOOC. we inject 5% of synthetic anomalies in the validation and test splits with the procedure described in $3.2
Reference benign edges are sampled within the same data split and with probabilities proportional to U.5 with Oe representing the number of occurrences of edge e in that split. For C, T-C, S-C, and T-S-C anomalies, following Huang et al. (2023b), we adopt cosine similarity as the comparison metric (d 332 with K=10 randomly sampled edge messages; For S-C anomalies, W=20.
Implementation details. We optimize parametric models with Adam (Kiggma & Ba 2015 for 50 epochs, with an early stopping patience of 20. We perform a grid search for learning rate in the range [10-3 10-9) Batch size is always 200 except in EdgeBank and EdgeBank where we set it to to prevent inaccurate look-ups within the same batch. Models have ~100k trainable parameters, with hyper-parameterss adapted from Yu al. (2023). We repeat each experimental run with three weight initializations, with the exception of DARPA-THELL TEIA where only use initialization due compute limitation. When introducing synthetic anomalies, also average results across three randomly seeded sets of injected anomalies.
5.2 Towards Link Anomaly Detection
Result Conditioning context. To demonstrate the efficacy of the contextual conditioning (see $4.2) we show that no link prediction method trained with Eq. 2 can detect contextual anomalics without contex- tual conditioning, while the revised optimization of Eq enables them to do so. We use TEMPSYNTHGRAPI and compute the AUC performance of anomaly detection for contextual anomalies only when training TGN with Eq. [ vs Eq. [
we report the result in Tab. 2 without conditioning on contextual information.
The table shows that contextual anomalies go undetected
Result Improved training regime. To demonstrate the efßi cacy of the improved negative sam- pler (sce $4.3). we show that the re- vised sampler of Eq. 5 encourages the model to learn enhanced rep- resentation of structure, time, and context compared to a random neg- ative sampler (Eq. D. We use TEMMSYNTHGRAPH and compute the AUC performance of anomaly detection on various anomaly types for models trained with the negative
Table 2: AUC of TGN trained under with and without conditioning context and improved training regime TEMPSYNTHGRAPI,
Conditioning on context
Improved training
Anomaly w/o (Eq. 2
w/ (Eq.
w/o (Eq.
w/ (Eq.
89.29
90.04.1.94
c
52.05
97.2110.22
66.49
97.24
T-C
89.51+1.00
97. 97.61
S-C
97.67
97.83+
T-S-C
97.36
98.2440.22
The table shows that the improved negative sampler boosts detection performance on all types of anomalies.
sampler of Eq. [ vs. Eq. 5 we report the result in Tab. 2
5.3 Synthetic Graphs, Synthetic Anomalies
Tab. 3 reports the link anomaly detection AUC on TEMPSYNTHGRAPH with synthetic anomalies. Our key findings are follows.
Anomalies on multiple dimensions are easier to detect. Anomalies that deviate from the norm in multiple respects are more easily identifiable. For example, Tab. 3 shows consistently that T-C anomalies are casier to detect compared to and T anomalies. Similarly, S-C anomalies are less detectable than S-C anomalies. All results in Tab. support this finding within the confidence intervals, corroborating the
9

---

Published in Transactions on Machine Learning Research (10/2024)
Table 3: AUC for all models on TEmpSynthGraph with synthetic anomalies of the five categories intro-
duced in 3.1
Bold and underlined values mark first- and second-best performance across anomaly types.
Anomaly type
EdgeBankss
EdgeBankww
TGN
CAWN
TCL
GraphMixer DyGFormer
T
60.8610.18
60.8710.17
90.0411.1
87.91
94.3210.90
96.21
85.18
49.02
49.52+0.06
97.24
90.92+*.93
92.49
98.32
98.5240.16
T-C
6o.02
61.02+0.11
97.61
94.86+2.34
96.80
98.89
97.8344.22
s-C
98.43
48.4340.04
97.8310.60
99.03
97.75-0.36
98.51
99.15.00.10
T-S-C
99.03
99.03
98.2440.22
99.03
98. 98.38
99.0220.01
99.03
validity of the graph generation process of $3.3 in TEMPSYNTHGRAPH, edges that are abnormal in multiple dimensions become more anomalous with respect to the normal behavior.
GraphMixer excels in modeling temporal dynamics. GraphMixer demonstrates superior performance in detecting and T-C anomalies on TEMPSYNTHGRAPH. As temporal anomalics differ from their benign counterparts solely through tinestamp variations (see $3.2), models require a robust representation of time to better detect them. We note that GraphMixer employs fixed time-encoding function [Cong et al. 2022), differently than other methods which use learnable ones (Rossi et al. 2020 Wang et al. 2021 2020)
In alignment with findings from Cong et al. (2022) fixed time-encoding functions improve performance by enhancing the stability of training and leading to better modeling of time.
DyGFormer and CAWN are superior modeling structure. DyGFormer and CAWN are well suited for identifying structural anomalies, i.e., S-C and T-S-C, achieving AUCs>99. CAWN is the only method that considers two-hop neighbors to generate node embeddings, suggesting that broader receptive fields increase methods' ability to model structure. On the her hand, DyGFormer considers only one-hop neighbors, but it features a neighbor co-occurrence encoding scheme (Yu et al. 2023) capturing correlations between node pairs and yielding weak structural information about two-hop paths between the nodes of interest. This additional information contributes DyGFormer's superior performance in detecting structural anomalies.
5.4 Real Graphs, Synthetic Anomalies
We report link anomaly detection AUC scores for models on WIKIPEDIA, REDDIT, MOOC, ENRON, and UCT with synthetic anomalies in Tab. 4 Below are the key findings from the table.
Findings TempSynthGraph generalize to real graphs with synthetic anomalies. As in Tab. 3 no single temporal graph method universally outperforms others across all datasets and anomaly types: even on a single anomaly type, no method consistently ranks first in Tab. g across all datasets. Similarly, GraphMixer demonstrates superior performance in detecting temporal anomalies also on real graphs, while DyGFormer and CAWN exhibit superior performance in identifying structural anomalies.
Real graphs may not. exhibit. strong consistencies. On TRMPSYNTHGRAPHH best-performing models consistently achieve AUCs>96 (Tab. 3) on each anomaly type, demonstrating the ability of models to accu- rately identify anomalies in graphs with clear consistencies. Conversely, in real graphs, the best-performing models often exhibit significantly lower detection rates (Tab. 4. indicating a lack of consistencies along the three dimensions. Detection of anomalies in REDDIT is notably poor, with CAWN achieving the highest result at AUC=81.12. This implies that REDDIT lacks regular temporal patterns, i.e., Reddit users tend to post at irregular times. In WIKIPEDIA, no method detects anomalies satisfactorily, with the best value being at AUC=84.03, highlighting lack of consistent contextual patterns. Note that edge attributes in WIKIPEDIA are a low-dimensional representation of the text contained in the page edit, suggesting that these edits carry limited contextual information. In UCI and ENRON, detection of structural anomalies is challenging, with the top two-per woo-performing models yielding AUC=86.45 and 18 respectively. This suggests, counter-intuitively, that in the communication networks participants may tend to communicate without following strict structural rules. MOOC seems be the most consistent network across all three aspects, with all best-performing methods scoring with AUCs>96. These findings highlight the irregular
10

---

Published in Transactions on Machine Learning Research (10/2024)
Table 4: AUC for models on WIKIPEDIA, REDDIT, MOOC, ENRON and UCI with typed synthetic anomalies 3.2 Bold and underlined values mark first- and sccond-best performing models across rows. We do not report results ENRON and UCI for C and T-C anomalies as these datasets lack edge attributes.
Anomaly type
Datasets
EdgeBankk
TGN
CAWN
TCL
GraphMixer
DyGFormer
WIKIPEDIA
58. 20.40
60.2910.27
85.1810.47
10.31
84.54-20.41
84.5510.20
86.63 86.6310.31
REDDIT
55.31 58.9610.1e
76.93
81.1210.27
77.98 10.43
76.6210.30
76.1611.68
MOOC
50.73=0.18
50.53
96.184o.15
95.9540..58
95. 95.71
95. 95.13+0.40
94.42+0.50
ENRON
53.66 13.6620.11
54.85
84.1912.37
89.6711.0
28.0612.15
91.92.0.. 10.48
74.8015.40
UCI
51.97.1..0
51.38 1..17
85.7211.06
83.0811.46
83.08
87.73 10.78
80.95
Avg. Rank
6.60
2.60
2.20
3.20
2.80
4.20
WIKIPEDIA
56.48 18.08
58.07 10.14
19.67
84.03
19.10
79.09 19.49
84.22
10..66
REDDIT
61.23 10.42
71.8610.24
03.81
13.33
95.3211..8
93.9410.27
94.81 10.45
MOOC
40.51
44.7940.65
99.36++.0o
98.40+0.45
20.20
98.7440.89
99.57+0.00
Avg. Rank
6.67
6.33
3.00
4.00
2.67
4.00
1.33
WIkIfediA
58.26
60.73
85, 85.0220.45
86.29 46.29
84. 84.4222054
84.85
86,24+0.45
REDDRT
54.82
58.46.0.26
93.39+0.40
91.9642.61
94.9440.27
94.87+0..2
93.75+0.90
MOOC
50.37 10.30
50.22 10.54
99.58 10.14
19.34
99.49 10.28
99.72 L0.18
99.64
Avg. Rank
6.67
6.33
3.33
3.67
3.33
2.33
2.33
WIKIPEDIA
23.29=0.24
91.0820.28
86.0420.51
93.59+0.65
73.45
70. 12..56
94.88...22
REDDET
94.78.1..
90.85.1.12
90.61
94.79 1.14
93.50 93.50.0.22
10.22
90.98 10.46
96.50 1.0.0.0000
MOOC
20.03
69.49 29.4
87.41+0.a9
96.65.0.24
81.50
79.5442.31
80.5
ENRON
85.03=0.79
85. 85.15
75.37+4.05
06.96
63.47:1.45
63.7741.54
88. 18:118:000
UCI
82.23 10.13
81.06
63.83 63.8312.22
86.45 1.42
60.16 12.44
1.884
84.66 1.50
Avg. Rank
3.80
4.80
4.80
1.60
5.40
6.00
1.60
WIKIPEMNA
94.61 20.00
92.33=20.00
90.40
97.37+0.23
92.41 1.44
97.0444..40
98.892..10
REDDIT
96.53 10.00
92.44.10.00
97. 42.2...
98.08 10.44
96. 96.42.0.22 00.28
98.08 10.12
97. 97.951...s
MOOC
80.37
76.6220.02
99.8 99.85
99.73.20.6
99.77-20.0
99.96 20.02
99.82+.004
T-8-C
ENRON
97.41=0.07
97.39
74.90
20.46
93.06=20.09
95.90++0.42
91.89 2:0 20.90
UCI
85.65
84.07 10.00
89.62
94.51 1..42
92. 12.21..
93.73
96.11
Avg. Rank
4.40
5.80
5.00
3.00
4.80
2.20
2.80
nature real graphs; supporting the use of synthetic graph generation methods, such as Alg, 1 to assess and compare the capabílities of models in controlled scenarios. We present comparison of the statistical properties of synthetic and real-world graphs in Appendix D
5.5 Real Graphs, Organic Anomalies
Tab. 5 reports the link anomaly detection AUC on LANL and DARPA-THELA with organic anomalies. Here we describe the main findings.
Organic anomalies in LANL and Darpa-Theia resemble synthetic structural anomalies. DyG- Former and CAWN exhibit the best performance on real datasets with organic anomalies, matching their good performance in detecting S-C and T-S-C synthetic anomalies in $5.4 In contrast, EdgeBanky, weakly outperforms a random baseline: achieves AUC<67 in LANI and AUC-53 in DARPA-THEIE This dif- ference implies that while methods apt at detecting structural anomalies work well, memorization- based approaches such as EdgeBank are not well-suited to detect them. In particular, the extremely high propor- tion of unseen test edges in DARPA-THER and LANL requires learning from non-structural aspects of the CTDG (Reha al. 2023 Poursafaei et al. 2022
Significant disparities in model performances. There are considerable gaps between the top- hop-performing and lowest-perforning lcarning-base models in LANL and DARPA-THERS This indicates that careful seloc- tion of an appropriate method is crucial when dealing with real-world graphs containing organic anomalies: the best learning-basee method outperforms the worst one by 6.8 and 21.39 AUC points, respectively.
11

---

Published in Transactions on Machine Learning Research (10/2024)
Table 5: AUC for models on LANL and DARPA-THEEE with organic anomalies. Bold and underlined values mark first- and second-best performing models across rows.
Datasets
EdgeBankw
TGN
CAWN
TCL
GraphMixer
DyGFormer
LANL
66.84
17.54
94.3410.06
89.31
88. 18.74
94.07
DARPA- -HEIA
52.72
77.18
84.18
90.30
69.03
90.42
Avg. Rank
6.50
4.50
2.00
2.50
4.50
1.50
6 Future Directions
We discuss future directions for CTDG link anomaly detection in this section.
Opportunity 1: New CTDG datasets with organic anomalies. Previous research highlights the importance of datasets with labeled organic anomalies for method comparison (Liu et al. 2022 Han et al. 2022). While synthetic anomalics are useful, they cannot fully substitute organic oncs. In CTDG, only LANL (Kent) 2016) and DARPA (Reha et. al. 2023) with organic anomalies exist, both from the cybersecu- rity domain. The limited and narrow testbed hinders comprehensive method comparison. Thus, developing datasets across diverse domains is essential for robust method evaluation and exploring innovative approaches.
Opportunity 2: Explainability of methods.
The ability to explain anomalies is crucial for practitioners (Panjei al. 2022 especially in safety-critical
sectors like healthcare (Ukil et al. 2016), financial services (Ahmed et al. 2016 and self-driving car manu-
facturing (Bogdoll et al. 2022). While efforts to clucidate temporal graph methods exist (He ct al. 2022 Vu
& Thai 2022 Longa et al. 2023 only recently have explainable methods for CTDGs been introd uced (Xia
et al. 2022). ) Explainability enhances our understanding of methods and their predictions, providing deeper
insights into organic anomalies. Comparing patterns between synthetic and organic anomalies allows drawing parallels, which improves method understanding and highlights distinctive features of organic anomalies.
Opportunity 3: Reducing the gap between organic and synthetic anomalies. In $5.5 we observe that certain synthetic anomalies resemble organic ones, such as structural anomalies LANL and DARPA- THEIA. Specific anomaly detection methods excel at detecting particular types ($5.3). These findings support using synthetic anomalics as proxics for model selection, reducing reliance on organic anomalics. This can be achieved by using domain knowledge about prevalent anomaly types or by automating pipelines map organic anomalies to matching synthetic types. This approach allows selecting detection methods based synthetic anomaly validation sets, strcamlining experinentation.
Opportunity 4: Expansion of taxonomy in CTDG beyond the edge-level anomalies.
Anomalies in static graphs are traditionally studied at various levels: edges, nodes, sub-graphs, and entire graphs (Ma et al. 2021 A promising future research direction involves extending analysis and taxonomy of anomalics in CTDG beyond the edge level.
7 Conclusions
We presented a pioneering analysis of link anomaly detection in continuous-lime dynamic graphs, addressing a gap the current literature. Our comprehensive taxonomy edge anomaly types and corresponding generation strategics, coupled with a synthetic graph generation process, enables better evaluation and understanding of anomaly detection in CTDG. Our experimental findings, together with a discussion on future research opportunities, enable further research in learning anomaly detection methods for CTDGs.
12

---

Published in Transactions on Machine Learning Research (10/2024)
References
Emmanuel Abbe. Community detection and stochastic block models: recent developments. Machine Learning Research, 18(177):1-8,, 2018.
Journal of
Mohiuddin Ahmed, Abdun Naser Mahmood, and Md Ratiqui Islam. A survey of anomaly detection tech- niques in financial domain. Future Genemation Computer Systems, 55:278-2888 2016.
Sambaran Bandyopadhyay, Lokesh, and Narasimha Murty. Outlier aware network embedding for attributed networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 12-19, 2019.
Siddharth Bhatia, Bryan Hooi, Minji Yoon, Kijung Shin, and Christos Faloutsos. Midas: Microcluster- based detector of anomalies in edge streams. In Proceedings of the AAAT conference on artificial intelligence, volume 34, pp. 3242-3249, 2020.
Siddharth Bhatia, Rui Liu, Bryan Hooi, Minji Yoon, Kijung Shin, and Christos Faloutsos. Real-time anomaly detection in edge streams. ACM Transactions on Knowledge Discovery from Data (TKDD), 16(4): 2022.
Daniel Bogdoll, Maximilian Nitsche, and J Marius Zöllner. Anomaly detection in autonomous driving: A survey. In Proceedings of the IEEE/CVE conference on computer vision and pattern. recogràtion, pp. 4488-4490 2022.
Lei Cai, Zhengzhang Chen, Chen Luo, Jiaping Gui, Jingchao Ni, Ding Li, and Haifeng Chen. Structural temporal graph neural networks for anomaly detection in dynamic graphs. In Proceedings of the 30th ACM international conference on Information & Knowledge Managemend, pp. 3747-3756, 2021.
Kyunghyun Cho, Bart Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop Syntar, Semantics and Structure in Statistical Translation, pp. 103 111, 2014.
Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do really need complicated model architectures for temporal networks? In The Eleventh International Conference on Learving Represenlations, 2022.
DARPA. Transparent computing engagement 3 data release. E3. md 2020.
https:///gitubb.com/aaaa
Crampparett-Computinng/ wttp://ttinnnoooo/mmtteeeeeeeeee
Kaize Ding, Jundong Li, Rohit Bhanushali, and Huan Liu. Deep anomaly detection on attributed networks. Tn Proceedings of the 2019 STAM M International Conference on Data Mining, pp. 594 602. STAM, 2019.
Yingtong Dou, Zhiwei Liu, Li Sun, Yutong Deng, Hao Peng, and Philip S Yu. Enhancing graph neural network-based fraud detectors against camouflaged frandsters. Proceedings of the 29th ACM interna- tional conference on information knowledge management, pp. 315 324, 2020.
Dhivya Eswaran, Christos Faloutsos, Sudipto Guha, and Nina Mishra. Spotlight: Detecting anomalies in streaming graphs. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1378 1386, 2018.
Haoyi Fan, Fengbin Zhang, and Zuoyong Li. Anomalydae: Dual autoencoder for anomaly detection attributed networks. In ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 5685 5689. IEEE, 2020.
Anuroop Gaddam, Tim Wilkin, Maia Angelova, and Jyotheesh Gaddam. Detecting sensor faults, anomalies and outliers in the internet of things: A survey on the challenges and solutions. Electronics, 9(3:5111 2020.
Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. Dyngem: Deep embedding method for dynamic graphs. Xio preprint arXiv 1805 11273, 2018.
13

---

Published in Transactions on Machine Learning Research (10/2024)
Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. Adbench: Anomaly de- tection benchmark. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 32142 32159. Curran Associates, Inc., 2022. URL https::/prooceddingg neurips heurips.cc/paper files//pperr0022/iiee 1:9975516622267787722226--88-
and Benchharrss pdff
Wenchong He, Minh N Vu, Zhe Jiang, and My T Thai. An explainer for temporal graph neural networks. In GLÖBECOM 2022 IEEE Global Communications Conference, pp. 6384 6389. IEEE, 2022.
Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Matthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovec, Michael M Bronstein, Guillaume Rabusseau, and Reihaneh Rabbany. Temporal graph bench- mark for machine learning temporal graphs. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023a.
Yihong Huang, Liping Wang, Fan Zhang, and Xuemin Lin. Unsupervised graph outlier detection: Prob- lem revisit, new insight, and superior method. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pp. 2565 2578, IEEE, 2023b.
Ming Jin, Yuan-Fang Li, and Shirui Pan. Neural temporal walks: Motif-aware representation learning on continuous-time dynamic graphs. Advances in Neural Information Processing Systems, 35:19844-19886, 2022.
Alexander D. Kent. Cyber security data sources for dynamic networkk research, chapter Chapter pp. 37-65. 2016. doi: 9781786340757 _0002
0002. URL http://pppp.......oooooooaaaaaaa
Isaiah J King and H Howie Huang. Euler: Detecting network lateral movement via scalable temporal link prediction. ACM Transactions on Privacy and Security, 2023.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1269-1278, 2019.
Fanzhen Liu, Shan Xue, Jia Wu, Chuan Zhou, Wenbin Hu, Cocile Paris, Surya Nepal, Jian Yang, and Philip S Yu. Deep learning for community detection: Progress, challenges and opportunities. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 4981-4987. International Joint Conferences on Artificial Intelligence Organization, 7 2020. doi: 10. 11//966//jaai20933
URL https: org/10. /0/ 24963/ijcai. 20200/92322333 Survey track.
Kay Luu, Yingtong Dou, Yue Zhao, Xueying Ding, Xiyang Hu, Ruitong Zhang, Kaize Ding, Canyu Chen, Hao Peng, Kai Shu, al. Bond: Benchmarking unsupervised outlier node detection on static attributed graphs. Advances in Newral Information Processing Systems, 35:27021 27035, 2022.
Antonio Longa, Veronica Lachi, Gabriele Santin, Monica Bianchini, Bruno Lepri, Pietro Lio, franco scarselli, and Andrea Passerini. Graph neural networks for temporal graphs: State of the art, open challenges, and opportunitics. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https:
Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Chuan Zhou, Quan Sheng, Hui Xiong, and Leman Akoglu. A comprehensive survey on graph anomaly detection with deep learning. IEEE Transactions on Knowledge and Data Engineering, 2021.
Amirhossein Nadiri and Frank W Takes. A largo-scale temporal analysis of user lifespan durability on the reddit social media platform. In Companion Proceedings of the Web Conference 2022, pp. 677-685, 2022.
14

---

Published in Transactions on Machine Learning Research (10/2024)
Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-time dynamic network embeddings. In Componion proceedings of the the web conference 2018, pp. 969 976, 2018.
Egawati Panjei, Le Gruenwald, Elcazar Leal, Christopher Nguyen, and Shejuti Silvia. survey on outlier explanations. The VLDB Journal, (5):977 1008, 2022.
Pictro Panzarasa, Tore Opsahl, and Kathleen M Carley. Patterns and dynamics of users' behavior and interaction: Network analysis of online community. Journal of the American Society for Information Science and Technology, 60(5):911 932, 2009.
Zhen Peng, Minnan Luo, Jundong Li, Huan Liu, Qinghua Zheng, et al. Anomalous: A joint modcling approach for anomaly detection on attributed networks. In IJCAI, pp. 3513 3513-3519, 2018.
James W Pennebaker, Martha E Francis, and Roger J Booth. Linguistic inquiry and word count: Liwe 2001.
Mahway: Lawrence Erlbaum Associates, 71(2001):2001, 2001.
Farimah Poursafaei, Shenyang Huang, Kellin Peirine, and Reihaneh Rabbany. Towards better evaluation for dynamic link prediction. volume 35, pp. 32928 32941, 2022.
Jakub Reha, Giulio Lovisotto, Michele Russo, Alessio Gravina, and Claas Grohnfeldt. Anomaly detection in continuous-time temporal provenance graphs. In Temporal Graph Learning Workshop @ NewrIPS 2023, 2023. URL
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. arXiv preprinl X0x0 10637, 2020.
Jitesh Shetty and Jafar Adibi. The enron email dataset database schema and brief statistical report. Infor- mation sciences institute technical report, University of Southern California, 4(1):120-125, 2004.
Amauri Souza, Diego Mesquita, Samuel Kaski, and Vikas Garg. Provably expressive temporal graph net- works. Advances in Neural Information Processing Systems, 35:32257-32269, 2022.
Georg Steinbuss and Klemens Böhm. Benchmarking unsupervised outlier detection with realistic synthetic data. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(4):1-20, 2021.
Arijit Ukil, Soma Bandyoapdhyay, Chetanya Puri, and Arpan Pal. Tot healthcare analytics: The importance of anomaly detection. In 2016 IEEE 30th international conference on advanced information networking and applications (AINA), pp. 994 997. IEEE, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Minh N Vu and My Thai. On the limit of explaining black-box temporal graph noural networks. arXiv preprinl ar Xix: 22. 00952, 2022.
Lu Wang, Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, and Hongxia Yang. Tel: Trasso dynamic graph modelling via contrastive learning. arXiv preprint ar Xrx:: 2021.
Yanbang Wang, Yen-Yu Yu Chang, Yunyu Liu, Jure Leskovee, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In International Conference on Learning Representations, 2020.
Wenwen Xia, Mincai Lai, Caihua Shan, Yao Zhang, Xinnan Dai, Xiang Li, and Dongsheng Li. Explaining temporal graph models through an napliaer-aviggaoo framework. In The Eleventh International Confer- ence on Learning Representations, 2022.
15

---

Published in Transactions on Machine Learning Research (10/2024)
Zhiming Xu, Xiao Huang, Yue Zhao, Yushun Dong, and Jundong Li. Contrastive attributed network anomaly detection with data augmentation. In Pacific- Asia Conference on Knowledge Discovery and Dala Mining, pp. 444 457. Springer, 2022.
Junting Ye and Leman Akoglu. Discovering opinion spammer groups by network footprints. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Por- tugal, September 7-11, 2015, Proceedings, Part I 15, pp. 267-282. Springer, 2015.
Le Yu, Leilei Sun, Bowen Du, and Weifeng Lv. Towards better dynamic graph learning: New architecture and unified library. In Thirty-seventh Conference Neural Information Processing Systems, 2023. URL
https://openreeiew http/viwwwweeeeeeemee net/ formm?orrmm/m (ddruummdddxddiiiHbbb.....
Xu Yuan, Na Zhou, Shuo Yu, Huafei Huang. Zhikui Chen, and Feng Xia. Higher-order structure based anomaly detection on attributed networks. In 2021 IEEE International Conference on Big Data (Big Data), pp. 2691-2700. IEEE, 2021.
Yue Zhao, Zain Nasrullah, and Zheng Li. Pyod: A python toolbox for scalable outlier detection. Journal of Machine Learning Research, 20(96)11-7, 2019. URL http://jmlr........////// http:///mrrrrr/r/pppss////////// html
Li Zheng, Zhenpeng Li, Jian Li, Zhao Li, and Jun Gao. Addgraph: Anomaly detection in dynamic graph using attention-based temporal gen. In IJCAI, volume 3, pp. 2019.
Michael Zipperle, Florian Gottwalt, Elizabeth Chang, and Tharam Dillon. Provenance-based intrusion detection systems: A survey. ACM Computing Surveys, 55()):1-36, 2022.
A Detailed Experimental Settings
We report here details of the datasets, methods and model configurations used in
A.1 Datasets
In our experiments, we use the proposed synthetic graph ($3.3) five datasets collected by Poursafaei et al. the Los Alamos National Lab (LANL) dataset (Kent) 2016 and the DARPA-THEIA dataset (Reha
2022
et al.
2023
We report overview of each dataset's statistics in Tableb and describe them here:
TEMPSYNTHGRAPH comprises N = 10, 000 nodes divided into M 10 communitics of equal size. The underlying static graph is generated using a stochastic block model (Abbe 2018). featuring six times more static edges within communities than between them. The temporal graph consists of roughly = 1, 000, 000 temporal odges, where each unique edge (i.c., an edge between a given pair of nodes) occurs X = 50 times on average. The graph spans fMAx = 10 steps. The parameters for sampling edge time spans, l4s and Tts, are configured make edge spans, on average, 1% of the entire graph time span, with a standard deviation of 0.5%. The timestamps of temporal edges are perturbed with g(t) equaling 5% of the average inter-event time of the edge: år". Nodes are assigned to one of C' 5 types. The edge o. message dimension is D 100. The function EXPECTEDMESSAGEL/ k, D) of Alg, returns the mean edge messages 2 where the 1th element is computed as follows:
j.k (1) -(1 if () - < (C)-
(1
(mm)
otherwise.
creating orthogonal mean edge messages, as these are constructed by concatenating blocks of zeros to a single block of ones, that each class pair has ones in different. location of the vector. In practice,
16

---

Published in Transactions on Machine Learning Research (10/2024)
classes pairs' edge message would look follows:
(mm) =
M1,1
M1,2 = 0,100.0,,1111111111.0000000
Pe.e = |0,0,0,0,0,0,0,000 1,1,111)
the length of consecutive block of Is (or 0s) C.. In sampling edge messages, the mean edge messages are perturbed with Gaussian noise with standard deviation ((m) 0.05, Note that orthogonality between mean edge messages not a general requirement but we use it here to guarantce small overlaps between randomly perturbed edge messages belonging classes, with the goal of isolating contextual anomalies. Less strict implementations of EXPECTED can be used in principle.
WIKIPEDIA (Kumar al. 2019 bipartite interaction graph encompassing edits made to Wikipedia pages within one-month timeframe. Users and pages represented nodes, while links signify editing interactions along with corresponding timestamps. Each link is accompanied by 172-dimensional Linguistic Inquiry and Word Count feature (Pennebaker et al. 2001 that is derived from the content of the edit.
REDDIT (Kumar el al. 2019) is bipartite network that documents user posts under various subreddits over the course of one month. Users and subreddits serve as nodes, and the connections between them represent timestamped posting requests. Each link is accompanied by a 172-dimensional Linguistic Inquiry and Word Count feature (Pennebaker al.) 2001). derived from the content of the respective post.
MOOC (Kumar et al., 2019: bipartite interaction network, consisting of actions done by students in MOOC online course. Students and course units (e.g., videos and problem sets) are represented nodes, while a link within the network signifies student's 's interaction with particular content. unit and is associated with 4-dimensional feature.
ENRON (Shetty& & Adibi 2004) contains email correspondences exchanged among ENRON energy company employees over three-year period. The dataset has edge features.
UCI (Panzarasa et al. 2009) is an online communication network among students of the University of California at Irvine. Nodes represent university students, and links are messages posted by students. The links have no edge features.
Los Alamos National Lab (LANL) dataset Kent 2016 comprises authentication events occurring over 58 consecutive days within the corporate internal computer network of the Los Alamos National Laboratory. Computers are denoted as nodes, and the links between them represent authentication events. Events associated with the red team, which exhibit bad behavior, are labeled as malicious links - organic anomalies in $ The datasct lacks odge features. We pre-process the LANL dataset according to the methodology outlined in King & Huang (2023), but without combining edges into discrete graph snapshots. Due to the substantial size of the dataset (1.6 billion edges), evenly subsample the benign edges to obtain approximately 1,000,000 total edges.
DARPA-THEIA [Reha et al. 2023 is system-level level data provenance graphs, constructed from the audit logs collection made by DARPA Engagement 3 (DARPA 2020). The datasct contains both benign and malicious activities, where the latter are carried out by a red team and represent organic anomalies. Nodes represent files, processes, and sockets, while links correspond to Limux system calls with timestamps. Each link is associated with a 47-dimensional edge feature, describing the system call and the involved nodes.
A.2 Temporal Graph Learning Models
We select the following representative temporal graph learning models to evaluate them on link anomaly detection task, we also refer the reader to Huang et al. (2023a) for extended descriptions of these methods, which this section borrows from:
17

---

Published in Transactions on Machine Learning Research (10/2024)
Table 6: Statistics of the datasets. All datasets have a time granularity of Unix timestamps.
Datasets
Domain
# #Nodes
#Edges
#Edge features
Bipartite
Duration
# #Unique Steps
TEMPSYNTHGRAPH
Synthetic
10,000
1,002,325
100
False
10" steps
997,256
WIKIPEDIA
Social
9,227
157,474
172
True
month
152,757
REDDIT
Social
10,984
672,447
172
True
month
588,915
MOOC
Interaction
7,144
411,749
4
True
17 months
345,600
ENRON
Social
184
125,235
False
3 years
22,632
UCI
Social
1,899
50,835
False
days
58,911
LANL (ours)
Cybersecurity
13,081
1,030,147
False
58 days
1,027,178
DARPA-TREIA
Cybersecurity
1,043,224
8,416,187
47
False
247 hours
1,766,816
TGN (Rossi et al. 2020 is a general framework for learning on continuous-time temporal graphs. keeps a dynamic memory for each node and updates this memory when the node observed in an event. This is realized by memory module, message function, a message aggregator, and a memory updater, To obtain the temporal representation of node, embedding module aggregates the node's memory and memories of its neighbors.
CAWN WWng al. 2020) initially extracts multiple causal anonymous walks for node of interest. These walks serve lo explore the causality within network dynamics and operate relative node identities, making the model inductive. Subsequently, it employs recurrent neural networks to encode each walk and aggregates the resulting representations to derive the final node representation.
TCL (Wang et al. 2021) utilizes a transformer module to create temporal neighborhood representations for nodes engaged in an interaction. Then, it models the inter-dependencies by employing a co-attentional transformer at a semantic level. To achieve this, TCL incorporates two distinct encoders to derive rep- resentations from the temporal neighborhoods surrounding the two nodes forming an edge. Additionally, during the training process, TCL employs temporal contrastive objective function preserve the high- level semantics of interactions in latent representations of the nodes.
GraphMizer (Cong et al.) 2022 is simple model architecture that comprises three components: a link-encoder that summarizes information from temporal links with multi-layer perceptions (MLP), node-encoder that summarizes node information with neighbor mean-pooling, and link classifier that performs link prediction based on the outputs of the encoders. It is noteworthy that none of these components incorporate graph neural network modules. Additionally, GraphMixer introduced a fixed time-encoding function that encodes any timestamp as an easily distinguishable input vector.
DyGFormer Yu et al. 2023) is a caanformer-based architecture for dynamic graph learning. It first ex- tracts historical hop interactions of the nodes participating in the interaction, yielding two interaction sequences. Then, it utilizes a neighbor co-occurrence encoding scheme to model correlations between the two nodes, based on the two interaction sequences. DyGFormer splits the interaction sequences enriched with the neighbor co-occurrence encoding into multiple patches and feeds them into transformer for capturing long-term temporal dependencies. Finally, the node representations are derived by averaging the corresponding outputs of the transformer.
EdgeBank (Poursafaei et al. 2022 is simple heuristic that stores all observed edges in memory. Given an edge, EdgeBank,, predicts it as positive if the corrosponding node pair is found in the memory, otherwise, EdgeBank, predicts it as negative.
EdgeBank (Poursafaci et al. 2022 is version of EdgeBank that only memorizes edges from limited time window in the recent past. Therefore, it has a strong recency bias and forgets edges that occurred
long ago.
A.3 Model Configurations
We present the syper-parameter settings of temporal graph learning models that are fixed across all the datasets. For each method, always search through the following learning rates: 1e-3, 3e-4, 1e-4, 3e-5,
18

---

Published in Transactions on Machine Learning Research (10/2024)
1e-5, 3e-6, 1c-6. We execute experiments cluster with 1000 Intel(R) Xeon(R) CPUs @ ~2.0GGHz. Note that, since experiments for DARPA-THEIA would take months under our model configuration and hardware setup, we run EdgeBanks and EdgeBanktw with batch size of 100 (rather than 1, see $5.10, and models on this dataset are stopped after 25 days of exccution time. In practice only TGN does not converge in this limespan. The results for the best checkpoint by the 25th day are reported. The rest. of the model configurations are as follows:
TGN:
Dimension of time encoding: 50
Dimension of node memory: 50
Dimension of output representation: 50
Number of sampled neighbors: 10
Neighbor sampling strategy: recent
Number of graph allention heads:
Number of graph convolution layers: 1
Memory updater: gated recurrent unit (Cho et al. 2014
Dropout rate: 0.1
CAWN:
Dimension of time encoding: 25
Dimension of position encoding: 25
Dimension of output representation: 25
Number of attention heads for encoding walks:
Number of random walks: 36
Length of each walk (including the target node): 2
Time scaling factor a: 1e-6
Dropout rate: 0.1
TCL:
Dimension of time encoding: 50
Dimension of depth encoding: 50
Dimension of output representation: 50
Number of sampled neighbors:
Neighbor sampling strategy: recent.
Number of attention heads:
Number of transformer layers: 2
Dropout rate: 0.1
GraphMixer:
Dimension of time encoding: 60
Dimension of output representation: 60
Number of sampled neighbors: 20
Neighbor sampling strategy: recent
Number of MLP-Mixer layers: 2
Time gap: 2000
Dropout rate: 0.4
DyGFormer:
19

---

Published in Transactions on Machine Learning Research (10/2024)
Dimension of time encoding: 50
Dimension of neighbor co-occurrence encoding: 20
Dimension of aligned encoding: 20
Dimension of output representation: 50
Maximal length of input sequences: 32
Patch size:
Neighbor sampling strategy: recent
Number of attention heads: 2
Number of transformer layers: 1
Dropout rale: 0.1
B Computational Efficiency
Table contains training plus validation time of the temporal graph methods on TEMPSYNTHGRAPH. The most computationally efficient methods our study are TCL and DyGFormer, both based on the transformers architecture (Vaswani al. 2017j. GraphMixer and TGN exhibit significantly longer training and validation times, approximately three times slower than the aforementioned methods. CAWN emerges as the slowest among the considered techniques, primarily due to the sampling of random walks at two-hops (compared to the remaining methods being 1-hop).
Table 7: Training and validation times of temporal graph methods per epoch on the synthetic graph. Results are reported in minutes and averaged over 20 runs. Measurements are performed on Ubuntu machine equipped with one Intel(R) Xeon(R) CPU E5-2658A v3 @ 2.20GHz with 24 physical cores.
TGN
CAWN
TCL
GraphMixer DyGFormer
173.37215.01 Time [min] 14..2
41.2513.35 145.8225.62 52.31
C Detailed Experimental Results
Table 8: AP for all models on TeMPSYNTHGRAPH with synthetic anomalies of the five categories introduced in 3.1 Bold and underlined values mark first- and second-best performance across anomaly types.
Anomaly type
EdgeBank EdgeBank
TGN
CAWN
TCL
GraphMixer DyGFormer
T
16.77
16.77+0.a7
35.64
37.90
53.31+
64.76.4..1
37.70=1.00
C
4.74_0.00
4.7440.00
19.81
44.74+10.84
31.5743.50
58.9744.00
61.1123.80
T-C
17.1020.25
17.11
59.08
53.68_1.80
54.72444.0
68.5141.60
48.94-4440
s-c
70.98
70.98
48.69
71.61
56.2744.52
64.72+2.00
77.66=2.00
T-S-C
72.01
72.02
60.04 a0
72.40 00.28
64.4242.68
70.72+0.68
72.48 2..27
We report the counterparts of the experiments reported in Table 3
4 results with Average Precision and
Recall@k metrics in Table g
10 and
13 For Recall@k, is set the number of known anomalies
for the considered data split.
20

---

Published in Transactions on Machine Learning Research (10/2024)
Table 9: Recallåk for all models on TEMPSYNTHGRaPH with synthetic anomalies of the five categories introduced in 3.1 Bold and underlined values mark first- and second-best performance across anomaly types.
Anomaly type
EdgeBanktw
TGN
CAWN
TCL
GraphMixer DyGFormer
T 24.8640.35
24.87
38.83 18.88
45, 45.25
59.98
69.33_u.4t
48.8 40.87
c
3.61+
3.61 +0.20
48.34+1.61
49.15=a.48
$1.87=a.75
57.64
60.18
T-C
24.9940.36
24.9620.39
59.2842.44
57.1823.08
53.81
67.78_1.
48.7644.35
S-C
70.2520.07
70.25
47.01
50.5
55.94_4.84
63.53
73.2141.03
T-8-C
70.93+0.40
70.83
59.21
22.26=0.22
63. 23..22
70.17
72.02
Table 10: AP for models WIKIPEDIA, REDDIT, MOOC, ENRON and UCI with typed synthetic anomalies Bold and underlined values mark first- and second-best performing models across rows. We do not report results on ENRON and UCI for C and T-C anomalies as these datasets lack edge attributes.
3.2
Anomaly type
Datasets
EdgeBank
EdgeBank..
TGN
CAWN
TCL
GraphMixer
DyGFormer
WIKIPEDIA
6.71
6.95
16.26
16.97
14.731..900
14.27 1..33
17. 17.2910.24
REDDIT
5.9840.09 6,55
11.8420.35
16.06
12.59+a.a1
12.05
12.47+1.22
MOOC
4.83
4.8120.02
64.07
61.4443.58
37.5 37.5642.95
62, 6849.8
53.81+4.35
ENRON
6.03
6.3620.10
15.8443.77
35.8448.18
25.05+20.12 72
42.84
12.09
UCI
4.
4.90
17.55
17.88
15.1141779
21.91
15.74
Avg. Rank
6.60
6.40
3.20
2.00
3.60
2.60
3.60
WIKIPEDIA
6.06
6.2620.04
13.9140.58
17.23
11.99
12.7520.44
16.57
REDDIT
20
12.4820.15
43.0943.19
38.0318.15 13
57.25
46.2522.00
49.9424.16
MOOC
4. 4.44+0.02
4.40
92.75+1.82
70.01
85.75
79.30+
94.18 2
Avg. Rank
6.67
6.33
3.00
3.67
3.00
3.67
1.67
WIKIPEDIA
6.66+0.02
7.11
15.76+0.67
17.06
14.43+ 7o
15.17=0.40
17.31
REDDIT
5.
6.40
49.9443.15
40.7045.40
55. 15.29.. 14.
50.24
44,4419.00
T-C
MOOC
4. 4.80
4.7820.05
90.5413.17
82.5613.31
85.85 16
88.85
93.96
Avg. Rank
6.67
6.33
2.67
4.00
3.33
3.00
2.00
WIKIPEDIA
30.43-0.22
23.62=0.17
21.2942.10
30.45+2.54
10.33
7.45=0.20
30.85 1.56
REDDIT
39.76.10.18
23.64.0.09
31.5412.71
34.4916..2
48.9612.0 12.13
34.0012.12
35.93
MOOC
8..01
7.88 10.02
56.6017.47
70.56
47.97
45.04_8.43
56.78
S-C
ENRON
44.08
38.44-1.16
19.09
17. 17.62+
6.6640.49
6.99
20,20
20.20
UCI
13.3440.oa
12.36=0.08
6.99+0.41
16.1741.86
6,23
5.33_p.ss
16.98
Avg. Rank
3.00
4.80
4.60
2.80
4.80
6.00
2.00
WIKIPEDIA
31.70.1..00
24.57
31.03
12.39
64.79.5.22
35.29
69.28 15.87
84.0612.44 12.86
REDDIT
41.90.1o.00
24.8610.00
64.67 15.00
67.52 13.64
54.7413..1
59.84.1.41
67.22
MOOC
11.30400.0
9.66+40.01
96.3642.25
95.90+0.79
95.77+1.79
93.51 11.4
94. 04.66+2.48
T-8-C
ENRON
63.69
56.27=0.13
21.75++8.2
49.3247.40
39.35+4435
52.55
27. 24.1444444
UCI
14.83 10.02
13.55
26.03 19.34
39.94 18.54
35.1 13.26
33.70 15.86
52.72 18.31
Avg. Rank
4.80
6.00
4.40
2.40
4.00
3.60
2.80
21

---

Published in Transactions on Machine Learning Research (10/2024)
Table 11: @ecall@k for models on WIKIPEDIA, REDDIT, MOOC, ENRON and UCI with typed synthetic anomalies ($3.2. Bold and underlined values mark first- and second-best performing models across rows. We do not report results on ENRON and UCI for and T-C anomalies as these datasets lack edge attributes.
Anomaly type| Datasets
EdgeBank
EdgeBankw
TGN
CAWN
TCL
GraphMixer
DyG DyGForer
WIKIpeDIA
13.75.0.49
22.39
18.64
18.53
14.74g1.73
20.67
17.13
REdDrT
12.0020.49 9.8820.25
15.63 16
20.7020.58
15.99
15.6521.06
15.90
MOOC
6.54
5.86
58.95+1.46
42.70
57.42+2.80
62.85
56.49
ENRON
10.2620.18
13.2440.41
16.80 40
12.0227.20
28.7347.35
48.91
16.19
UCI
5. 1.226
4.61
20.06 09
22.30
17.58
25.55
18.63
Avg. Rank
6.20
6.80
3.20
1.80
3.60
2.40
4.00
WIKIPEDIA
10.50
9.71
17.18
19.5621.70
13.03
15.39
18.09
REDDIT
20.
18.57
50.22
43.89
58.45
53.61+1.81
54.89+2.n2
MOOC
3.50
3.67+0.22
89.48+3.04
69.6949.05
84.67+5.07
81.1744.77
91.62+1.56
Avg. Rank
6.33
6.67
3.00
3.67
3.00
3.67
1.67
WIKIpediA
41.85
11.840..a
17.74+1.1
18.58
14.04+1.76
16.79+0.54
18.1640.78
REDDIT
21.23
9.78+0.09
54.59+1.98
46.71+s.08
57.13+2.62
$7.47+1.60
48.7549.25
T-C
MOOC
40.00
6.20
88.90
84.06
84.3344.44
89.5242.36
90.32 +2.08
Avg. Rank
6.00
7.00
3.00
3.67
3.67
2.33
2.33
WIKIPEDIA
29.97=0.95
20.51
25.86+3.50
32.15
11.25++3.36
5.45+
31.42+2.83
REDDIT
41.27:10.4
24.3910.15
38.53 84
37.3716.90
52.2210.8
41.82133.0
37.50.1.e
MOOC
8.95
8.01
64.09+2.91
72.80
61.3444.02
53. 53.69
62.85+1.97
S-C
ENRON
58.50
49.4
49.49+1..5
27.52 84
13.57+3.a1
6.51
7.77+0.58
19.73
UCI
12.65
13.1710.79
6.42
15.30
6.35 68
5.3611.07
17.41
Avg. Rank
3.40
3.60
3.00
4.80
5.40
3.00
WIKIPEDIA
33.50
15.12
37.55
63.17
37.7717.05
66.33
76.20
REDDIT
41.7110.1r
10.99
67 14.94
67.3414.45
53.8312245
63. 47 14.18
69.49 14.71
MOOC
16.73
10.18
94.26+2.84
90.62+1.24
91, 91.80+2.30
93 65.404
92,95
T-S-C
ENRON
56.66 1..67
47.39 13.23
27.6817.60
49.56.3.40
43.1914.55
54.09 12, 12.25
22.03
UCI 13.91
10.04
27.95
42.88
13.83
37 17.62.77.8.
53.87 12.67
Avg. Rank
5.00
6.40
3.80
3.40
4.00
2.80
2.60
Table 12: AP for models on LANL and DARPA-THEIA with organic anomalies. Bold and underlined values mark first- and second-best performing models across rows.
Datasets
EdgeBank BageBank
TGN
CAWN
TCL
GraphMixer
DyGFormer
LANL
1.01
0.82
5.77
6.13
1.11
3.85
7.08
DARPA-THEIA THEIA
0.01
0.01
0.07
0.02
0.01
0.04
0.08
Avg. Rank 5.50
6.50
2.50
6.00
3.50
1.00
Table 13: Recall@k for models on LANL and DARPA-THELL with organic anomalies, Bold and underlined values mark first- and second-best performing models across rows.
Datasets
EdgeBank
EdgeBankw
TGN
CAWN
TCL
GraphMixer
DyGFormer
LANL
0.27
0.27
12.30
5.61
4.55
5.88
9.09
PARPA-THER
0.00
0.00
0.77
0.00
0.00
0.77
0.77
Avg. Rank
5.00
5.00
1.00
4.00
4.50
2.00
1.50
22

---

Published in Transactions on Machine Learning Research (10/2024)
TEMPSYNTHGRAPH
WIKIPEDIA
REDDIT
10a
10
10
109
10a
Froquenty
Frequency
Prequency
10F
102
10
10
104
10"
10
3010
Node degreo MOOC
Node degroe
Node degroe
ENRON
UCI
10
10a
10a
Frequencc
Frequency
Frequency
10a
10a
10
10F
10
10
2000
6000
25
Node degree
Node degree
Node degree
Figure 2: Distributions of node degrees in synthetic and real-world graphs.
D Statistical Properties of the Synthetic and Real-World Graphs
We present analysis of the statistical properties of the synthetic graphs introduced in $3.3 and the real- world graphs listed in App. A.I In particular, we analyze properties related to the structure, context, and temporal dynamics of the graphs, highlighting consistencies and irregularities in and across these three dimensions.
Structure. To analyze and compare the graphs for their structural properties, compute the distributions of node degrees and the joint degree matricos, defined as the relative number of edges between vertices of degree i and degree for every pair (i.j), of the static versions of the graphs (i.e., the graphs without timestamps). These statistics are displayed Figure 2 and Figure 3 respectively. Notably, both statisties reveal Gaussian distributions and across node degrees in the synthetic graph, confirming the desired structural consistency this controlled setting. Conversely, distributions of and across node degrees are less structured within and inconsistent across the real-world datasets. Notably, all real-world datasets have long-tailed distribution of node degrees, with top degrees ranging from 111 (EnRON) to 6695 (MOOC), Moreover, node degrees in WIKIPEDIA, REDDIT, and UCI follow exponential distributions, whereas node degrees in the ENRON dataset follow noisy Gaussian distribution. MOOC is globally weakly interconnected except for varions outlier nodes with degrees between and 6695. Moreover, the similarity between the degrees of neighboring nodes constituting the joint degree matrices depicted suggests the datasets' structural homogeneity and consistency, which Figure 3 reveals to be strong in TEMPSYNTHGRAPH, weaker yet existing in ENRON and UCI, and at most very weak in WIKIPEDIA, REDDIT, and MOOC.
Context. We investigate the contextual consistencies of graphs by analyzing the distributions of standard deviations a of edge messages across timestamps. Higher standard deviations indicate edges with inconsis- tent edge messages over time. this view, Figure 4 shows that edge messages in real-world graphs exhibit substantially lower contextual consistency than the synthetic graph. Specifically, TEXpSYNtHGRaPH fea- tures standard deviations concentrated around 0.05, whereas WIKIPEDIA, REDDIT, and MOOC feature exponential distributions of standard deviations with tails ending around 9, 5, and 20, respectively. Note that ENRON and UCI have standard deviations of zero due to the absence of edge messages in these datasets.
Time. We analyze the temporal patterns of the graphs by examining the distributions of standard deviations of inter-event times for edges that reoccur multiple times. Wider distributions indicate graphs with less
23

---

Published in Transactions on Machine Learning Research (10/2024)
TEMPSYYTHCRPPH
WIKIPEDIA
REDDIT
0
Node degroe.
Node degroc
Node degroee
0.004
1000
200
0.02
0.002
0.002
0.001
2000
0.00
0.000
0.000
10
200
400
1000
2000
Node degree
Node degree
Node degree
MOOC
ENRON
UCI
0.0008
0.00100
Node degroe
0.00075
Node degroe
0.002
Node degroe
0.0006
100
4000
0.00050
0.0001
0.001
0.00025
200
0.0002
6000
100
0.00000
0.000
0.0000
2000
4000
6000
50
100
200
Node degree
Node degree
Node degree
Figure 3: Joint degree matrices of synthetic and real-world graphs.
TEMPSYNTHCRAPI
WIKIPEDIA
REDDIT
10
Froquency
10a
Frequency
10a
Prequency
10
102
10
10
10A
10
108
10a
10
0.046
0.048
0.050
0
of Edge Messages
Edge Messages
a Edge Messages UCI
MOOC
ENRON
10
10a
10
Prepercy
Frequency
Prequency
10a
10a
10
10
20
00.0
-0.25
0.00
0.25
0.50
-0.50
-0.25
0.00
0.25
0.50
of Edge Messages
0. of Edge Messages
a of Edge Messages
Figure 4: Distributions the standard deviations of edge messages across timestamps for edges in synthetic and real-world graphs.
consistent intervals between re-occurrences of the edges. As shown in Figure 5 inter-event times are notably less consistent real-world graphs compared to the synthetic one. Specifically, TEMPSYNTHGRAPI exhibits standard deviations below 10*, whereas the standard deviations for WIKIPEDIA, REDDIT, MOOC, ENRON, and UCT span multiple orders of magnitude, reaching up to 107.
24

---

Published in Transactions on Machine Learning Research (10/2024)
TEMPSYYNTICRAPH
WIKIPEDIA
REDDIT
10a
10
10a
Freqency
10
Frequency
Frequency
10
10
10
109
10
109
2000
4000
6000
8000
0.00
0.25
0.50
0.75
1.00
1.25
0.00
0.25
0.50
0.75
1.00
1.25
a Irter Event Times
a Initrn Evvent Times
0 Inter-Fvent Fvvent Times
MOOC
ENRON
UCI
10a
10a
Frequenc
10a
Prequency
10a
Frequency
10
108
10a
0.00
0.25
0.50
0.75
1.00
1.25
di of Inter- Event Times
x10
a of Inter- Even- Times
x107
0 of Inter Event Times
Figure 5: Distributions of the standard deviations of inter-event times for edges synthetic and real-world graphs.
25


Reviews:
[{"review_id": "OmoGsyU2tr", "reviewer": "Reviewer_fVbf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:** The paper is well-written and easy to understand. It provides clear definitions of different types of anomalies with straightforward examples. Furthermore, it clearly distinguishes between link prediction and link anomaly detection, emphasizing the importance of considering different types of anomaly detection. The paper also introduces effective negative edge sampling methods to address this challenge, enabling state-of-the-art (SOTA) models to perform well on these tasks.\n\n**Weakness:** The paper does not provide sufficient empirical comparisons between considering different types of anomalies and not doing so. While I understand that this comparison might be challenging, I believe it would significantly strengthen the proposed work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the lack of existing methods for anomaly detection in continuous-time dynamic graphs, particularly those dealing with various types of anomalies (contextual, structural, and temporal). The authors propose several key contributions: they introduce a method to generate and inject typed anomalies into graphs. By doing so, they create realistic scenarios for evaluating anomaly detection algorithms.\nAlso, the paper extends the optimization problem of link prediction to that conditioning link existence on contextual edge attributes.\nThen they provide a comprehensive experiment that benchmarks many SOTA models on both synthetic and real-world datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "None", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1fxoAi7D9g", "reviewer": "Reviewer_emDe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n=======\n\n* Authors presented a novel taxonomy for edge anomaly types that enhances current understanding on synthetic graph generation and anomaly dectection in continuous time dynamic graphs.\n\n* Extensive experiments on real and synthetic datasets help to elucidate the advantages of the proposed methodology.\n\n\nWeaknesses\n=========\n\n* Apart from the last paragraph on section 4.2, little has been discussed on possible other potential alternatives for incorporating contextual information into the training process.  Are there any other options? If so, how does it compare the one used in paper?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a framework for anomaly detection on edges of dynamic graphs.  In addition to tackling a very interesting and challenging research problem, the authors main contributions are:\n* They propose a framework that leverages structural, temporal, and contextual features for dectecting edge-level anomalies on graphs\n* On the back of such framework, the authors propose methods for injecting anomalies into graphs as well as generating continuous-time dynamic graphs\n* They extend link prediction learning methods by conditioning edge existence on contextual edge attributes\n* They perform extensive experiments with synthetic and real-world dataset that show strong evidence for validation of the proposed methods\n\nIn summary, tasks revolving dynamic graphs estimation (i.e., dynamic link prediction, node fault dectection, graph forecasting, time-varying graph clustering) are challenging and underexplored topics; in this manuscript, the authors made a meaningful step towards advancing the practical knowledge on the field.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "As I mentioned previously, it would be relevant to have a paragraph discussing potential alternatives for incorporating the additional edge contextual information into the graph training process.", "broader_impact_concerns": "No concerns on ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BhtqrRytdE", "reviewer": "Reviewer_KNpY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**: The authors conducted a thorough review of the related work, analyzing their limitations. The proposed benchmark, including a fine-grained taxonomy and dataset generation algorithms, is clearly described.\n\n**Weaknesses** (see *Requested Changes* for details): \n1. The necessity for proposing the new taxonomy is not well highlighted or convincing. While the authors discuss the limitations of existing taxonomies for anomalies in dynamic graphs, I would appreciate examples of real applications on CTDGs that necessitate the new taxonomy introduced in this paper.\n2. The motivation behind introducing algorithms for generating synthetic CTDG and injecting anomalies are not well conveyed.\n3. The significance of the studied problem (edge-level anomaly detection) is not well conveyed.\n4. Some statements and conclusions require more supporting evidence.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors proposed a benchmark for anomaly detection in continuous-time dynamic graphs (CTDG), including a new taxonomy and algorithms for generating synthetic CTDGs and injecting corresponding anomalies. In the experiments, they adapted and compared seven existing CTDG learning methods using this benchmark, and gave analysis of the experimental results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The authors need to enhance their motivation of proposing the new taxonomy. I understand that existing taxonomies are not comprehensive - as almost all taxonomies are - I am looking for explanations that clarify what inspired the authors to design this new taxonomy in such a specific way. For examples, *\"previous taxonomies cannot adequately address the studied anomaly detection tasks on CTDGs, and there are emerging real-world graph applications that necessitate detecting edge-level anomalies with the proposed fine-grained detail.\"* Meanwhile, the authors need to emphasize the importance of the studied problem (edge-level anomaly detection), for instance, *what is its application?* \n2. The authors need to clarify their motivation for introducing a synthetic CTDG generation algorithm. I understand that the algorithm ensures consistencies across multiple dimensions, but it is unclear why such perfect consistency is deemed essential for the task. Expecially, real-world CTDG datasets do not exhibit such perfect consistency.\n3. In **Definition 1** (Continuous-time dynamic graph), is the \"non-decreasing\" hypothesis a conventional setting for CTDG? I believe some works account for scenarios where edges are removed over time [1], where the time attribute in interactions is represented as time spans rather than timestamps. If it is not conventional, the authors should highlight that supposing \"non-decreasing\" is specific to this work.\n4. In Section 5.4, where the authors concluded **\"real graphs may not exhibit strong consistencies\"**, the authors are suggested to present statistical differences of synthetic and real graphs to support this conclusion. For example, *their distributions of node degrees and temporal attributes*. \n\n[1] Nguyen et al. Continuous-Time Dynamic Network Embeddings, WWW'18, https://dl.acm.org/doi/10.1145/3184558.3191526", "broader_impact_concerns": "There is no ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper was reviewed by three expert reviewers. The reviewers agreed that the paper provides clear definitions of the different types of anomalies and a thorough experimental evaluation on both synthetic and real-world datasets. However, the reviewers also raised concerns mainly about the motivation of the work (do we actually need a new taxonomy?), the intuition behind the proposed taxonomy (why is this the right taxonomy and not others?) and the considered setting (a continuous-time dynamic graph is defined as a sequence of non-decreasing chronological interactions).  The authors comprehensively responded to the reviewers' comments, and all reviewers are now positive about the paper. I thus think that the paper is ready for publication. Since regular submissions consist of no more than 12 pages of main content, I request the authors to make sure that the conclusion fits within the 12-page limit.
Decision: Accept (poster)