dataesr/scientific-paragraphs-categorization huggingface dataset:
The first term maximizes the probability of co-occurrence between center and context nodes, whereas the second term minimizes the probability of co-occurrence between context nodes and negative center node samples. Following this process, users who regularly authenticate to similar entities are close in embedding space. Finally, edge prediction (i. e. authentication prediction) is done between every pair of nodes a and b by using a logistic regression on embeddings a • b, where • is the Hadamard product. The authentication subset of LANL dataset and PicoDomain custom dataset are used for evaluation and the proposed method outperforms other unsupervised ML techniques, on both true positive and false positive rates. To overcome malicious logins and credential-based lateral movements, MLTracer  [77]  models login activity from logs as a heterogeneous information network (HIN) and extract contextual semantic with meta-paths.
For each type of metapath, they apply a CNN along with co-attention to learn vector representations, and a fully-connected layer predicts malicious edges given as input the concatenation of source and destination node embeddings. The lateral movement detection problem is also tackled by LMTracker  [78] , where the authors also model the interactions between entities using a heterogeneous graph. The metapath2vec++ model  [94]  computes all node embeddings using meta-paths and random walks. An autoencoder then tries to reconstruct the embeddings, in an attempt to find outliers in embedding space when the reconstruction of malicious examples fail. The final task is to predict a potential lateral movement path, namely a malicious path along multiple edges in the graph. For training, only benign samples from LANL and CERT datasets are leveraged.
Experiments are conducted on authentication logs from LANL 2015 dataset, with multiple encoders such as GAT, GCN or GraphSAGE, along with various RNNs like GRU and LSTM. Paudel et al. [19]  also leverage the temporal dimension of graphs for network-based anomaly detection. They propose PIKACHU, an unsupervised dynamic graph embedding technique that considers both the dynamic edges and nodes. The network is modeled as a stream of graphs, where a temporal dimension represents the superposition of discrete graph snapshots, with possibly different structures. No features are used, except the IP addresses and timestamps required to build the dynamic graph. Temporal random walks are used in combination with Skip-gram model to generate embeddings that encode the spatial and short-term temporal dependencies in the dynamic graph.
A GRU-based auto-encoder also captures long-term temporal dependencies. An anomalous edge can then be detected when it has a low probability to exist between two nodes u and v at a given timestamp T. This probability is expressed in Eq. 27 and 28. p v = P(v|u, N (u)) T , ( 27 ) p u = P(u|v, N (v)) T , (28) where p v and p u respectively represent the probability that there exists an edge (v, u) and (u, v). N (i) is the neighborhood of a given node i. This probability is estimated by the softmax function and cross-entropy is used as loss to train the model. Finally, an anomaly score is computed based on p v and p u : e score = (1 -p v ) + (1 -p u ) 2. ( 29 ) In their experimental setup, the authors use the LANL and DARPA OpTC datasets, where 1 snapshot corresponds to 1 hour of data. Three dynamic graph-based anomaly detection methods have been compared to PIKACHU and it notably achieves a lower false positive rate and higher AUC.
4) NETWORK INTRUSION DATASETS This section aims to review the different network intrusion datasets that were employed in previous studies. A comprehensive listing of these datasets is also provided in Table  2. CAIDA  [95] : Between 2008 and 2019, the Center for Applied Internet Data Analysis (CAIDA) recorded passive network traces from high-speed monitors on a commercial backbone link. Hundreds of Gigabytes of requests were recorded over these years and can be accessed via pcap files. These data are usually used as background traffic for creating synthetic cyberattack detection datasets  [65] ,  [75]. Zhou et al. [65] : In their botnet detection method based on GCN, Zhou et al. released a graph-based dataset made of both synthetic and real botnet topologies within a real background network traffic.
The CAIDA 2018 passive network traces were used as background traffic and random subsets of nodes were chosen to embed botnet topologies using real botnets or overlaid P2P topologies. A total of 4 synthetic topologies (DE BRUIJN, KADEMLIA, CHORD, LEET-CHORD) and 2 real botnet topologies (C2 and P2P) generated and divided into 6 labeled datasets. Each dataset contains 960 undirected graphs with self-loops and is composed in average of 140,000 nodes and 700,000 edges. No additional flow features are provided in this dataset and existing works rather leverage representation learning to learn directly embeddings from graph topology, as done in  [66]  and  [67]  with C2 and P2P datasets. CTU-13  [96] : CTU-13 was delivered by CTU University, Czech Republic and is made of network traffic captures containing benign traffic and Botnet attacks.
Two types of data are available for this dataset: packet-based pcap captures (more than 850M packets) and flows captured with Argus in Netflow format (around 20M bi-directional flows). The dataset consists of 13 attack scenarios exploiting various protocols, divided into 13 files. A graph can possibly be built using the source and destination IP addresses of either the packet or flow data. However, flow data are in practice much more used in graph approaches due to the large size of packetbased captures. [97] : CIC-IDS2017 is a network dataset proposed by the Canadian Institute of Cybersecurity (CIC), composed of benign and attack network flows. For each flow, 80 features have been extracted into pcap and CSV files using CICFlowMeter  [98]  for 5 days in a simulated environment. Seven types of web attacks are present in the dataset: Brute Force, HeartBleed, Botnet, DoS, DDoS, Web Attack, Infiltration.
CIC-IDS2017 CSE-CIC-IDS2018  [99] : CSE-CIC-IDS2018 is a network dataset born from a joint project between the Communications Security Establishment (CSE) and the Canadian Institute for Cybersecurity (CIC). The data cover examples from the same seven attack types as CICIDS2017, and CICFlowMeter was also used to extract 80 flow features that are available in pcap format.
UNSW-NB15 [100]: This dataset was released by the Cyber Range Lab of UNSW Canberra. Network flows were captured in a private environment, resulting in 2,218,761 benign flows (87. 35%) and 321,283 attack flows (12. 65%) VOLUME 11, 2023  divided into 10 classes: Benign, Analysis, Backdoor, DoS, Exploits, Fuzzers, Generic, Reconnaissance, Shellcode, Worms. A comprehensive range of data formats are available, including packet-based (pcap files) and flow-based (CSV, Zeek, Argus) network logs. This dataset is known to suffer from class imbalance and class overlap  [101] , meaning that these two issues should be addressed before usage. As an example,  [102]  used bagging, undersampling and synthetic minority oversampling to tackle the class imbalance problem, resulting in a better classification accuracy.
BoT-IoT  [103] : Also developed by the Cyber Range Lab of UNSW Canberra, this dataset was created by simulating a network environment and running multiple botnet attacks on it. Both packet-and flow-based data are captured and made public in pcap, Argus and CSV file formats. The pcap files store around 70GB of packet-based data and CSV files store 16. 7GB of flow-based data. The overall dataset contains 477 benign flows (0. 01%) and more than 3. 6M attack flows (99. 99%). Four categories of attacks are present in the dataset, including DDoS, DoS, OS and service scan, as well as keylogging and data exfiltration, for a total of 10 labeled attacks. [104] : ToN-IoT is another dataset created by the Cyber Range Lab of UNSW Canberra. It is composed of 3 categories of datasets: Telemetry of IoT and IIoT sensors, Windows and Ubuntu OS metrics and Network traffic.
These data were captured from a simulated and realistic network. For NIDS applications, the provided network dataset leveraged Zeek to capture 796,380 benign flows (3. 56%) and 21,542,641 attack flows (96. 44%), along with 44 flow features extracted from Argus and saved in CSV files. Nine attacks are represented: DoS, DDoS, backdoor, injection, MITM, password, ransomware, scanning and XSS. [105] : NF-BoT-IoT, NF-ToN-IoT, NF-CSE-CIC-IDS2018 and NF-UNSW-NB15 have been introduced as part of a standardization process that aims to use Net-Flow as the default standard for NIDS datasets (NF-prefix stands for NetFlow). Indeed, all four initial datasets have few features in common, making the benchmark of ML methods impossible on each of them. The authors therefore propose a new version of these datasets, using only 8 NetFlow features (12 by considering source and destination IP addresses and ports) instead of the original dissociated features.
NF-datasets • NF-UNSW-NB15: 1,550,712 benign flows (95. 54%) and 72,406 attack flows (4. 46%) divided into 10 classes: Benign, Fuzzers, Analysis, Backdoor, DoS, Exploits, Generic, Reconnaissance, Shellcode, Worms. • NF-BoT-IoT: 13,859 benign flows (2. 31%) and 586,241 attack flows (97. 69%) divided into 5 classes: Benign, Reconnaissance, DDoS, DoS, Theft. • NF-ToN-IoT: 270,279 flows (19. 6%) and 1,108,995 attack flows (80. 4%) divided into 10 classes: Benign, Backdoor, DoS, DDoS, Injection, MITM, Password, Ransomware, Scanning, XSS. • NF-CSE-CIC-IDS2018: 7,373,198 benign flows (87. 86%) and 1,019,203 attack flows (12. 14%) divided into 7 classes: Benign, Brute Force, Bot, DoS, DDoS, Infiltration, Web Attacks. • NF-UQ-NIDS: 9,208,048 benign flows (76. 77%) and 2,786,845 attack flows (23. 23%) divided into all previous classes.
The same authors also provided a second version of these datasets (with -v2 suffix)  [88]  that integrates a total of 43 flow features instead of the original 12. [107] : CIC-Darknet2020 is a network dataset used for VPN and Tor applications classification, developed by the Canadian Institute for Cybersecurity. It is composed of 77 flow features extracted with CICFlowMeter from 134,348 attack scenarios (84,67%) and 24,311 benign activities (15,33%), divided into 9 categories: Benign, Audio-Stream, Browsing, Chat, Email, P2P, Transfer, Videao-Stream, VOIP.
CIC-Darknet2020 LANL-2015  [108] : The Los Alamos National Lab (LANL) dataset is made up of 58 consecutive days of data gathered from the Los Alamos National Laboratory's internal computer network. The dataset is composed of 5 types of data: red-team activities, network flows, process start and stop events, window-based authentication events, and DNS lookups. A typical APT campaign produced 1,648,275,307 events across 17,684 Windows machines, including 305 compromised computers and 749 harmful events. With this dataset, graphs can be built thanks to the use of either network flows or authentication data events. DARPA OpTC  [109] : DARPA OpTC contains more than 17 billion network-based and host-based events following APT-like scenarios, generated in an enterprise network by DARPA, where 3 attack scenarios are represented: powershell empire staging, data exfiltration and malicious software upgrade.
All samples follow the eCar format, which is an extension of MITRE's CAR data model  [110] , where events are identified in temporal space using a 3-tuple (object, action, fields), inspired from object-oriented programming. CSIC 2010  [111] : The CSIC 2010 dataset is composed of 36,000 normal and 25,000 attack HTTP/1. 1 requests generated from an eCommerce website developed in the Information Security Institute of CSIC. Multiple types of attacks are present in the dataset, such as SQL injection, buffer overflow, information gathering and XSS. A graph can be built based on the HTTP requests, where two nodes are the host and destination URLs and where an edge represents a request. ISCXIDS2012  [112] : Initiated by the Information Security Centre of Excellence (ISCX) at the University of New Brunswick, this dataset is composed of 7 days of network activity monitoring from a simulated realistic environment where normal bahaviors and attack scenarios happen on specific days.
Multiple kinds of attacks are present: network infiltration from inside, HTTP DoS, DDoS with IRC Botnet, SSH BruteForce. Roughly 125,000 attack packets and 2. 1M benign packets are recorded. For each day, a pcap file can be downloaded. Multiple standard features are available, including packet payload and total number of bytes sent or received. PicoDomain  [113] : PicoDomain is a light-weight dataset containing network flows captured with Zeek in a small-size network environment with multiple departments and Windows Active Directory. The attack campaign was registered during 3 days and follows the Mandiant Attack Lifecycle (MAL), which comprises multiple cyclic phases appearing during adversarial campaigns. This dataset aims to be compact (∼ 220MB of Zeek logs) and useful for small analysis research projects, where small computation time and memory consumption are required.
Sequences of system calls can be monitored on a host device, and transformed into graphs to represent the sequential relations between calls. A wide variety of host-based data monitoring tools is publicly available and leveraged in research (e. g. Windows and Linux event logs, CamFlow  [124] , Sysmon [125], auditd  [126] ). On top of CamFlow, Flurry  [127]  is a framework that captures both kernel-level and user-level data to build provenance graphs for downstream graph representation learning experiments.
Three types of relations are taken into consideration while creating the graph: causal and sequential relationships within a day, logical relationships among days, and logical relationships among objects. Only benign examples are required to train the model as the goal here is to detect anomalies (i. e. suspicious behaviors). A custom approach based on random walks and word2vec is used to extract the context of each node and to construct embeddings. Using this technique, close nodes (i. e. log entries with close relationship) will be embedded close in the embedding space. Finally, they apply a clustering method with threshold to separate benign and anomalous nodes in latent space. Log2vec has been compared to 11 baselines on the CERT dataset and 2 methods on the LANL dataset. In both cases, Log2vec widely outperforms the baseline methods.
THREATRACE  [48]  is a real-time intrusion detection framework also based on provenance graphs. Each node is a system entity (e. g. file or process) and an edge is a system call associated with a timestamp identifying the event time. The model is inspired by GraphSAGE to aggregate the nodes and to build the embeddings. However, only benign examples are required here. The idea is to learn normal behaviors to detect anomalous ones in embedding space, without requiring anomalous examples. A total of 7 state-of-the-art algorithms are used as baselines, where 3 are anomaly-based host threats detectors such as Unicorn  [118] , ProvDetector  [133]  and StreamSpot  [134] , and 4 are anomalous log detectors like Log2vec  [22]  or DeepLog  [135]. On the StreamSpot, Unicorn and DARPA TC datasets, THREATRACE outperforms all methods, especially in terms of false positive and false negative rates.
Due to the complexity of finding real-life APT scenarios, the authors introduce a module responsible for creating synthetic attack graph samples by extracting threat knowledge from public CTIs and Tactics Techniques and Procedures (TTPs). The graphs are generated from threat reports written in natural language by domain experts and are then embedded into real benign provenance graphs to enhance the authenticity of data. Multiple evaluation strategies are compared on private data and on the public DARPA TC dataset. On the private dataset, the model is outperformed by CONAN  [137] , a state-of-the-art APT detection method based on hand-designed rules. However, APT-KGL outclasses other baselines by a large gap on DARPA TC despite a relatively low precision (i. e. high number of false positives).
The final node embedding is computed using a simple dot product as follows: h k v = a k v. hk v. (34) The given model is evaluated on the graph classification task by only aggregating the 1-hop neighborhood and by using as readout function a sum along all node embeddings. The loss is computed with binary cross entropy and experiments were done on StreamSpot and Unicorn datasets, where both original methods are outperformed. However, no other GNNbased methods were compared here. Huang et al. [121]  push the host-based intrusion detection task further by introducing an unsupervised method that considers multiple modalities at node-and edge-level. An heterogeneous directed graph is built from system interactions, where nodes are processes or files and edges are events such as a process accessing a file or a process forking other processes.
Finally, a deep SVDD model  [141]  is leveraged to compute a global score based on the graph embedding (obtained via mean-pooling), and a local score based on node embeddings. Considering these two scores in the loss allows the model to detect attacks from both local and global areas in the graph. The proposed model outperforms other baselines such as GCN and GraphSAGE. However, the experiments are done only on a private enterprise dataset.
3) HYBRID HOST AND NETWORK INTRUSION DETECTION Although rarely evoked in current graph-based intrusion detection works due to the lack of real-world datasets, some researchers are interested in leveraging hybrid approaches, where both host-and network-level data are taken into consideration when building the graph. To our knowledge, Li et al. [51]  are the first authors to propose a graph deep learning method that leverages both kinds of data to detect complex attacks such as APTs. They propose a framework that comprehensively captures the behaviors of the full APT lifecycle by building an Intrahost Provenance Graph (IPG) to capture host-based features, and an Interhost Interactive Graph (IIG), that models all network-based communications among hosts. The IPG is created using a similar heterogeneous graph as in previous works (i.
The IIG model is trained in a self-supervised fashion using negative sampling by maximizing the anomaly score of edges between the malicious hosts discovered in the IPG and by minimizing the anomaly score of benign edges. Thus, the loss takes into consideration the predictions of both IPG and IIG models. These two methods have respectively been evaluated separately on the StreamSpot and LANL datasets. However, the complete architecture using both models has not yet been experimented due to the lack of datasets.
4) HOST-BASED DATASETS The extensive research that was undertaken in HIDS over the last decade led to the emergence of large datasets that researchers can leverage to build various representations for downstream tasks. In this section, we describe the datasets that were leveraged in the studied papers, also summarized in Table  4. DARPA TC  [143] : DARPA TC dataset was created as part of the DARPA Transparent Computing program, that aimed to develop technologies to record and preserve the provenance of systems for security and system research. More precisely, multiple experiments involving APT attacks have been conducted by the DARPA and the recorded provenance system logs are, since then, publicly available. In most approaches studied here, the data are transformed into provenance graphs and the interactions between entities is learned with graph representation learning.
StreamSpot  [144] : This dataset hosted on GitHub  [145]  is composed of 6 * 100 system call flow graphs derived from 5 normal behavior scenarios and 1 attack scenario. The normal behavior graphs are represented as benign syscall flows of activities such as checking emails in Gmail, browsing CNN. com, downloading files and watching videos on YouTube. The attack scenario is made of malicious syscall flows from a drive-by download attack triggered from a malicious URL browsing that exploits a victim host and gains root access. The dataset is by default graph-oriented, meaning that it has been designed with the aim to be used as graph structures and in a real-time environment. Multiple node types exist such as socket, file or memory, whereas edges represent actual system calls like read or fork.
Unicorn  [118] : The Unicorn dataset consist of SC-1 and SC-2 datasets, that are created in a private lab environment following the cyber kill chain model. The system provenance was captured with CamFlow during benign and attack activities and over the course of three days. Therefore, provenance graphs can be intuitively built from these datasets. BETH  [146] : The BETH dataset provides more than 8 million data points collected from 23 honeypots in a network. Attacks have been conducted and both network and kernel-level host data were collected, labeled into benign, unusual and evil classes. A graph can be built using the interactions between events and processes available in logs  [130]. ADFA-LD  [147] : A host intrusion detection dataset based on Linux system calls that was developed by the Australian Defence Force Academy.
All data were gathered from hosts with normal background behaviors such as web browsing or LaTeX document editing. A total of 5,925 examples are present in the dataset, already separated into train/validation/attack sets, including 7 classes: Benign, Hydra-FTP, Hydra-SSH, Adduser, Java-meterpreter, Meterpreter, Web shell. CERT  [148] : The CERT Insider Threat Test Dataset is a synthetic insider threat dataset provided by Carnegie Mellon University  [149]. In the last v6. 2 release version, five scenarios are proposed such as a user logging into another account and searching for files, a user working after hours, etc. Five types of entities exist in the dataset: LDAP, device, email, http and logon with four to five features such as id, date, user, pc, url. More than 135M operations with 101.
Formally, given a training dataset D = {(x 1 , y 1 ),. , (x n , y n )} and a model f , the attacker seeks to modify D to obtain a new dataset D ′ = {(x ′ 1 , y 1 ),. , (x ′ n , y n )}, where x ′ i ≈ x i for all i, such that the performance of f on D ′ is degraded or manipulated in a specific way. This can be achieved, for instance, by injecting new samples or by flipping the labels of some examples  [153]. In contrast, during inference (i. e. , when the model is already trained), an attacker may leverage evasion attacks to modify the input data and deceive the model. Precisely, given an input sample x and a model f , the attacker seeks to find a perturbed sample x ′ such that f (x ′ ) ̸ = f (x) and x ′ ≈ x. The goal of evasion attacks is to cause the model to misclassify malicious samples as legitimate.
This adversarial attack works in black-box scenario, meaning that only the output prediction is known by the attacker. The proposed HAA method is evaluated using the UNSW-SOSR2019 dataset  [157] , and results show that it can reduce classification precision by more than 30%. The findings suggest that this approach is an effective strategy for implementing level-aware black-box adversarial attacks against GNN-based IDSs in IoT environment. The authors of paper  [54]  presented in Section III-B1, verify the robustness of their GNN model against flow-based adversarial attacks. They notably modify flow-level features to test the robustness of each system, such as incrementing the packet size of attack-related flows, or increasing the inter-packet arrival times, to serve traffic at lower rates.