# ADAL-NN: Anomaly Detection and Localization Using Deep Relational Learning in Distributed Systems
## 4 Implementation
### Experiments  
We performed the following steps for the experiments of anomaly detection and localization:  
1. Datasets: We utilized three different datasets for our experiments: Hadoop Distributed File System (HDFS), Big-Log (BGL), and Papertrail. These datasets are widely used in anomaly-detection research and provide a diverse range of log data for evaluation purposes.
2. Preprocessing: Before conducting the experiments, we preprocessed the datasets to prepare them for anomaly detection. This involved cleaning the data, removing irrelevant characters, and transforming the log messages into a structured format for analysis.
3. Feature extraction: In order to extract meaningful features from the log data, we applied various techniques such as tokenization, dictionary-based grouping, and embedding. These techniques allowed us to represent the log data in a structured and informative manner.
4. Experimental steps: 1. Splitting the data: We partitioned the preprocessed datasets into training, validation, and testing sets via 8:1:1. The training set was used to train the  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Log Type** & **No of Messages** & **No of Templates** & **Seq** & **Perf** \\ \hline HDFS & 11,176,738 & 31 & ✓ & ✗ \\ \hline BGL & 4,747,963 & 377 & ✓ & ✗ \\ \hline Papertrail & 2,949,569 & 109 & ✓ & ✓ \\ \hline \hline \end{tabular}
\end{table}
Table 2: Log datasets.  
models, the validation set was used for hyperparameter tuning, while the testing set was employed to evaluate the performance of the trained models in detecting anomalies. Model selection: We selected a range of anomaly-detection models to compare with our proposed NN model. These models included Support Vector Machines (SVM), Isolation Forest (IM), Principal Component Analysis (PCA), Neural Networks (NN), Autoencoders (AE), Variational Autoencoders (VAE), and Recurrent Neural Networks (RNN). Each model was chosen based on its popularity and effectiveness in anomaly-detection tasks. Model training: For each selected model, we performed training using the training set of the respective dataset. The models were trained to learn the normal behavior of the log data, capturing patterns and structures that characterize normal system operations. Parameter tuning: We conducted parameter tuning for each model to optimize its performance. This involved adjusting the model-specific parameters, such as the number of hidden layers, learning rate, and regularization parameters to achieve the best possible anomaly-detection results. Model evaluation: After training the models, we evaluated their performance using the testing set. We calculated various evaluation metrics, such as precision, recall, and F1 score, to assess the effectiveness of each model in detecting anomalies. We considered the weighted average F1 score as the main metric for the evaluation part. Experimental environment: Hardware: The experiments were conducted on a high-performance computing cluster comprising multiple computing nodes. Each node was equipped with sufficient computational resources, including multi-core processors and a substantial amount of memory, to handle the processing requirements of the experiments. Software: We used open-source machine learning libraries and frameworks, such as scikit-learn for machine learning, and Pytorch for deep learning, to implement and train the different models. These libraries provided the necessary tools and functions for data preprocessing, model training, and evaluation. Programming environment: The experiments were conducted using Python programming language. Python offers a wide range of libraries for machine learning and data analysis, making it suitable for implementing and evaluating anomaly-detection models.