# ADAL-NN: Anomaly Detection and Localization Using Deep Relational Learning in Distributed Systems
## 4 Implementation
### Discussion
Additionally, the method's integration of anomaly identification and localization components further strengthens its robustness by providing a comprehensive understanding of the detected anomalies. Lastly, the proposed method demonstrates superior generalization capabilities. Through the utilization of large-scale datasets such as HDFS, BGL, and Custom, the model is trained on diverse log data, enabling it to learn and generalize patterns across different domains and applications. This generalization capability allows the method to effectively adapt to new and unseen log data, making it highly applicable in real-world scenarios where log data patterns may vary. The incorporation of explainable AI techniques further enhances the model's generalization by providing transparent and interpretable explanations for anomaly detection and localization, ensuring its applicability across various domains and facilitating decision-making processes. In summary, the proposed method offers significant advancements in terms of efficiency, robustness, and generalization. By leveraging optimized techniques, deep learning models, and explainable AI components, the method achieves faster processing, accurate anomaly detection, comprehensive localization, and adaptability to diverse log data, making it a highly promising approach in the field of anomaly detection and localization. This research provides a case study of a real-world system, Hadoop, to further illustrate the identification and localization of anomalies. The \(R\) graph of a Hadoop cluster is illustrated in Figure 6. Hadoop consists of six distinct types. The relationship between types is represented by arrow lines, with the 1:1 relationship indicated by a red dashed line and the 1:n relationship indicated by a black line. For instance, if an application creates multiple blocks, their relationship is 1:n. In our research, the 1:1 correlation is eliminated. Following that, we provide a real-world anomalous instance. To begin, a series of incoming logs progressively launches the \(R\) graph as the \(R_{i}\) graph. In reality, application 01 \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Training Time** & **Real Time** \\ \hline NN & 41.58 & 10.75 \\ \hline SVM & 6.77 & 2.07 \\ \hline IM & 14.76 & 2.0 \\ \hline PCA & 8.55 & 8.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Offline and online time on HDFS data utilizing distinct models for offline and online time. generates a total of 14 blocks and 16 containers. In Figure 7, we only display the simplified \(R_{i}\) graph with three blocks, three containers, one app attempt, one attempt r, and one attempt m due to space constraints. This research first identifies anomalies in log data organized by block ID and labels blk 01 as an anomalous block. Following that, an exception is detected in block 03; consequently, the ADAL-NN identifies block 03 as aberrant as well. Ultimately, the ADAL-NN method detects a killing signal in container 02, which is labeled as flawed. In Figure 7, all of the incorrect instances in A are colored red. The graph \(Ri\) is then subjected to heuristic searching. The ADAL-NN reports the anomalous instances blk 01, blk 03, and container 02 because they are leaf nodes in the \(Ri\) graph. Consequently, we can identify anomalies in Hadoop at the instance level (blk and container in this example). Such information can greatly aid developers in identifying the source of the issue. We can see, for instance, that not all blocks and containers are affected by these anomalies. In slave clusters, we can assume that a node failure is possible. The associated logs of blk 01, blk 03, and container 02 indicate that slave 2 is experiencing a network issue. Therefore, when we encounter anomalies, this research can aid us in swiftly identifying and analyzing root causes.