# ADAL-NN: Anomaly Detection and Localization Using Deep Relational Learning in Distributed Systems
###### Abstract  
Modern distributed systems that operate concurrently generate interleaved logs. Identifiers (ID) are always associated with active instances or entities in order to track them in logs. Consequently, log messages with similar IDs can be categorized to aid in the localization and detection of anomalies. Current methods for achieving this are insufficient for overcoming the following obstacles: (1) Log processing is performed in a separate component apart from log mining. (2) In modern software systems, log format evolution is ongoing. It is hard to detect latent technical issues using simple monitoring techniques in a non-intrusive manner. Within the scope of this paper, we present a reliable and consistent method for the detection and localization of anomalies in interleaved unstructured logs in order to address the aforementioned drawbacks. This research examines Log Sequential Anomalies (LSA) for potential performance issues. In this study, IDs are used to group log messages, and ID relation graphs are constructed between distributed components. In addition to that, we offer a data-driven online log parser that does not require any parameters. By utilizing a novel log parser, the bundled log messages undergo a transformation process involving both semantic and temporal embedding. In order to identify instance-granularity anomalies, this study makes use of a heuristic searching technique and an attention-based Bi-LSTM model. The effectiveness, efficiency, and robustness of the paper are supported by the research that was performed on real-world datasets as well as on synthetic datasets. The neural network improves the F1 score by five percent, which is greater than other cutting-edge models. log parsing; deep relational learning; natural language processing; anomaly detection +
Footnote †: journal: Medical  
interleaved logs as they simultaneously process or execute operations [2]. This can make it difficult to manually analyze the logs and to identify anomalies, particularly as the system grows in size and complexity. Frequently, automated tools and techniques for log analysis, such as machine learning algorithms or statistical analysis methods, are required to address this issue. These tools can help identify patterns and correlations in the log data that may indicate the presence of anomalies, as well as provide insight into the components or subsystems responsible for the anomalies. Maintaining the stability and performance of these complex systems requires the detection of anomalies utilizing neural networks and their localization utilizing relation graphs. Localization refers to the process of identifying the specific components or subsystems responsible for the anomalies. Anomaly detection involves identifying unusual or abnormal system behavior. Recent research has focused extensively on data-driven and log-based anomaly detection, with or without localization [3; 4; 5; 6; 7; 8; 9; 10; 11]. Current methodologies are based on assumptions that are difficult to confirm in a real-world production setting. When employing the aforementioned techniques in a production setting, the following three issues must be considered:  
* Dependencies in logs. In distributed systems, logs are utilized to monitor the statuses and significant events of active instances or entities. Insufficiently, the current methodologies only correlate them with one component. Using log data, developers can identify dependencies between various components or subsystems in a distributed system and use this information to locate anomalies. By correlation of logs with a similar identifier on distributed components, developers can gain a deeper understanding of the relationships and interactions between various system components and use this knowledge to identify and localize anomalies. * Modifying the logs. In reality, log formats in software systems change frequently, by as much as 45 percent over the lifetime of a software system [9; 12]
* Hidden performance problems.