m ABC : Multi-Agent Blockchain-inspired Collaboration for Root Cause Analysis in Micro-Services Architecture (2024)

Abstract: Root cause analysis (RCA) in Micro-services architecture (MSA) with escalating complexity encounters complex challenges in maintaining system stability and efficiency due to fault propagation and circular dependencies among nodes. Diverse root cause analysis faults require multi-agents with diverse expertise. To mitigate the hallucination problem of large language models (LLMs), we design blockchain-inspired voting to ensure the reliability of the analysis by using a decentralized decision-making process. To avoid non-terminating loops led by common circular dependency in MSA, we objectively limit steps and standardize task processing through Agent Workflow. We propose a pioneering framework, multi-Agent Blockchain-inspired Collaboration for root cause analysis in micro-services architecture (mABC), where multiple agents based on the powerful LLMs follow Agent Workflow and collaborate in blockchain-inspired voting. Specifically, seven specialized agents derived from Agent Workflow each provide valuable insights towards root cause analysis based on their expertise and the intrinsic software knowledge of LLMs collaborating within a decentralized chain. Our experiments on the AIOps challenge dataset and a newly created Train-Ticket dataset demonstrate superior performance in identifying root causes and generating effective resolutions. The ablation study further highlights Agent Workflow, multi-agent, and blockchain-inspired voting is crucial for achieving optimal performance. mABC offers a comprehensive automated root cause analysis and resolution in micro-services architecture and significantly improves the IT Operation domain.

## introduction micro-services architecture (msa) decomposes an application into a series of independent nodes, inter-* corresponding author. acting through lightweight communication mechanisms @xcite @xcite . a key component in maintaining msa is root cause analysis (rca), which aims to find the root cause of alert events and enhance system robustness plays a significant role in avoiding data leaks and program failure analysis @xcite . compared with traditional architectures only containing one central service, rca in microservices architecture (msa) has become extremely difficult as faults continue to propagate between service nodes and alerts become increasingly complex @xcite . in figure 1 , alert event arises on a, while the alert event root cause node is i with fault propagation path @xmath0. rca identifies the root cause of faults, i here, by tracing back to the origin node and even further analyzing metrics. existing approaches such as traceanomaly @xcite , and mepfl @xcite with lack of mechanism are unable to handle circular dependencies (e.g. @xmath1. large language models (llms) like gpt @xcite and their integration with multi-agent exhibit remarkable analytical and problem-solving capabilities, which are essential for identifying and addressing the root cause of fault in complex msa @xcite @xcite . although rca-copilot @xcite , rcagent @xcite , and d-bot @xcite have improved rca tools with event matching and information aggregation, they struggle with the hallucination problem and the common cross-node fault (e.g. @xmath2. to tackle the above issues, we introduce mabc, a groundbreaking framework designed to revolutionize rca. to solve diverse cross-node faults, we introduce multi-agents with diverse expertise and extensive software knowledge to analyze a wide range of data and navigate through node dependencies, which fully consider the propagation of failures in dependencies. to mitigate hallucination in llms, we integrate a blockchain-inspired voting system in the mabc that uses multi-agent collaboration and community voting for high-quality content assurance. inspired by blockchain governance @xcite , this process is transparent and community-driven, enhancing decisionmaking correctness through a decentralized structure. the mabc employs dynamic weight adaptation for fairness and includes penalties for inactive or inaccurate agents, alongside a cap on weight concentration. this ensures accuracy, fairness, and reliability in content generation through decentralized, professional multi-agent assessments and repeated verifications. to address the non-terminating loop led by circular dependency, we objectively limit the number of steps and standardize task processing through agent workflow based on task difficulty and dynamic context perception. by harnessing the power of llms within multi-agent blockchaininspired collaboration, mabc conducts rca and resolution development in msa, unlike previous methods. specifically, 1) an alert event arises due to access function blockages or monitoring system alarms in msa. 2) alert receiver (a 1 ) chooses and forwards the alert event with the highest priority. experimental results on the public aiops challenge dataset and our created train-ticket dataset demonstrate superior performance in identifying root causes and effective resolution development, compared to existing strong baselines. the ablation study further highlights agent workflow, multiagent, and blockchain-inspired voting is crucial for achieving optimal performance. generally, the main contributions of this work are as follows: ## conclusion in this paper, we introduce mabc, a framework that improves alert events resolution in complex msa by combining multi-agent systems, llms, and blockchain voting. we also develop the trainticket benchmark, an open-source dataset for rca in msa. experimental results on the aiops challenge and train-ticket datasets show mabc's effectiveness in identifying root causes and providing solutions, with the agent workflow and voting mechanism being crucial. mabc enhances root cause analysis, boosting system reliability and operational efficiency. future work will focus on enhancing components, incorporating more data sources, and improving agent collaboration, aiming to make mabc essential for it operations.