Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems
Authors: Shaokun Zhang
Ming Yin
Jieyu Zhang
Jiale Liu
Zhiguang Han
Jingyang Zhang
Beibin Li
Chi Wang
Huazheng Wang
Yiran Chen
Qingyun Wu
Venue: ICML 2025 spotlightposter (2025)
Keywords: failure attribution
multi-agent systems.
Primary area: deep_learning->large_language_models

Abstract: Failure attribution in LLM multi-agent systems—identifying the agent and step responsible for task failures—provides crucial clues for systems debugging but remains underexplored and labor-intensive. 
In this paper, we propose and formulate a new research area: automated failure attribution for LLM multi-agent systems.
To support this initiative, we introduce the Who\&When dataset, comprising extensive failure logs from 127 LLM multi-agent systems with fine-grained annotations linking failures to specific agents and decisive error steps.
Using the Who\&When, we develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.  The best method achieves 53.5\% accuracy in identifying failure-responsible agents but only 14.2\% in pinpointing failure steps, with some methods performing below random.  Even SOTA reasoning models, such as OpenAI o1 and DeepSeek R1, fail to achieve practical usability. These results highlight the task's complexity and the need for further research in this area. Code and dataset are available in https://github.com/mingyin1/Agents_Failure_Attribution.

Which Agent Causes Task Failures and When?
On Automated Failure Attribution of LLM Multi-Agent Systems
Shaokun Zhang" *1 Ming Yin 22 Jieyu Zhang' Jiale Liu"
Zhiguang Han Jingyang Zhang? Beibin Li
Chi Wang
Huazheng Wang?* 78 Yiran Chen 2 Qingyun Wu
Abstract
Tasks
LLM Multi-
Failure attribution in LLM multi-agent sys- tems-identiffing the agent and step responsible for task failures- providess crucial clues for sys- tems debugging but remains underexplored and labor-intensive. In this paper, we propose and for- mulate a new research area: automated failure attribution for LLM multi-agent systems. To sup- port this initiative, we introduce the Who&When dataset, comprising extensive failure logs from 127 LLM multi-agent systems with line-grained annotations linking failures to specific agents and decisive error steps. Using the Who& When, we develop and evaluate three automated failure attri- bution methods, summarizing their corresponding pros and cons. The best method achieves 53.5% accuracy in identifying railosleresponsible agents but only 14.2% in pinpointing failure steps, with some methods performing below random. Even SOTA reasoning models, such as OpenAl ol and DeepSeek R1, fail to achieve practical usability. These results highlight the task's complexity and the need for further research in this area. Code and dataset are available in the public repository.
Agent System
Assistant
Coding
Mach
Failure Logs
Manual
Antomateed Palluro Alnbbutii
Necia Sxpatioo
Laomsir Cool
Or
Expert-rree
Efficiene
Good
Good
Fallure Respotsible
Good
Dedisive
Ernur
Agent
Step
Figure When developing LLMs Lowered multi-agent systems, failure attribution atrribution-idenniffiing system components responsible task failures based on evaluation results has received limited attention in existing research. This process is typically performed manually, demanding substantial labor and specialized expertise. In this study, we explore the potential for automating this process.
strated remarkable potential across various domains, includ- ing coding (Hong al., 2023), scientific discovery (Gha- farollahi & Buehler, 2024), and addressing complex real- world challenges (Fourney et al., 2024).
Once constructed, these systems are typically refindd through an iterative process when they fail in specific scenar- ios: evaluation against established benchmarks, followed by manual failure attribution and system refinement. This cycle repeats until the desired outcomes are achieved. Failure attribation-iideniffiin the components of the system that directly lead to task failures-is crucial step that serves the foundation for guiding improvements. Despite its im- portance, this process is largely overlooked in mainstream research, which typically leaves it as a manual task requir- ing significant labor, such as analyzing complex historical logs and navigating the technical intricacies of the system. Moreover, mapping benchmark evaluation results to failure components is heavily dependent domain expertise, im- posing additional requirements practitioners. As systems grow in complexity, this challenge becomes increasingly difficult due to the growing number of components that must considered when conducting failure attribution.
1. Introduction
In recent years, the reframing Large Language Mod- els (LLMs) agents and built agentic multi-agent sys- tems cons-co of interactive, -.o--p agents col- laborating to achieve shared goals-has garnered significant attention (Hong et al., 2023; Li et al., 2023a; Wu et al., 2023). These purposefully designed agentic systems have demon-
'Equal contribution 'Pennsylvania State University DDuke University 'University of Washington "Nanyang Technologi- cal University 'Meta "Google DeepMind 'Oregon State Uni- versity AG2AI, Inc.. Correspondence to: +Shaokun Zhang <shaokun zhang @ psu.edu>>
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Previous manual efforts involve a con-straightforward way

---

Title Suppressed Due to Excessive Size
to facilitate failure attribution in multi-agent systems: de- veloping increasingly fine-graines benchmarks, with the hope that more metrics will enable quicker failure attribu- tion (Zhuge et al., 2024). For example, DevAI (Zhuge et al., 2024) introduces a coding benchmark that incorporates di- verse delivery requirements, offering a more nuanced eval- uation compared to the widely used SWE-Bench (Jimenez et al., 2023), which focuses exclusively on final resolution rates. Despite these advancements, the process of failure attribution based on benchmark results remains a manual process, merely providing more metrics as reference points without fundamentally addressing the underlying challenges. With increasingly comprehensive benchmarks, a fundamen- tal question remains unanswered: which components of the agentic system require improvement?
cific reasons behind failures, which can further facilitate targeted system refinements. We believe that Who& When can serve as foundational resource for driving progress in automated failure attribution research.
Additionally, we construct and evaluate several automated failure attribution approaches on the Who& When. Our find- ings reveal the strengths and limitations of each method, as well as their performance across different conditions, in- cluding model variations, historical context lengths, and the presence or absence of query labels. The results underscore the complexity of using LLMs for failure analysis in multi- agent systems. For example, the best- best-performing method achieved only 8.77% accuracy in identifying decisive error steps within the hand-crafted agentic system.
We argue that evaluation and failure attribution should be tightly integrated, adhering to the principle that "evaluation is not an end in itself, but a means to improvement." (Scriven, 1991) More research efforts should focus on bridging the substantial gap between evaluation results and failure attribu- tion, which currently relies heavily on manual labor. Draw- ing inspiration from the LLM-as-a-judge paradigm (Gu et al., 2024; Zheng et al., 2023), which leverages LLMs to replace human effort in evaluation, we propose to bridge the gap between evaluation and failure attribution by harnessing the comprehensive judgment capabilities of LLMs. Specifi- cally, we propose and formulate a new research problem: au- tomated failure attributions in LLM multi-agent systems. When failures occur under specific scenarios during evalu- ation, the goal is to automatically identify the components responsible for these failures without human intervention. We believe this research could serve as a substitute for man- ual failure attribution, enabling human resources to focus on improving system functionality rather than performing time-intensive diagnostics as shown in Figure!
2. Problem Formulation: Automated Failure Attribution in Multi-Agent Cooperation
In this section, we introduce decisive errors and formu- late the automated failure attribution problem. We adopt the widely-adopted turn-based LLM multi-agent proto- col (Hong et al., 2023; Li et al., 2023a; Wu et al., 2023).
Background. Considering a LLMsspowered multi-agent system M with a group of N agents, denoted as N [1,2,..N), that operate at discrete time steps. These agents are taking actions in a turn-based protocol, meaning that exactly one agent performs an action at each time step. Formally, the system described as:
M= = N, S, S.
M
(I)
Here, S is the set of possible states. A is the global action set; each agent i E A can typically perform actions from some subset A, C A. e(() is a function that indicates which agent is active at time f, thus specifying the turn-based rule. P P(st+1 | St, at, o(t)) is the state-transition probability, given that only one agent s(l) acts at time
To advance the research in this area, we introduce the Who& When benchmark, comprising extensive failure logs annotated with fine-grained failure details for addressing real-world tasks, where these logs are collected from 127 LLMs-powered multi-agent systems. This benchmark in- cludes both algorithmically generated and hand-crafted multi-agent systems, encompassing a wide range of realistic scenarios. Each failure log accompanied by meticulous annotations, specifying the failure-responsible agent, the corresponding failure step, and the reasons for failure in plain language. The primary task involves pinpointing the agent most accontaabl for the failure and the exact step where the error occurred. Predicting the failure-responsible agent serves as a fundamental requirement for failure attri- bution, given that agents are the basic units of multi-agent systems. Extending this to the specific failure step predic- tion imposes higher level of requirement, enabling more fine-grained failure attribution, such as uncovering the spe-
We employ @(t) to denote the agent that takes an action a at time step t. A full trajectory 5 can be written as: T = (so. dos $1, 01, ..., ST), where T is a terminal time step or when the system enters a terminating state.
Decisive Error and Objective. We use tuple (i, to denote mistake in a trajectory, which means agent is active at time t, and its action a is deemed a mistake (e.g.. wrong reasoning etc.). A trajectory may contain multiple mistakes, but not all of them result in overall failure. We employ ZT) to denote the result of a trajectory T.
1, if the system ultimately fails,
7(7)=
(2)
0, otherwise.
2

---

Title Suppressed Due to Excessive Size
Suppose the original trajectory 7 is failure, i.e., ((r) 1. Considering the following scenario, if correcting the mistake made by agent at time t: we replace oe with a "correct" action āz. The steps prior to step t remain unchanged. while the actions following are adjusted accordingly to ensure correctness. This process generates a modified trajectory:
Specifically, each instance in Who& When includes the fol-
lowing entry: (1) Query: A query from GAIA (Mialon et al., 2023) or AssistantBench (Yoran et al., 2024), describing real-world question. (2) Failure log: The full conversation log of a specific system as it fails to solve the query. (3) Agentic system information: For
-errithm--geerttedd sys- tems, including system prompts, tools, and agent names, all tailored to this specific query. (4) Annotations: An annota- tion of the agent responsible for task failure, specifying the step where the failure occurred, along with plain-language explanation of why the failure took place. An example of the instance in this benchmark could be found in Appendix C.
70.0 (T),
(3)
where I(a,)) denotes the intervention. If in the modified trajectory we obtain Z(7(4.0) = 0 (success), then the error (i, is said to be a decisive error. Formally, we define the decisive error indicator (T)(T) as
To better reflect our definition of decisive error in Section 2, we design three metries to evaluate the performance of vari- ous failure attribution methods: (1) Agent-Level Accuracy: This metric measures the percentage of correctly predicted failure-responsible agents by failure attribution algorithms. (2) Step-Level Accuracy: This metric quantifies the per- centage of correctly identified decisive error steps. It im- poses higher requirements on the algorithms compared to the first metrie. (3) Step-Levvel Accuracy with Tolerance: To account for slight deviations, this metric allows toler- ance range for mistake step predictions. If the predicted step falls within the specified tolerance range of the actual mistake step, the prediction is considered correct.
11, if = 1 and ((F(10)) = O, (.,((T)
(4)
0, otherwise.
In words, (T, =1
Fixing agent i's mistake at time & changes ((T) from 1 (fail) to 0 (success). For- mally, the decisive error could be defined as agent-time pairs (i", (*) such that (r) = 1, where i* represents the agent responsible for the system failure, and t" repre- sents the exact time step at which the critical mistake occurs. agent and the
We refer to these as the miilure-responsiiee decisive error step, respectively across the paper.
In practice, multiple decisive errors may occur within tra- jectory. In our study, we address this situation by identifying the earliest error in time as the principal cause of failure. Specifically, we define an objective to determine:
3.1. Agentic Systems Constructions
Who&When includes two types of agentic systems: algorithm-generacee agentic systems and one meticulously hand-crafted agentic systems, totaling 127 agentic systems equipped with diverse tools for evaluation.
C(T) ((,,1) 1 (,, = 1}.
(5)
(i', (*) = arg min (
1.
A Algorithm-Generated Agentic Systems. To ensure an adequate number of agentic systems for the Who& When datasets, we first employ the CaptainAgent algorithm (Song et al., 2024) from the AG2 library to automatically gen- erate agentic systems for each data instance sourced from the validation sets of the GAIA (Mialon et al., 2023) and AssistantBench (Yoran et al., 2024) benchmarks. Specifi- cally, it constructs a team of agents tailored to given task, assigning appropriate agent names, prompts, and necessary tools. The system iteratively optimizes the agents' con- figuration until the task is successfully completed. In the Who& we select only the final multi-agent config- urations, along with the corresponding execution history, as these represent the optimized solutions for each query. All agents within the constructed systems, as well as the CaptainAAeett algorithm itself, are based on the CPT-4o on 2024 10-000--10000 -1l-previie version. Additionally, since the primary objective of the Who& When is to capture mis- takes made by agents that lead to failures in solving real-
which selects the pair (i", t*) yielding the highest decisive error indicator with earliest time step. In this study, the research problem focuses on the automatic identification of the (i", (t) in LLMs-powered multiple agent systems.
3. The &ho& When Dataset
To advance research in this area, we introduce a dataset called Who&When. This dataset comprises extensive fail- ure logs from 127 LLM multi-agent systems including both Aggoritmm-geeeratee and human-crafted systems. These logs are carefully annotated with labels that identify the failure- reponsible agents and the decisive error steps in agent co- operation directly responsible for moollem-solving failures. Additionally, each annotation is supplemented with natural language explanations, culminating in 184 distinct failure annotation tasks. The dataset is specifically designed to detect the failure-reponsibe agents (who) and the corre- sponding steps (when) within each failure log.
hntp://gihhhhccomaggaiingg
3

---

Title Suppressed Due to Excessive Size
50
30
Algecilim Genersted Systems
204nd84n Bn3d0wmm Odmfogg
Annotion Cost (bour)
25-
Hand Crafted Systems
Uncertainty Rate (%)
33.3
50.0
20-
18.2
25-
12.5
14.
23,0%
2.1.1%
16.7
N/A
16-
10-
10-
N/A
10.0
18.2
20- Rn
0dmlogp3
204m68444 4n
Re:diwingg
Admiogps3
204nd84n 8r3d0wmg Odmlogg3
(a) Annotation labor cost.
(b) Uncertain annotation percentages.
(c) Disagreement rates in voting.
Figure 2. Statistical analysis of the annotation process: (1) Total labor for annotations in human hours. (2) The proportion of uncertain annotations to total annotations during the second round. (3) Initial disagreement rates between annotators (note that we make sure reach consensus through careful discussion and voting process afterwards). These results highlight the challenges involved in performing manual failure attribution.
world problems within agentic systems, we retain only those agentic systems that fail to successfully address the queries associated with each data instance from these benchmarks.
reasoning behind the mistake in natural language. Addition- ally, each expert is required to categorize their annotations into two groups: those they are undoubtedly confident are correct and those they have any uncertainty about. Round I:: In the second round, people are instructed to make an agreement on all the uncertain annotations from Round II. For these uncertain annotations, rative discussion to reach a consensus. We do not simply follow the principle of majority rule; instead, we aim to ensure that everyone is persuaded and that consensus is ultimately reached. Round III: In the final round, a cross- validation procedure employed. Each expert is asked to go through another expert's annotations assess the con- sistency of the annotation standards. If any discrepancies issues with the annotations are identified, the experts engage in further discussion and, if necessary, re-annotate the data according to the established guidelines until a con- sensus is reached. Incorporating the viewpoints of multiple annotators and ensuring consensus among them, we aim to accurately reach the actual ground truth, as suggested by previous studies (Clemen, 1989; Zhuge al., 2024).
Hand-Crafted Agentic Systems. In addition algorithm- generated systems, Who&When also includes meticu- lously hand-crafted, mature multi-agent system, Magnetic- One (Fournyy et al., 2024), to ensure the representation of realistic and highly refined agentic systems. Magnetic-One generalist agentic system designed to handle a broad range of tasks. comprises five carefully crafted agents, each specializing in distinct capabilities, such operat- ing web browser navigating local files. We evaluate Magnetic-One using the validation set from the Assistant- Bench (Yoran et al., 2024) benchmark, aggregating its fail- ure logs for subsequent annotation. We also test Magnetic- One on a randomly sampled subset of 30 instances from the GAIA (Mialon et al., 2023), incorporating the correspond- ing execution failure logs into the dataset. We exclude the rest of the GAIA dataset due to the complexity of annotating the long context logs produced by Magentic-One.
engage in collabo-
3.2. Decisive Error Annotation
3.3. Analysis
After obtaining the failure logs of various agentic systems, we introduce annotation procedure to identify the deci- sive error failure and decisive eтror step. To ensure precise annotation, we conduct multiple rounds of annotation per- formed by three human experts in Al agent (whose identities are anonymized as Odmfogp3, 8n3d0wmg, and 204nd84n).
Annotating the decisive error agent and identifying the spe- eific step of the error is challenging for both non-expert people and domain experts. The annotators must parse com- plex logs, follow the problem-solving logic of each agent, and assess whether each action is correct or if it misleads the entire problem-solving process. For example, if an agent uses web browser to gather essential information for problem-solving, annotators must check the browser history and visit each website determine whether the failure is due to unavailable information the website or because the agent failed to retrieve it. As shown in Figure 2(a), three annotators spent 30.9, 30.2, and 23.2 human hours, respec- tively, to complete the annotations. This demonstrates that
Round I: the first round, we distribute the failure logs from all agentic systems for each query equally among three experts. To ensure consistency, we provide the experts with standardized annotation guideline as shown in Appendix F. Each expert is tasked with annotating three elements: the single erroneous agent primarily responsible for the task failure, the specific step at which the error occurred, and the

---

Title Suppressed Due to Excessive Size
the annotation process is very time-consuming, leading us to consider doing research on automated failure attribution.
the query and the full failure log, the LLM is tasked with determining whether the mistake occurred in the upper half or lower half of the failure logs. Once this decision is made, the LLM is provided with the selected half of the log and the process is repeated iteratively until a single step is identified. The three algorithms and corresponding prompts are detailed in Appendix A and Appendix G.
Additionally, in many data instances, it's not just one agent that makes mistakes, but several agents. People need to iden- tify these mistakes and select the most severe ones, which can directly lead to problem-solving failures as formulated in Section 2. Since the severity of mistakes may be subtle and even subjective at times, the process becomes even more difficult. As shown in Figure 2(b), we present the uncertain annotation percentages for three individuals. The uncertain percentages across different annotators range from 15% to 30%. We also visualize the disagreement rates between different individuals when voting on each other's s uncertain data in Figure 2(c). We can see some disagreement remains before discussing to make the agreement, further highlight- ing the difficulties involved in the annotation process.
4.2. Settings
Scenario. We conduct experiments under two distinct settings to simulate various realistic scenarios. (1) With Ground Truth: In this setting, the final ground truth of the query that the agentic system is attempting to resolve is available to the LLMs. Our focus here is on the typical AI system development cycle, where it is common practice to use a development dataset with ground truth to identify and debug potential errors in experimental systems. (2) Without Ground Truth: In the second setting, the final ground truth of the query unavailable. In this scenario, LLMS are employed perform failure attribution in agentic systems based their running logs. This capability can also be viewed as form of self-reflection (Huang et al., 2023; Shinn et al., 2024), which contributes to the improve- ment of multi-agent systems. Throughout this paper, unless otherwise specified, all results are reported as the average accuracy across these two scenarios.
4. Can LLMs help identify When and Which agent causes task failures?
As revealed in Section 3.3, detecting the failure- railure-repoonsible agent and corresponding failure step in agentic system are often subtle, requiring significant human effort. Given these challenges, we were thinking of performing automated fail- ure attribution, using LLMs themselves to detect these errors and provide signal for human to perform essential improve- ment. In this section, we set up experiments to answer fundamental question: Can LLMs help identify when and which agent causes task failures in multi-agent systems?
Models. The primary experiments are conducted using the GPT-4c model, unless otherwise specified. Addition- ally, we also incorporate several other models, including both open-source (such as the Llama and Qwen series) and closed-source models (CPT series), to ensure the con- sistency of the conclusions drawn from the experiments. Additionally, we employ advanced reasoning models, i.e.. OpenAl ol and DeepSeek RI, assess the performance of reasoning models failure attribution tasks. The results of these evaluations are provided in Appendix 4.8.
4.1. LLMs for Failure Attribution in Agentic Systems
To answer the question mentioned above, we propose three judgement methods for automated failure attribution in agen- tie systems. Through extensive experiments, we demon- strate that each method has distinct advantages and limita- tions, and they can be applied either independently or in combination. Furthermore, we analyze the performance of these methods across various scenarios and constraints, highlighting their applicability in different contexts.
4.3. Overall Performance
(1) All-at-once: An LLM is provided with a query and the complete failure log, and it is tasked with identifying the failure-responsible agent as well as the specific step where the decisive error occurred. (2) Step-by-step: An LLM is provided with a query, and the failure log is presented step- by-step. At each step, the LLM is tasked with determining whether a mistake has occurred in the current step. If mistake is identified, the judging process terminates, and the responsible agent's 's name along with the current step number are returned as the output. Otherwise, the process continues until the final step is reached. (3) Binary search: Alternatively, this method uses a receptive field approach that lies between the previous two methods. Starting with
We first perform experiments to compare the performance of three failure attribution methods on Who& When dataset with GPT-4o model. The results are reported on Table 1.
Agent-Level Accuracy Relies on Large Receptive Field. As shown in Table 1, all-at-once significantly outperforms the other two failure attribution methods in agent-level ac- curacy. Specifically, its agent-level accuracy is 19.13% and 20.69% higher than step-by-step when judging with ground truth, and 25.1 5% and 20.69% higher when judging without ground truth, respectively. The performance of the binary search method falls between these two approaches.
whenjudging
These results can be attributed to the fact that predicting the
5

---

Title Suppressed Due to Excessive Size
With Ground Truth
Without Ground Truth
Agentic Systems Types
Algorithm Generated
Hand Crafted
Algorithm Generated
Hand Crafted
Random
Agent-Level Accuracy
29.10
12.00
29.10
12.00
Step-Level Accuracy
19.06
4.16
19.06
4.16
All-at-Once
Agent-Ievel Accuracy
54.33
55.17
51.12
53.44
Step-Level Accuracy
12.50
5.26
13.53
3.51
Step-br by-Step
Agent-Level Accuracy
35,20
34.48
26.02
32.75
Step-Level Accuracy
25.51
7.02
15.31
8.77
Binary Search
Agent-Level Accuracy
44.13
51.72
30.1
36.21
Step-Level Accuracy
23.98
6.90
16.59
6.90
Table 1. Performance of the three failure attribution methods the Who& When dataset with and without labels. evaluated on the GPT-4o model. For agent-level accuracy, all-at-once outperforms binary search, which in turn surpasses step-hy-step. Conversely, for step-level accuracy, step-hy-step achieves the best performance, followed by binary search and then all-at-once.
All-at-Once
Step-by-Step
Binary Search
Fine-Grained Predictions Boost Better Step-Level Ac- curacy. In terms of step-level accuracy, the all-at-once approach performs obviously the worst, even with its aver- age performance falling below that of random method. This outcome underscores the method's practical infeasibility. By contrast, the step-by-step approach achieves the highest performance, outperforming other methods in step-level ac- curacy
Agent-Level Acouracy (%) GPT-4-turbo
Step-Level Accuracy (%)
GPT-4-turbo
Qwen-2.5
GPT4o-mini 2.5-22
GPT-Ao-mini
of 4 cases. The performance of the binary search method still falls between these two approaches.
Qwen-2:5-78
Ll1-8-3.1-8b Qwen-2.5-76
Llamb-3.1-8b
These two results can be attributed to the 'space-in-the needle' problem, where LLMs often struggle to retrieve specific information from long contexts (Nelson et al., 2024). The all-at-once method has access to the largest context when making decisions about the decisive error step, but this lead diffieulties in pinpointing the exact failure step within long history. In contrast, the step-by-step method processes the context incrementally, allowing for more focused decision decision-making. The binary search method performs at a level between these two approaches.
Figure 3. Performance comparison of three failure attribution meth- ods different models in both two metrics. We found the conclu- sion is mostly consistent with Table 1.
failure-responsibbl agent requires the judge L.MSs sider broader context, including the behaviors of multiple agents. Since all-at-once has access to the entire conver- sation log when making the final judgment, its prediction of the failure-responsible agent is more accurate. In con- trast, the step-by-step method processes the conversation history incrementally, while the final decision can be made with incomplete information, thus resulting in lower perfor- mance. Moreover, all failure attribution methods outperform the random baseline, highlighting that these approaches are nontrivial and affirming the necessity of involving LLMs for failure attribution.
Incrementally processing context en- ables better step-level failure attribution since LLMs struggle to retrieve information from long contexts.
Finding 2:
Impact of Ground Truth on Failure Attribution. We also observed that failure attribution accuracy is higher for three methods when ground truth are available, compared to when judgments are made without ground truth in all cases all metrics. Although the answers to users' queries may not serve as definitive 'golden labels' for each agent's correct behavior, they provide useful reference signal for the judgment LLMs. For instance, if an agent leads the system in completely wrong direction, with no possibility of reaching the correct final answer, the label information can directly help alert the judgment LLMS this error.
Providing broader failure log context enables more accurate agent-level failure attribution by incorporating more complete information.
Finding 1:
6

---

Title Suppressed Due to Excessive Size
Al-l-Orce
120
Step Stp
many Search
Accuracy (%)
Distance
Distanca
80
Accuracy
Length Level
Length Level
Aggorihm-GGenrraeed
(b) Hand-Crafted
(a) Agent-Level Accuracy
(b) Step-Level Accuracy
The distances between human- annotated decisive error steps and the predicted steps for each date instance on failure logs from both algorithm -egierattgee and hand-crafted systems.
Figure 5.
Figure 4. Comparison of three failure attribution methods applied failure logs from hand-crafted systems the Who& When, evaluated under varying failure log lengths across both metrics.
than agent-level accuracy. Furthermore, the step-by-step performance decline particularly pronounced compared to the other two. We also analyze the distances between annann--dntttedd decisive error steps and the predicted steps for each data instance, as shown in Figure 5. These results demonstrate that the step-by-step method outperforms the other two methods in accurately predicting the decisive error steps. However, as context length reaches its maximum, all three failure attribution methods converge to near 0%, as shown in Figure
Without such intervention, the entire system might proceed in the wrong direction without any external warning.
Consistency of Conclusions Across Various LLMs. addition the GPT-4o model, conducted evaluations on other LLMs, including open-source models (e.g., the Llama series and Qwen series) well closed-source models (e.g., the GPT series). Due to the significant compu- tational and token costs, we only perform experiments on hand-crafted agentic systems from Who& When which has fewer failure logs. The results of three methods are shown in Figure We found that the phenomena observed in Table hold consistently across different LLMss. Specifically, for agent-level accuracy, the ranking is: all-at-once, followed by binary search, and then step-by-step. Conversely, for step-level accuracy, the ranking is: step-by-step, followed by binary search, and then all-at-once.
Failure attribution performance de- clines as context length increases, with step-level accuracy being more sensitive.
Finding 4:
4.5. Step-Level Accuracy Under Different Tolerances
Toler.
All-at-Once
Step-by-Step
Binary Search
+ 12.07
14.66
13.79
2 19,83
16.38
18.97
The pros and cons of different failure attributions methods in this study mostly consis- tent across different LLMs.
Finding 3:
+
30.17
18.10
22.41
+ 4
37.07
31.90
31.89
5 43.10
33.62
36.21
4.4. Performance Across Varying Context Lengths
Table 2. Step-level accuracy with different tolerances on the failure logs of hand-crafted agentic systems from Who& When dataset.
investigate the relationship between the length of failure logs and the corresponding failure attribution performance. Specifically, the failure logs of hand-crafted agentic systems from the Who& When dataset divided into five levels, with context length progressively increasing from Level to Level 5. Specifically, Level spans 5-17 steps, Level 2 covers 19-29, Level 3 includes 31-49, Level ranges from 51-91, and Level 5 spans 93-130 steps. Both agent- level and step-level judgment performances across the three evaluation methods are presented Figure Algorithm- generated systems excluded from this analysis due to their limited maximum step count of 10, which prevents meaningful divisions of context length.
In practice, directly identifying the exact decisive step is not always necessary; is often sufficient determine range of steps where the mistake might occur. In this section, we show the performance of the three failure attribution methods under varying tolerance conditions on the failure logs hand-crafted agentic systems from the Who& When dataset. Algorithm-generant
systems are excluded from this analysis because their maximum step count is limited to 10, and increasing the tolerance would lead artificially inflated accuracy.
As shown in Table 2, our findings show that step-by-step achieves the highest performance when the tolerance is set to or 1. However, as the tolerance increases, the advantages of all-at-once become more pronounced, while the benefits
Our findings indicate that all three methods exhibit decline in both metrics as context length increases. Notably, step- level accuracy is more sensitive to context length changes

---

Title Suppressed Due to Excessive Size
Ground
All-at-Once
positive, all three methods still yield meaningful insights from statistical perspective. In practice, these statistical re- sults provide more actionable basis for system refinement compared to focusing solely on single data instances.
Hisbale Percentege (50)
50-
Ground Tuth
Mizzlk Porcenaue (56)
All-ac- ncc
30-
10-
20-
Finding 6:
The three baseline methods are more effective at performing failure attribution at statis- tical level than at an instance level.
Step-by-Step
Binary Search
Notall Percentage (h)
Step-ity Step
Nizala Percentage (36)
Binery Search
4.7. Can We Combine Multiple Failure Attribution Methods?
30-
10
Cost Token Num
Agent-Level Step-Level
Metrics
Accuracy
Accuracy
Binary Search
34,659
43.97
6.90
& All-at-Once
17,106
57.02
4.39
Step-by-Step 87,720
35.96
7.90
Figure 6. Histogram of the actual and predicted failure-responte responsbbe agents for all three methods. We present only the failure logs of hand-crafted systems in Who&When aggregate the largest number of results for one multi-agent system. Number 0, 1, 2, 3 represents Assistant. Fintt,iileee wetisstttt,ieiiiiiiee FileSurfer, Orchestrator and webSur fer respectively.
Hybrid Method (T&A)
149.177
57.02
12.28
Table 3. Comparison of the three failure attribution methods with
hybrid approach that combines all-at-once and step-by-step on the failure logs of hand-crafted systems from the Who&Wher dataset. The hybrid method achieves the highest performance in both two metries but incurs the highest token costs.
of step-by-step diminish. Compared to all-at-once, step-by- step demonstrates better alignment with accurate predictions when high precision is required.
We then investigate whether hybrid method could leverage the advantages of both two different methods, all-at- all-et-once and step-by-step. The former excels at failure-responsible agent predictions, while the latter better at accurately predicting the decisive error step. Specifically, we start by prompting all-at-once to predict the rislureeresponsible agent and then use step-by-step detect the mistake step in the actions step taken by the identified failure-responsibe agent. To evaluate this, we perform experiments on the hand- crafted systems from the Who& When dataset considering the token cost. The results are shown on Table
Allowing tolerance in failure attribu- tion enables broader context processing methods to achieve competitive step-level accuracy.
Finding 5:
4.6. A Statistical Viewpoint on Failure Attribution
This study primarily perform experiments single-data- level failure attribution in LLM-powered multi-agent sys- tems, i.e., identifying the specific component (referred to as the failure-ne resiurs-beppo agent) and the precise location (the decisive error step) responsible for task failure in single data instance. This practice indeed mirrors human proce- dures for failure attribution and could serves as founda- tional tool for deriving statistical-level conclusions. There- fore, we think of whether these methods could be applied to entire datasets to extract meaningful statistical results.
We observe that the hybrid method outperforms all methods in step-level accuracy. This improvement is attributed to the all-at-once narrowing the range of possible failure steps by excluding action steps taken by other agents, thereby significantly reducing the difficulty of prediction for step- by-step. However, the hybrid method comes with notable drawback: requires running two algorithms sequentially, Compared to making judgments with single algorithm, this approach incurs higher computational costs.
In Figure 6, we show the histogram of actual and the pre- dicted failure-responsible agents for all three methods. We only show the failure logs of hand-crafted systems from Who& When to aggregate the largest number of results for one system type. We observe that the single agent that make the most decisive emrors predicted by all methods to con- sistent with the ground truth (agent 3). Moreover, the top two failure-responsibl agents predicted by three methods are also consistent with the ground truth in most cases (2 out of 3). These experiments demonstrate that, although the instance-level failure attribution results are not highly
Combining different failure attribution methods allows leveraging their respective strengths for better performance.
Finding 7:
4.8. Strong Reasoning Model for Automated Failure Attributions
We fianlly examine whether reasoning models OpenAl ol and DeepSeek RI (DeepSeek-AL. 2025) can enhance the automated failure attribution process. However, the original
8

---

Title Suppressed Due to Excessive Size
GPTT-0
OpenAI ol
DeepSeek R1
Accuracy Agent-Level Level Agent-Level Step-Level Agent-Level Step-Level
All-at-Once
54.31
4.39
41.38
10.34
56.90
3.45
Step-by-Step 33.62
7.90
36.21
13.79
32.76
6.90
Table 4. The performance of the automated failure attribution methods with reasoning mechanism with strong reasoning models.
prompt used in our experiments was flagged by OpenAT's policy as violating usage guidelines. Therefore, we imple- mented minor modifications to the prompt while preserving its original intent. For DeepSeek RI, we employed the same prompt as used in other experiments ensure consistency. The results are shown in Table 4.8. We don't 't include binary search because it doesn't include reasoning mechanisms in their prompt. We perform experiments on hand-crafted agentic systems of Who& When. The results indicate that stronger reasoning models do not necessarily outperform standard models. Although it provides some improvement, but still far from practical usability. For instance, DeepSeek RI underperforms GPT-4o in three out of four cases, and OpenAl ol fails to consistently surpass GPT-4o across all metrics. These findings highlight the inherent challenges of failure attribution. In contrast, integrating reasoning mechanisms into the prompt yields significant performance improvements across all metrics and cases, as shown in Figure 7. This demonstrates that replacing the base model alone does not guarantee better outcomes.
van Schaik & Pugh (2024), who employ LLMs as evalua- tors in the context of text summarization which also heavily relies on human efforts. In the field of agentic systems, related research includes Shinn et al. (2024), who adopt the concept of LLMs-as-judges to analyze task feedback signals and guide corrective actions. Similarly, Zhuge et al. (2024) demonstrate the use of LLMS to provide detailed evaluations of agentic systems within their proposed De- vAI dataset. Despite these advancements, failure attribution remains a manual process, with evaluation results serving only as a reference for such attributions
Reward Models Most reward models (RMs) are designed either to predict human preference rankings for outputs gen- erated by large language models (Zhong et al., 2025) or to evaluate the reasoning process step by step, rather than assessing only the final answer (Cui et al., 2025; Lightman et al., 2023; Wang et al., 2023; Zheng et al., 2024). A num- ber of studies have proposed training process-level reward models that evaluate the correctness of intermediate rea- soning steps produced by a single LLM (Cui et al., 2025; Lightman et al., 2023). For instance, Math Kath-Shepherd (Wang et al., 2023) employs automatically generated supervision data to assign reward scores to each step in solving math- ematical problems. Similarly, ProcessBench introduces a benchmark of step-by-step solutions annotated by human experts, identifying the location of errors within mathemati- cal problem-solving processes. In this setting, models are tasked with detecting the earliest erroneous step or confirm- ing that the entire solution is correct. However, these works focus primarily on constructing reward models for evaluat- ing the outputs of individual L.M,s, rather than identifying the errors in complex agentic systems.
5. Related Works
LLM Multi-Agent Systems. An emerging research focus examines using LLMs (Achiam et al., 2023; Wang et al., 2024) as central controllers to develop LLM agents that inter- act with the external world beyond text domains (Deng et al., 2024; Xie et al., 2024; Zhang al., 2024b; 2025). While single-agent systems (Yao et al., 2022; Zhang et al., 2023a; 2024a) excel in specific tasks, they struggle with challenges requiring collaboration and collective intelligence. To ad- dress this, studies have explored LLM--ooered multi-agent systems, where multiple interactive agents work concur- rently (Hong et al., 2023; Li et al., 2023a). These systems leverage the specialized skills and roles of individual agents, enabling collaborative -oblin for complex tasks by simulating real- wolld cooperation patterns.
6. Conclusion
In this study, we propose and formulate a new research area: automated failure attribution in LLM multi-agent systems, an area that has been largely overlooked in current research. To advance this field, we introduce the Who& When dataset, which consists of 127 multi-agent systems with extensive failure logs meticulously annotated with failure details. Fur- thermore, we develop and evaluate three automated failure attribution methods, highlighting the challenges and com- plexities of this task. Our findings underscore the significant difficulty of automated failure attribution and emphasize the urgent need for further research in this emerging area.
LLM for Judging. Numerous studies have explored the use of large language models (LLMs) as evaluators as- sess various tasks based on pre-detined standards (Fu al., 2023; Gu et al., 2024; Hu et al., 2024; Li et al., 2023b; Liu et al., 2023; Thakur et al., 2024). For instance, Chan et al. (2023); Zheng et al. (2023) utilize LLMs to evaluate the performance of LLMs in chat conversation scenarios, which would otherwise incur significant labor costs if performed by humans. Another notable example Miao et al. (2023);

---

Title Suppressed Due to Excessive Size
Impact Statement
Gu, J., Jiang, X., Shi, Z.,, Tan, H., Zhai, X., Xu, C.,, Li, W., Shen, Y., Ma, S., Liu, H., al. A survey llm-as-a- judge, arXiv preprint arXiv:
Our approach has societal implications, both positive and negative. On the positive side, our work contributes to the efficient development of multi-agent systems powered by LLMs, enabling their application across wide range of do- mains. Incorporating mechanisms for failure attribution and conduct corresponding improvement, these advancements have the potential to enhance LLM multi-agent systems significantly. However, the work also introduces potential risks. For instance, granting these systems the ability to modify external environments, such as executing code on computers, could lead to unintended consequences.
2024.
Hong, S., Zheng, X., Chen, J., Cheng, Y., Wang. J., Zhang. C., Wang. Z., Yau, S. K. S., Lin, Z., Zhou, L., et al. Metagpt: Meta programming for multi-agent collabora- tive framework. International Conference on Learning Representations, 2023.
Hu, Z., Zhang, J., Xiong, Z., Ratner, A., Xiong, H., and Krishna, R. Language model preference evalu- ation with multiple weak evaluators. arXiv preprint arXiv:2410. 12869, 2024.
References
Huang, J. and Chang, K. C.-C. Towards reasoning in large language models: A survey, In Findings of the Associa- tion for Computational Linguistics: ACL 2023, 2022.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L.,, Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., and Zhou, D. Large language mod- els cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798, 2023.
Chan, C.-M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., Fu, J., and Liu, Z. Chateval: Towards better evaluators through multi-agent debate. International Con- ference on Learning Representations, 2023. 20233.
Jimenez, E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. Swe-bench: Can language mod- els resolve real-world github issues? International Con- ference on Learning Representations, 2023.
Clemen, R. T. Combining forecasts: A review and annotated bibliography. International journal forecasting, 1989.
Cui. G., Yuan, L., Wang. Z., Wang, H., Li, W., He, B., Fan, Y., Yu, T., Xu, Q. Chen, W., et al. Process re- inforcement through implicit rewards. arXiv preprint arXiv:2502.01456. 2025.
Li, G., Hammoud, H., Itani, H., Khizbullin, D., and Ghanem, B. Camel: Communicative agents for" mind" exploration of large language model society. Advances Neural In- formation Processing Systems, 36:5 06:51991-52003 2023a.
DeepSeek-AL. Deepseek-rl:
Incentivizing reasoning ca- pability in lims via reinforcement learning deepseek-ai. arXiv preprint arXiv:2501.1294 2025.
Li, X., Zhang, T., Dubois, Y., Taori, R., Gulrajani, I., Guestrin, C., Liang, P., and Hashimoto, T. B. Alpacaeval: An automatic evaluator of instruction-following models, 2023b.
Deng, X., Gu, Y., Zheng, B., Chen, S., Stevens, S., Wang, B., Sun, H., and Su, Mind2web: Towards a general- ist agent for the web. Advances in Neural Information Processing Systems, 2024.
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. In International Conference on Learning Representations, 2023.
Fourney, A., Bansal, G., Mozannar, H., Tan, C., Salinas, E., Niedtner, F., Proebsting, G., Bassman, G., Gerrits, J.. Alber, J., al. Magentic-one: A generalist multi- agent system for solving complex tasks. arXiv preprint arXiv:2411.0442 2024.
Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., and Zhu, C. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303. 16634, 2023.
Fu, J., Ng. S.-K., Jiang, Z., and Liu, P. Gptscore: Evaluate as you desire. Proceedings of the 2024 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, 2023.
Mialon, G., Fourrier, C., Swift, C., Wolf, T., LeCun, Y., and Scialom, T. Gaia: a benchmark for general ai assistants International Conference Learning Representations, 2023.
Miao, N., Teh, Y. W., and Rainforth, T. Selfcheck: Using Ilms to zero-shot check their own step-by-step reasoning. International Conference on Learning Representations, 2023.
Ghafarollahi, A. and Buehler, M. J. Sciagents: Automating scientific discovery through aulti- intelligent graph reasoning. arXiv preprint arXiv:2409.0 05556, 2024.
10

---

Title Suppressed Due to Excessive Size
Nelson, E., Kollias, G., Das, P., Chaudhury, S., and Dan, S. Needle in the haystack for memory based large language 2024.
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 2024.
models, Xrxiv preprint
Scriven, M. Evaluation thesaurus. Sage Publications, 1991.
Yoran, O., Amouyal, S. J., Malaviya, C., Bogin, B., Press, O., and Berant, J. Assistantbench: Can web agents solve realistic and time-consuming tasks? In Proceedings of the 2024 Conference on Empirical Methods in Natural Lunguage Processing, 2024.
Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: Language agents with verbal rein- forcement learning. Advances in Neural Information Processing Systems, 2024.
Song, L.,, Liu, J., Zhang, J., Zhang, S., Luo, A., Wang, S., Wu, Q. and Wang, C. Adaptive in-conversation team building for language model agents. arXiv preprint arXiv:2405. 19425, 2024.
Yu, T. and Zhu, H. Hyper-parameter optimization: A review of algorithms and applications. arXiv preprint Siii:20003056883 2020.
Zhang, J., Krishna, R., Awadallah, A. H., and Wang, C. Ecoassistant: Using I1m assistant more affordably and accurately. arXiv preprint arXiv:2310.03046, 2023a.
Thakur, A. S., Choudhary, K., Ramayapally, V. S., Vaidyanathan, S., and Hupkes, D. Judging the judges: Evaluating alignment and vulnerabilities in Ilms-as- judges. arXiv preprint arXiv:2406. 12624, 2024.
Zhang, S., Jia, F., Wang, C., and Wu, Q. Targeted hyper- parameter optimization with lexicographic preferences over multiple objectives, In International Conference on Learning Representations, 2023b.
van Schaik, T. A. and Pugh, B. A field guide to automatic evaluation of Ilm tim-generated summaries. In Proceedings of the 47th International ACM SIGIR Conference on Re- search and Development in Information Retrieval, pp. 2832-2836, 2024.
Zhang. S., Zhang, J., Ding, D., Garcia, M. H., Mallick, A., Madrigal, D., Xia, M., Rühle, V., Wu, Q., and Wang. C. Ecoact: Economic agent determines when to register what action. arXiv preprint arXiv:241
Wang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., Tang, X., et al. A comprehen- sive survey of small language models in the era large language models: Techniques, enhancements, applica- tions, collaboration with Ilms, and trustworthiness. arXiv preprint ar arXiv:2411.033300 2024.
2024a.
Zhang. S., Zhang, J., Liu, J., Song, L., Wang, C., Krishna, R., and Wu, Q. Offline training of language model agents with functions as learnable weights. In International Conference on Machine Learning, 2024b.
Wang, P., Li, L., Shao, Z., Xu, R., Dai, D., Li, Y., Chen, D., Wu, Y., and Sui, Z. Math-shepherd: Verify and reinforce Imms step-by-step without human annotations. arXiv preprint arXiv:2312.08923 2023.
Zhang. S., Dong, Y., Zhang, J., Kautz, J., Catanzaro, B., Tao, A., Wu, Q., Yu, Z., and Liu, G. Nemotron- tool-nl: Tool-using language models with reinforced rea- soning. arXiv preprint arXiv:2505.00022 2025.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thoughh prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, pp. 24824-44837, 2022.
Zheng, C., Zhang, Z., Zhang, B., Lin, R., Lu, K., Yu, B., Liu, D., Zhou, J., and Lin, J. Processbench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559, 2024.
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Zhang, S., Zhu, E.,, Li, B., Jiang, L., Zhang, X., and Wang, C. Autogen: Enabling next-gen Ilm applications via multi-agent conversation framework. Conference on Language Modeling, 2023.
Zheng, L., Chiang, W.-L.,, Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z.., Li, Z., Li, D., Xing. E., et al. Judging Ilm-as-a-judge with mt-bench and chatbot arena. Ad- vances in Neural Information Processing Systems, 2023.
Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T. J., Cheng, 7., Shin, D., Lei, F., et al. Os- world: Benchmarking multimodal agents for open-ended tasks in real computer environments. arXiv preprint arXiv:2404.07972. 2024.
Zhong, J., Shen, W., Li, Y., Gao, S., Lu, H., Chen, Y., Zhang, Y., Zhou, W., Gu, J., and Zou, L. A comprehensive survey of reward models: Taxonomy, applications, challenges, and future. arXiv preprint arXiv:2504.12322 2025.
Zhuge, M., Zhao, C., Ashley, D., Wang, W., Khizbullin, D., Xiong, Y., Liu, Z., Chang, E., Krishnamoorthi, R., Tian, Y., al. Agent-as-a as-a-judge: Evaluate agents with agents, arXiv preprint arXiv:2410.100 10934, 2024.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. React: Synergizing reasoning and acting in language models. ICLR, 2022.

---

Title Suppressed Due to Excessive Size
Appendix
A. Algorithm Details
A.1. Notations
We then provide more details on the Step-by-Step and Binary Search failure attribution methods. To begin, we define some notations used in the algorithms. We employ Q to denote the query provided to the system. == (2,,2..... Ln) denotes the failure log consisting of n entries where each entry 4 specifies the action taken at time step by one agent. A s" denotes the agent responsible for the task failure and the decisive error step respectively.
A.2. Details Step-by-Step
Algorithm 1 Step-by-Steep
Require: Query Q, failure log - (1 12............... (n)
Ensure: Responsible agent A", error step s*
1: for i E (1,2,..., n do
2: Provide Q and (l,..., 4 to LLM
3: if L.LM indicates error at step then
4:
5: Identify responsible agent A in
6: Return A", s*
7: end if
8: end for
9: No error found
A.3. Details of Binary Search
Algorithm 2 Binary Search
Require: Query Q, failure log I. = (4, 12, 1n)
Ensure: Responsible agent A", error step 8* Initialize low +- 1, high + n while low < high do
low + high
mád-
Extract log segment L' < Utons Now + (mia]
Provide Q and L' to LLM
if LLM indicates error in L' then high + mid
else
low mid +
end if
end while
s* low, identify responsible agent A' in (,- Return s
B. Additional Experiments
B.1. Ablation of Reasoning Prompts
LLMs have shown incredible reasoning ability (Huang & Chang, 2022; Wei et al., 2022; Yao et al., 2024), considering these, in both the all-at-once and step-by-stey approaches, we explicitly require the LLMs to not only conduct failure attributions also specify the reasons for these attributions within the prompt. We don't 't include binary search here because doesn't tt
12

---

Title Suppressed Due to Excessive Size
Reroinng Srascring
Reseiing
Reainning Basooling
Rosiening
Basooiing
Accennyy
Accusany
Aucusas
VecoTerAil
Sig-
Skplp-Skp
Usac-be-lll
Sapay-ii
Osac-ie-Alll
SaptorSip
(a) Ag.eGereette Agent-Level (b) Alg.-Generated Step-Level (c) Hand-Crafted Agent-Level (d) Hand-Crafted Step-Level
Figure 7. Ablation of the explicit reasoning prompts in all-at-once and step-by-step. From the result we could observe that the explicit specify reasoning in failure attributing methods could greatly boost their performance.
include reasoning mechanisms in their prompt. We only want binary search do simple classification task. To investigate the impact of these reasoning prompts the failure attributions, we conduct additional experiments where the reasoning prompt is removed, allowing the LLMs to directly provide the judgment results. We make comparisons and the results are shown in Figure 7. We observed a significant drop in performance after removing the explicit reasoning prompts for failure attribution in both metrics. For example, in Aggorithm-generttee by 7.4% for the all-at-once method. For the step-by-step method, the step-level performance drops 4.4%. These results highlight the necessity of incorporating additional reasoning mechanisms in failure attributions.
multi-agent systems, the agent-level accuracy decreased
C. More Details of Who& When
C.1. Overview
Algorithm-Generate
Hand-Crafted
GAIA
Assistan(Bench
GAIA
AssistantBench
Total Number
98
28
30
28
Maximum Agent Number
5
4
Minimum Agent Number
3
2
Maximum Log Length
130
129
Minimum Log Length
5
5
8
Table 5. Additional details about the Who& When benchmark: We present the total number of tasks for each category, along with the maximum and minimum number of agents and log lengths.
We then provide more details about the Whe&When dataset, which comprises 184 failure annotations tasks from both hand-crafted and algorithn-generaaed agentic systems. These failure logs encompass diverse scenarios with varying numbers of agents and interaction lengths. In Table 5, show the total number of data instances for each category, along with the maximum and minimum number of agents and log lengths. We also visualize the information of each data instance in Figure 8. Note that due to task overlap, some data points may appear sparse in the visualization. We also show an failure task example in Figure 9.
C.2. Data Distribution
123
Total Log Length
Log
Number Agents
Number Agents
(a) Algorithm- Algorithm-Generated Generated
(b) Hand-Crafted
Figure The number agents involved and the total length each failure log instance in the Who& When dataset. Note that due task overlap, some data points may appear sparse in the visualization
13

---

Title Suppressed Due to Excessive Size
C.3. Data Example
Usez Quezy
What's the lowest price Single Parily house vae sold in Queen Anne in danuary 2023"
Eyetes
Eystie
Detalacipplatton Eepatt: To achlave tazk hand, madd be tellow the plan providecria cllas. vicelless pesl catate teamactiog dasa 2023 Firet, leste neplore understint ito oteuslaro lake pythonke ricol, lo1 explose the de file to inderstand ite ieport
to fellow paratble:inii innon- Ask deddurfer search for the loveat: price Singin Family heuse soid in Queen Anne platforme like 81llos, Redrin, local county property recorde- Ar- Sumaeriie rindings and identiry the levaal peicw from. the avuilable deta.
eg
simple tonlly
Ovees
Bingle
20239),
named
2023 uning real estate
"ceal escace cransacctons coeplooe cre creet astate tre mecticaa, cat' ha 'slatet's procaed witn chis first step
Cosputer_ termical: "oxiceodor Insooution failod)
Assistant: to address this, suggest the follewing
bacaknalyaia Repect: since vo doote have the cccract file path or the File stsele, HEMES
appesach
azopp:
"Decisive "Rozson": "hee Datakna lysis, Expert generates synthetic data to address user queries, which is incerrect is this retrieve Insteed, shoute leverage capability wellatle ansuers.",
respansibla apent": "Catafnalysis, Expert"
"failere, respensiblee.gpneeeeeeeeee "Assistant" "becisive arwerr saapp"" "B2" arrer
the
direction contacting the offices for the result. Testead, the agents should solve the pres lens by thresetves."
Figure 9. task example from Who& When, where we annotate suilure-responsiin agents and their corresponding error steps within the failure logs. Each annotation includes natural language explanation of the failure reason for reference.
D. Brief Cost Analysis
We then present a brief analysis of the computational costs associated with three failure attribution methods. We focus solely on input tokens, as the contribution of output tokens such the agent name and error step number is small. We also ignore the mirror token difference between one-time instruction from different methods. We let C' to denote the cost of query Q and corresponding instructions of methods. We employ L = (4,,2, .... In), where each entry I has an average token of T7.
D.I. All-at-Once
In the all-at-once method, the LLM receives the full context in a single input. The total input token cost is:
Costall-at-on = C + n Tj
(6)
This method incurs the lowest cost as it requires only single inference step.
D.2. Step-by-Step
In the Step-by-Stey method, the LLM processes the failure log incrementally. At each step i, it receives query, instructions and the log segment (4,,..,73) 7,). The process terminates when the decisive error step it is found.
bostztee-bb-stee = Σ (C+i-TT) T)) = C 1" (1*+11
2
In the worst case, i n,, either when no error is detected or the decisive error occurs in the final step.
D.3. Binary Search
In the Binary Search method, the LLM operates logarithmic fashion by iteratively splitting the failure log into halves. At each step i, the segment of the failure log processed by the LLM has size of approximately g/r, where n is the total number of log entries. Therefore the total cost at interaction is C + a-T, The Binary Search continues until the search space narrowed down to a single step, requiring [log, (n)) iterations. Therefore the cost of binary search is:
14

---

Title Suppressed Due to Excessive Size
[10) (C+ [loga C+
log]
n-
Cost Cost RinorySeaarc
(8)
2
2
D.4. Cost Summary
In summary, the costs associated with the three methods are influenced by three key factors: the size of the failure log (n)) the average token count per log entry (Ti), and the decisive error step (i"). The choice of method should align with the user's budget and specific use case requirements. Among the methods, the all-at-once approach incurs the lowest cost as it requires only single inference step. In contrast, the costs of the binary search and step-by-step methods are highly dependent on the specific scenario, particularly the distribution of decisive error locations and the total length the failure log.
E. plypepparseeeer
Hyperparameters play a critical role determining the performance of machine learning algorithms (Yu & Zhu, 2020; Zhang et al., 2023b). In this paper, the hyperparameters we utilize are divided into two categories: those used for Who& When data construction and those employed for automated failure attribution algorithms. For data construction, we adopt the default settings CaptainAgent and Magentic-One One from their official libraries (AG2 and Autogen). One notable setting is that the maximum iteration count for CaptainAgent is limited to 10, whereas Magentic-One allows up 30 rounds. It is important to highlight that the agent's 's thought processes are excluded from the round count, which contributes to longer failure log lengths, discussed in Appendix C. For the inference hyperparameters of other large language models (LLMs), we adhere to the default configurations specified in their official documentation.
F. Annotation Details
In Figure 10, present our standardized annotation guidelines used by all annotators. The guidelines clearly define criteria for identifying failure-responsible agents and decisive error steps. Annotators instructed to document any uncertainties in their annotations for subsequent group discussion and voting.
Annotation Guideline
Fallure Responsible Agent:
Select the single agent that ahould be directly responsible for this your zeasons.
your mind.
some subteitivity, but be prepazed give
b) Don't be too strict. If there exist agents that do redundant steps and mistakes.
there agenta nake obvioas mistakes, decide single agent your mind.
If multiple agenta nake mistakes, choose the that made the moat nistake
Decisive eEror step:
decide single mistake agent, decide single step. The Kistake step nust be nade by the mistake agent.
b) If the mistake agent nakes mistakes in multiple steps, choose the first
Failure Reasons:
Firet, use natural langsage to describe the reason. R.g.. \ibe agent wrote the wrong code*. Make the reader could understand the annocations.
Others:
Accurately zecord the time labeling.
b) Kark all annotation if you have any unsertain, and then we need to vote and discuss later.
Figure 10. The guideline making annotation. maintain consistent annotation guidelines across annotators.
15

---

Title Suppressed Due to Excessive Size
G. Prompts
list the prompt templates for all three attribution methods in this section. Please refer to our code base for more details.
G.1. Prompts of All-at-Once
You are an Al assistant tasked with analyzing multi-agent conversation history when solving real world problem. The problem is: {problem).
Identify which agent made an error, at which step, and explain the reason for the error.
Here's the conversation: (failure log)
Based on this conversation, please predict the following:
1. The name of the agent who made a mistake that should be directly responsible for the wrong solution the real world problem. If there no agents that make obvious mistakes, decide one single agent in your mind. Directly output the name of the Expert.
2. In which step the mistake agent first made mistake. For example, in conversation structured as follows:
{
"agent a": "xx",
"agent b": "xxxx"
"agent c": "xxxxx",
"agent a": "XxXXXXx" }.
each entry represents 'step' where agent provides input. The "x' symbolizes the speech of each agent. If the mistake is in agent c's speech, the step number is 2. If the second speech by 'agent a' contains the mistake, the step number is 3, and so on. Please determine the step number where the first mistake occurred.
3. The reason for your prediction. Please answer in the format: Agent Name: (Your prediction)
Step Number: (Your prediction)
Reason for Mistake: (Your reason)
You are an Al assistant tasked with analyzing a multi-agent conversation history when solving a real world problem. The problem is: {problem).
The Answer for the problem is: (ground truth).
Identify which agent made error, at which step, and explain the reason for the error.
Here's the conversation: (failure log]
Based on this conversation, please predict the following:
1. The name of the agent who made a mistake that should be directly responsible for the wrong solution to the real world problem. If there are no agents that make obvious mistakes, decide one single agent in your mind. Directly output the name of the Expert.
2. In which step the mistake agent first made mistake. For example, in conversation structured follows:
{
"agent a": "xx",
"agent b": "xxxx",
"agent c": "xxxxx",
"agent a": "XXXXXXx" }.
each entry represents 'step' where agent provides input. The 'x" symbolizes the speech of each agent. If the mistake is in agent c's speech, the step number is 2. If the second speech by 'agent a' contains the mistake, the step number is 3, and so on. Please determine the step number where the first mistake occurred.
3. The reason for your prediction. Please answer in the format: Agent Name: (Your prediction)
Step Number: (Your prediction)
Reason for Mistake: (Your reason)
16

---

Title Suppressed Due to Excessive Size
G.2. Prompts of Binary Search
You are an Al assistant tasked with analyzing a segment of a multi-agent conversation. Multiple agents are collaborating to address a user query, with the goal of resolving the query through their collective dialogue. Your primary task identify location of the most critical mistake, and determine the single step in the conversation where this error occurs, ultimately leading the failure in resolving the user's query. The problem address is follows: (problem).
Review the following conversation range frange description): :: {sliced log).
Based on your analysis, predict whether the error is more likely to be located in the upper or lower half of the segment. lower half is defined as the range lower half range and upper half defined as the range upper half range. Please simply output either "upper half" or "lower half". You should not output anything else.
You Al assistant tasked with analyzing segment of multi-agent conversation. Multiple agents are collaborating to address user query, with the goal of resolving the query through their collective dialogue. Your primary task is to identify location of the most critical mistake, and determine the single step in the conversation where this error occurs, ultimately leading to the failure in resolving the user's 's query.
The problem to address is as follows: {problem). The Answer for the problem is: (ground truth).
Review the following conversation range {range description): {sliced log).
Based on your analysis, predict whether the error is more likely be located in the upper or lower half of the segment, lower half is defined as the range lower half range and upper half is defined as the range upper half range. Please simply output either "upper half" or "lower half". You should not output anything else.
G.3. Prompts of Step-by-Step
You are an AI assistant tasked with evaluating the correctness of each step in an ongoing multi-agent conversation aimed at solving a real-world problem.
The problem being addressed is: [problem].
Here is the conversation history up to the current step: (failure log].
Your task is determine whether the most recent agent's action contains error that could hinder the problem- solving process. Please respond with 'Yes' "No" and provide clear explanation for your judgment. Note: Please avoid being overly critical in your evaluation.
Attention: Respond in the format:
1. Yes/No. 2. Reason for the judgment.
You are an Al assistant tasked with evaluating the correctness of each step in an ongoing multi-agent conversation aimed at solving real-world problem.
The problem being addressed is: (problem).
Here is the conversation history up to the current step: (failure log).
The Answer for the problem is: {ground truth).
Your task is to determine whether the most recent agent's action contains an error that could hinder the problem-
solving process. Please respond with 'Yes' or "No" and provide a clear explanation for your judgment.
Note: Please avoid being overly critical in your evaluation. Attention: Respond in the format:
1. Yes/No. 2. Reason for the judgment.
17


Reviews:
[{"review_id": "JoXVF3Rd7K", "reviewer": "Reviewer_BgjX", "summary": "The paper introduces a new research problem of automated fault attribution in multi-agent systems. The task includes identifying both the agent and the corresponding step that lead to task failure. To study this task, a new benchmark dataset called Who&When is created by manually labeling 127 failure logs. Three prompting-based approaches are proposed and evaluated on the benchmark. Experimental results indicate that even SOTA LLMs struggle on this task.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable time and insightful feedback! Due to text limit, additional experiments are shown in anonymous link: **https://shorturl.at/JSeJd**. \n\n**[1. Discussion needed for verifiers & Can [3] be leveraged?]**\n\nThank you for the suggestions! We acknowledge the necessity of discussing verifiers. We will add a dedicated subsection in related work, including **all the references you mentioned**. We also kindly emphasize that failure attribution is a fundamentally different research problem, and verifier, like PRMs are not directly applicable:\nPRMs are designed to reward well-structured reasoning chains in tasks like Math. In contrast, our failure attribution targets multi-agent systems, where turn sequences do not necessarily form a coherent reasoning chain. Single agent step may involve multiple reasoning steps—or none—while different agents may address distinct subtasks, with turn-taking reflecting task decomposition rather than unified reasoning. So, ProcessBench does not apply to our task.\n\nIn response to your suggestions, we evaluated these PRMs in [3] for failure attribution under the same setting as Tab. 1 and shows the results in **https://shorturl.at/JSeJd**. It shows that no PRM outperforms All-at-Once and Step-by-Step in two metrics. This performance gap further underscores the fundamental difference between the objectives of PRMs and the proposed failure attribution task. \n\n**[2. Task formulation is subjective.]** \n\nWe kindly clarify that the final annotation is not subjective. The disagreement number in Fig. 2 reflects only the **initial stage** of annotation, where approximately 20% cases were flagged as ambiguous. In subsequent stages, annotators rigorously followed the deterministic formulation in Sec. 2, engaging in discussion and cross-validation to ensure the final annotations were accurate.\n\n**[3. Why is the task challenging?]**\n\nIt is challenging because:\nAgentic logs are very difficult to interpret. On average, each failure case includes 30 steps and 4,695 words and links to a total of 48 tools. Annotators must manually reconstruct agent actions, which requires understanding the complex action logic and manual run tool to verify behavior.\n\nAnnotators may need to manually solve the task—either to obtain a \"golden trajectory\" for reference or to resume from intermediate states to assess whether a step constitutes a decisive error (as defined in Sec. 2), which demands replicating the agent’s conditions over multiple steps and is often hard. \n\n**[4. The results suggest the data do not contain self-correction. & How often is the decisive error step the same as the earliest error?]**\n\nSelf-corrections are common in Who&When, which is reflected in the low average step-level accuracy of 14.2% in Tab. 1. The decisive-error step might occur after the prediction step of Step-by-Step.  In light of your suggestions, we conducted additional round of annotations—using the same procedure—solely on the hand-crafted systems (considering limited time) to identify the first-error step. We then compared it to the decisive error step and reported the overlap percentage in **https://shorturl.at/JSeJd**, which shows that in nearly half of the cases, they do not overlap.  \n\n**[5. Dataset is small.]**\n\nWe plan to expand the dataset in extended work. However, we respectfully argue that the current dataset is non-trivial and sufficient for drawing meaningful conclusions. Our main findings—such as the relative ranking of three methods across metrics—remain robust regardless of dataset scale. In Tab. 1, these rankings hold across both algorithm-generated and hand-crafted systems. The dataset includes 201 distinct agents, 4,092 steps, an average of 4,695 words per trajectory, and 48 external tools. Annotation required 84.3 hours by three experts. which offers a rich testbed for evaluating failure attribution methods.\n\n**[6. Were the results averaged over multiple runs?]**\n\nWe did not initially run multiple LLM trials because: (1) the randomness has minimal impact given the task’s difficulty, (2) small variations don’t affect the paper’s core conclusions—such as the relative ranking of methods- and (3) extreme cost. In response to your suggestion, we conducted five more runs under the same setting as Tab. 4 and report the results in **https://shorturl.at/JSeJd**. We found the findings remain consistent.\n\n**[7.  Are the disagreements within a range of steps?]**\n\nNo, the ambiguous decisive error between annotators does not exhibit a correlation with their positional distance.\n\n**[8. Regarding Fig. 4 and 5]**\n\nWe will replace Fig. 5 with a table to improve clarity. Additionally, we will clarify the step ranges in Figure 4: Level 1 spans 5–17 steps, Level 2 covers 19–29, Level 3 includes 31–49, Level 4 ranges from 51–91, and Level 5 spans 93–130 steps.\n\n**Thank you for your time and consideration! We sincerely hope that you find our responses convincing and would consider increasing the rating.**", "claims_and_evidence": "The main claim made in the paper is that a new task for automated failure attribution is proposed, which is important and challenging. Their experimental results with SOTA LLMs highlight the challenging nature of this task. However, there are some issues with the task.\n\nThere is no discussion of existing work on verifiers, particularly process reward models (PRM)[1, 2], that also aim to identify errors at the step-level. How is the proposed task different from this line of work? I understand that the proposed task is to identify the root-cause step that caused failure, rather than “all” erroneous steps, as done in existing work. But, can existing methods and datasets [3] least be leveraged for this task? For example, a simple baseline could be to choose the earliest erroneous step identified by PRM. \n\n            The other sub-task is to identify the failure-responsible agent. However, once the decisive error\n\nstep is identified, identifying the corresponding agent becomes trivial. \n\n       b.  The current task formulation seems highly subjective as shown in Figure 2, with up to 50%\n\ndisagreement between human annotators. The authors mention that consensus was reached via discussions. But, is the task well-defined or does is a different task design required, such as choosing a set of problematic steps instead of a single one? Without this understanding, the practical utility of the current task and benchmark data remains questionable. \n\n[1] Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., ... & Cobbe, K. (2023, May). Let's verify step by step. In The Twelfth International Conference on Learning Representations. \n\n[2] Wang, P., Li, L., Shao, Z., Xu, R. X., Dai, D., Li, Y., ... & Sui, Z. (2023). Math-shepherd: Verify and reinforce llms step-by-step without human annotations. arXiv preprint arXiv:2312.08935.\n\n[3] Zheng, C., Zhang, Z., Zhang, B., Lin, R., Lu, K., Yu, B., ... & Lin, J. (2024). Processbench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559.", "methods_and_evaluation_criteria": "Given the high disagreement between annotators, some more discussion around reliability of the ground truth data would be useful. Why exactly is the task challenging? Are the disagreements between annotators usually within a range of steps?\n\nThe dataset size is quite small.\n\nProposed “step-by-step” method seems to assume that the agent cannot correct its actions.  The high performance achieved by this method could suggest that the annotated logs do not contain self-correction steps, even though it is a popular strategy [4]. A more diverse benchmark dataset containing such LLM strategies would help draw more general conclusions.\n\n[4] Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2023). Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies. arXiv preprint arXiv:2308.03188.", "theoretical_claims": "Cost analysis in Appendix E is useful and correct.", "experimental_designs_or_analyses": "Figure 5 error bars are too large to draw any meaningful conclusion.\n\nWere the results averaged over multiple LLM runs? What is the variance across runs?", "relation_to_broader_scientific_literature": "The paper builds upon existing work on identifying step-level errors in LLM execution logs, and extends it to pinpoint the most severe error that led to failure. \n\nProposed LLM-as-judge approaches, such as binary search, could potentially be useful for other long-text evaluation tasks.", "essential_references_not_discussed": "Existing work on identifying step-level errors is missing, even though main task proposed in the paper is to identify the decisive failure step.\n\n \n\n[1] Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., ... & Cobbe, K. (2023, May). Let's verify step by step. In The Twelfth International Conference on Learning Representations. \n\n[2] Wang, P., Li, L., Shao, Z., Xu, R. X., Dai, D., Li, Y., ... & Sui, Z. (2023). Math-shepherd: Verify and reinforce llms step-by-step without human annotations. arXiv preprint arXiv:2312.08935.\n\n[3] Zheng, C., Zhang, Z., Zhang, B., Lin, R., Lu, K., Yu, B., ... & Lin, J. (2024). Processbench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559.", "other_strengths_and_weaknesses": "Strengths\n\nThe problem is well motivated\n\nExtensive analyses were conducted. Analysis and findings on consistency of the performance of proposed methods across LLMs is particularly useful.\n\nFor weaknesses, please refer to all issues above mainly around task formulation, dataset reliability and generality.", "other_comments_or_suggestions": "Lines 036-040 and 047-059 are weirdly phrased", "questions_for_authors": "How often is the decisive error step same as the earliest error made by any agent?\n\nWhat length values do the 5 levels in Figure 4 correspond to?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N3KGwJGB5d", "reviewer": "Reviewer_Fdny", "summary": "This paper introduces automated failure attribution for LLM-powered multi-agent systems, addressing the problem of identifying which agent causes a task failure and at which step the decisive error occurs. The authors formally define this research area, propose Who&When, a dataset with annotated failure logs from 127 LLM-powered multi-agent systems, and evaluate three automated attribution methods. Their experiments show significant performance variability related to context length and model choice, highlighting that current LLM-based methods are not yet practically usable and underscoring the need for further research.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the insightful comments! Please find our response to your comments below.\n\n**[Re Weakness 1: Why not make a comparison/do some analysis with Agent-as-Judge in your experimental section?]**\n\nWe appreciate this suggestion; however, the research objective of our failure attribution task significantly differs from that of Agent-as-Judge, making a direct comparison unsuitable. Specifically:\n\n- **Distinct Research Objectives:** Agent-as-Judge employs LLM agents primarily to evaluate the completeness and quality of coding outputs generated by LLMs. Its main focus is on assessing delivered coding projects rather than identifying errors within agentic systems themselves.\n\n- **Different Target Objects:** In Agent-as-Judge, the evaluation targets the final outputs (coding projects) produced by LLMs. Conversely, our failure attribution research specifically examines agentic systems, explicitly aiming to pinpoint the responsible agents leading to failures and identify the exact decisive-error step. \n\nGiven these substantial differences, we have not incorporated Agent-as-Judge into our current experimental setup. We will elaborate further  on these differences  in the related work section.\n\n**[Re Weakness 2 & Question 1: Need more explanations for Table 4 results.]**\n\nThank you for the suggestion. We would like to clarify that the most strong reasoning model (OpenAI O1) still achieves the best performance on three out of four metrics, as highlighted in bold in the original table. However, we do observe instances where the reasoning model performs worse than GPT-4o. This could be attributed to several factors: reasoning models excel at tasks requiring complex logical inference, such as challenging mathematical problems, but they are not necessarily optimized for parsing intricate logs or identifying errors in complex agent-based systems, as these tasks fall outside their training objectives. Developing effective failure attribution methods is an important avenue that merits further research.\n\n**[Re Weakness 3: Why not combine different failure attribution methods? ]**\n\nWe would also like to clarify that we did conduct experiments combining different failure attribution methods; these results are presented in Table 3. Our findings indicate that the Hybrid Method indeed achieves the best performance across both metrics.Thanks for the feedback! \n\n**[Re Weakness 4: Experiments in Section B.2 should be incorporated in main experiments but not Appendix.]**\n\nThanks for the suggestions. We will consider moving it to the main paper after all additional experiments are done for these models. \n\n**[Re Suggestion 2 & Question 2: The experiment's setup.]**\n\nWe conducted these experiments using the same experimental setup as described in the ablation studies in Section 4.4. Due to cost considerations, we did not use the complete dataset for these experiments. Thank you for your suggestion; we will clarify this point further in future revisions.", "claims_and_evidence": "(1) Problem Establishment: The authors rigorously formalize and motivate the automated failure attribution problem in LLM-powered multi-agent systems (Section 2). (2) Key Findings: The extensive experiments in Section 4 yield seven important insights, such as the relationship between context length and failure attribution performance. To me, these insights meaningfully guide future advancements in the field.", "methods_and_evaluation_criteria": "The authors mainly investigate three automated failure attribution methods they proposed in this work on the Who\\&When benchmark. Additionally, they also proposed two main metrics they established in this area, i.e, agent-level accuracy and step-level accuracy. The benchmark, the evaluation metrics and the methods they proposed makes senses to me and well-motivated.", "theoretical_claims": "The authors’ theoretical analysis are mostly around problem formulation and the cost analysis in Appendix. The problem formulation makes senses and rigorous. I also checked the Appendix and the conclusion are also correct.", "experimental_designs_or_analyses": "I have discussed the experimental designs in the previous parts. I like the finds listed in experimental section like the correlation between context length and failure attribution performance.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "**Strengths**\n\n(1) The paper is overall well-structured and well written. The explanations are easy to follow and the logic flows are reasonable. The quality of the paper is very good. \n\n(2) The problem itself is very interesting and important. The problem formulation is mostly clear (see comments before) and the authors provide intuition about how it this problem is important. I am convinced by these arguments.\n\n(3) The authors propose the first benchmark named Who&When for failure attributions with established metrics. These metrics are well-motivated and makes senses to me. I have also checked the anonymous repository associated with the paper. The dataset quality is great and the annotations are also in a high-standard. \n\n(4) The authors perform extensive experiments on the proposed benchmarks and the conclusions are meaningful and could potentially guide the research in this new area.\n\n**Weakness**\n\n(1) Why not making a comparison/ do some analysis with Agent-as-Judge in your experimental section?  If I understand correctly, the agent-as-judge could also be applied to this area with some mirror modifications. \n\n(2)  Surprisingly, I see DeepSeek R1 performs worse than GPT-4o in step-level accuracy, while OpenAI O1 model is worse than GPT-4o in Agent-Level accuracy. Does the authors have some explanations for that? Some discussion needed to be included in the paper. If these conclusions hold, do you have some plan to incorporate model selection in the failure attribution procedures? \n\n(3) Different methods seem to have different advantages. For example,  all-at-once is good at picking up mistake agent and step-by-step is good at picking up mistake step. Why not combining them?", "other_comments_or_suggestions": "1. Experiments in Section B.2 should be incorporated in main experiments but not Appendix. Reasoning models are also LLMs and it should be compared with other models in Figure 3. \n2. The experiments in Section B.2 are conducted on subset of  Who&When or the entire dataset? I am not able to find any information about this on the paper.", "questions_for_authors": "(1) Why the reasoning models perform worse than GPT-4o? Does the author have some explanation for this? \n\n(2) The experiments in Section B.2 are conducted on subset of the Who&When datset or the entire dataset? I could not find any information about that on the paper. \n(3) Why not performing experiments on the entire Who&When on all models in the experimental section? The experiments setting should be consistent.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vwLpeUVfSr", "reviewer": "Reviewer_micc", "summary": "The paper explores automated failure attribution in LLM multi-agent systems. It introduces and formulates a new research of identifying the agent and specific step responsible for task failures within agentic systems. The research introduces the Who&When dataset, which contains failure logs from 127 LLM multi-agent systems, annotated to link failures to particular agents and error steps.\nThe paper evaluates three automated failure attribution methods, demonstrating their strengths and limitations. The best method achieved 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps, underscoring the complexity of the task.\nThe authors argue that evaluation and failure attribution should be integrated and that more effort is needed to bridge the gap between evaluation results and failure attribution. They propose leveraging LLMs for automated failure attribution to reduce the need for manual analysis and enable human resources to focus on improving system functionality.\nOverall, the findings highlight the challenges of using LLMs for failure analysis in multi-agent systems and the need for further research in this area.\n\n# update after rebuttal I maintain my score of Accept and think authors discussed an important and interesting topic with proper experimental approach.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable time and insightful feedback! We address each of your questions in our responses below.\n\n**[Re Comment 1 on Claims & Comment 4 on Experiments: Need to specify the scope. ]** \n\nWe will incorporate all your suggestions and make the following clarifications and revisions in future versions, Specifically: \nWe will explicitly state at the beginning of Section 2 that the initial failure attribution task targets turn-based LLM multi-agent systems, aligning with prior work and widely adopted agentic frameworks [1][2]. We will also emphasize that the protocol is rigorously defined, as detailed in the Background of Section 2.\nTo better inform readers about the scope and diversity of Who&When, we will expand the statistical overview in Section 3. This will include the total number of agents, a summarization of entire action steps, and detailed tools information—facilitating better adaptation to specific use cases.\n\n[1] Wu, Qingyun, et al. \"Autogen: Enabling next-gen llm applications via multi-agent conversation.\" \n[2] Li, Guohao, et al. \"Camel: Communicative agents for\" mind\" exploration of large language model society.\"\n\n**[Re Comment 2 on Claims:  The subjectivity in annotation.]**\n\nThe disagreement number reported in Figure 2 reflects only the initial stage of annotation. Annotators are required to reach a consensus by strictly adhering to the problem formulation in Section 2 rather than relying on subjective judgment. During debating, expert annotators must justify their positions based solely on this formal definition, ensuring precision and unambiguousness. We will clarify this in the revised Section 3.2.\n\n**[Re Comment 1 on Methods: Need to provide empirical results that demonstrate the cost calculations.]**\n\nThank you for highlighting this detail! We have provided empirical cost calculations for all methods in Table 3. The results indicate that the Hybrid Method incurs the highest cost, followed by Step-by-Step, Binary Search, and All-at-Once. We would be happy to discuss them if you have further questions!\n\n**[Re Comment 1 on Theory: The practical utility of mathematical notations.]**\n\nThe mathematical notation introduced in Section 2 aims to rigorously define failure-responsible agents and decisive error steps. These definitions (1) enable readers to clearly understand the research problem without ambiguity, and (2) establish strict guidelines for subsequent annotations in constructing the Who&When dataset.\n\n**[Re Comment 1 on Experiments:  Details of Random search.]**\n\nThe random baseline is constructed using a uniform selection strategy over all available options. Step-level accuracy is defined as the inverse of the average number of steps per problem, and agent-level accuracy as the inverse of the number of agents per problem. Final baseline performance is obtained by averaging these values across all data. We will include this explanation in the revised manuscript.\n\n**[Re Comment 2 on Experiments:  Adjusting the x-axis in Fig. 8.]**\n\nThank you for the suggestion. We will adjust the x-axis in the revised manuscript by removing non-integer values for clarity.\n\n**[Re Comment 3 on Experiments: Does it force annotators to choose an agent even when none can be found?]**\n\nYes. We force annotators to choose one agent with their best rather than passively accepting others’ opinions; we also explicitly require annotators to highlight any uncertain annotations as shown in guideline (b). This helps maintain accuracy and eliminate biases. We will clarify this intention further in our revised manuscript.\n\n**[Re Weakness 1 & Suggestion: The constructed dataset is somewhat limited & future work should consider expanding the dataset.]**\n\nThank you for the constructive suggestions! A key direction for future work is expanding the dataset to further benefit the research community. We also kindly emphasize that Who&When already covers a substantial scope: it includes 127 LLM-based multi-agent systems, 201 agents using 48 tools, and 4,092 action steps. The annotation process required 84.3 hours of expert effort across three annotators.\n\n**[Re Other Comment:]**\n\nWe sincerely appreciate your detailed feedback and will incorporate all your suggestions. Specifically:\n\n**(1), (5), (7), (8):** We acknowledge the typographical errors and will correct verb tenses, third-person singular usage, and the redundancy in \"both two.\"\n\n**(2):** We will add explicit references to Figure 1 at lines 48 and 70 when introducing \"manual failure attributions\" and \"automated failure attribution.\"\n\n**(3), (4), (6)** We will revise all these unclear phrasings in accordance with your recommendations!\n\n**(9):** We will rephrase for clarity: \"Although strong reasoning models yield improvement on the Who&When dataset, their performance remains insufficient for practical use in failure attribution.\"\n\n**(10):** We will remove non-integer x-axis values and revise the instruction to: \"Record all uncertain annotations.\"", "claims_and_evidence": "In general, the paper is well structured and written, and most of the claims are logical and sufficiently substantiated.\nMain claims are: \n1) Automated failure attribution for multi-agent LLM systems is underexplored and novel, but it is important for debugging purposes. Especially with increasing complexity of these systems. \n\nTheir claim is supported by literature overview demonstrating reliance on human labor that is resource-intensive. The authors further support it with the empirical results of human hours involved in data labelling.   \n\n2) A new dataset is constructed and annotated containing logs from 127 multi-agent systems  with 184 failure annotation tasks claiming to advance the research in this area.\n\nThe dataset indeed is thoroughly analysed and used to provide three methodologies applied to it. \n\n3) Current three automated methods achieve modest accuracy (53.5% accuracy at identifying agents responsible for failure and 14.2% at pinpointing exact step), highlighting significant complexity in automation of this task. \n\nThe experimental evidence is clear, it thoroughly outlines findings that can be logically followed. \n\t\nSomehow problematic claims:\nThe paper explores a relatively new research area, supported by a newly introduced dataset, which contains annotations from 3 annotators and from a single hand-crafted agent system and multiple automatically generated agentic systems. Several findings exclude logs from automated ones (with well justifiable notes but still). This limits the diversity and generalisation raising questions if these results will hold for a broader variety of agentic systems. Saying that, I still think it is important to raise issues and problems but expressing those limitations in the paper will help to caveat it for the audience. \n\nThe second issue worth raising is the obvious subjectivity in annotation. While authors raise that the consensus was built after, it highlights that the 'ground truth' might be noisy and further highlights that the accuracy results can be unreliable.", "methods_and_evaluation_criteria": "Overall, the approaches and metrics used in the study are appropriate for addressing the complexities and challenges of failure attribution in LLM multi-agent systems. While the results are quite modest, I appreciate the complexity and documentation of those weak results. \n\nOne issue in the appendix discusses computational costs for various failure attribution methods as mathematical function of the size of the failure log, the token count and error step. However, providing empirical results that demonstrate these costs in practice would substantiate these theoretical calculations. Just as the hybrid method's token expense is discussed in a practical context in the paper, showing how these costs manifest in experiments would be helpful.", "theoretical_claims": "The paper primarily focuses on empirical evaluations and practical methodologies rather than providing detailed theoretical proofs. Its main contributions involve introducing the Who&When dataset and evaluating three methods for automated failure attribution, highlighting their strengths and weaknesses through empirical results.\n\nGiven this focus, the paper does not present theoretical proofs that require verification for correctness. Instead, it relies on experimental results to support its claims. The evaluation metrics and empirical analyses serve as the basis for the paper's conclusions.\n\nTherefore, there are no theoretical proofs within the paper that need to be checked for correctness.\n\nSaying that, there is a problem foundation that introduces mathematical notations for automated failure attribution that I thoroughly checked.\n\nOne comment: the paper indeed introduces mathematical notations, however, it's practical utility in the paper is unclear, given that the authors almost do not utilise it for any theoretical derivations. The notation appears primarily illustrative rather than operational.", "experimental_designs_or_analyses": "The experimental designs and analyses are mostly  sound and valid for the problem at hand. \n\nThere are concerns for representativeness of accuracy due to limited data and annotators disagreement (only three people). \n\nA few suggestions and comments: \n\t1) The paper does not clearly explain how the random baseline was constructed. Detailed information on this would enhance understanding and evaluation.\n\t2)  Figure 8 appears to have non-integer values for the number of agents ( we can't expect to have 1.5 or 4.5 agents). Adjusting the x-axis to display only integer values would be more appropriate.\n\t3) I was puzzled by Annotation guideline (Figure 10) point c). Does it force to choose agent even when none can be found by an annotator?  This can potentially lead to higher levels of uncertainty and some annotators in the debate biasing the rest. \n\t4) In Section 2 the formulation of the problem defines a turn-based protocol. While it can be seen as a problem scope for this paper, an introduction in mathematical formulation sounds like this is the only way LLM agents can act, which is not true. Clearly specifying that this is the scope for this paper will make it more clear.", "relation_to_broader_scientific_literature": "The paper contributes to the broader scientific literature by addressing the challenge of automated failure attribution in large language model (LLM) multi-agent systems. Here are the key contributions and their relation to prior findings and ideas:\n\n1. Automated Failure Attribution aspect:\n Prior research has largely focused on using LLMs for various evaluation tasks, leveraging LLMs to reduce human labor. The paper introduces the concept of automated failure attribution within LLM multi-agent systems, proposing methods to identify the agent and steps responsible for failures. This extends the application of LLMs from evaluation to a more diagnostic  role.\n\n2. Who&When Dataset construction:\nExisting datasets often focus on evaluating model outputs in isolation without detailed annotations linking failures to specific agents and steps (like DevAI and SWE-Bench). The Who&When dataset fills this gap by providing failure logs from 127 multi-agent systems, with annotations that link failures to specific agents and errors. While this dataset supports more detailed and systematic failure analysis than previously possible, it still lacks of comprehensiveness. It can be better positioned as a first step towards automated failure attribution rather than finalised dataset that can be used to solve the problem. \n\n3. Evaluation of Failure Attribution Methods:\n Manual failure attribution has been the norm (Gu et al., 2024; Tan et al., 2024; Zheng et al., 2023), with a significant labor cost and potential for human error. This paper evaluates three automated methods—All-at-Once, Step-by-Step, and Binary Search—demonstrating their respective advantages and limitations in identifying failure-responsible agents and steps. This empirical evaluation provides a first step towards understanding of the effectiveness of these methods, offering a benchmark for future improvements.", "essential_references_not_discussed": "In my opinion, all the required references are there contributing to a structured and logical flow.", "other_strengths_and_weaknesses": "The paper presents notable strengths in focusing on a new and increasingly critical area of research—automated failure attribution in LLM multi-agent systems. The growing complexity of agentic workflows means that effective debugging is essential. The authors have taken a significant first step by identifying this problem and introducing a new dataset and a few methods designed to address failure attribution. The currently weak performances of the proposed methods only further highlight the complexity of the task. The empirical evaluations of the All-at-Once, Step-by-Step, and Binary Search methods pave the way for future advancements and provide a crucial foundation for ongoing research.\n\nThe paper also has its weaknesses. The constructed dataset is somewhat limited, which may restrict the generalizability of the results. The accuracy of the proposed methods is primarily based on this limited data and involves only a few annotators, leading to potential issues with the reliability and validity of the ground truth. This limited scope could result in higher levels of uncertainty and potential biases in the findings. To ensure more robust and widely applicable results, future work should consider expanding the dataset and involving a more extensive and diverse pool of annotators, agentic designs and logs to establish clearer and more certain ground truth data and robust findings.", "other_comments_or_suggestions": "Here is a list of various typos and unclear sentences:\n\t1) Page 1 line 039, second column: manual efforts involve(s)\n\t2) Couldn't find a reference to Figure 1 in the text\n\t3) Page 4 lines 180-181: \"We exclude the entire GAIA dataset\" - but prior to that you say that you use randomly sampled instances of GAIA. It is confusing. Do you mean you exclude the rest of GAIA dataset? \n\t4) Page 4, lines183-184, second column: \"for both normal people and domain experts\" :) It assumes that domain experts are abnormal. Suggest to change it to non-experts. \n\t5) Page 5, line 221, first column: \"we are thinking of performing\" - assuming the rest is in past tense, it would be better to keep it as past tense. After all, you have already reported it. \n\t6) Page 12 Figure 7 description \"the explicit specify reasoning\"\n\t7) Page 12 line 630, \"We only want(s)\" - same in line 653\n\t8) Page 12 line 634 \"attribution in both ~two~ metrics\" \n\t9) Page 12 lines 655 \"Although may provide some improvement<…> - reads like a comment, needs to be a full sentence. \n\t10) Page 13 line 665 'specify\" -> specifies\n\t11) Figure 8 x-axis does not make sense with non-integer values\nFigure 10 Others b) needs rephrasing and checking.", "questions_for_authors": "No other questions", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces the novel problem of automated failure attribution in LLM multi-agent systems, proposing the Who&When dataset and evaluating three methods to identify both the responsible agent and the decisive error step in failed trajectories.


The paper's strengths include:

- Novelty and importance of the problem formulation: The work introduces a well-motivated research area with strong potential impact in debugging LLM multi-agent systems (Reviewer micc, Reviewer Fdny, Reviewer BgjX).

- Introduction of a high-quality benchmark dataset: The Who&When dataset is carefully designed and annotated, providing a rigorous testbed for future research (Reviewer micc, Reviewer Fdny).

- Thorough experimental evaluation and clear empirical findings: The authors present comprehensive results with multiple methods, metrics, and ablations, yielding practical insights such as the influence of context length and model architecture (Reviewer micc, Reviewer Fdny, Reviewer BgjX).

- Strong writing and structured presentation: The paper is well-organized and accessible, with clear problem definitions, detailed methodological explanation, and thoughtful discussion (Reviewer micc, Reviewer Fdny).

Before the rebuttal, common concerns raised by reviewers included the limited dataset diversity and small size (Reviewer BgjX, Reviewer micc), subjectivity and potential ambiguity in the annotation process (Reviewer BgjX, Reviewer micc), and missing connections to related verifier-based work such as PRMs and Agent-as-Judge (Reviewer BgjX). These concerns were directly addressed in the rebuttal with additional experiments, clarification of annotation protocols, justification of task distinctions, and detailed discussion of scope, all of which were acknowledged by reviewers as sufficient.

Therefore, AC recommend acceptance of this paper.
Decision: Accept (spotlight poster)