Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems
Authors: Fulin Lin
Shaowen Chen
Ruishan Fang
Hongwei Wang
Tao Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-Agent System
Autonomous Agents
Efficiency
Large Language Models
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: While Multi-Agent Systems (MAS) excel at complex tasks, their growing autonomy with operational complexity often leads to critical inefficiencies, such as excessive token consumption and failures arising from misinformation. Existing methods primarily focus on post-hoc failure attribution, lacking proactive, real-time interventions to enhance robustness and efficiency. To this end, we introduce SupervisorAgent, a lightweight and modular framework for runtime, adaptive supervision that operates without altering the base agent's architecture. Triggered by an LLM-free context filter, SupervisorAgent intervenes at critical junctures to proactively correct errors, guide inefficient behaviors, and purify observations. On the challenging GAIA benchmark, SupervisorAgent reduces the token consumption of the Smolagent framework by an average of 29.68% without compromising its success rate. Extensive experiments across five additional benchmarks (math reasoning, code generation, and question answering) and various SoTA foundation models validate the broad applicability and robustness of our approach.

Published as a conference paper at ICLR 2026
STOP WASTING YOUR TOKENS: TOWARDS EFFICIENT RUNTIME MULTI-AGENT SYSTEMS
Fulin Lin Shaowen Chen' Ruishan Fang2
Hongwei Wang
Tao Lin
!Zhejiang University Westlake University *State Key Laboratory of CAD&CG, Zhejiang University
[fulini. 24, hongweiwand @intl. 2 zju. edu. en swenchen2zju.co edu.
(fangruishan, lintao) @westlake, edu.
ABSTRACT
While Multi- Systems (MAS) excel complex tasks, their growing auton- omy with operational complexity often leads to critical inefficiencies, such as excessive token consumption and failures arising from misinformation. Existing methods primarily focus on post-hoc failure attribution, lacking proactive, real- time interventions enhance robustness and efficiency. To this end, we introduce SUPERVISSRAGENT lightweight and modular framework for runtime, adaptive supervision that operates without altering the base agent's architecture. Triggered by an LLM- free adaptive filter, SUPERVISOR to proactively correct errors, guide inefficient behaviors, and purify observations. On the challenging GAIA benchmark, SUPERVISOR AGENT reduces the token con- sumption of the Smolagent framework by an average of 29 29.68% without compro- mising its success rate. Extensive experiments across five additional benchmarks (math reasoning. code generation, and question answering) and various SoTA foundation models validate the broad applicability and robustness of our approach.
intervenes at critical junetures
INTRODUCTION
The advent of powerful Large Language Models (LLMs) has catalyzed significant advancements in Multi-Agent Systems (MAS) (Liu al., 2025a; Gao et al., 2025), enabling them to achieve remark- able performance across diverse and challenging domains such as mathematical reasoning (Shang al., 2025), code generation (Lu et al., 2025). and complex question answering (Luo et al., 2025). This progress has spurred research into sophisticated agent architectures, including self-evolving systems that learn from feedback and experience (Shi al., 2025b; Liu al., 2025b), and dynamic topologies that adapt to task complexity (Li et al., 2025a;b). However, critical paradox has emerged: as these systems grow more capable and complex, they often become less robust and economically viable (Wu et al., 2025a; Huang et al., 2025). Systemic inefficiencies incur prohibitive computational costs, while intricate interactions introduce vectors for unpredictable failures (Zhang et al., 2025e).
This lack of robustness stems from the operational complexity of modern MAS, which introduces a significant reliability challenge (Tian et al., 2025). The long chain of interactions inherent in these systems creates fertile ground for error propagation (Dong et al., 2025; Shen et al., 2025). For instance, a single piece of misinformation generated by an agent, a common risk with today's ' powerful yet occasionally hallucinatory foundation models (Kalai et al., 2025; Farquhar et al., 2024), can be committed to memory and subsequently poison the reasoning of all downstream agents (as explained in Figure la). These vulnerabilities mean that even a state-of-the-art MAS fail tasks well within its theoretical capabilities, simply due to a lack of operational robustness (Chen et al., 2024).
Furthermore, the issue of economic inefficiency is a major barrier the real-world deployment of MAS (Wang et al., 2025a). We identify two primary sources of this inefficiency. First, agents often struggle with long observations, such as verbose web pages or tool outputs, which flood their context windows. This not only inflates token costs but can also obscure critical information, causing the agent to lose focus and derail its task execution (Hosseini et al., 2025). Second, agents may adopt sub-optimal strategies, entering into repetitive action loops choosing unnecessarily complex paths to solution (Cemri et al., 2025), further wasting computational resources (see Figure la).
Work was done during Fulin's 's visit to Westlake University.
1 Corresponning authors.

---

Published as a conference paper at ICLR 2026
MINE
MANE
AGNET
AGGET
MAMET
MAMET
4GMET
AGNET
Error Propagation
corracted by SupervisorAgent
Chaeraatio with BITER
Conniteed shhrnddkon
MAS
Page Down
BOWRT
nn' Page Dowe
Page Down
Page Dows
Page Down
Paga
SMAS
Inefficient Behavior
guided by SupervisorAgant
Culbiin provieed q.errccsige
Supenitarkgant improves Goth Cificene Consistancy
Bias
AGNET
Conciseness
AGNET
Noise
Efficiency
Ang. Biak soliene
35% tokens saved
Redundancy
Holloonztion
Pactusiity
A
Misifomation
AGNET
Inefficiency
Volid
AGNET
B
C2Ok cokera
63% variance reduction
MEMORY
Error
TOOL
NEWORY
TOOL
Inndssance
(d)
Corvectness
Multi-Agent System
Suporvised
Takan Coel per Tank
Multi-Agent System (ours)
Robustness & Efficiency
SupervisorAgent (ours)
Agent
Error Occurence
AlCunz) (correct,
guidance, (correcs.guidancs,
ficution)
Row Observatio
Tool
Filter
Excessive Observation
Window
AlGarmbel (correct)
Row Obeanctionn Guidance
Memorr
Inffficient Behavior
Alcustom) Eorgrove, gridenoe))
The SUPERVISORAGENT Framework: Concept and Impact. (a) Illustrative examples of common failure modes in MAS, including error propagation and inefficient loops, and the corresponding intervention by our SUPERVISOR AGENT. (b) An overview of conventional MAS, highlighting the high-risk interaction loci (agent-agent, agent-tool, agent-memory) where such failures occur. (c) The core workflow of our SUPERVISOR- AGENT, which monitors these interactions to provide real- time intervention. (d) The resulting Supervised MAS (SMAS), which integrates the SUPERVISORAGEN GAIA (Level 2), where SMAS (blue) reduces token cost by 35% and variance by 63% versus baseline (red).
Figure I:
enhance robustness and efficiency.
(e) Performance on
To address these intertwined challenges, we propose SUPERVISORAAGEN a lightweight and modular framework that enhances the robustness and efficiency of Multi-Agent Systems (MAS) through real-time supervision (see Figure lc). Incorporating an adaptive filter, SUPERVISOR AGENT enables proactive process control, exemplified by its GAIA Level 2 performance in Figure intervenes at critical junctures to mitigate key operational risks: it conducts proactive error diagnosis, provides pragmatic guidance for inefficient behaviors, and performs adaptive observation purification to reduce contextual noise from long observations.
adaptively
In summary, our main contributions are:
1. We propose and implement SUPERVISS ISORRGENT, novel, lightweight, and non-intrusive meta- agent framework for real-time MAS supervision. It improves agent robustness and efficiency through proactive error correction, inefficiency guidance, and adaptive observation purification, without altering the base agents' architecture.
2. We conduct extensive experiments the challenging GAIA benchmark and demonstrate signif- icant Pareto improvement. When applied to the Smolagent framework (Roucher et al., 2025), SUPERVISOR AGENT reduces token consumption by an average of 29.68% while maintaining competitive task success rates.
We validate the general applicability of our approach across five additional benchmarks spanning mathematical reasoning, code generation, and question answering. Our method consistently delivers substantial efficiency gains, highlighted by 23.74% token reduction on HumanEval alongside an accuracy improvement. The framework's 's effectiveness is further confirmed across various foundation models, including the GPT-4.1, Gemini-2.5 -pro, and Qwen3 series.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
The increasing complexity of Multi-Agent Systems (MAS). Recent advancements in Large Language Models have spurred the development of increasingly sophisticated Multi-Agent Systems (MAS) capable of tackling complex, multi-step tasks (Tran et al., 2025; He et al., 2025). Frameworks like Tongyi DeepResearch (Team, 2025c), AgentOrchestra (Zhang et al., 20251), and Aime (Shi et al., 2025a) exemplify this trend, introducing complex features such as hierarchical structures (Zhu al., 2025; Cheng al., 2025), dynamic agent management (Wu et al., 2025b; Zhang et al., 2025g), and end-to-end training (Li et al., 2025b; Ye et al., 2025). However, this escalating architectural complexity invariably introduces significant challenges in maintaining operational robustness and computational efficiency. which we address in this work.
Failure attribution and robustness. A significant body of work has emerged to address the chal- lenge of MAS robustness, primarily focusing on post-hoc failure attribution (Zhang et al., 2025e). Systems like Aegis (Song et al., 2025) and SHIELDA (Zhou et al., 2025) propose taxonomies for failure analysis, while AgenTracer (Zhang et al., 2025b) and A2P (West et al., 2025) introduce meth- ods to better trace the root causes of task failures. While valuable, these methods are fundamentally reactive, analyzing failures after they have occurred. In contrast, our (UPERVISORAGEN) is designed for proactive, real-time intervention, aiming to detect and mitigate high-risk steps before they lead to systemic failure.
Efficient Multi-Agent Systems. Another stream of research targets the efficiency of MAS, a critical factor largely driven by token consumption. Most approaches focus on design-time optimization. Some prune the system's architecture by eliminating agents with AgentDroopou (Wang et al., 2025b) or communication links with SafeSieve (Zhang et al., 2025d). Others generatively construct efficient prompts (Han al., 2025) or agent topologies from the outset, as seen in MetaAgent (Zhang et al., 2025h), MaAS (Zhang et al., 2025a), and HiVA (Tang et al., 2025). A second direction, context compression, aims to reduce token count by summarizing or distilling observations (Chen et al., 2025; Mou al., 2025). Our work is orthogonal to these methods. Instead of focusing static design or message content, we introduce runtime process control. SUPERVISOR AGENTT addresses dynamic inefficiencies during execution, complementary approach that can enhance existing systems.
3 PRELIMINARY
In this section, we first establish a formalism for our proposed Supervised Multi-Agent System (SMAS). We then detail the core components of our framework: the SUPERVISORAGGE action space and the contextual information it leverages for decision-making.
3.1 A FORMALISM FOR SUPERVISED MULTI- AGENT SYSTEMS
Our work is predicated on the idea that the complex, often chaotic, interactions within Multi-Agent System (MAS; see Figure lb) can be actively managed to improve both robustness and efficiency. To formalize this, we introduce the concept of a Supervised Multi-Agent System (SMAS; see Figure 1d).
Definition (Supervised Multi-Agent System (SMAS)) A SMAS is a Multi-Agent System aug- mented with meta-level control agent, henceforth referred to the Supervisor. The Supervisor's objective is to monitor agent interactions in real-time, proactively detecting and mitigating opera- tional risks without altering the core logic of the agents oversees. In this work, we implement this conceptual Supervisor as a concrete agent named SUPERVISORAGENT.
The fundamental unit of supervision is the interaction, which occurs when an agent engages with other system components. We categorize interactions into three primary types:
Communication or delegation between agents. In architectures like ReAct (Yao et al., 2023), where an agent's 's output becomes another's input, this channel is highly susceptible to the propagation of hallucinated or erroneous information (Shen et al., 2025);
Agent-Agent Interactions:
2. Agent-Tool Interactions: The invocation of external tools or APIs This interaction is a primary source of external information, but it is also fraught with risks, including factually incorrect, irrelevant, outdated data that corrupt the agent's 's context (Qian et al., 2025);
3

---

Published as a conference paper at ICLR 2026
3. Agent-Memory Interactions: The retrieval of information from short- long-tern teng-term memory stores. While crucial for self-evolving systems, memory introduces the hazard of acting upon stale flawed information from past experiences (Xiong et al., 2025).
3.2 THE SUPERVISOR AGENT'S CONTEXT WINDOW
To make informed decisions, the SUPERVISORAGEN is provided with rich, real-time snapshot of the MAS's state, which we formalize as the context window.
Definition 2 (Context Window). The standard context window, W, is tuple of five key elements:
W (N, Qg Q1 S).
where N the name of the agent under review, Qg and Qi are the global and local tasks, Ti is the local trace of agent N's s recent actions and observation summaries, and S is summary of the agent's 's latest interaction step. For diagnosing system-wide inefficiencies, we augment this to an extended context window Wer W }T where Tg the global trace of all agent interactions.
3.3 THE SUPERVISORAGENT'S ACTION SPACE
The role of the SUPERVISORAGENT is to diagnose high-risk interactions and execute tar- geted intervention (Figure 2c). We define three primary intervention contexts, e E =
Cemorr Cinffficient, Cexcessive which activate of three core supervision strategies:
Proactive Error Correction: Triggered by Carorr this strategy aims to diagnose the root cause of explicit error and provide direct fix verification task resolve it.
Guidance for Inefficiency: Triggered by Cinefficient- this strategy provides pragmatic, course- correcting hints for sub-optimal behaviors, while also critically permitting productive, albeit repetitive, processes to continue via an approve action.
Adaptive Observation Purification: Triggered by Cexcessive, this strategy refines excessively long noisy observations to improve the hignal-to-noll ratio for the agent.
These strategies are implemented by selecting an action from the global action space The specific subset of permissible actions, A(c), is formally defined by the intervention context as follows:
poorecctobservation. guidance, run runverificaatin) if Ceroor
A(c)
supprove.provide prodiddeaeeddeg
if Cinffficcntt
(correct_observation)
if Cexcessive
The implementation of each action is detailed in Section 4.3.
4 METHODOLOGY
Building upon the formalism of a Supervised Multi-Agent System (SMAS) introduced in Section 3, now detail the architecture and operational workflow of our SUPERVISOR AGENT (illustrated in Figure 2). Our methodology is structured around three fundamental questions: What to supervise, When supervise, and How supervise. We defer the specific implementation details, including all hyperparameters and prompts, to Appendix A.3 and A.7.
4.1 WHAT TO SUPERVISE: HIGH-RISK INTERACTION POINTS
The primary targets for our supervision the three high-risk interaction points defined in our preliminary formalism (Section 3.1, see also Figure 2a): Agent-Agent, Agent-Tool and Agent- Memory interactions. These points are the primary channels through which errors and inefficiencies are introduced and propagated throughout the system. Our goal is to monitor these specific channels to maintain the operational integrity of the MAS.
4

---

Published as a conference paper at ICLR 2026
Adpative Filter
Context Window W
Agent
Interaction Summay
Observation
Global Task
Local Task
Is erar
Yew
Error Occurence
(b)
Global Trace
Local Trace
Action Space A
A Coneet
Cuudenne (welfcatioon
Raw diservation
Guidaanea
Ia inafficient
Yes
Behavior
observation
Veriiiation
W:
Approve
supeen supervision interr
Approve
Guidance
Guidance
0
excessive
Excessive observation
Correct
Q
cmmply contoxt window
Approve
Verification
(c)
Orea WWAA
Manager Agent
kmp:
Manager Apent
Ougas: Unable deternie
Observatio
enigy Mar
Swurch
Adaption
cilic beladke
Total taken
420,587
70%
1,412 509
MAS
C welloee beheeer
Total
45%
23
Window
SupervisorAgent
HOW
monitor
monitor
monitor
Supervision
Manager Agant
Frbin
Nanage: Ageni
Output
Agent
nocss eiseralt
bahaddo
Supervision
Agent
10) SMAS (urs)
5
Search Agent
i Monitor process
The architecture and workflow of SUPERVISORAGENT. (a) The LLM-free free adaptive filter for identifying high-risk interactions. (b) The context window, aggregating goals and traces for situational awareness. (c) The spectrum of intervention actions, from simple approval to intensive verification. (d, e) Case study on a GATA task, comparing the baseline MAS (d) with our SMAS (c), which cuts steps by 43% and token cost by over 70%. (f) The supervise workflow for an interaction, from filtering to final supervision action.
Figure 2:
4.2 WHEN TO SUPERVISE: THE ADAPTIVE FILTER
While naive approach might monitor every interaction, the associated computational cost is prohibitive and would undermine our goal of improving efficiency. Therefore, the cornerstone of our framework lightweight, LLM-free adaptive filter (see in Figure2a) designed to trigger supervision only at critical junctures (see case studies in Figures 2d and 2e). This approach ensures resources are deployed judiciously, maximizing impact while
that the SUPERVISOR minimizing overhead. The filter is designed to be fast and heuristic-based, monitoring the MAS for three pre-defined, high-risk scenarios:
Error occurrence: The manifestation of an explicit error (e.g., in tool use code execution) is a critical trigger. Unlike current MAS that often pass the full error log into a cluttered context for subsequent agent to debug, filter immediately flags these events for focused, real-time intervention.
An agent may enter loop of sub-optimal repetitive actions that, while not explicit errors, lead to high token consumption and latency. Our filter is designed to detect such patterns, such as an agent repeatedly using the page_dowl action instead of more direct search strategy.
Inefficient behavior:
Excessive observation length: Interactions with tools can return excessively long and noisy observations (e.g., raw HTML) that inflate costs and distract the agent. Our filter identifies such cases for immediate information purification.
5

---

Published as a conference paper at ICLR 2026
4.3 How TO SUPERVISE: GEMORY-AUGMENTED MULTI-LEVEL INTERVENTION
Once high-risk interaction is flagged, SUPERVISORAGENT leverages rich context window and spectrum of intervention strategies to deliver nuanced, effective response.
Memory-augmented context window. To make an effective decision, a supervisor must possess more comprehensive understanding of the system's state than any single agent. This why AUPERVISORAGENT is conceptualized with its own memory module, not simple monitor. As illustrated Figure 2b, this achieved through dynamic context window W, which aggregates the global task Qg. the agent's local task Qi, interaction summary S, and its recent local action trace T. Crucially, for diagnosing complex inefficiencies, SUPERVISORAGENT also accesses the global trace Tg. granting it holistic perspective that transcends the limited view of any individual agent. This elevated viewpoint is what enables it to provide genuinely strategic guidance.
spectrum of intervention actions. With this rich context, SUPERVISOR selects an action from multi-level action space A, adapting intervention intensity tailored issue severity (Figure 2c). These actions range from minimal nudge to a comprehensive correction:
approve: A minimal intervention that permits productive, albeit repetitive, agent behavior to continue. Primarily used the inefficient context, its purpose is to avoid disrupting a process that is pragmatically the best path forward from its current state.
provide_geed -uiideneu semi-intrusive action that steers an agent away from a sub-optimal strategy or logical flaw. This action appends concise, directive hint the existing observation, correcting the agent's reasoning path without altering the core context data.
correct correct_obsere observation: A direct and forceful intervention that refines the agent's sensory input. is the sole action for excessive observations, where it purifies the content, and is also used in error contexts to fix factually incorrect data. This action replaces the original raw observation entirely with cleaned and corrected version.
run uin_vrrffication The deepest intervention, used in complex error contexts when internal infor- mation is insufficient. It invokes verification sub-agent for external fact-checking advanced debugging, returning a definitive, verified result.
5 EXPERIMENTS
5.1
EXPERIMENTAL SETUP
We empirically validate the effectiveness of SUPERVISORAGENT through series of extensive experiments. We begin by outlining our evaluation metrics, datasets, and baselines. For a more detailed description of the experimental settings, please refer to Appendix A.2.
Datasets. We evaluate method diverse suite of six benchmarks spanning three domains. Our primary benchmark is the challenging GAIA validation set (Mialon et al., 2023), which pro- vides comprehensive test of MAS's general problem-solving capabilities. To demonstrate broader applicability, we use five additional benchmarks: for mathematical reasoning. we use AIME 2024 (HuggingFaceHHH, 2024) and random subset of 600 samples from GSM8k-Han (Gao et al., 2022): for code generation, we use the full HumanEval (Chen et al., 2021) and MBPP (Austin et al., 2021) datasets; and for question answering, we use a subset of 800 samples from the DROP (Dua et al., 2019) dataset, following the sampling strategy of prior work (Zhang et al., 2025c).
On several benchmarks, we compare SUPERVISOR AAGERI against comprehensive set of agentic systems equipped with web-browsing and code execution capabilities. These baselines fall into two categories: (1) single agent execution methods: including vanilla LLM, Self Consistency CoT (3 answers) (Wang et al., 2023), and CodeAgent (Roucher et al., 2025); and (2) multi-agent systems, including Smolagent (Roucher et al., 2025), OAgents (Zhu al., 2025), MetaAgent (Zhang et al., 2025h), OWL (role playing) (Hu et al., 2025), and AWorld (Xie et al., 2025; Yu et al., 2025). Detailed descriptions of these baselines are provided in Appendix A.2.2.
Baselines.

---

Published as a conference paper at ICLR 2026
Table Overall performance the GAIA validation set. Our SMAS consistently reduces the average token cost comparing to Smolagent baseline while achieving competitive pass @k success rates.
Method
Avg. Acc.
Avg. Tokens
Acc.
Tokens (K)
1,2 Acc.
1.2 Tokens (K)
Acc.
1,3 Tokens (K)
CodeAgent
40.00
120.40
56.60
92.84
34,88
131.90
23.08
138.5
OWL
45.40
111.07
56.56
67.72
43,02
110.36
29.16
209 34
OAgents
49.09
340.50
66.04
260.27
47.67
19:23
Smolagent
50.91
$27.76
62.26
298.51
53.49
619.59
19:23
691.35
AWorld
60.00
128.27
67.92
69.51
62.79
164.08
34.62
133.65
puss
Smologent
5091
$27.76
62.26
298.51
$3.49
619.59
19:23
691 33
SMAS (ours)
50.91
371.12 L20089
62.26
258.28
51.16
404.96
26.92
489.22 120239
p
Smolagent
58,
467.19
69.81
275.85
59,30
548.02
30.77
589.92
SMAS (ours)
73.58
270.07
420.97 123/89
1380%
529.20 110.990
puss
Smolagent
61.82
502.40
21.70
282.14
63,95
605.05
34 62
611.87
(ours)
11.2/8
369.52 .2644%
75.47 13.779
276.84 11.8%
62.79
409.05 $2.378
12.54
Implementation details. To assess model-agnosticism, we test SUPERVISOR AGENT with multiple foundation models. For the demanding GAIA benchmark, primarily GPT-4.1 as the base model for all agents, and evaluate (UPERVISORAAGEN when powered by GPT-4.1 (OpenAI, 2025), Gemini- 2.5-pro-0605 (Team, 2025a), and Qwen3-235B-2507 (Team, 2025b). For all other benchmarks, we employ the efficient and powerful Qwen3-32B (Team, 2025b) for both the base agents and the SUPERVISORAGENT to assess performance in more resource-come constrrined setting.
Testbed selection. We selected Smolagent as our primary experimental testbed, which provides flexible framework upon which build our agentic systems (SMAS). Critically, Smolagent's capabilities stem primarily from its internal agentic interactions rather than powerful external tools(e.g web APIs or solvers). This provides an ideal, controlled environment to isolate and evaluate the direct impact of our SUPERVISOR AGENT on an agent's ' core reasoning and communication processes.
Metrics. For GAIA and the code generation benchmarks, we report the standard pass @k metric. For our main baseline, Smolagent, we report pass@ 1, 2, and 3. For math reasoning, we report the final solve rate (%). For question answering, we report the F1 score for DROP. all experiments, we meticulously track and report the total token consumption as a primary measure of efficiency.
5.2 RESULTS AND ANALYSIS
Significant efficiency gains with competitive accuracy. The main experimental results, presented in Table confirm the substantial benefits of SUPERVISORAGENT. On the GAIA validation set, when integrated with the Smolagent framework, SUPERVISORAGEN, achieves an average token reduction of 29.68% % at pass @ while maintaining a statistically equivalent success rate. Notably, the efficiency gains are even more pronounced on more difficult tasks, with token savings reaching 32.39 % on Level and 30.10% on Level tasks at @3
Across the other five benchmarks, SUPERVISOR generally achieves Pareto improvement (see Table 2). mathematical reasoning, raises the AIME solve rate by 6.67% while cutting token costs by 18.92 %. In code generation, it maintains competitive accuracy on HumanEval and further reduces token use by 23.74%. %, likely due to ability to streamline repetitive debugging cycles. Occasionally, SUPERV ISOPERVIOORAREE AGENT may overcompress long contexts during purification, causing minor accuracy or Fl drops on certain benchmarks. These results underscore SUPERVISORAGENT'S ability to act universal efficiency enhancer across diverse problem domains.
Model-Agnostic generalization. To demonstrate that the benefits of SUPERVISORAGENT are architectural rather than model-specific, we evaluated it with three different powerful LLMs as its inference engine on GAIA. As shown in Figure 4b, SUPERVISOR significant token savings and maintains robust performance across all models, including 1PT-4.1, Gemini-2.5 2.5-ii--..ppp- pro, and Dwen3-235B. This validates that our supervision framework is model-agnostic component that enhance wide variety of L.M-ppoerred agent systems.
consistently yields
7

---

Published as a conference paper at ICLR 2026
Table 2: Generalization across diverse benchmarks. SUPERVISORAGENT consistently reduces token costs while maintaining or improving accuracy on tasks spanning mathematical reasoning. code generation, and question answering. All reported gains are relative to the Smolagent baseline.
Method
Metries
GSM-hard
AIME
HumanEval
MBPP
DROP
Acc/ F1 (96
67.17
26.67
76.82
80.09
76.36
Vanilla
Avg. Tokens (K)
0.37
2.01
0.28
0.27
0.46
Ace/ P1 (%)
69.01
30.00
77.78
81.26
77.72
CoT SC (3-shot)
Avg. Tokens (K)
2.62
14.26
1.42
1.29
2.73
Acc/ F1 (9))
72.48
33.33
90.74
79.08
79.85
OWL
Avg. Tokens (K)
15.67
56.11
31.87
54.80
11.47
Acc/ / Fl (%)
72.14
26.67
74.08
79.86
78.16
MetaAgent
Avg. Tokens (K)
4.35
6.24
2.59
6.39
1.43
/cc/FI (%)
74.33
30.00
92.07
85.68
81.08
Smolagent
Avg. Tokens (K)
11.59
59.14
40.91
111.07
12.01
Acc/ Fl (%)
75.50
36.67
92.68
84.43
79.80
SMAS (ours)
Avg. Tokens (K)
10.55 18.92s
47.95 J18.22
31.19 (23.7496
103.71 16.62%
11.34
Level paga
Level
Level
Variance Comparison
1.25
1.25
$ 1.00
Lend 2MA5
00
01 p00
0.7
0.50
0.50
01 ma82
0.25
0.25
0.25
90AS
S9A5
V/A
SPAS
11 00.00
Level 2 pogg
Level 2 pu-22
109
Level 2 px33
12 0200
12
12
200AS
XAS
STAG
FAG
SPAS
MAS
poit
polp
ln6
p3
3.5
1.0
pm
m3m
0.0
3.0
3005
3944
3995
Kilon Valunce
Ic31
Figure 3:
SUPERVISORAGEN enhances performance consistency on the GAIA benchmark. (a) Violin plots of token cost distributions, revealing the more compact and predictable performance of our Supervised MAS (SMAS). (b) A direct comparison quantifying the substantial reduction in token cost variance achieved by SMAS across all difficulty levels.
Improving robustness and performance consistency, Beyond average performance, we define robustness as the consistency of an agent's performance. As illustrated by the violin plots in Figure 3, SUPERVISOR AGENT significantly reduces the variance token consumption per task. The distributions for the SMAS are visibly shorter and wider, indicating more concentrated and predictable performance profile. The bar chart on the right further quantifies this, showing a marked decrease in token cost variance, especially for the more complex Level 2 and tasks. This demonstrates that our method not only makes the MAS more efficient on average but also more reliable and less prone extreme resource consumption outliers.
Ablation study. We conducted an ablation study on the full GAIA validation set to isolate the impact of SUPERVISORAGEEN full framework with w/o Correction (Proactive Error Correction), w/o Guidance (Guidance for Inefficiency), and w/o Purification (Adaptive Observation Purification) reveals distinct roles. Pu- rification is the primary driver of efficiency; disabling drastically reduces token savings (from 29.68% 15.96%). Conversely, removing Correction Guidance results in the most significant
three core strategies (Table 3, Figure 4a). A comparison of the
8

---

Published as a conference paper at ICLR 2026
Table 3: Ablation study of SUPERVISORAGENT components the full GAIA validation set.
Method
Avg. Acc.
Avg. Token
Level Avg. Token
Level Avg. Token
Level 3 Avg. Token
Smolagent
50,91
527,759
298,506
619,591
691,331
+ SMAS (wlo Correction)
47.88
354,226
221,515
363,871 J41
592,852 j14.208
SMAS (wio Guidance)
48.48
363, 644
131.006
253,591
015.8%
419.913
_3223%
401.861
J41.5%
+ SMAS (w/o Purification)
49,70
443,520
270,058
19.85%
502,937
600,582 113.19
+ SMAS
50.91
371.119
120,68%
258,279
J13.488
404,955
.34.64%
489,222
(29:3396
(idevl Region)
[Baseling]
100%
600k
Aocatacy
29.68%
14.95%
14.32%
80%
800k
Accurscy (60
Removing Putificaiin
Cost, Accurncy)
462.24
c0%o
STL
40%
Average
Removing Guidanne Accurasy)
200k
Methods
Smelagent Baeett Our Coneettr)
Rerioving Correction (1) Accussy)
100k
Ours hrss Putboster)
Ours (Pal Supenduot)
350k
375k
403k
425k
45tk
425k
Sook
525k
Smolagent (GPT-4:3)
Ours (GPT-4.1)
Our (Gomiri-2.5pp))
Ours (Qwen-3)
0%
Token Cost (Average per Task)
(a) Ablation study on GAIA.
(b) Model Generalization of SUPERVISORAGENT.
Figure Ablation study and model generalization of SUPERVISORAGENT. (a) Ablation study on challenging GAIA tasks, dissecting the distinct contributions of each module to the framework's 's overall efficiency and robustness. (b) Validation of modellaggosticcssi showing that SUPERVISORAGEN consistently delivers token savings across diverse foundation models.
accuracy drops, confirming their necessity for robustness. This underscores synergistic design: while Purification minimizes cost, Correction and Guidance ensure task success, justifying their marginal overhead. These benefits are particularly pronounced on high-cost tasks (see Appendix A.4.2).
MAS-Agnostic generalization. To verify MAS-agnostic feature of SUPERVISORAGENT, we integrate SUPERVISORAGERS into two distinct multi-agent system frameworks: AWorld (Xie et al., 2025) and OAgents (Zhu et al., 2025), and evaluate their performance on the subset of GAIA benchmark(top- 10 most loken-intensive tasks per GAIA level). The results, presented in Table 4, indicate that SUPERVISOR AGENT consistently enhances the performance of both frameworks, underscoring its versatility and effectiveness across different MAS architectures.
Specifically, integrated with AWorld (Xie et al., 2025), our SMAS((World) achieved superior average accuracy over both the original AWorld (without Guard) and the Guard-enabled version. Furthermore, SMAS(AWorld) demonstrated substantial token efficiency, saving 36.54 % on average versus AWorld (with Guard). With savings reaching 48.38% on Level tasks, it confirms SMAS's ability to enhance tool-intensive MAS. Applying SUPERVISORAGEE general applicability, reducing average token consumption by 39.36% while maintaining competitive accuracy. Interestingly, the largest token reduction (50.19%) occurred on Level tasks, exceeding the savings on Level 43).3.).. While this might reflect OAgents' inherent proficiency on harder broad utility.
to OAgents (Zhu et al.. 2025) further validated its
tasks, the significant overall savings underscore
Overhead analysis. Crucially, all efficiency gains reported in this work represent net savings. fully As detailed in Table 6, the supervisor itself incurs
accounting for the cost of modest overhead, averaging only 15.45% of total token usage, which validates its lightweight design. Regarding latency, the supervisory interventions introduce an average increase of less than one minute and half per task (Table 7). We consider this temporal cost justifiable trade-off given the substantial economic savings achieved in complex multi-agent workflows. comprehensive overhead analysis is provided in Appendix 1.4.1.

---

Published as a conference paper at ICLR 2026
Table 4: froos-rramework performance of SUPERVISORAGENT. Evaluated on GAIA subset (top- 10 most token-intensive tasks per level).
Method
Avg.
Avg. Token
Avg. Token
Level 2 Avg. Token
Level Avg. Token
Smolagent + SMAS
40.00
1,446,526
933,013
2,037,437
1,369,131
46.67 16.67%
721,332 150.1%%
522,364 J4401
960,694 152.85%
680,939 150.20%
AWorld (without Guard)
23.33
155,239
50,851
217,332
166,500
AWorld (with Guard)
30.00
353,738
135,413
463,083
376,878
AWorld (with SMAS)
36.67 16667
224,480 126,54%
90,569 135.12%
355,051 123.33%6
194,561 148.30%
OAgents
46.67
$30,939
430,852
359,511
802,454
SMAS
46.67
321,957
214,604 .9019%
274,875 (23
476,793 (40
6 DISCUSSION AND CONCLUSION
Supervisor as a foundational MAS component. Our work positions AOPERRISORAGENT as a foundational component for future Agent Systems, akin to established modules like memory banks and tool-usage frameworks. By providing real-time, adaptive supervision, SUPERVISORAGENT alleviates critical challenges of robustness and efficiency that are pervasive across diverse MAS architectures. Its modular design allows for seamless integration with existing systems, enhancing their performance without necessitating fundamental changes to their core logic. This underscores the potential of supervisory agents as universal enhancers of MASs, capable of elevating both reliability and cost-effectiveeess across a wide range of applications.
Comparison with related supervisory agents. A related concept is the Guard agent in AWorld (Xie et al., 2025), which is invoked at key steps primarily for factual verification to enhance task accuracy. While valuable, its scope differs significantly from method. SUPERVISOR AGENT adopts broader objective of improving overall system efficiency and robustmess through continuous (albeit adaptively filtered) monitoring and wider range of interventions, including error correction, inefficiency guidance, and observation purification, complementing the Guard's focus on accuracy.
Broader insights. Our work also yields critical insights for the broader field. First, we discovered that seemingly "noisy" information, such as HTML structure and truncation cues, serves as a vital signal for ReAct-style agents. The overly aggressive purification can paradoxically harm perfor- mance. This highlights fundamental trade-off between information density and the preservation of envirommental texture. Second, our focus on token cost underscores the need for a more holistic efficiency evaluation for MAS. A comprehensive analysis must also account for the frequency and complexity external tool API calls, which offload significant burdens from the MAS. This very trade-off informed our choice of Smolagent reasoning, rather than powerful external tools, provided a controlled environment to isolate and evaluate our SUPERVISORAGENT impact the interaction process itself.
primary testbed its reliance internal agentic
Future directions. These insights inform several promising avenues for future work. First, moving beyond heuristic rules, exploring learning- baanidg-bbsedd adaptive filter could enable more precise, dynamic control over supervisor invocations. This aligns with the broader goal of developing a self-evolving, memory-augmentted version of SUPERVISOR AGENT. Second, further research should focus on mitigating the latency overhead introduced by supervisory calls to enhance real-time applicability, alongside creating sophisticated purification techniques that address the "noise-as-sic signaall"" trade-off. Finally, developing a universal resource consumption metric for MAS remains a critical open chal- lenge. Ultimately, we posit that incorporating such real-time, meta-level supervision a foundational component for building the next generation of truly scalable and reliable MAS.
Conclusion. In this work, we introduced SUPERVISORAGENT a lightweight and non-intrusive meta-agent framework that enhances the robustness and efficiency of Multi-Agent Systems. Through real-time, adaptive supervision, SUPERVISOR AGENT mitigates common failure modes and reduces computational overhead using three core strategies: proactive error correction, pragmatic inefficiency guidance, and adaptive observation purification. Our extensive experiments demonstrate a significant Pareto improvement. On the challenging GAIA benchmark, SUPERVISOR reduces token consumption by an average of 29.68% while maintaining competitive task success rates, a crucial step towards building more practical and scalable agentic systems.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported in part by the National Key Research and Development Program of China (2024YFF0007003), Research Fund for International Scientists National Natural Science Foundation of China 723307107955 National Natural Science Foundation of China (NSFC) under No. 62576285, 62276230, Rescarch Center for Industries the Future (RCIF) at Westlake University, and Westlake Education Foundation.
ETHICS STATEMENT
Our work aims to improve the reliability and efficiency of Multi-Agent Systems, a crucial step for developing practical and beneficial autonomous technologies. We believe that by introducing mechanism for real-time supervision, our framework provides a paradigm not only for performance optimization but also for enhancing the safety and predictability of future agentic systems. Our research was conducted on publicly available benchmarks, did not involve private user data, and adheres the ICLR Code of Ethics.
REPRODUCIBILITY STATEMENT
We are committed to ensuring our work is reproducible. The core architecture and logic of SUPERVI- SORAGENT are detailed in Section with theoretical formalisms Section 3. For direct replication, we provide all implementation details and final prompts in Appendix A.3, A.7, and our code is available at https: /ttps:///gthub rics used in extensive experiments (Section 5) are all based publicly available benchmarks, allowing for direct comparison and validation of our results.
The datasets and met-
REFERENCES
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021. URL https: aaraaaaxvvvvro..//aaa org/ abs 21.. 07732.
Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph Gonzalez, and Ion Stoica. Why do multi-agent Ilm systems fail?, 2025. URL httph:tt///// //apps::////xv....// org/abs/ 2503. 13657.
Junzhi Chen, Juhao Liang and Benyou Wang. Smurfs: Multi-agent system using context-efficient dfsdtt for tool planning. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 3281 1898-,988, 2025.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Ka- plan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss. William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, 1lya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code, 2021. URL https: /ttpaaaiiivv/..///////eee//aaaaa erg/ ht///abs/2220 C3374.
Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming Ilm agents via poisoning memory or knowledge bases, 2024. URL https: : /arxiiiiivv///////aa/////aa org.oog//aoo.. /bs/ 2407. 12784.
11

---

Published as a conference paper at ICLR 2026
Yuyang Cheng, Yumiao Xu, Chaojia Yu, and Yong Zhao. Hawk: A hierarchical workflow framework for multi-agent collaboration, 2025. URL /attps://arxiv.... wttp://wwwwr///////...o.o /t0.///0022000007..
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint vrXiv22100.41666 2021.
Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. A practical memory injection attack against Ilm agents, 2025. URL org/ aaaaa//2500 03704.
//axxvv.
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 2378-,378, Minneapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10. 10.18653//0 10.18653/v1/119. 1246. URL https: a/aclanthe 1 www.ogggy....oooo.ooo.o........o org/N19-124//1 9-246/
Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017)x625 630, jun 2024. ISSN 1476- 4687. doi: 10. 10.1003444188-0004-00000 s s41586 024 07 4201-0.
186-024-0742----6
URL https: /t////wwww/ www. www.nature...e.rm..mm..www nature. com/ articles/
Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, Hongru Wang, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, Qihan Ren, Cheng Qian, Zhenghailong Wang, Minda Hu, Huazheng Wang, Qingyun Wu, Heng Ji, and Mengdi Wang. A survey of self-evolving agents: On path to artificial super intelligence, 2025. URL https:
//arxive orgo abs/ / 2507. 10.07.2100 21046.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. arXiv preprini 1XXi:2211.110335 2022.
Yichen Han, Bojun Liu, Zhengpeng zhou, Guanyu Liu, Zeng Zhang, Yang Yang, Wenli Wang, Isaae N Shi, Yunyan, Lewei He, and Tianyu Shi. Mapgd: Multi-agent prompt gradient descent for collaborative prompt optimization, 2025. URL https: /aaxxiivvvvv.////////aa/////aavv org/ htgg/bs////r abs 22002 11361.
Junda He, Christoph Treude, and David Lo. LIm-based multi-agent systems for software engineering: Literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology. 34(5):1-30, 2025.
Peyman Hosseini, Ignacio Castro, Iacopo Ghinassi, and Matthew Purver. Efficient solutions for an intriguing failure of LLMs: Long context window does not mean LLMs can analyze long sequences flawlessly. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert (eds.), Proceedings of the s1 International Conference on Computational Linguistics, pp. 1880-1891, Abu Dhabi, UAE, January 2025. Association for Computational Linguistics. URL https: /aaaccaanttt aaclannthoogg... org/ 2025 col moaiinnmmiin. 128/.
Mengkang Hu, Yuhang Zhou, Wendong Fan, Yuzhou Nie, Bowei Xia, Tao Sun, Ziyu Ye, Zhaoxuan Jin. Yingru Li. Qiguang Chen, Zeyu Zhang, Yifeng Wang. Qianshuo Yc, Bernard Ghanem, Ping Luo, and Guohao Li. Owl: Optimized workforce learning for general multi-agent assistance in real-world task automation, 2025. URL https wwwpivivvvv.... org. abs 22005 23885.
Jen-tse Huang, Eric John Li, Man Ho Lam, Tian Liang, Wenxuan Wang, Youliang Yuan, Wenxiang Jiao, Xing Wang. Zhaopeng Tu, and Michael R. Lyu. Competing large language models in multi-agent gaming environments. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR), 2025.
HuggingFaceH4. Aime 2024 dataset.
http:://auggingaacccco////
Huggi inngggnnFFceeH444 / / 22024, 2024.
12

---

Published as a conference paper at ICLR 2026
Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. Why language models hallucinate, 2025. URL
org/ abs/2509. 04664.
Shiyuan Li, Yixin Liu, Qingsong Wen, Chengqi Zhang, and Shirui Pan. Assemble your crew: Automatic aglti- communication topology design via autoregressive graph generation, 2025a.
URL https: /arxiv.vvvv..////////.a////aaav. org/ abs//bb//22 2507 182..
Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, Xinpeng I Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, Hongxuan Lu, Tianrui Qin, Chenghao Zhu, Yi Yao, Shuying Fan, Xiaowan Li, Tiannan Wang. Pai Liu, King Zhu, He Zhu, Dingfeng Shi, Piaohong Wang, Yeyi Guan, Xiangru Tang, Minghao Liu, Yuchen Eleanor Jiang, Jian Yang, Jiaheng Liu, Ge Zhang, and Wangchunshu Zhou. Chain-of-agents: End-to-end agent foundation models via multi-agent distillation and agentic rl, 2025b. URL https: hittps://ariiv http/vaxivvv..i.... org. wwwp/og.////./oooo../..../a.../t 2508. 13167.
Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongzhang Liu, Shaokun Zhang, Kaitao Song, Kunlun Zhu, Yuheng Cheng, Suyuchen Wang, Xiaoqiang Wang, Yuyu Luo, Haibo Jin, Peiyan Zhang, Ollie Liu, Jiaqi Chen, Huan Zhang. Zhaoyang Yu, Haochen Shi, Boyan Li, Dekun Wu, Fengwei Teng, Xiaojun Jia, Jiawei Xu, Jinyu Xiang, Yizhang Lin, Tianming Liu, Tongliang Liu, Yu Su, Huan Sun, Glen Berseth, Jianyun Nie, Ian Foster, Logan Ward, Qingyun Wu, Yu Gu, Mingchen Zhuge, Xinbing Liang, Xiangru Tang, Haohan Wang, Jiaxuan You, Chi Wang, Jian Pei, Qiang Yang. Xiaoliang Qi, and Chenglin Wu. Advances and challenges in foundation agents: From brain inspired intelligence to evolutionary, collaborative, and safe systems, 2025a.
URL https: /aaxiv.viii..///ii///eaa///a.avv org/ abs / 250 4. 01990.
Yuhang Liu, Pengxiang Li, Congkai Xie, Xavier Hu, Xiaotian Han, Shengyu Zhang, Hongxia Yang. and Fei Wu. Infigui-rl: Advancing multimodal gui agents from reactive actors to deliberative reasoners, 2025b. URL https: /ttp/aawww../aaaaaaaaeeea/aaeaea
wrgg/as//s...ooooo...ee..eeeseee 2504 14239.
Xu Lu, Weisong Sun, Yiran Zhang, Ming Hu, Cong Tian, Zhi Jin, and Yang Liu. Requirements development and formalization for reliable code generation: A multi-agent vision, 2025. URL https://arxiie org/ abs/2500 2500 18675.
Yajie Luo, Yihong Wu, Muzhi Li, Fengran Mo, Jia Ao Sun, Xinyu Wang, Liheng Ma, Yingxue Zhang, and Jian-Yun Nic. An entity linking agent for question answering, 2025. URL https: wawxiv.org/aaaaa......a 2508 03865.
Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general assistants, 2023. URL wtttthttss/ttiiiiie http///xwww..rgaaaa/a/aaaaaaase/ 12983.
Xinyi Mou, Chen Qian, Wei Liu, Xuanjing Huang, and Zhongyu Wei. Ecolang: Efficient and effective agent communication language induction for social simulation, 2025. URL https: larxiv.orggabs//5055.. 001.555506666000 06904.
OpenAI. Introducing gpt-4.1 in the api. OpenAl blog post, Apr 2025. URL httthttst////eee com/ http///pxxtpp-11/1///
Cheng Qian, Emre Can Acikgoz, Hongru Wang, Xiusi Chen, Avirup Sil, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. Smart: Self-aware agent for tool overuse mitigation, 2025. URL https:
11435.
Aymeric Roucher, Albert Villanova del Moral, Thomas Wolf, Leandro von Werra, and Erik Kaunis- mäki. 'smolagents': a smol library to build great agentic systems. http::///gtthii com/ huggingface/ momolaagnntt. 2025.
Ning Shang, Yifei Liu, Yi Zhu, Li Lyna Zhang, Weijiang Xu, Xinyu Guan, Buze Zhang, Bingcheng Dong, Xudong Zhou, Bowen Zhang, Ying Xin, Ziming Miao, Scarlett Li, Fan Yang, and Mao Yang. rstar2-agent: Agentic reasoning technical report, 2025. URL abs/2508 20722.
Xu Shen, Yixin Liu, Yiwei Dai, Yili Wang, Rui Miao, Yue Tan, Shirui Pan, and Xin Wang. Under- standing the information propagation effects of communication topologie in Ilm- based multi-agent systems, 2025. URL https: /arxivvvvvvv.//////v///////a//v www..oovvoooooooooooooooooooooor abs 22005.55550000 23352.
13

---

Published as a conference paper at ICLR 2026
Yexuan Shi, Mingyu Wang. Yunxiang Cao, Hongjie Lai, Junjian Lan, Xin Han, Yu Wang, Jie Geng, Zhenan Li, Zihao Xia, Xiang Chen, Chen Li, Jian Xu, Wenbo Duan, and Yuanshuo Zhu. Aime: Towards fully- aulyy-autonomous multi-agent framework, 2025a. URL 2507. 11988.
Yucheng Shi, Wenhao Yu. Zaitang Li, Yonglin Wang, Hongming Zhang, Ninghao Liu, Haitao Mi, and Dong Yu. Mobilegui-rl: Advancing mobile gui agent through reinforcement learning online environment, 2025b. URL
org/orgg///.... abs 2507.05720.
Kevin Song, Anand Jayarajan, Yaoyao Ding, Qidong Su, Zhanda Zhu, Sihang Liu, and Gennady Pekhimenko. Aegis: Taxonomy and optimizations for overcoming agent-environment failures in IIm agents, 2025. URL www.ttttt/tt/. /ttp:////aavvv./// org/ abs 2508 2 19504.
Jinzhou Tang, Jusheng Zhang, Qinhan Lv, Sidi Liu, Jing Yang. Chengpei Tang, and Keze Wang. Hiva: Self-organized hierarchical variable agent via goal-driven semantic-topological evolution, 2025. URL https: / arxiv. /2509///b/29 00189.
Gemini Team. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025a. URL htthtttt///arxi 2507.06261
Qwen Team. Qwen3 technical report, 2025b. URL /Rt.htppss//aaxii..o///////////2
Tongyi DeepResearch Team. Tongyi-deepresearch. https:///itthu.. baal:/taaacbccccm/aaaaaaaaaaaaa. DeepResearch, 2025c.
Fangqiao Tian, An Luo, Jin Du, Xun Xian, Robert Specht, Ganghua Wang, Xuan Bi, Jiawei Zhou, Ashish Kundu, Jayanth Srinivasa, Charles Fleming, Rui Zhang, Zirui Liu, Mingyi Hong, and Jie Ding. An outlook on the opportunities and challenges of multi-agent ai systems, 2025. URL betps://arxxor org/orgg///aooo /200..250.. 18397.
Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O'Sullivan, and Hoang D. Nguyen. Multi-agent collaboration mechanisms: A survey of llms, 2025. URL https:
06322.
Ningning Wang. Xavier Hu, Pai Liu, He Zhu, Yue Hou, Heyuan Huang, Shengyu Zhang, Jian Yang, Jiaheng Liu, Ge Zhang, Changwang Zhang, Jun Wang, Yuchen Eleanor Jiang, and Wangchunshu Zhou. Efficient agents: Building effective agents while reducing cost, 2025a. URL https: //arxiv..rg./ abs 2508. 02694.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- cry, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models, 2023. URL https: http//axivvv.///////ee//a/a org/ abs 1217..1177..
Zhexuan Wang. Yutong Wang, Xuebo Liu, Liang Ding, Miao Zhang, Jie Liu, and Min Zhang Agentdropout: Dynamic agent climination for token-efficient and high- perffrrrrommccc Ilm-based multi-agent collaboration, 2025b. URL https: / /awxiviivvv///////v///aaa////aav vv.iv... org/ abs 220...503..00 18891.
Alva West, Yixuan Weng, Minjun Zhu, Zhen Lin, and Yue Zhang. Abduct, act, predict: Scaffolding causal inference for automated failure attribution in multi-agent systems, 2025. URL https:
//arxiv.orgg/bs/ abs 2509. 10401
Chen Henry Wu, Rishi Shah, Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried, and Aditi Raghu- nathan. Dissecting adversarial robustness of multimodal Im agents, 2025a. URL https: / //arxiv.org/aba.... wwaprr/abb///ee 2406.128111
Feijie Wu. Zitao Li, Fei Wei, Yaliang Li, Bolin Ding, and Jing Gao. Talk to right specialists: Routing and planning multi-agent system for question answering. 2025b. URL https: //arxiv. orgg abs /2001. 07813.
Zhitian Xie, Qintong Wu. Chengyue Yu, Chenyi Zhuang, and Jinjie Gu. Profile-aware maneuvering:
A dynamic multi-agent system for robust gaia problem solving by aworld, 2025. URL https: / www.vi.rggabb///eo 25088. 09889.
14

---

Published as a conference paper at ICLR 2026
Zidi Xiong, Yuping Lin. Wenya Xie, Pengfei He, Jiliang Tang, Himabindu Lakkaraju, and Zhen Xiang. How memory management impacts Ilm agents: An empirical study of saperience-following behavior, 2025. http:: // arxiv. org/ absp//as../55/ 16067.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023. URL https:/ /appss///xxiii org/abs/ 2210. 03629.
Rui Ye, Shuo Tang, Rui Ge, Yaxin Du, Zhenfei Yin, Siheng Chen, and Jing Shao. Mas-gpt: Training Ilms to build Ilm-based multi-agent systems, 2025. URL 03686.
Chengyue Yu, Siyuan Lu, Chenyi Zhuang, Dong Wang, Qintong Wu, Zongyue Li, Runsheng Gan, Chunfeng Wang. Siqi Hou, Gaochi Huang, Wenlong Yan, Lifeng Hong, Aohui Xue, Yanfeng Wang, Jinjie Gu, David Tsai, and Tao Lin. Aworld: Orchestrating the training recipe for agentic ai, 2025. URL https: araii. org/ abs 2508. 20404.
Guibin Zhang, Luyang Niu, Junfeng Fang, Kun Wang, Lei Bai, and Xiang Wang. Multi-agent architecture search via agentic supernet, 2025a. URL https: wwwpw.pgwwwaaaaaaaaaoo/////.aaaa /200.. 04180.
Guibin Zhang, Junhao Wang, Junjie Chen, Wangchunshu Zhou, Kun Wang, and Shuicheng Yan. Agentracer: Who is inducing failure in the Ilm agentic systems?, 2025b. URL htttt:///arxii brg/abs/2505 03312.
Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingehen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. Aflow: Automating agentic workflow generation, 2025c. URL https: http://axxvvvvoovv
abs/2410.1074
Ruijia Zhang. Xinyan Zhao, Ruixiang Wang, Sigen Chen, Guibin Zhang, An Zhang. Kun Wang, and Qingsong Wen. Safesieve: From heuristics to experience in progressive pruning for Ilm-based multi-agent communication, 2025d. URL https: /ttp:v///////va/aaaaaaee org. org/ abs /500. 11733.
Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, and Qingyun Wu. Which agent causes task failures and when? on automated failure attribution of I1m multi-agent systems, 2025e. URL https: //a //arxivv .rg/ abs/ / 2505. 00212.
Wentao Zhang, Liang Zeng, Yuzhen Xiao, Yongcong Li, Ce Cui, Yilei Zhao, Rui Hu, Yang Liu, Yahui Zhou, and Bo An. Agentorchestra: A hicrarchical multi-agent framework for general-puu -urppse task solving, 2025f. URL https: / arxiv. org/ abs/ 2506. 12508.
Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 13378-233385 2025g.
Yaolun Zhang, Xiaogeng Liu, and Chaowei Xiao. Metaagent: Automatically constructing multi- agent systems based finite state machines, 2025h. URL holLhttss////tttt
abs ab2/2507 /507.
22606.
Jingwen Zhou, Jieshan Chen, Qinghua Lu, Dehai Zhao, and Liming Zhu. Shielda: Structured handling of exceptions in Ilm-driven agentic workflows, 2025. URL abs/25 07935.
He Zhu, Tianuui Qin, King Zhu, Heyuan Huang, Yeyi Guan, Jinxiang Xia, Yi Yao, Hanhao Li, Ningning Wang, Pai Liu, Tianhao Peng, Xin Gui, Xiaowan Li, Yuhui Liu, Yuchen Eleanor Jiang, Jun Wang, Changwang Zhang, Xiangru Tang, Ge Zhang. Jian Yang. Minghao Liu, Xitong Gao, Jiaheng Liu, and Wangchunshu Zhou. Oagents: An empirical study of building effective agents, 2025. URL https: //tpxiv...v////////////////./aav org/abs /etp///bb/50..... 2506. 15741.
15

---

Published as a conference paper at ICLR 2026
CONTENTS
1 Introduction
2 Related Work
3 Preliminary
3.1 A Formalism for Supervised Multi-Agent Systems.
3.2
The SUPERVISOR AGENT'S Context Window
3.3
The CAPPERVISORRGNNT Action Space
Methodology
4.1
What Supervise: High-Risk Interaction Points
4.2
When to Supervise: The Adaptive Filter
4.3
How to Supervise: Memory-Augnnented, Multi- Intervention
5
Experiments
5.1
Experimental Setup
5.2
Results and Analysis
6 Discussion and Conclusion
10
Appendix
17
A.I
LLM usage
17
A.2
Experimental Setup
17
A.2.1
Datasets
17
A.2.2
Baselines
18
A.3
Implementation details
19
A.3.1
What to Supervise: The ActionStep Object
19
A.3.2
When to Supervise: The Prioritized Adaptive Filter
19
A.3.3
How to Supervise: The Intervention Pipeline
21
A.4
Extended Experimental Analysis
A.4.1
Overhead analysis of SUPERVISORAGENT
21
A.4.2
Performance Analysis on Voken-Intensive Scenarios
22
A.5
Failure Mode Analysis
23
A.6
Case Study
24
A.7
Prompts
26
16

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 LLM USAGE
The large language model (LLM) was utilized as writing assistant during the preparation of this manuscript. Its application was strictly limited to improving the clarity and grammatical accuracy of the text. Specific uses included rephrasing sentences for better flow and translating initial concepts and drafts from Chinese to English. All core scientific contributions, including the conceptualization of our SUPERVISORAGEN framework, the design of the methodology and experiments, and the analysis and interpretation the results, are solely the work of the authors. The authors take full responsibility for all claims and the final content of this paper.
A.2 EXPERIMENTAL SETUP
A.2.1 DATASETS
Here, we provide a detailed introduction to the datasets used in this paper:
GAIA (Mialon et al., 2023) serves as a benchmark designed to evaluate next-generation LLMs that possess enhanced capabilities through the incorporation of tools, efficient prompting strategies, and access to external search resources. This benchmark comprises over 450 challenging questions, each with a clear and unequivocal answer, necessitating varying degrees of tooling and autonomy for resolution. Accordingly, the questions are categorized into three distinet levels: Level is expected to be solvable by proficient LLMs, while Level 3 signifies a substantial increase in the model's iss  capabilities. Each level includes a fully public development set for validation purposes, as well as test set containing private answers and associated metadata. In our experiments, we utilize the test set, which encompasses 164 tasks.
GSM-hard (Gao et al., 2022) is an advanced version of the GSM8K mathematics reasoning dataset (Cobbe et al., 2021), This enhanced dataset presents models with increased challenges, featuring larger numerical values and more complex relationships within the problems.
AIME-2024 (HuggingFaacet4, 2024) is a dataset comprising problems derived from the Amer- ican Invitational Mathematies Examination (AIME) 2024. AIME is prestigious mathematics competition for high school students, recognized for its challenging problems that span various mathematical domains. This benchmark serves multiple purposes: evaluates the mathematical reasoning capabilities of L.Mss, assesses their problem-solving abilities on complex mathematical challenges, and investigates Al performance on structured mathematical tasks.
HumanEval (Chen et al., 2021) is a dataset released by OpenAl that includes 164 programming problems, each containing function signature, docstring, body, and several associated unit tests. These problems were handwritten to ensure that they were not included the training dataset for code-generation models. This benchmark is crucial for evaluating code-generation models, providing a structured set of challenges in Python that facilitates the assessment of both the quality and correctness of code produced by language models.
MBPP(Mostly Basic Python Problems Dataset) (Austin et al., 2021) comprises approximately 1, 1,000 crowd-sourced Python programming problems that are specifically designed to be solvable by entry-level programmers. The dataset covers essential programming fundamentals and standard library functionalities. Each problem includes task description, corresponding code solution, and three automated test cases.
DRoP(Data Retrieval Open Answering) Dua et al. (2019) is a reading comprehension benchmark that requires discrete reasoning over paragraphs. This dataset consists of 96,000 questions developed through crowd sourcing and adversarial methods. It challenges systems to resolve references within the questions, which may point to multiple input positions. The tasks entail performing discrete operations, such addition, counting, and sorting, necessitating substantially more comprehensive understanding of paragraph content than that demanded by prior datasets, In our experiment, we sampled 800 tasks for evaluation.
17

---

Published as a conference paper at ICLR 2026
A.2.2 BASELINES
Vanilla is the original Large Language Model (LLM) that processes input using only the question and basic prompt, without any prompt engineering or external tool integration. This straightfor- ward approach emphasizes the model's inherent capabilities in handling natural language tasks. By operating in this simplistic manner, Vanilla L.LM serves as a critical baseline for evaluating the performance of more advanced techniques that incorporate sophisticated prompt strategies or additional tools, thereby providing valuable insights into the effectiveness of various methodologies in natural language processing.
Self- seff-Consistency) (Wang et al., 2023) serves as baseline for enhancing the reasoning capabilities of language models. This approach generates multiple reasoning chains, which are then aggregated produce coherent summary. By leveraging self-consistency, CoT- SC improves the reliability of the model's s outputs, allowing for better performance in complex reasoning tasks. This structured process facilitates deeper analysis of the model's thought processes, providing a foundation for comparing more advanced reasoning strategies and understanding their impact on overall performance.
CoT--SCCCaiinoff-TTT
MetaAgent (Zhang et al., 2025h) is groundbreaking framework designed to automatically construct multi-agent systems by specifying the objectives of a given task. A distinctive feature of MetaAgent is its ability to generate these multi-agent systems without relying on external training data. This capability allows the produced multi-agent systems effectively address all scenarios within the specified task domain. The underlying architecture of the Mueti-Agent System is based on Finite State Machines(FSM)) which facilitates structured decision-making and state transitions, thereby enhancing the system's operational efficiency and adaptability.
OWLnOpen Web Language) (Hu et al., 2025) serves as a foundational framework for knowledge representation in multi-agent systems. By enabling agents to process and reason over complex data in a machine-readabl format, OWL is crucial for facilitating interoperability among diverse agents. It allows for the creation of ontologies that define intricate relationships and constraints within the environment, thereby enhancing collaborative behaviors among agents. The expressive power of OWL supports advanced inference capabilities, empowering agents to share knowledge effectively and make informed decisions. This framework establishes a robust baseline for evaluating and enhancing the performance of multi-agent systems in various applications.
Smolagent (Roucher et al., 2025) is a lightweight library designed to facilitate the development and implementation of Al agents that can think and operate using code. It emphasizes simplicity and efficiency, enabling users to create multi-agent systems with minimal code. Smolagent's architecture allows for smart threading, dependency management, and context sharing, making ideal for orchestrating complex tasks. By providing a streamlined framework, Smolagent serves as a foundational model for evaluating the performance and capabilities of more advanced agent-based based systems in various applications.
OAgents (Zhu et al., 2025) is a modular multi-agent framework that conducts a thorough empirical study of key agent components (planning, memory, tool use, test-time scaling) on benchmarks such GAIA and BrowseComp. It delivers great performance among open-source agent frameworks. Importantly, OAgents builds on the lightweight agentience model provided by Smolagent (which emphasises code-based agent orchestration and minimal overhead) and extends it with fine- fine-grained task decomposition, dynamic workflow adaptation, multi-source web browsing and more extensive tool and memory modules.
AWorld (Yu et al., 2025) is an open-source framework for large-scale agent- environment interac- tion, designed to operationalize the "learning from practice" paradigm in agentic AI. It features a hierarchical multi-agent architecture composed of specialized agents such as the Execution Agent, which performs primary reasoning and tool-use operations, and the Guard Agent, which intervenes at critical steps to verify and refine intermediate outcomes. AWorld adopts a modular design supporting dynamic supervision, context tracking, and distributed orchestration, enabling efficient coordination across diverse tasks and environments. By treating agents and tools as interchangeable components within unified orchestration layer, it facilitates flexible composition. concurrent execution, and fine- -rained control over reasoning workflows, illustrating a scalable and extensible paradigm for constructing adaptive multi-agent systems.
18

---

Published as a conference paper at ICLR 2026
Table 5: Hyperparameter settings for the Heuristic-BBi Saaristie-BBsed Adaptive Filter across different benchmarks. The symbols correspond the definitions Algorithm
Condition
Parameter (Symbol)
GAIA
HumanEval
MBPP
AIME
DROP
GSM-Hard
Inefficient
Step Check Interval (Taep)
8
6
4
Loop Detection Window (Toop)
5
5
3
3
Excessive
Length Threshold (Tea)
3000
A.3 IMPLEMENTATION DETAILS
In this section, we provide detailed description of how the conceptual framework of SUPER- VISORAGENT is implemented in our codebase. Our implementation is centered around the supervise_arde
function, which serves the primary entry point for all supervisory actions. We structure our explanation following the same Whut, When, and How logic presented in our main methodology.
A.3.1 WHAT TO SUPERVISE: THE ACTIONSTEP OBJECT
Our supervision targets the discrete interaction steps performed by each agent within the MAS. In our framework, every such interaction encapsulated a data structure we refer to as an Act inttioette ep object. This object contains all relevant information for single step, including the agent's 's thought process prrodel_outpure
the executed _ool_calls, the resulting observations, and an error attribute which populated if exception occurs. Our SUPERVISOR AGENT is implemented as a callback function that intercepts every Act onStep object generated by any agent in the system.
A.3.2 WHEN TO SUPERVISE: THE PRIORITIZED ADAPTIVE FILTER
To avoid the prohibitive cost of constant intervention, we employ a lightweight, LLM-free adap- tive filter. This filter is implemented as prioritized conditional chain the beginning of the supervise andervise_and___e aad_aaanddd__a_aeeeeeeaaaaaaaeee correct function. It evaluates each Act ionStep to determine if supervi- sion warranted. The conditions are checked in the following order of precedence:
The highest priority is to check if the observation contains a final report from sub-agent (identified by the presence of "<summary_or of soff_ff__ooo__ __rrk=""""" string). If so, triggers the specialized Adaptive Observation Purificat strategy to distill the findings for the manager agent.
Sub-Agent Completion:
2. Error Occurrence: If the step. error attribute is not None, the Proact i ve Error Correct i on strategy is triggered. Our implementation includes a defensive check to ensure this does not fire for known, non-critical tool failures that the base agent can handle.
Inefficient Behavior: If no error is present, we then check for inefficiency using our heuristic- based check for ineef fn fficiency function. This function detects patterns such hard loops (identical actions and observations) and excessive step counts for a given sub-task, triggering the Guidance for Ineffi cieccy strategy.
4. Excessive Observation Length: Finally, if none of the above conditions are met, the filter checks if the length the step. observeeeeeo rvat ons string exceeds pre-defined threshold Ten (3,000 characters in our implementation). If it does, the general type of Adaptive Observation Pur cat on strategy is activated.
If none of these trigger conditions are met, the step is approved by default, thereby avoiding any unnecessary LLMe supervision overhead. The filter's sensitivity governed by three key hyperparameters: Taep and Toop modulate the detection of inefficient behaviors, while Ten defines the threshold for identifying excessive observations. The complete logic of this heuristic-based mechanism is formalized in Algorithm 1.
Sensitivity analysis and hyperparameter configuration. Table 5 details the hyperparameter settings for our fteuristicBBased Adaptive Filter across different benchmarks. We conducted a sensitivity analysis the excessive observation length threshold (Zen) using the representative subset
19

---

Published as a conference paper at ICLR 2026
Algorithm 1 Fleuristic-Based Adaptive Filter Logic
Input:
Current execution history H. (sequence of steps), Current observation o, Error status
Require: Hyperparameters: Tstep (step check interval), Noop (loop detection window), Teen (length threshold)
Output: Boolean flag trigger, Intervention context
I: trigger False
2: c <- None
Priority I: Check for explicit runtime errors
3: i is True then
4: return (True, Cerror)
5: end if
Priority Check for inefficient behaviors
6: N + Length(H))
7: if N and N (mod Teee) then
Periodic strategy check
8:
return (True, Cuefficient)
9: end if
10: if VN Toop then
Repetitive loop detection
11: Arecen < GeallssTTolCalls(/4 window = TToop)
12: if [nUniuue(AAuead
then return (True, Cinefficiem) )
13:
14: end if
15: end if
Priority 3: Check for excessive information
16: if Lengtho)) > Tea then
return (True, Gexcessive 18: end if
19: return (trigger, c)
Token Cost
Accuracy
50.0%
1600k
1400k
46.
46.67%
46.57%
47.5%
Token Consumption
45.0%
1200k
Accuracy (%)
43.33%
1000k
800k
721k
730k
600k
561k
37.5%
35.0%
32.5%
Smolagent
1000
3000
5000
7000
Excessive Threshold
Figure 5: Sensitivity analysis on excessive threshold (7m)). Evaluated on GAIA subset (top-10 10 most token- intensive tasks per level).
of the GAIA benchmark (top-10 most token-intensin tasks per level), as illustrated in Figure 5. The results indicate that our method maintains robust performance across a wide range of threshold values, demonstrating its adaptability without significant degradation. Although slight performance peaks are observed at Men 1000 and Men = 5000, we selected 3000 as the default value. This choice strikes prudent balance between sensitivity (catching enough noise) and specificity (preserving useful context), ensuring optimal performance across diverse tasks while minimizing the risk of over-intervention.
Configuration for OAgents. For the OAgents framework (Zhu et al., 2025), we adjusted the parameters Tstep 6, Noop 3, and The 10000. The significantly higher Ten (compared to 3000
20

---

Published as a conference paper at ICLR 2026
in Smolagent) is necessitated by OAgents' architecture, which tends to generate extensive verbose outputs due to its complex tool usage and memory retrieval modules. A higher threshold essential here to effectively identify truly excessive information without triggering false positives on standard OAgents operations.
Adaptation for AWorld: An MCP-Based Approach. Unlike the external heuristic filter used in Smolagent, the integration with AWorld (Yu et al., 2025) leverages its native tool-use architecture. We implemented the SUPERVISOR AGENT Model Context Protocol (MCP) service, allowing to be dynamically discovered and invoked by the AWorld agent. This adaptation involves three key modifications:
We refined AWorld's system prompt to enforce protocol where the agent must invoke the SUPERVISORAGEN, during critical phases-Specificall, "Information Gathering" and "Thinking Process Reviewing". This ensures the SUPERVISOR.AGENT acts as mandatory gatekeeper for logical consistency.
Mandatory Invocation via System Prompt:
Capability- Routing: We defined specific MCP schema where the SUPERVISORAGENT broadcasts its capabilities, including Error Root Cause Diagnosis, Strctureed Informat on Synthesis and Workflor Effi ciency Assessment. This allows the AWorld agent to match its current execution status (c.g., encountering an exception or synthe- sizing search results) with the appropriate Supervisor function.
Trigger Mechanism: Instead of counting token length, the trigger is semantic. Explicit returns from AWorld's system
triggers for error aoroo_aaallsis,, while the and ine ff caanccaanalllaeeeeeeeeeeeeeaeeaa modes are seamlessly integrated into the MCP process flow facilitate output verification.
sub_agent_resulls ceast__rsuu___sthhhheee
A.3.3 How TO SUPERVISE: THE INTERVENTION PIPELINE
Once the adaptive filter flags an interaction, the supervise and aad__correcttt function executes stree-stage intervention pipeline:
1. Context Aggregation Before making a decision, the Supervisor aggregates context window (W). This process involves retrieving the global task (G) and the agent's local task (L), formatting the agent's recent local action history (Ti) via the Format_local__aettt_ttooo__a___a and generating summary the current step (S) using the _unma r ze ze_ interacteion function. For inefficient behavior, the full global trace (Tg) is also included.
for prompt function,
LLM-based Decision Making The aggregated context is then compiled into a specialized prompt tailored to the triggered supervision type (e.g., Proactive Error Correct ion). This prompt instructs our main model (e.g.. GPT-4.1) to analyze the situation and return a structured JSON object containing its analysis chosen act i in (from the set (approve, correct correct_observal obwerrattttteeeeeee.e ion, action.
run veri cat ion)). and the necessary pa parmeters to execute that
3. Action Execution
The returned JSON is parsed, and the chosen action is executed.
The original step. observations is entirely replaced with the new_observatil roo provided in the parameters. A "[Supervisor's Note: ...J" is prepended to inform the agent of the modification.
correct_ cetrect_obbeeraticnn
guidance: The guidance string from the parameters is appended to the end of the existing step. observat intttionn ions, leaving the original sensory data intact while providing corrective hint.
provide leooiide_gidanneeeeeeeeeee
run_veri fication. The task parameter passed a dedicated, fully- verification agent, and its conclusive findings are appended to the step. observations.
A.4 EXTENDED EXPERIMENTAL ANALYSIS
A.4.1 OVERHEAD ANALYSIS OF SUPERVISORAGENT
Token overhead analysis The token overhead SUPERVISOR itself is shown Table and Figure 6. We analyze the token consumption the GALA validation under different pass @
21

---

Published as a conference paper at ICLR 2026
Table 6: Token efficiency analysis GAIA validation set. Comparison of token consumption different pass settings.
Method
Avg. Tokens (K)
L1 Tokens (K)
1,2 Tokens (K)
L3 Tokens (K)
pass
Smolagent Supervised MAS
527.76
298.51
619.59
691.33
314.07 140.99
220.63 126.09%
342.18 L44.77%
411.58 440.47%
Supervised MAS (NET)
371.12 129,689
258.28
(13.48%
404.96
J34.648
489.22
(29.23%
pass
Smolagent Supervised MAS
467.19
275.85
548.02
589.92
329.51 (29.47%
231.96 115.91%
354.21 (35.374
446.64 124.29%
+ Supervised MAS (NET)
389.55 (16.62%
270.07
12.10%
420.97
123.18%
529.20 110.29%
pass
Smolagent Supervised MAS
502.40
282.14
605.05
611.87
312.06 137.89%
236.28 J16.25%
342.36 J43,22%
366.31 140.13%
Supervised MAS (NET)
369.52 126.45%
276.84 11.88%
409.05 332.399
427.72 130.108
Token Saved
Supervisor 60k
Smulsgont 528k
Analageno 487k
502k
SMAS
Supervisor 57k
SMAS
Supervisor
SMAS 33IK
57t
GAIA pass@1
GAIA pass@2
GAIA pass@3
Figure 6: SUPERVISORAGENT overhead on the GAIA benchmark.
settings. The results indicate that integrating SUPERVISOR leads significant reduction in overall token usage across all complexity levels (L1, L2, L3) and pass @ k configurations. Specifically, SUPERVISOR AIGNNTTRRRRTTTT achieves average token saving of 35.95% across all settings compared the baseline Smolagent. This substantial decrease in token consumption highlights SEPERVISSORAGETT effectiveness in optimizing the multi-agent system's 's efficiency by reducing unnecessary interactions and streamlining the reasoning process.
Notably, even when accounting for the additional tokens introduced by SUPERVISOR AGENT'' super- visory interventions, the net token consumption remains significantly lower than that of the bascline (already reported in main content). And the token overhead of SUPERVISOR about 15.45% % in average of the total tokens used in the Smolagent baseline. This demonstrates that the benefits of improved efficiency and reduced redundancy far outweigh the costs associated with supervision.
only contains
Latency Overhead Analysis. Table and Figure 7 analyzes the temporal impact of SUPERVI- SORAGENT. While integrating the supervisor introduces average latency increase of 37.27 %. this translates to an absolute delay of less than 1.5 minutes per task. Crucially, the ablation study reveals that Adaptive Observation Purification is the primary driver of this latency. Notably, the w/o Purification variant exhibits a runtime nearly identical to the baseline (236.21s vs. 233. 233.96s). This indicates that the overhead is strictly tied the processing of excessive information. We consider this strategic trade-off:: exchanging a modest temporal cost for substantial economic (token) savings and enhanced system robustness.
A.4.2 PERFORMANCE ANALYSIS ON TOKEN-INTENSIVE SCENARIOS
Complementing the comprehensive ablation study on the full GAIA benchmark (Table 3), this section zooms in the most demanding scenarios: the top- 10 most token-intensive tasks per GAIA level. This analysis aims to evaluate the scalability and robustness of SUPERVISORAGENT under extreme computational loads.
22

---

Published as a conference paper at ICLR 2026
Table Ablation study of SUPERVISORAGENT components regarding Latency the GAIA validation set. Average latency (in seconds) reported for different complexity levels.
Method
Avg. Avg. Latency (S)
Level Avg.
Level Arg. Lat. Level Avg. Lat.
Smolageut
50.91
233.96
155.83
247.47
348.58
SMAS (wlo Correction)
47.88
280.77 130rss
193.98
124.885
276.50 111.76
471.81 135.599
SMAS (wlo Guidance)
48.48
271.95 136248
211.04
135439
291.22
117,88
332.38 LA45%
+ SMAS (w/o Purification)
49.70
236.21
nawes
137.00
11200N
252.01 11,898
386.19
100.796
SMAS
50.91
321.15
233.11
320.93
501.31
Smolagent
w/o Correction
wil Guidance
W/o Purification
+SMAS
500
400
Latency (s)
300
Th
100
Average
Level 1
Level
Level
Figure 7: SUPERVISORAGENT latency on the GATA benchmark.
Table 8: Ablation study of SUPERVISORAGENT'S components on the subset of GATA benchmark (top-10 tasks per GAIA level).
most
Method
Avg. Acc.
Avg. Token
Level I Avg. Token
Level 2 Avg. Token
Level 3 Avg. Token
Smolagent
40.00
1,446,526
933,013
2,037,437
1,369,131
SMAS (wlo Correction)
40.00
719,075
15023%
426,786
.54266
755,543 us299%
974,895
138.99
+ SMAS (wlo Guidance)
40.00
706,831
15:
453,623
_SL38%
913,109 155.18%
753.761
144.05%
+ SMAS (wlo Purificution)
46.67-6u76
851,747 Jil.e
585,411
L37228
990,769
151.37
979.061
49%
SMAS
46.67 16.7%
721,332
150136
522,364
.4005
960.694
680,939
Amplified Efficiency and Robustness. As detailed in Table 8, the benefits of SUPERVISOR are significantly amplified high-complexity regimes. While the average token reduction on the full dataset is 29. 68,,, SMAS achieves remarkable 50.13% reduction on this intensive subset. More importantly, unlike the full dataset where accuracy remains stable, SMAS yields distinct accuracy improvement of 6.67% (from 40.00% to 46.67%) on these hard tasks. This suggests that as task complexity and context length increase, the Supervisor's 's interventions become indispensable not just for cost-saving, but for enabling the agent to complete tasks that were previously intractable due to context overflow or reasoning derailment.
Component Contribution in Extremes. The ablation results this subset reinforce the synergistic roles of our three strategies. The w/o Purification variant shows a drastic drop in efficiency (savings drop from 50.13% to roughly 40%), confirming that Adaptive Observation Purification is the primary countermeasure against the exponential token growth in complex tasks. Meanwhile, the removal of Correction Guidance leads sharp decline in accuracy back the baseline level (40.00%), verifying that these modules are the key safety guardrails that allow the system to navigate long-horizon tasks successfully.
A.5 FAILURE MODE ANALYSIS
While SUPERVISOR AGENT demonstrated robustness across benchmarks, it relies on backbone LLMMs and is thus subject to their inherent limitations. We identify three primary failure modes and their implications.
Information Loss during Purification. The Adaptive Observation Purification module faces a trade-off between context reduction and information preservation. In extreme observed in the
23

---

Published as a conference paper at ICLR 2026
OAgents framework, where single observations exceeded 200,000 characters, the Supervisor risks hallucinating or omitting critical details during compression. However, our empirical results suggest that the system's 's resilience to context overflow generally outweighs the cost of granular information loss, as evidenced by the overall token savings and success rates (see in Table 4).
Ineffective Guidance and Loops. The Inefficiency Guidance module may occasionally provide suboptimal advice or fail to break a stubborn loop. To mitigate the risk of the Supervisor itself becoming source of latency (e.g., engaging in infinite correction loop with non-responsive agent), we enforce a hard constraints of maximum two guidance interventions per sub-task. While this design prioritizes bounded latency over guaranteed resolution for the hardest tasks, it effectively prevents runaway costs.
Variance in Trigger Frequency across Backbones. Contrary to the assumption that supervisor acts uniformly, our analysis reveals that SUPERVISORAGENT exhibits different operational behaviors depending on the backbone model's capability. For instance, shown in logs with Qwen3-235B. less capable models trigger the Error Correction module significantly more often due to frequent basic failures (e.g., malformed JSON tool calls). This frequent firing increases the Supervisor's token overhead, partially explaining the lower net token savings compared to GPT-4.1 (see in Figure 4b). Conversely, stronger models may trigger supervision enough to their subtle logic errors. This highlights that while the framework is model-agnostic, the efficiency gains are correlated with the backbone L.M''s adherence to instruction following.
rarely if the heuristic filter is sensitive
A.6
CASE STUDY
GAIA Benchmark Case Information
Task ID: 200244448-----4--0000036--600000
Level:
The brand that makes these harnesses the dogs are wearing the attached pic shares stories from their ambassadors their website. What meat is mentioned in the story added Dec 8th 2022?
Question:
Attached iamge: 5b2a14e8- 5b4444446665544----0244440099913
Smolagent
Activate first search aagent: Failure attribution: page_ down
pown 10 times and end up searching
Output of first search agent:
Observations: ### Task outcome (short version)
Despite a detailed investigation and navigation through the RC Pets official website and blog section, I was unable to locate the ambassador story posted on December 8th, 2022. Therefore, I cannot report any mention of meat in its content.
2. Activate another search agent:
Plan: turn access historical versions of the website
page_ page_down for times
Then web search other resources
Output of search agent
Observaticns: ### Task outcome (short version)
Despite detailed investigation and navigation through the RC Pets official website and blog section, I was unable to locate the ambassador story posted on December 8th, 2022. Therefore, cannot report any mention of meat in its content.
24

---

Published as a conference paper at ICLR 2026
Activate third search agent:
Plan: Thought: Since all direct website and archive attempts failed, will extend my search to include RC Pets' official social media (Instagram, Facebook, Twitter) and broader web, as brands often share ambassador stories across platforms and tag date posts.
web_search for 8 times
Output of search agent
Observations ### 1, Task outcome (short version):
dence was found of any RC Pets anbassador or partner story posted on December 8th, 2022, on official RC Pets Instagram, Facebook, Twitter, or the broader web. No blog posts, community group stories, ambassador reposts regarding RC Pets for that date menth could be located, and no relevant content mentioning meat was found.
Original final answer:
Final answer: After an exhaustive investigation across available RC Pets channels (official website, blog, and secial media), Wayback Machine, pet influencer blogs, hashtags, and pet community forums, ambassador story repost by RC Pets or affiliated
ambassadors/partners could be found for December 8th, 2022. story mentioning any meat was identified for the date or month, and no content relevant to the request is accessible through any official or community web source. A11 logical avenues for retrieving the answer attempted and exhausted.
Reformulated answer: Unable to determine
Smolagent + SupervisorAgent
Supervision type: Basic eetrrcctetttion - 8 times
Supervision type: befficiency_nnalyyse times
First time
Providing guidance to agent: 'Stop paging through blog manually. Instead, use the webl_search tool or the Ruffwear website's s internal search to find the specific ambassador story posted December 8th, 2022. You cculd search for RRfffwear ambassador stery December has date-filter or text search. Once you locate the precise article, examine it for any menticn of meat, noting a quote for context.'
guidancc:
2022'
check if the blog section
Analysis: The agent is stuck repeatedly using 'down' without making meaningtul progress toward locating the December 8 th, 2022 story among the 'ambassadors' blog entries. There are 82 pages, and there's no indication the agent is close to the correct blog post. This paginated browsing is highly inefficient when direct search or filter by date would be much faster. It's overwhelmingly more efficient blog post by date (' Decembeer 8th, 2022') using the webl_search tool or any site search functicnality if available.
instead search
specific
Second time
approve
Supervision type: sub agent result synthesis
time
25

---

Published as a conference paper at ICLR 2026
Original output length: 47902
Lenghh after process: 1438
content .""
**# Official Ruffwear Ambassador Stcries Section
Ruffwear ambassador stories are shared in 'Explored: A Ruffwear Blog'
story Posted December 8,
Title: "Snow Camping With Theresa & Cassie"
Author:
Theresa silveyra
Date Published: December 8, 2022 ("2022-12-08 08:00:01 -0800")
Story CRL: http:////faaaa....o//s//////////
with-theresa--assii
### Meat Mentiened in the Stery
The only specific meat mentioned is "bacon".
Specific qucted context from the story:
"Of course, barely a minute had passed before Cassie's longing eyes and dangling tongue signaled
desire to play perhaps, desire for treats. She led the way back, her fluffy white tail raised high excitement, ready to take on this new year and, more likely, ready to devour the New Year's Day bacon being cooked in camp."
### Confirmat
of Accuracy
The article and details are confirmed by both page metadata and di rect content search.
Searches for "meat", "chicken", "beef", "turkey", "pork", "fish "salami", and "sausage" yielded other results; only bacon was present.
IH Additional Context
Author attribution and publicaticn date are validated. Other foods or treats are referenced, but bacon the only explicit meat in the text.
### Direct Answer
The meat mentioned in the Ruffwear ambassador story posted December 8th, 2022 bacon.
A.7 PROMPTS
Base prompt for SUPERVISOR
base_prompt EuI"
Role: You are expert supervisor in multi-agent system. Your role is to monitor agent's actions, alignment with the main goal, correct errors, and optimize the workflow.
Objective: The overall objective (Global Task) is: "gllcbal_task)
Agent context:
You are currently reviewing an action from the agent (agent_nand) I.
This agent's specific sub-task (Local Task) is: [local_task)', Here is the current local execution trace: (local trace_strr
26

---

Published as a conference paper at ICLR 2026
Here is the summary of the agent's ss latest thought process and the resulting observation: {sunmary] [eurrent_state_sumaal
Rules:
1. Assess Necessity: First, assess if intervention is truly necessary. If the agent's action and observation are correct and productive, use the "approve" actien. Avoid unnecessary interventions.
2. Be Decisive: when an intervention is needed, choose the most effective action to move the project forward.
3. Output Format: Your response MUST be valid JSON object.
Actions:
Your available actions are:
'approve': The agent's action is correct and requires no changes
coorrect_observation The observation contains errors or can be significantly improved (e.g., filtered, summarized, extracted).
You will provide a corrected versicn.
The observation is correct, but the agent's step flawed. You will provide hint corrected reasoning to guide the agent.
thinking
You have doubts about the factual accuracy of the observation and need an external assistant to verify it.
Your response be
object with the following ng structure:
"analysis"; "Your brief analysis of the situation, explaining your reasoning for the chosen action.", "action": "ONE of the available actions: l'approve', correct__bbervationn,, 'provice_guidance', "run_verffication" "parameters";
"non_obscrvation": "IF acticn is 'correet__ooervatien"" provide the refined observation here.", "guidance"; "IF action is hint or instruction for the agent's next thought process. ", "task": "IF action is 'run_verification provide the verification question for the assistant.*
provide clear
nnn
Prompt for "error_occurrence"
Base Prompt:
Addtional Prompt:
expert Debugger and AI Diagnostician. Your primary goal is to understand the root cause of an errer and provide the most effective solution to get the agent back on track.
**oole** :Yo
**Situation*::
The agent's last action resulted in critical error, which is detailed in the "sumnary" the agent's action below. **pproval is not an option; you must intervene.**
Your Debugging Framework (MANDATORY)
27

---

Published as a conference paper at ICLR 2026
Before generating your JSON output, you MUST follow this structured thinking process:
**Step 1:
Analyze the Error.:
-What the precise error message and type (e.g., TTcll 'Python Exception', 'APIError)??
;
**Step 2:
Examine the Contexte
Review the Ibcal_executionttraec and the agent's 'thought
process leading to the error.
What was the agent *trying* to accomplish?
Was the tool call or code it executed ( ('summary' section) syntactically correct but logically flawed?
Step 3: Root Cause Diagnosisms
Based on the error and context, what is the single most likely root cause?
(e.g., "The agent passed natural language string to tool expecting mathematical expression. "The agent is trying access file that does not exist.")
**Step 4: Formulate a Solution Strategy**
Based the root cause, determine the best intervention: error can be fixed by correcting the agent' ** next thought process or acticn**, choose
This is the most common case for logical errors. If the error was caused by faulty information in the previous observation that the agent is acting upon, choose 'correct_observath
If you lack critical information to solve the errer and need to consult an external source, choose run veriviiiiii......
AA--- YOUR ACTIONABLE OUTFUT (JSON) ---AA
Based on your diagnosis, provide your final decision in the JSON format.
Prompt for "inefficiency_behaviiii
Base Prompt:
Additional Prompt:
f"nn
: You are a pragmatic and experienced AI workflow strategist Your primary goal is to ensure the agent team achieves its task in the most efficient way *.from its current state...
**Situations=:Ar inefficiency trigger has been activated for agent "agentt_namei_ **This is flag for you to review, NOT confirmation of problem.** The agent might be engaged in necessary, methodical process.
**Global execution Trace*:: @global__trac__e
**--- Your Decision Framework (MANDATORY)
Before generating your JSON output, you MUST follow this structured thinking process:
28

---

Published as a conference paper at ICLR 2026
**Step 1: Goal Plan Inference**
Based on the 'Global Execution Trace', what is the agent's immediate, implicit plan?
(e.g.r "The agent is clearly trying to collect all rews of data table by repeatedly using 'page_down'." ".
**Step 2: Progress Assessment**
Is the agent making tangible progress towards its inferred goal?
each new step yielding new, relevant informaticn (even it's just more rows of the same table)?
How close is the agent to completing this sub-task? (e.g., It is page 10 of 13, it is very close to getting all the data.")
** Step 3: Cost-Benefit Analysis of Intervention
**Compare two costs**:
**Cost A+*: The estimated cost (time, tokens) of letting the agent **continue*. its current, perhaps clumsy, path to completion.
Cost BA*: The estimated cost of *einterruptingu the agent, guiding it to new path, and having it **start over that new path.
the agent "one step away" from solving its sub-task? If so, interrupting it is almost always the wrong decision, even if a theoretically "better" path exists.
**CRITICAL QUESTION=::
**Step 4: Decision and Justi catificationa Based on the analysis above, decide between 'approve" and provide guiideceuddd'...
YOUR ACTIONABLE OUTFUT (JSON)
You must choose ONE of the following two actions:
*+1. If you decide the agent should continue:*.
**Conditione:: The agent is making clear, incremental progress AND is close to completing its sub-task (Cest A Cost B).
**Action*:: MUST be '"approve"',
**Analysis*:: Briefly explain *why+ the agent's current path, while perhaps repstitive, is the most pragmatic way ferward from its current state. (e.g., "The agent is methodically paginating through table to gather all data. Altheugh repetitive, this is a valid and necessary process. It is page 10 of 13 and about to succeed. Intervention would be disruptive.")
If you decide the agent is truly stuck:** ++2.
+*Conditions:: The agent is in non-productive leep (e.g., getting the same observation repeatedly) OR alternative path is overwhelmingly more efficient. and the agent not close to finishing (Cost в << Cost A)
**Action*:; MUST be pprovide_guiidncee
**naayysis*:: Briefly explain the root cause of the irefficiency.
**Guidance**: The 'quidance' parameter MUST contain a clear, concrete, and actionable instruction that represents significantly better strategy. (e.g., "Instead of scrclling, use the 'web_search' tool with the query 'who had the most for the 1977 Yankees' to get the answer directly.")
29

---

Published as a conference paper at ICLR 2026
Prompt for "excessive observation length"
E*nn
# Role: AI Agent Observation Compresser
You are specialized data compression model for an AI agent. Your sole purpose is to process raw observations (HTML, text, etc.) and reduce their token count while strictly preserving their structural integrity and all potentially useful information.
**| Core Principles Pf*.
1. **Context sontexx-Agnostiic: You have NO knowledge of the agent's overall goal or past actions. Do NOT try to infer the task. Your compression must be generic and unbiased, preserving information that could be useful for ANY potential task.
2. **Frescrvation Over Compressicn:** It is critically important to avoid over-sumarrzzation. Losing a potentially key piece information is greater failure than not compressing enough. The output must retain enough detail for the agent to make informed decisions.
3. **Structural Integrity: The output's structure (headings, lists, paragraphs, HTML hierarchy) must mirror the input's structure. Do not merge distinct sections.
4. **Freserve Metadata**: Always keep leading lines like "Address " verbatim.
...." " "Viewport:
#Compression R
Based on the type of content, apply the following rules:
### *xpppe 1: For HIML Content**
Your goal to simplify the HTML to its semantic and structural core, removing presentation-foused noise. 1. **Simplify Tags:** Remove nen-essential attributes.
**REMVE attributes like:** 'class', 'id', 'style', onclick', 'onmouseover', and any 'data-x' or 'js-x' attributes. These are primarily for styling and scripting, not for content structure.
**KEEP essential attributes:** 'href', 'sre', 'alt', title', 'aria-label', 'placeholder', 'value', These attributes contain crucial information for navigaticn and interaction.
2. +.Remove Non-Visible Content:** Completely remove 'script >', '<style>', and HTML comment blocks.
3. **Preserve Content:* Keep ALL text content within tags exactly as it is. Do not summarize the text inside the HIML.
4. **hittspacc::* Condense multiple spaces, newlines, and tabs in the HTML structure into a single space where appropriate improve reacability without losing structure. **xxample::.
**rriginal:** '<td < style='padding >x;'><a http://wwww//////////sss///see// title="Some bink">25</a></ta
**Compressed: '<ta><a href="/viki/ link" title="
Some Link /ink///2</a><</t>>
### ++yppe 2: For Plain Text content*
Your goal is make the text more concise without lesing factual information or its original layout.
**Retain Key Information:** Fully preserve all named entities (e.g., people, organizations, locations), numbers, dates, codes, IDs, and any factual data.
1.
2. **Condense Prose:** For aescriptive sentences paragraphs
rephrase them to be more direct. Remove filler words,
30

---

Published as a conference paper at ICLR 2026
reduncant phrases, and overly elaborate adjectives. Hewever, do NOT eliminate the sentence entirely.
3. **Maintain Structure:** If the input text has multiple paragraphs, bullet points, or numbered lists, the output MUST have the same structure. paragraph.
flatten list into single
* *Example:**
**Original:** "The company, officially known as The International Business Machines Corporation (IBM), is very large and influential American multinational technology corporation that has its headquarters located in Armonk, New York, and it was originally founded all the way back in 1911."
**Compressed: "The International Rusiness Machines Corporation (IEM) an American multinational technology corporation headquarterec in Armonk, New York, founded in 1911."
**UV Final Instruction Ui+.
Process the following observation according to the rules above. Provide only the compressed output, without any extra text, explanation, preanble. (observation) ввв
Prompt for "result synthesis"
# this is for synthesis the final answer from sub-agent Base Prompt:
Additional Prompt:
E"n
R=ooe You are an expert Intelligence Analyst working for a manager agent. Your task is to process a verbose report from a sub-agent (e.g., search specialist) and synthesizc a direct, comprehensive, and clean answer for your manager.
YOUR INPUTS
**1 The Manager's s Request (Immedi ate Goal)
""(occal_tee task)
+22 The Overall Mission (Global Goal)**: +*: "[global_task)"
**3, The Sub-Agent's Full Field Report (Raw Observation) **;
(summary)
(Note: The 'sumnary' variable here centains the sub-agent's full, multi-part answe)
*---- YOUR CRITICAL TASK
Your sole task is to read the ENTTRE "Field Report" (including the short version, detailed version, and the summary of work) and synthesize single, clean, and self-contained response that fully and comple:ely** answers the "Manager's Request".
**Critical Rule for Synthesis**:
**Freserve Semantis Structure**: when synthesizing, you MUST maintain the original information's s hicrarchy. If the source
31

---

Published as a conference paper at ICLR 2026
contains headings, chapters, articles, or numbered/bulleted lists, these structural elements **MUST be preserved** in your output to give context to the data points belcw them. ** Do not flatten structured document into simple, unstructured block of text.**
Your Internal Thought Process (MANDATORY) ..:
1. **Deconstruct the Manager's s Request**: What are the specific pieces of information the manager is asking for? Create mental checklist.
**Scan Entire Report*:: Read -agent's s report to find the answers for your checklist. The most valuable details are often in the "extremely detailed version"
parts the sub
the "summary of work".
3. vaSynthesize, Don't Just Extract*:: Combine the findings into a coherent, fluent, and direct answer. not simply copy the "short version". Your answer must be comprehensive enough to prevent the manager from needing ask follow-up questions.
Example+*:
+anagerr's a Request*:: "Find the number of encoder layers in the BERT-Base model."
**Sub-Agent's Report*:: (A long containing "Short version: layers", "Detailed version: Sectiio of the paper states 1=12 for BERT-Base..... etc.)
*+Your Ideal Synthesized Output**: "The BERT-Base model has 12 encoder layera (I-12), speci in Section of the criginal paper by Devlin al., 2018."
* :Actionns: action MUST be cecerect_observation
PParameter=w:Providde your final, synthesized in the " new_observation" parameter.
32


Reviews:
[{"review_id": "HgBIHMvc8A", "reviewer": "Reviewer_hEWr", "summary": "The paper addresses two critical challenges in MASs, 1) Operational Inefficiency (measured through token usage here), 2) Robustness Failures (error propagation, misinformation, etc.). Unlike prior methods that focus on post-hoc error attribution and diagnosis, the paper introduces SupervisorAgent, a meta-agent, that monitors the interactions in an MAS (agent-agent, agent-tool, agent-memory), uses an LLM-free \"adaptive-filter\" to determine high-risk interactions, and if invoked, the SupervisorAgent intervenes through actions like proactively correcting errors, guiding agents away from inefficient behaviors (like loops), and purifying excessively long or noisy observations to reduce token load.\n\nThe authors implement the SupervisorAgent over the SmolAgents framework, and compare against strong baselines across several benchmarks, finding that the intervention reduces token consumption by average as much as 29.45% without losing accuracy. The paper concludes with an ablation study of the various SupervisorAgent interventions, finding that purification leads to efficiency gains, whereas correction and guidance are necessary for performance.", "questions": "1. Do the reported token costs for SMAS in all figures/tables take into account all supervisor token costs?\n2. Can the authors describe the overhead introduced by SupervisorAgent, not only in terms of the token costs, but also in terms of wall-clock times, or number of additional LLM calls invoked (especially relevant since once a SupervisorAgent intervention will be invoked, the agent system's execution will need to be halted).\n3. Can the authors' comment on the interaction of SupervisorAgent when implemented in other MAS frameworks, which could include significant interaction with external environments (for example, through MCP)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a novel, lightweight, non-intrusive monitoring of MAS. This could potentially become a major MAS development pattern going ahead.\n2. The paper is well-to-read, and educational from the perspective of understanding MAS design, issues with current MAS, and nicely introduces the proposed intervention.\n3. Evaluation: The authors validate the intervention across 6 benchmarks, and across 3 leading LLMs, including open and proprietary models.", "weaknesses": "1. The overhead introduced due to SupervisorAgent is not described in detail. \n2. The details about the working of adaptive filter are not described. Since one of the core features of the SupervisorAgent is \"lightweight\" monitoring, the authors should clearly describe how the adaptive filter works without LLMs, especially to identify \"inefficient behavior\" which seems to be a highly subjective criteria unlike the other 2 high-risk interactions identified.\n3. Many MAS proposed in the past have included an additional agent that acts as a verifier or quality-assurance agent (for example, ChatDev includes a \"Tester\" agent). However, one issue with such interventions is that often the verifier can itself fail, introduce misinformation, or propagate errors. The authors do not discuss the failure modes or failure rate of complex SupervisorAgent actions like \"correct_observation\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fTVFuFEwII", "reviewer": "Reviewer_vopL", "summary": "This paper addresses two critical and intertwined problems in modern Multi-Agent Systems (MAS): operational inefficiency (excessive token consumption) and lack of robustness (failures from error propagation). The authors argue that existing methods are either post-hoc (analyzing failures after they occur) or design-time (optimizing the agent architecture itself), with a lack of tools for proactive, real-time intervention .\n\n\n\n\nTo fill this gap, the authors propose SUPERVISORAGENT, a lightweight, modular meta-agent framework designed for runtime supervision. The framework's core is a lightweight, LLM-free adaptive filter that monitors agent interactions. This filter triggers interventions only at critical junctures, which it identifies as: 1) explicit error occurrences, 2) inefficient behaviors (like loops), or 3) excessive observation lengths.\n\n\n\n\n\nWhen triggered, the SUPERVISORAGENT (which is an LLM) analyzes the system's state using a rich context window and performs one of four actions: approve, provide_guidance, correct_observation, or run_verification .\n\n\nThe authors demonstrate the system's effectiveness by applying it to the Smolagent framework on the GAIA benchmark, achieving a 29.45% reduction in token consumption without any loss in task success rate. The paper further validates this approach's generalizability across five other benchmarks (for math, code, and QA) and multiple state-of-the-art foundation models .", "questions": "1. Do the token savings reported in Table 1 and 2 (e.g., the 29.68% on GAIA)  account for the tokens consumed by the SUPERVISORAGENT itself? If not, what is the net token savings for the entire system (base agents + supervisor)?\n\n2. The ablation study (Table 3) shows that \"Purification\" is the main source of token savings, while \"Correction\" and \"Guidance\" are key for accuracy . This suggests that for tasks where the baseline is already robust (high accuracy), a \"Purification-Only\" supervisor might offer the best cost-benefit. Have you considered this variant?\n\n3. The \"Excessive Observation Length\" trigger is a hardcoded \"3,000 characters\". How sensitive is the framework's performance (both token savings and accuracy) to this hyperparameter?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper's framing of MAS inefficiency as a runtime process control problem is a fresh and valuable perspective.\n\n\nThe hybrid \"LLM-free filter + LLM supervisor\"  is a very strong and practical design choice that balances cost and capability.\n\n\nThe 29.45% token reduction on GAIA with no accuracy loss is a headline-worthy result, strongly supported by the data.\n\n\n\nThe ablation in Table 3 is a highlight, perfectly justifying the three-part design of the supervisor's intervention strategies (Purification for efficiency, Correction/Guidance for robustness) .\n\n\nThe method is shown to be model-agnostic (Figure 4b) and benchmark-agnostic (Table 2), proving it is a general framework and not a one-off trick.", "weaknesses": "1. The paper's main claim is \"Stop Wasting Your Tokens\", and it reports a ~30% token reduction. However, it appears this 30% saving applies only to the base agents (Smolagent). The paper never states the **token cost of the SUPERVISORAGENT itself**. The supervisor is an LLM (e.g., GPT-4.1)  and is called every time the filter is triggered. To make a true claim of efficiency, the paper must report the net token savings (i.e., Baseline_Tokens - (SMAS_Agent_Tokens + Supervisor_Agent_Tokens)). Without this, the primary claim of token reduction is incomplete.\n\n2. The paper focuses exclusively on token efficiency. However, the proposed framework introduces an extra LLM call into the execution loop every time an intervention occurs. This will \"stop the world\" and add wall-clock latency. It's possible the system saves tokens but becomes significantly slower to run. A comprehensive \"efficiency\" analysis should include latency, especially when the interventions are designed to cut down on inefficient loops (which also consume time).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wv47CSHEwF", "reviewer": "Reviewer_UCU6", "summary": "This paper addresses critical challenges in MAS, such as excessive token consumption and error propagation, by introducing SUPERVISORAGENT, a lightweight, non-intrusive meta-agent framework for runtime supervision. The approach is innovative in its use of an LLM-free adaptive filter to trigger targeted interventions, enabling proactive error correction, inefficiency guidance, and observation purification without modifying the base agents' architecture. The experiments are comprehensive, spanning multiple benchmarks and models, and demonstrate efficiency gains on a specific baseline. However, the paper's claims of a \"significant Pareto improvement\" are undermined by unfavorable comparisons to SOTA baselines presented in its own results, raising concerns about the method's broader applicability and impact.", "questions": "Why was SUPERVISORAGENT not applied to SOTA baselines like AWorld? If integrated, could it further reduce AWorld's token costs (e.g., from 128K to below 100K) while maintaining its 60% accuracy? If incompatible, please explicitly discuss this limitation in the paper.\n\nHow does SUPERVISORAGENT's own overhead (e.g., from context aggregation or LLM calls in interventions) factor into total token costs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper effectively highlights underexplored pain points in MAS, such as runtime inefficiencies from error propagation and excessive observations, which lead to high token costs (for example, up to 2 million tokens on GAIA tasks). This framing is fresh and relevant, building on recent MAS advancements while addressing a critical paradox: increased autonomy often reduces robustness and economic viability.\n\nSUPERVISORAGENT is a lightweight, non-intrusive framework that integrates seamlessly with existing MAS without architectural changes. Its LLM-free adaptive filter for triggering interventions (e.g., at agent-agent, agent-tool, or agent-memory points) is efficient and proactive, focusing on runtime process control rather than static optimizations. The action space (e.g., approve, provide_guidance, correct_observation, run_verification) is well-formalized, and the memory-augmented context window enables holistic oversight. This design is modular and broadly applicable, as validated across diverse benchmarks and models (e.g., GPT-4.1, Gemini-2.5-pro, Qwen3).", "weaknesses": "The paper claims a \"significant Pareto improvement\" (reduced tokens without compromising success rates), but this holds only against the weak Smolagent baseline (50.91% accuracy, 527.76K tokens reduced to 371.12K). In contrast, the paper's own Table 1 shows SOTA baseline AWorld achieving higher accuracy (60.00%) at much lower cost (128.27K tokens). The supervised system (Smolagent + SMAS) is thus both less accurate and more expensive than AWorld, contradicting the Pareto claim. While the authors justify Smolagent for isolating \"agentic interactions\" over external tools, this does not excuse avoiding direct SOTA integration or comparison. It makes the method's impact limited.\n\nThe LLM-free filter relies on hardcoded heuristics (e.g., detecting repetitive actions or observations exceeding 3,000 characters). These thresholds lack justification, sensitivity analysis, or ablation, raising concerns about brittleness and potential overfitting to Smolagent or GAIA. Without robustness tests (e.g., varying thresholds across datasets), the filter's generalizability is questionable.\n\nToken reductions vary inexplicably between main results (29.68% on full GAIA) and ablation on 30 high-token tasks (50.13%). This suggests benefits are skewed toward long-tail cases, potentially masking uneven performance. The abstract and conclusions claim \"average\" savings (e.g., 29.45%) without clarifying this distribution, which could mislead readers and requires more transparent analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OXIgSnFRIf", "reviewer": "Reviewer_WiAV", "summary": "The paper introduces SUPERVISORAGENT, a lightweight, runtime “supervisor” that sits on top of a multi-agent system (MAS). A non-LLM prioritized adaptive filter detects four trigger conditions (sub-agent completion, error occurrence, inefficient behavior, excessive observation length) and, when activated, a three-stage intervention pipeline decides among four actions: approve, provide_guidance, correct_observation, run_verification. The goal is to reduce token usage and improve robustness without modifying underlying agents. Experiments on GAIA (Smolagent) and several code/math/QA benchmarks report sizable token savings with roughly maintained—or sometimes improved—accuracy, and an ablation suggests observation “purification” is the main driver of efficiency, while guidance/correction help accuracy.", "questions": "1. What fraction of ActionSteps trigger each condition on GAIA (by level) and on each external benchmark? Please report a table of invocations per action and supervisor token cost so readers can reason about net savings.\n\n2. Why was the ablation study limited to the 30 most token-intensive tasks from the GAIA validation set? Do the individual contributions of the Correction, Guidance, and Purification modules remain consistent when evaluated across the entire dataset, including less token-intensive tasks?\n\n3. The current filter is rule-based. Have you considered a learning-based approach for the supervisor? For example, could a policy be trained to decide when and how to intervene, potentially leading to a more adaptive and less heuristic-driven system?\n\n4. Have you tested a tool-rich agent configuration (e.g., retrieval-augmented browsing with multi-source aggregation)? If the majority of cost/latency comes from tools rather than LLM tokens, does the supervisor still help?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper addresses a critical and highly relevant challenge in the field of agentic AI. The prohibitive token cost and lack of operational robustness are major barriers to the real-world deployment of complex Multi-Agent Systems. The work's focus on runtime efficiency is a valuable contribution.\n\n2. The empirical validation is extensive and rigorous. The authors test their method on six different benchmarks spanning multiple domains, which strongly supports the claim of general applicability.\n\n3. The paper highlights reduced variance and a more compact per-problem token distribution after supervision—an under-discussed but practically important property for cost predictability. \n\n4. The prioritized LLM-free filter (sub-agent summary → error → inefficiency → length) is a neat engineering idea to avoid constant, costly oversight while still catching the most harmful failure modes. The concrete heuristics (e.g., length threshold ≈3,000 chars; loop/inefficiency checks) are spelled out sufficiently to reproduce.", "weaknesses": "1. The core idea—a meta-controller that detects errors/loops/long observations and either prunes context or issues guidance—resembles prior runtime oversight, routing, or budgeted-reasoning controllers. The paper would benefit from a sharper contrast to existing runtime monitors/filters (as opposed to offline compression or architectural agent redesign). As written, the conceptual delta may feel incremental.\n\n2. The authors justify Smolagent because it relies less on “powerful external tools,” isolating the MAS. But in many real MAS workloads, tool calls dominate; it is unclear whether the same gains hold with tool-heavy agents, where tool output is the main token pressure and mis-tooling is the dominant failure mode. The paper acknowledges tokens meticulously but not end-to-end cost/latency including tool latencies and external compute.\n\n3. The authors justify using Smolagent as the base framework because its capabilities stem from \"internal agentic interactions rather than powerful external tools\". While this provides a controlled environment, it also means the baseline might be inherently less efficient and robust. Applying SUPERVISORAGENT to a weaker baseline could inflate its perceived benefits.\n\n4. The prioritized conditions (e.g., 3,000-char cutoff; loop detectors) appear hand-tuned. How sensitive are results to these thresholds across tasks and models? The paper claims model-agnosticism but does not deeply analyze failure cases where the supervisor fires too often/too rarely on different backbones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)